Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
You have to decide
21 Plays5 months ago
Code migration sounds like a technical task, but it’s a strategic minefield.
In this episode, we explore how code migration projects fail not because of bugs, but because of broken decision-making and unclear architecture.
You’ll hear about raw stories from engineers at Slack, Datadog, and other world-class teams who’ve battled through the chaos of code migration under pressure.
If you’re relying on AI to do the thinking for you, this episode will make you rethink everything.
Whether you’re planning a rewrite or deep in the trenches, you’ll learn how to survive your next code migration, and maybe even lead one right.
Get Tern Stories in your inbox: https://tern.sh/youtube
Recommended
Transcript
The Risks and Opportunities of System Migrations
00:00:00
Speaker
All system migrations are terrifying. If you haven't done one, you probably don't want to because you get it wrong, it's game over security-wise almost immediately. If you get the design wrong as a small company, it can genuinely kill the company. AI has brought an enormous lift in that noise floor. If every engineer can generate 2x, 3x, 5x more code, they obsess over software design. And they do this because bad software design is the Achilles heel.
00:00:24
Speaker
They're using ai to learn more. They're finding ways to see data, to move quickly, and make higher quality decisions. Because that's the bottleneck, right? Today's AI coding tools aren't built for engineers.
00:00:35
Speaker
Look, I love that Bolt, Lovable, and V0 make it easy for anybody to build a web app. I think it's awesome to watch GitHub's keynote and hear them talk about the next billion developers, 20 times more developers than there are today.
00:00:48
Speaker
And I have tried so many AI IDEs. And as a former PM, you what's exciting? I get to prototype. I get to do my own design. and I don't have to involve engineers to get an idea out there.
00:01:00
Speaker
But I was an engineer once. These tools aren't built to do engineering. They steer you away from the act of designing systems and the craft of building software. So. This is Turn Stories, and I'm your host,
AI and the Future of Software Development
00:01:12
Speaker
T.R. Jordan.
00:01:12
Speaker
Today, i want to take a step back. I want to share some of the lessons from people I've talked to in my job at Turn as founder who haven't shared their lessons publicly, who haven't come on this show yet, talked about everything that they've done at their past job and how successful it was.
00:01:30
Speaker
Because since I've started this company, I have talked to dozens of people who are using AI to work on code migrations and ambitious projects. And their stories paint a clear picture of where software development is headed and what we'll need to be successful in an AI-nated world.
00:01:47
Speaker
So let's start with paradox. And here's what I can't square. you ask who's the most cracked coder you know, and you'll hear breathless stories about how they've got half a dozen agents running, they're manically multitasking, they're cranking out code 10 times faster than anyone else on their team.
00:02:04
Speaker
They're opening PRs in random order and no one can keep up. First of all, that's a little bit red flag to me. But more importantly, that whole world doesn't make sense.
00:02:15
Speaker
The best engineers that I've worked with, they demand focus. They need flow because they think deeply about a single problem and they pour over the details and then they solve it.
00:02:27
Speaker
They obsess over software design. And they do this because bad software design is the Achilles heel. If you've worked at a large company, you've seen this in action.
00:02:38
Speaker
Those companies are there in spite of the bad decisions they made early. Yeah, 80% of the company is flying, creating features, shipping, the marketing department can't keep up with all the launches.
00:02:49
Speaker
But if you ask the team to go change the color of a button in the admin panel, everyone kind of stares at each other, grimaces, and realizes it's going to take a quarter to make even the smallest change.
The Importance of Software Design
00:03:00
Speaker
And those are the successful companies. If you get the design wrong as a small company, it can genuinely kill the company. Look, if you work at a small company, yeah, you got to get the market right. You got to keep your users happy, but also,
00:03:12
Speaker
If you've made foundational assumptions about how the software should be built, where you didn't anticipate how it might grow, you're not going to be able to change the software. You're not going to be able to change the product.
00:03:23
Speaker
You're not going to be able to ship the features you need to keep up because it's hard to build successful software. And look, I've got sympathy for teams that are painted into a corner and have to rewrite. I've been there.
00:03:34
Speaker
When I had sold my first startup, I ended up managing a team that had to rewrite the reporting system of... a multi-million dollar SaaS business. And you know what?
00:03:45
Speaker
We had to eat a quarter. And we chose to eat that quarter not because it was exciting. We thought it was going to be easy. Everyone was mad about it. But for the previous year, we had been trying to add features to a reporting system that simply wasn't designed to support it.
00:04:00
Speaker
Where somebody had he either sat down and made bad decisions because they didn't spend the time or they simply got unlucky, which happens. I understand that. We managed to get through that project. We shipped wonderful new reporting dashboard.
00:04:13
Speaker
Half the team, myself included, left after that because it was such a slog. It is a tough choice to try and figure out what to do with bad decisions. And as a result of that, the experience of good software engineers really comes through when you start a new project.
00:04:28
Speaker
They realize you can't just jump in and code. You need to think about the bones of the fact. And when I think about the world of AI, to go back to that paradox, the simplest explanation here is so confusing.
00:04:39
Speaker
It sure seems like a lot of people have fully delegated the craft of design to AI. We can't do that. yeah Yeah, AI has gotten strong enough that it can get your syntax right on the first try, that it can write your tests.
00:04:53
Speaker
But there is no way that a person running the AI has all the context to make a perfect decision, much less has taught an IDE like cursor or clog code enough to make those correct decisions.
00:05:04
Speaker
So the effect is that we are in this renaissance of tinkering and prototyping, but it's easier than ever to skip the hard work of architecture, feedback, and planning. And it's great to sit down on a weekend, describe the user experience, get some acceptance test running and get an app running, but that misses the part where you care about how the thing is built.
00:05:22
Speaker
So not only you need to make sure to think about the systems in order to make sure your company can succeed you can be successful in your job, but if you skip the part where you care about how it's built,
00:05:34
Speaker
That just feels wrong. And that is what I want to dive into today. Let's think a little bit. and Let me tell you what I have seen in the best companies and the best people today who are dodging this roadblock, who are figuring out how to get around it.
Decision-Making in Complex Projects
00:05:47
Speaker
I've been a part of a lot of gnarly and ambitious projects. The hardest part is always making the decisions. I have seen guests on this show, E-Edit Datadog, spent an enormous amount of time picking which product to go first to their new data store.
00:06:02
Speaker
How do you ship Python 3 to 150 microservices? Anthony told us that they made decisions about how they would ratchet progress, how they would lock in their work. And even at Slack, I ran a team of 20 engineers.
00:06:16
Speaker
Gov Slack was a three-year project. We had rebuild Slack and AWS GovCloud. You know what? We stalled for a year because we couldn't make the right decisions. We couldn't figure out what services to put first into the new infrastructure.
00:06:30
Speaker
And you know what? The way we got through that is through the work of great engineers. I worked with Sean Sabut, who's an amazing person. He sat down and he said, you know what? There's constraints here and I just need to make the decision about what we're going to do.
00:06:45
Speaker
And he wrote a document. He said, we're going to put in the foundational networking first. We're going to stand up service networking after that. We're going to build the data stores after that. we We had a plan. Yeah.
00:06:55
Speaker
It could be argued, it could be questioned, but once we had it, it unblocked not only 20 person cloud team, but 800 person engineering team at Slack to go tackle something hugely ambitious.
00:07:08
Speaker
I will forever be grateful to Sean for being willing to stick his neck out and say, this is how we're going to do it, even though he could have been wrong. I get it. It's scary to make these decisions. That's what comes up over and over again is because the problem is underspecified.
00:07:22
Speaker
The business always gives you constraints. You always have technical constraints. There's running code, there's running systems. But know what? There's still multiple choices you can make. There's not an easy right answer. You can't train an AI on to do the right thing when you can't even give it the right data or data about what the right answer is.
00:07:41
Speaker
So humans struggle with this too. There's almost always more than one viable choice. And even as you go through and make these decisions, it's not a one and done problem.
00:07:52
Speaker
You don't make the decision at the beginning of them the project and then say, this is how it's going to run. The decisions are fractal. They're continuous. There's always a detail that you need to hit. How we structure the indexes in a database can be incredibly long lived.
00:08:07
Speaker
What you choose not to fix is almost as important as what you choose to fix. We were talking about Slack's message migration with Madeline Wu Short just a couple of weeks ago, and you know what? She decided not to fix anything else she saw other than putting every message into text.
00:08:22
Speaker
And that's the only reason they got it done. That's the kind of work that you have to make that's going to be questioned in the moment. And in hindsight, makes you look like a genius. That learning those kind of things is engineering.
00:08:34
Speaker
You try and learn things as you go through the project. And the hardest things to learn are sometimes about the running system. What exists in production? Is there a logic bug or a race condition that's actually load-bearing?
00:08:45
Speaker
Until you ship the last feature flag, until you rip out the last piece of code and shut down the old infrastructure, there is something to be learned. There's a decision to be made. And some of these decisions can be can be made. And if they won't matter, but a few of them matter a ton.
00:09:00
Speaker
And you won't know until after you're done. So you have to care about every single one of them. And that's what I again, we hear this over and over and over again from the people we talk to at turn, from the guests that are on this show and anyone I've run into over the course of my 20 year career.
00:09:16
Speaker
And that's where I hear people struggling today. AI output looks viable because you skipped the part where you learned about the problem. They dive in immediately. They do change code.
00:09:27
Speaker
So they change code. And even if they write a little to-do list first, there's no space for planning. There's no space for you or the AI or combination of you and the AI to make thoughtful decisions.
00:09:38
Speaker
And that is where there is this enormous opportunity. But I have seen a few things that are absolutely inspiring. The best developers I know most interesting stories I've heard in the last year are the people who use ai weird.
00:09:53
Speaker
They are building mini products and they are wrapped perfectly around their brains for their problem to wrangle the complexity of the systems that they are looking at right now.
00:10:05
Speaker
They are learning using AI. Let me tell you a couple of these because almost every single one of them is absolutely unique. I talked to a security engineer. He works at a company with thousands of developers, hugely complicated product that absolutely heard of.
00:10:19
Speaker
He wires together half a dozen MCPs or more at a time because he's walking through off-system migration. All system migrations are terrifying. you haven't done one, you probably don't want to because you get it wrong, it's game over security wise almost immediately.
00:10:33
Speaker
So what he does is he's not only looking at the code, he's not only looking at the specs of the new system, he's looking at live logs, he's looking at runtime. He's trying to understand what can be changed about this particular call site.
00:10:46
Speaker
And then only then is he automating it. with the full context at hand. Making wonderful progress, no bugs, no incidents so
AI's Role in Complex System Management
00:10:54
Speaker
far. I talked to a product engineer. He is tasked with driving the technical strategy across a group of 50 engineers, principal.
00:11:01
Speaker
He sets up EM, PM, exec, and principal engineer agents within his IDE. He gives them roles, he tells them what they typically care about, and he has them argue with each other over product specs.
00:11:14
Speaker
He has them written, Mark, down in docs, right? You write an RFC, he wants to get something done. And what he'll find is that he'll find that the personas within the AI will actually expose new and different flaws in the plan.
00:11:27
Speaker
And because the AI can read so much, they'll have the executive ai arguing, well, here's our OKRs for the quarter. And you know what? I don't think that aligns with task 13. Maybe we should skip that one.
00:11:39
Speaker
And more often than not, it's right. That information allows the whole system to get better and the spec to get better. So they can actually deliver something that keeps everyone around him happy, which keeps him happy.
00:11:51
Speaker
I talked to an infra engineer who is almost the archetype of that cracked coder that I described earlier. She's got half a dozen terminals running at any given time, loves quad code, and is just cranking out PRs day in and day out.
00:12:05
Speaker
But it's not just her. She has figured out how to write runbooks for all of the infrastructure. Her entire team contributes to her. The only reason that she can keep all of that system running is because she spends most of her brain power working on the runbooks in order to drive context to her agents and her team's agents.
00:12:25
Speaker
They share rules, they share snippets, and they work within the system in order to drive their agents to go find the right context. This context, by the way, is things like live information from Kube Control or from Logstash. They're querying production systems.
00:12:40
Speaker
And that's the only reason they're making progress here. Because if you've worked in infrastructure, you know that it is extremely twitchy work where the state of the running system drives so much more than just the the code that you have on disk. And then the last one I heard recently is a tooling engineer who actually added auto-regenerating hierarchical descriptions to every repo and every subdirectory within his company's code base.
00:13:05
Speaker
The reason he did this is because he found that there was an enormous amount of information in comments and documents that was categorically important to even touch any of these repos.
00:13:16
Speaker
you didn't know that this frontend was the one frontend at the company that used PNPM instead of NPM, then you were going to mess it up and you going to be able to deploy it properly. And the AI ages were even worse about it.
00:13:28
Speaker
So by synthesizing the information that was necessary to work within any of those projects, suddenly it unlocked enormous amount of work of the people around him. Now, I'm not sure what you heard in those stories.
00:13:40
Speaker
What I heard... is that the common thread here is that they're using ai to learn more. They're finding ways to seek data, to move quickly and make higher quality decisions faster because that's the bottleneck, right?
00:13:52
Speaker
These are senior engineers who have done amazing things and they have scar tissue where they realize if they don't make good decisions, they will find themselves struggling to land the plane.
00:14:03
Speaker
Even worse, software ecosystem gets more complex every year. It's always harder year after year to get all the information you need in order to make those decisions. And even though that's true gradually, it's been true for as long as I've been working in and software, a step change happening right now.
00:14:19
Speaker
AI has brought an enormous lift in that noise floor. If every engineer can generate 2x, 3x, 5x more code, the amount of stuff you need to know that you don't know today has gone up enormously, exponentially.
00:14:32
Speaker
And the only way to look through that is, surprisingly, We have robots that can read novels in seconds. It's on us to use them. So beyond even interrogating and synthesizing data, that's all wonderful. Every one of these engineers that I talk to, they're using AI influence.
00:14:52
Speaker
It's not just decisions that they write down. They don't write a markdown doc and say, like oh, this is how we're going to do it. They set up the structures that allow everybody around them to succeed. They're setting the direction about what to build how to build it and I think the the best part about this is in this world, they get to lead from the front.
00:15:11
Speaker
They are still writing code. They're writing the important things and showing people how it is done without having to sit behind game of chart and a document and say, this is how you should do it. I don't have time to actually implement the code.
00:15:24
Speaker
We're in a new world where engineering can be both architecture and hands-on keyboard execution, and it is the most effective way to work. And that's what keep hearing. That is where I see the paradox breaking is that those engineers are not using a ton of parallelism and and a ton of AI in order just to crank out code.
00:15:45
Speaker
They're using it to make the decisions that makes their code better because they're building amazing things before they even write a line of code. So what does this mean for the future of software development? I think that's the blueprint, is that how do you take AI and make it part of the development process natively?
00:16:03
Speaker
Because the next decade of software is going to be even more ambitious than the last one. Building it is going to require a fundamentally different approach than even today's most AI for corporate developers have found.
00:16:15
Speaker
We have more tools than ever. We can create amazing experiences, but our users are divinely discontent. as I'm sure you are too. We expect so much more polish, so much more depth, so much more thought.
00:16:28
Speaker
Do you remember the first time you used an AI app? It was magic. Okay, Apple's notification summary has always been absolute garbage. But the first time you used ChatGPT, The first time you saw a meeting summary.
00:16:40
Speaker
The first time you used a chatbot and it wasn't obviously a computer on the other end of it. I don't know about you, but I can recognize AI slop. I know when it's inauthentic, when someone's done pass over their LinkedIn post with ChatGPT, when someone's written code that was vibe coded with three line prompt, we understand how that bar has gone up.
00:17:02
Speaker
The bar for products will be so much higher in the next decade because we have these tools to build amazing things. And we need to use them to not make things cheaper, to not build things faster and churn out shoddy products. Our users will demand that we build software that we could not even conceive building five years ago.
00:17:19
Speaker
And in order to do that, we need a new software development process. We need to create a space for the important work of making great decisions in what is obscenely noisy environment, with AI in the center of that as a true partner.
00:17:35
Speaker
And if we don't, we're not going to be able to create that software that our users expect. So let me talk about what that space looks like, because it is, for all the noise, it clear what needs to be done.
AI in the Creative Process
00:17:46
Speaker
This space exists at the beginning of the creative process. There's three pillars of this space. First one is it starts with exploration. You need to understand what's going on. This is where AI can play a huge role.
00:17:59
Speaker
Imagine a world where you can connect to your production systems, to your communication systems like Slack and email, searching the web, searching your internal company information, and truly connecting with the constraints of the system to understand what needs to be done and understanding what is the game before you even start to play.
00:18:18
Speaker
The second thing is how do you store this information? Today, you can store it in JIRA, you can write it in it in a document. Those are basically dead pieces of information. In the future, for lack of a better word, let's call them tasks.
00:18:29
Speaker
Tasks will be self-executing because you need a home for these tiny programs that say, hey, I know what I want to change. I know where it needs to change and here's how to change it. Reductively, this is a regex and a system prompt and that's all you need.
00:18:44
Speaker
But you need something that can fix its own problem. Because that is what a great plan looks like, is that it's not it's not simply the context and the desires. In order to build a technical plan, you need to tie it to the existing system.
00:18:58
Speaker
And that means in code with the ability to change code. And then finally, you need to execute it. This space needs to create a home for prioritization. There will be so much that we can do and we can't do it all today, if nothing else, because we don't want to overwhelm users.
00:19:15
Speaker
But we want to be able to ship many things in parallel and we need a space to understand, is it possible to take a plan, decide whether it's shovel ready? And among all the plans that are, what is the sequence?
00:19:29
Speaker
What are the teams that need to care about this and bigger companies? How do we play nicely with the chaos of a code base? where 30% of the lines of code have been written in the last month. This space might sound a little like project management or source control or observability.
00:19:43
Speaker
And it's not any of those things. This space exists at the intersection of those tools because it's not about the data of tickets or code or telemetry and making that available to every stakeholder in an organization.
00:19:55
Speaker
That's what existing tools excel at. This is about doing engineering that's fully informed by that data and fully leverages AI. That's not what those tools do today. That's not what any existing tool does.
00:20:08
Speaker
But if we do this right, we create this space, we not just steer AI into crafting good code, but we create wildly better software. We give every engineer a home to plan out the best version of the software they want to write, to truly do the craft of building and to design systems that will be responsive and strong in the face of any changes.
00:20:32
Speaker
And every engineer will get to stand on the shoulder of giants. Not just those that came before them, but the ones that stand next to them today. Where we can make AI work for multiplayer teams, which software has always been a multiplayer sport.
00:20:45
Speaker
We can make changes with small teams that were absolutely unthinkable to tackle even two years ago. And that's why we're building TURN. This is that space to do engineering, where you can decide not just what to build, but how to build.
00:20:58
Speaker
And hope that that resonates with you, because it resonates with everybody I've talked to. And it's based on the stories of the most successful people I've heard so far. If that sounds like something you're missing, i would love to show it to you.
00:21:11
Speaker
Leave a comment, get in touch, and let's talk about it and let's build this future together.