Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
More Code, More Problems: Software Delivery in the Agentic Era
1 Plays3 seconds ago
In this episode of the Kubernetes Bytes podcast, Bhavin talks to Rob Zuber, CTO at CircleCI about all things Software Delivery. The discussion starts by talking about the evolution of Software Delivery over the last decade, but then dives into how Vibe coding impacts CI pipelines. Rob also shares his insights about running an AI-first engineering organization at CircleCI. Listen to learn more!
Check out our website at https://kubernetesbytes.com/
Show Notes:
- https://www.linkedin.com/in/robzuber/
- https://circleci.com/
- https://circleci.com/blog/
Recommended
Transcript
Introduction to Kubernetes Bytes Podcast
00:00:03
Speaker
You are listening to Kubernetes Bytes, a podcast bringing you the latest from the world of cloud native data management. My name is Ryan Wallner and I'm joined by Babin Shah coming to you from Boston, Massachusetts.
00:00:14
Speaker
We'll be sharing our thoughts on recent cloud native news and talking to industry experts about their experiences and challenges managing the wealth of data in today's cloud native ecosystem.
00:00:30
Speaker
Good morning, good afternoon and good evening wherever you are. We are coming to you from Boston, Massachusetts.
Guest Introduction: Rob Zuber, CTO of CircleCI
00:00:35
Speaker
Today is June 17th, 2026. Hope everyone is doing well and staying safe.
00:00:41
Speaker
oh I hope you guys are watching the Soccer World Cup or the FIFA World Cup. I'm just catching up on highlights never really been a fan of soccer as much, but man, the the videos of, so so Boston is one of the city that's hosting the World Cup this time, right? So ah we have people from Scotland coming over and having a good old time, man, just looking at at those videos on Instagram.
00:01:04
Speaker
makes it look so much fun. um And then same with with the Norwegians. i think I think they are the next group of people that are that are having, i don't know, they're having fun in in Boston, going to Fenway games and staying there till like 3 a.m. m until the cops have to ask them to leave. So um I'm hoping that you guys are also having a good summer ah like everybody else who is supporting their team.
00:01:27
Speaker
Good luck to the team that that you are supporting. ah But for today's episode, right we have a great guest lined up for you. an interview with ah the chief technology officer of CircleCI, Rob Zuber. And we'll ask him all about how software delivery has ah evolved, especially in the age of ai but more ah ah more importantly, over the last decade, and how he adopts these new tools and systems inside his own ah engineering organization to to ah speed up development of CircleCI as a product.
00:02:00
Speaker
So without further delay, let's get Rob on the podcast. Hey Rob, welcome to the Kubernetes
The Evolution of Software Delivery
00:02:05
Speaker
Bytes podcast. Why don't you take a second and introduce yourself to our guests? yeah of sun Yeah. I mean, they can all be guests. We can all be in this together. um yeah Thanks so much. Thanks so much for having me. So I'm i'm Rob Zuber. I'm the CTO of CircleCI. i I've been at CircleCI coming up on 12 years. So it doesn't really feel like what I did before that matters. um You know, I love software. I love software delivery and ah excited to talk about all of it today.
00:02:34
Speaker
Oh, that's awesome. So you you said it first, right? Like you have been like close to 12 years at CircleCI. So I want to get your thoughts, right? Like you've been deep into this software delivery ecosystem. How has that landscape changed? And what's the biggest shift you're seeing right now, right in 2026?
00:02:51
Speaker
Yeah. Oh my goodness. Yeah. so So I'll talk about some others just to give some context to how big this change is. right like um When CircleCI started, I wasn't there, but I was building software at the time and kind of doing the same thing. it was ah It was an era of Ruby, Monoliths, Rails, primarily ah pushed to Heroku. That was the the problem that... And CircleCI ran your test for you between your, you know, Rails monolith and Heroku so you could stop using Capistrano.
00:03:19
Speaker
Um, it's a little shout out for anyone who was around then. And, um, and then, you know, things kind of went to mobile first and that's how I got to CircleCI was I had built to a mobile platform with a couple other folks, CI, CD platform. And then,
00:03:32
Speaker
um containerization, right? yeah We were using containers, LXC, under the hood, but then Docker came along and made that ah you know accessible to everybody. yeah um And so there was like...
00:03:44
Speaker
um That was a big shift. People moving to microservices, Kubernetes and orchestration, like all of those things were fundamental shifts in how people delivered software. And so that impacts how we deliver what we deliver, how we build for those customers, but never impacted our mission, right? We're trying to help people get software in front of customers fast with confidence, you know, be able to iterate speed wins in the marketplace, you know, speed is the moat, all the things that we keep saying, but have always been true.
00:04:15
Speaker
and um And none of those, I think, hold a candle to like, we stopped writing software. Right. And like, like the most, the most extreme version of what's happening right now. But, you know, we're, we're pushing the envelope as much as we can just to understand where this is going.
00:04:30
Speaker
um and What does it look like if people don't write a line of code, if they don't look at the
Navigating Microservices with CircleCI
00:04:35
Speaker
code, like what are the machines that we build to support sort of a ah generative approach to software.
00:04:41
Speaker
um how do we How do we truly get to that next layer of abstraction? Yeah. Right? Like... um
00:04:49
Speaker
there's kind of been like a decent amount of rhetoric as we've, ah you can see individuals thinking evolving online, right? About like, this is not myself. This is not abstraction. I think that's still true. yeah I trust my compiler because it does the same thing every time. If I just type random English into ah you know, an LLM, I'm going to get a different result every time. And that's okay if you build a system that's designed to handle...
00:05:12
Speaker
non-deterministic results, right? To use that as a fantastic piece of a bigger system. And let's be honest, humans also did not produce deterministic results, right? Like you, the product manager, tell me, the engineer, hey, let's go build this thing. And I'm like, I got you. And then I go build something that has nothing to do with what you wanted, right? Like that's a problem as old as software creation. And so then we tried to build systems and tools around that, right? And so I think that's, a again, the biggest fundamental shift that we've gone through as an organization.
00:05:42
Speaker
um and in the sense of of pursuing the same mission, but both very different implementation for our customers and not just the size of the shift, but the rate of continual change, right? Like we're in the middle of this saying, could go this way, could go that way. Both those things would be interesting, right? So how do we support our customers in that in that journey?
00:06:03
Speaker
OK, but like before we go into the ai discussion, right, because i do want to get your thoughts on it. ah Since you brought up like, hey, people are building monoliths using Ruby on Rails and then you had to help them transition over to microservices.
00:06:18
Speaker
Even even that was a huge, like again, it doesn't hold a candle to what's going on right now, as you said, but ah even back then, that seemed like a really huge shift. What was the adoption patterns? What were some of the challenges, right? Like, did you have to train and enable your your customers to, hey, this is this is okay. This is the new way of doing things and you should ah come on board or you'll get left behind. Like, how was that discussion?
The Role of AI in Software Delivery
00:06:42
Speaker
I would actually say that's one of the things that's different is that I mean, the jury's still out on microservices, right? Like people are still like, i don't know, we did it and then we went back or we never tried it because we wanted to do this instead or, you know, and um and so the adoption was a little more like,
00:07:04
Speaker
you know, what is the need of your company right now? Are you, you know, are you small and trying to move really quickly and therefore having a simpler, you know, smaller number of changes for any given sort of deliverable is valuable? Or are you operating at a really massive scale and therefore breaking out your system, you know, makes sense. trying to create more isolation of whether it's deploy pipelines or teamwork or whatever, right? And so um i i think...
00:07:29
Speaker
we didn't necessarily have to educate our customers because it was more like this might be right for you or it might not. And you could probably say that about, you know, AI now, butll we'll get that but we'll get to that later. But like everyone's adopted it. Like it's, it's a, you know, it's a,
00:07:48
Speaker
rush right everyone's trying so in that case it was a little more like we had to observe and we still have to do this just in a faster timescale now but we had to observe ah patterns that were emerging from our customers and understand okay like how do you want this to work what's going to support you the best here are some things that we we can do for you we did we did um because of the tooling changes, we we had to make some changes in how our platform was structured.
00:08:19
Speaker
um And like truthfully, we made our own shifts in terms of just our architecture. And, um you know, we were going through a huge amount of scaling and as a result, some breakage back in, you know, I guess it's like 2015, 2016, right? as like when,
00:08:33
Speaker
i think of that as like when containers were really starting to to blow up. yeah And then around 2017, everyone was like, okay, fine, Kubernetes, like the whole like, orchestrator war ended. And it was just like, this is the one. um So in that time, we were also as an organization scaling massively and and dealing with, you know, some of the challenges of that architecturally. And so we did our own partly breaking apart services, partly um redesigning how customers and builds and whatever interacted with our platform to...
00:09:07
Speaker
optimized performance for things like Docker building, which was just not a part of the, like building images you know was not a problem four years before that or whatever. Right. Yeah, exactly. I mean, people were like, they like, it was some weird thing that we were doing with LXC and containerization. No one knew what that was.
00:09:24
Speaker
And then all of a sudden everybody wanted to build a Docker image. And, um, And so we had to change, like, because of our layers of sort of virtualization, containerization conflicted at some point.
00:09:36
Speaker
ah Like Docker was originally built on LXC, but then it sort of branched off and used other tools and stuff. And so um we had to go through some some migration there. And there's another really interesting point in there for us, which I think is true for most developer tools companies, which is sometimes we choose to do things, right? We talked a little bit about our own microservices journey. Yeah.
00:09:59
Speaker
Not necessarily because they're the right thing for us, but because we need some experience. Do you know what I mean? like like yeah To support all these companies building services with Docker and whatever, and we're like, no, we're cool. We're still running Rails on Heroku. like We feel good about those decisions in hindsight, but sometimes we have to dabble in ways that as an engineering leader, I'd be like, put that down.
00:10:21
Speaker
We don't need that right now. You know what mean? Like, but, but it's actually really important for us to have a broad understanding of the different ways that people are building. So, you know, to your original question, we can support them through these kinds of transitions. and really Yeah. And then that's a great point. And because even at, at my day job, like when we talk to big enterprise customers, they're like, Hey, is your internal NetApp IT t team using this? Because if you're not drinking your own champagne or having that customer zero mindset, like, I don't think people want to talk about a vendor ah doing pitches about a new technology like hey is this real or not that we'll know that once we see your customers you know talking about it yeah yeah and it's such a fascinating thing um I think a lot of engineers I mean I'm an engineer by background and you know I got into this by building a tiny CICD platform for iOS like
00:11:12
Speaker
we love when we're scratching our own itch sort of thing. Like it's so it's so much easier to move quickly. um But you you also run the risk, you know, as customer zero of thinking that you are the model customer.
00:11:28
Speaker
And I know tons of organizations, whether it's software development tools or otherwise, who treat themselves as a model customer, but there's so many things about how they operate that don't represent the customer, right? Like,
00:11:41
Speaker
i um I heard this recently and and um with someone i was on a panel with that at OpenAI that you know the rhetoric internally is tokens are free. No other company is saying that right now. like No one is like, it's fine, just use as much as you want. and so you know, there's kind of like, I saw an article go by yesterday, I think about, we don't understand why people are so concerned about costs. It's like, how could you not see that? You know what I mean? and so same thing for us, like whether it's the like who pays the bill for RCI or um even how we, how we use the product, we build our product in a very specific way, which is built on sort of our principles. And every, every company has that identity, but it can be different. So you end up
00:12:26
Speaker
at some point trying to shape a toolkit that then allows your customer to refine it down to how they want to operate instead of saying, no, you're wrong about how to build software. Like do what we do, right? Like there's this fine balance of helping on people, giving them guidance versus being, you know, dogmatic about specific approaches that just aren't right for everybody.
00:12:49
Speaker
Yeah, makes sense. No, okay. So let let's now switch gears into AI, right? Like you said, you started with iOS CI CD platform. Now with, ah like I know, well like white coding became a very widely used term. I don't know if it's still popular or people have just called, are calling it coding agents, but let's talk about that, right? Like how has the role of CI CD changed because of all of this agentic generated code or model generated code?
00:13:16
Speaker
um How do you see that evolution and How does CI fit in? Yeah, such a good question. And you're right about vibe coding. Like we love in this industry,
00:13:27
Speaker
to be super pedantic about what a term means
Integrating AI Agents in CI/CD Processes
00:13:30
Speaker
and then have everyone use it differently until it means nothing. Right. Like we somehow do both at the same time. yeah um and And so I don't know what vibe coding means. Like if it's, I don't look at anything and I just lob it into production. Like that's a style of like, this might be great for a prototype that you're showing to your friend. Right. but like gets coding yeah I think, I think what matters now, regardless of what you call it, is that,
00:13:52
Speaker
serious software companies doing serious software work, however we want to describe that, are leveraging agentic coding, AI tooling as much as they can, yeah right? And it's a varied spectrum for sure, but as much as they can to try to accelerate deliver results faster.
00:14:13
Speaker
um And I think there's two roles that we really see ourselves playing in that. The first, as a result of the shifts that happened there, but first, as a gatekeeper, like a more traditional CI, you know, that's not going to production until you have some confidence in it.
00:14:30
Speaker
Now our ability to build confidence um requires some work on our part, which we're investing in pretty heavily, which is ah historically, you know, a human wrote the code, a human wrote the tests, another human looked and said, yeah, I see that there's test coverage here, et cetera. Now we can discuss how high quality that was. But there was some measure of sort of checks and balances, accountability on, by the time this goes into CI, when these tests pass and the linter passes and everything else, I'm going to have confidence that this is good.
00:15:03
Speaker
If an agent does that and no one has looked at it, like agents are, you know, optimizing towards an outcome. yeah You might end up in a state where you have a bunch of tests that do nothing or no tests at all and CI will pass, but that signal has been diluted by the quality of the testing, right? So actually measuring test quality, which I think is bigger than coverage, is a place where we've we've made some bets, we've made some investments and we continue to work on. So again, part one, gatekeeper, but it means a little more now. If we really want to like take our hands off the wheel, right?
00:15:38
Speaker
Sure, you could have your agent write a 2,000 line PR and someone could ostensibly read it, but they're not reading it, right? They're like grumbling and like checking Instagram and scrolling through and saying, i don't know, that looks fine.
00:15:51
Speaker
Ship it, right? So how do we how do we make it so that we don't even have to do that? That's like a whole thing that we're pursuing. But to do that, I think you need... again, more um deterministic ability to identify the quality of the testing itself.
00:16:04
Speaker
that's so yeah in In that framework, right? Do you see like red team agents or ah adversary agents trying to poke holes in the code or or come up with tests that make sure that the PR, like somebody scrolling on Instagram doesn't approve of PR. there is There is actually due diligence that's being done before something is added to production.
00:16:24
Speaker
Yeah, I think there's a bunch of signals that are, um um baked into your organization that you can encode in that, right? So some of it is deterministic um and some of it is deterministic driven by agents, if you will, which might sound weird, but like we we built a little thing, which is just a skill that's freely available to do mutation testing, right? Which is like a ah concept that's been around, I think since the seventies.
00:16:49
Speaker
But, you know, where you basically look at the code and modify around what you think are the edge cases, the code, and then see if it's caught by testing, ah right?
00:17:01
Speaker
So you're looking for dangerous parts of the code and then saying, i know there's supposedly test coverage here, but what if I actually break it, right? Does it get caught? And then, you know, you run the local test suites and when it passes that, you're like, that doesn't seem good. And then you push it to CI and then some of them will fail and you're like, great. And so you create all these mutations, right? Variants of the code and then filter out the ones that actually get caught. And then when when they aren't caught, say,
00:17:25
Speaker
great, let's add more testing to cover these particular conditions, right? So that that can't happen again. So that's like, um again, it's not totally deterministic, at least the way that we've done it, because we use an agent to generate all the cases, but that's what makes it scalable in a, you know, in a, to do something that again, we've talked about since the seventies, but no one is really like production productized, I guess. Right. So, so then, yeah, absolutely. You can use like, honestly, just asking your agent when it's done doing something to now critique it with supposedly slightly better results with a different model, but like, honestly, just put in the work, you know, we have like review skills and stuff like this internally as well. So
00:18:05
Speaker
and some of those things that we're baking into the product. So again, there's kind of deterministic views, there's non-deterministic views, but you want to build up as much confidence as possible. and then I alluded to um social signals in your organization, like who wrote the code, who who tested the code, right? Like, or who reviewed it, like all those things. Of course, those things are sort of dissipating. But if you know, another thing that we worked on was, um,
00:18:31
Speaker
we know who the most consistent reviewers are of a code base and we know what they say on that code base, right? Just extract all that, build that into context and apply it to all of your reviews, right? Like, why do I have the same staff engineer making the same comment on every PR? Like, that's annoying to them. They they don't feel like, oh, what a great sense of fulfillment that I just typed the same sentence. Like, they're probably trying write a macro, you know, so that they don't have to do it.
00:18:53
Speaker
And so there's a ton of opportunity around that. to you know and And what's interesting about that particular piece ah You know, there's a lot of code review, PR review products on the market today, but it feels quite late.
00:19:08
Speaker
You know, the PR is a surface designed for two humans to coordinate and discuss or multiple humans to coordinate and discuss a piece of code. I wrote the code with an agent. I have an agent available to me that can also do that assessment.
00:19:20
Speaker
Why am I putting it out to some human web interface so that agents with APIs can talk to each other and I can watch it scroll by? I should be doing something else of higher value, right? So I think that starts to push left is where that that lands, right?
00:19:34
Speaker
i would Why would I put up a PR? I mean, I don't know that the PR has a long lifespan from here on out, but why would I put up a PR just for agents to comment on it, right? i When I can review it before I ever push change.
00:19:47
Speaker
So I can pull all that down to the left. And that's where it said like classic CI gatekeeper part one, we're investing a lot. And we launched something recently called chunk sidecars in validation in the inner loop. So at the,
00:20:03
Speaker
point that the agent is just working yeah it completes a cycle we hook into that cycle fire off a much smaller version of the build that you would ultimately run for the pipeline whatever based on our understanding of what's changed and therefore which
AI Adoption Strategies at CircleCI
00:20:20
Speaker
tests matter based on a rapidly available sort of always on instantly available environment right that looks and feels like your ci environment but can be paralyzed scale but is directly attached to your agent and then give that feedback directly back to the agent.
00:20:33
Speaker
And then targeting the feedback, right? Like you could get all the logs or we could tell you out of the logs or the output, here's the thing that probably matters to you, agent, in order to make the next your next move, right? And if it all passes, great, keep going.
00:20:47
Speaker
But we found an issue. Here's the very specific issue. Use that to make an adjustment and then we'll try again, right? And to have all that happen even before a human intervenes, so that by the time you're thinking about, well, I'll keep calling it a PR, but like now I'm pushing it to CI to then get something into production, I have really high confidence that my changes are good. Now it becomes the multiplayer problem, right? Which is where CI comes in of your agent wrote 2,000 lines of code and my agent wrote 2,000 lines of code and they're totally overlapping and conflicting in all kinds of different ways and not your sort of classic
00:21:21
Speaker
get a conflict, right? Which it will be. like I have the little chevrons and I'm like, cool, or the angle brackets, whatever you want to call them. Great, easy to find. But what really matters is the conceptual conflicts, yeah right? You changed the contract of that function, but you didn't change the function definition.
00:21:39
Speaker
So now my stuff breaks because it thinks it's going to return this and it actually returns something else. And that, you know, so that gets caught in testing or whatever. And so how to... maintain velocity while merging those together, i think is a really... And do you think that's a problem for a human, like a staff engineer to solve or are there tools out out there that that does this like rebase or ah remove conflicts or resolve conflicts?
00:22:05
Speaker
Yeah, I think
00:22:10
Speaker
I get pretty good results yeah by telling my agent... please resolve the conflicts. Like these two, ah something merged since this, since we made this change, we, like I did any of it. Right. So like, since we did this thing, buddy, we worked on it together. Right. Like, so now can you pull the latest rebase deal with the get conflicts? Yes. But then also run the test, run the linter, check that the test still makes sense h and look for any issues and then identify what those are.
00:22:39
Speaker
Right. Like, yes, as a, as a, I'll call myself whatever. I'll pretend that I could be a staff engineer. right like yeah As a seasoned engineer, I could do that work.
00:22:49
Speaker
okay But if I understand enough about what's going on, and that's an interesting thing that we have to figure out, right then I can guide an agent through doing it.
00:23:03
Speaker
Or say, hey, like what do you see? Let's reason about it. Ask me questions. But you know like rather than saying, okay, now let me read 4,000 lines of changes, which is yours plus mine, right? Yeah.
00:23:16
Speaker
Looking for that one thing. like I think um one one area that i we're exploring all of this kind of simultaneously, this great thing about having a lot agents, um is, and I've talked to a lot of people who are looking into this or trying to understand it is some form of risk scoring, right? I'm trying to pick public examples, but ah Brian Scanlon from Intercom has ah a talk that he's done a bunch of times where they talk about their auto merge rate, right? Like what percentage of PRs that are agent developed yeah are just merged into production. Sorry, Intercom's now FIN, which is now Salesforce as of yesterday. So I guess i'll I just dated the, sorry for dating the podcast, but anyway, um so,
00:24:02
Speaker
has spoken about that specific thing. Like how do we continue to accelerate and how do we identify risk? But then within, it so if it like let's keep talking about 2000 lines, right? yeah There might actually be 1950 lines of super low risk boilerplate stuff.
00:24:19
Speaker
And one interesting thing. And if you ship me a PR today, i'm going to start at the top and go file by file. And by the time I get to those 50 interesting lines, I'm probably asleep. I'm on my third coffee.
00:24:34
Speaker
I'm like, whatever, this must be fine. Where I think we have the ability to say, go here. Don't worry about all this stuff. So like, like, we you know, we've been talking about it on a macro, like um quantized PR, and no PR, like this whole change set versus no change set. But I think within the change set,
00:24:53
Speaker
A huge amount of it is like, I've seen all this before. but Yes, there are tests. Yes, they cover this stuff. But this looks interesting. This looks important, right dear human reader.
00:25:04
Speaker
Come have a quick look at this. right Either it's a security issue or it's a piece of business logic that probably has opinions about it. right and I think in the way that we write and then in the way that we evaluate, validate, verify, whatever words we want to use,
00:25:19
Speaker
we want to flag the stuff that's important and let everything else go through. right Because if it's like 10,000 lines every morning when I wake up, like that's my job. yeah It's just reading PRs and it doesn't have to be all or nothing, I guess. I Okay, gotcha. You also, like in in one of the previous sections, where you spoke about chunk and the new thing that you guys introduced. Does it work with all coding agents? How, like, is it something that's available to everybody, even non-Circle CI customers? Like, can you talk about what that helps with? but Yeah, so um again, the goal is really just faster feedback, right? and Can I run my test suite or some portion of it?
00:26:00
Speaker
in a CI like environment. Right. um And I'll come back in a second, why that matters. yeah um And give feedback directly to the agent. Yeah. So it works with, I'll say all, but there's a new one every day. So like the, the, the mechanisms are very simple.
00:26:20
Speaker
I mean, it's, it's effectively a CLI that does the work behind the scenes. And all we do is like in I'll use Cloud Code as an example. a concept of stop hooks. We we enter or configure the stop hooks to use this, right? So most coding agents have some kind of tooling around that.
00:26:36
Speaker
um And we'll add support for others or people can, you know, kind of put it in themselves. SaaS, their agent to configure itself in most cases. That's what I always do, right? Okay. And so...
00:26:48
Speaker
Yeah, the idea is to get very quick, direct feedback. um it it is um You have to be a CircleCI user, so but you could sign up tomorrow for a free account and start using it. like yeah You don't have to have with an established sort of relationship with us or or whatever. told you um And...
00:27:06
Speaker
ah And yeah, again, the goal is to give the feedback directly to the the agent ah so that it I mean, you can run it by hand and sort of run all the your own tests in this remote environment. And the reason we want the CI-like remote environment, you know, I personally, as a, call myself a seasoned developer, right? Yeah. Ask the question, like, doesn't everybody just run their tests locally and they know that it all works by the time they push it to production, but our data says that that's not real right because what we've witnessed um when we put out this state of software delivery report and we're working on some new ones quarterly now because software is changing so fast um i think that's the cadence we're shooting for once every six months something because it's just like we used to do it every year and six month old data in this environment is just like that doesn't mean anything anymore right that's that's like eons ago might as well be talking about punch cards and so um
00:28:01
Speaker
What we noticed is, and I think people are feeling it, engineering leaders that I talk to anecdotally, significant rise in branch builds, meaning I did some work, I pushed the branch with a significant rise in failure rates on those branch builds, followed by meager increases, depending on the organization. Some organizations are blowing the doors up, but on average, which is never the story, on average, meager increases in main build. So like main is when people are getting stuff to production. ye And so what we're seeing is like when I think of a branch build failure, I think of surprises.
00:28:39
Speaker
I built a thing and I thought it was going to work. And I had to push it to CI to find out that it didn't. right So that might be, because it's complex and I don't have good understanding, it might be that my test suite is so big I can't run it locally. It might be a bunch of different reasons.
00:28:53
Speaker
We had an interesting case actually in the CLI that runs this thing where it was originally written in TypeScript. And the test suite passed on a Mac but failed on Linux. So like our builds failed, but they all ran for us locally.
00:29:04
Speaker
right Which is is like a, it's a real thing and you don't find out until you push it to CI. It was like a weird file system ordering like how it selects tests and there was a weird side effect between tests and we didn't discover it till CI and then, you know, luckily I use a Mac or a Linux development machine so I was like, cool, I'll debug it. Yeah, but But still, like that difference is real. So there's a bunch of different reasons. And um and so we want to make it possible to really increase that confidence. But running a full CI build, like people's you know total CI consumption is going up, ah is more time consuming, can be more expensive, depending on how your stuff is configured. So instead of running a full build on a branch, let's just execute the stuff that is high risk of failure.
00:29:46
Speaker
Okay. And be clear, now you're ready to go. Okay. So like you you're, because the investment you're going to make is kind of 30 seconds, again, in an instant on ready to go machine, as opposed to like, you know, most people's CI builds have a lot built into them. You're doing security scans and all this other stuff that maybe is not important on that first
Managing AI Model Updates and Costs
00:30:05
Speaker
pass.
00:30:06
Speaker
Right. But you're you're not getting that on your local machine for whatever reason, but definitely people are not. Makes sense. But, and you also brought up the concept of hooks, right? And I like, again, most people should know this, but for people that that don't write, like, can you talk about what those are? And then are there any other integration points in the way these agents are, our models are generating code?
00:30:23
Speaker
Yeah, there's, I'm not the world's leading expert on integration points, but so hooks and cloud code is, is something that I've spent a bunch of time with. And you basically have different events.
00:30:34
Speaker
The one that we use primarily is the stop hook, right? So at the point, that the agent has decided that it is done, right? But before, so it intercepts that before the sort of control is fully given back to the user, right?
00:30:50
Speaker
And so the agent thinks it's done, but if we return an error to the agent using the error codes, whatever, Then we can signal to it, this work is not complete.
00:31:01
Speaker
And here's some additional context. And that additional context is typically like this test failed. Here's the result of the test, whatever that might be. Or it didn't even run properly because you know you're calling the wrong command whatever, like was going to fix something.
00:31:15
Speaker
and um And so in the stop hook, you have a window to feed information back to the agent, it's it's I think the limits have loosened. It used to be a 60 second cap, like a timeout that was fixed, in the at least in Claude, which is really nice for us because it's sometimes it's good to have constraints. like How much value could we produce in 60 seconds is a really interesting question. right because people have been talking As long as I've been at CircleCI, 12 years, everyone it talks to me about their build time.
00:31:49
Speaker
but everyone has a different opinion, right? It has to be under 10 minutes to like, and it must be under two hours. I'm like, what are you doing in two hours? having a 60 seconds is like a really, really tight constraint to get you really valuable feedback and to say, okay, we're not going to run any of those tests. We're going to minimize the footprint. We're going to optimize little performance, all those sorts of things, which I think is great for us overall, but also great for the agents, right? So then ah there are other hooks like tool use, I think maybe pre and post tool use, um where you can say, oh, let's double check something or let's decide, you know, we and we're doing things like, um you know, checking if state has changed at all and whether it's worth even doing anything, right? So there's some, you know, so you can put in security bits, you can put in things like validation like this, um pre-commit, of course, pre-commit only works if you're using your agent to commit everything, right? But like, so there's a bunch of different tools or sort of integration points
00:32:46
Speaker
I think if you look at something like pie, it's even broader cause it's designed to be extended okay versus like, here's some hooks that you can integrate. Um, but everything has these sorts of opportunities to sort of customize your workflow or customize a workflow for your developers. And I think that's, um, something that we're witnessing now, probably partly based on where we sit, but, um,
00:33:12
Speaker
a lot of AI development has been very single player, right? Like you get this tool, this IDE, this whatever, and you use it. And then by the time you put up a PR, it's just like the old way, right?
00:33:23
Speaker
And the question is, can we extract value across teams and create acceleration and capability across teams, right? It was where things like hook configurations and, you know, skills, whatever.
00:33:35
Speaker
um So that it's not every individual developer feeling like a pioneer, you know, just exploring this unknown space. Reinventing the wheel. Yeah. Yeah. It's so interesting. Like i think all leaders bias there or have a bias based on their own point I mean, every human probably has a bias like, oh, everyone thinks like me, right?
00:33:58
Speaker
that And our early days of sort of AI enablement at CircleCI were, hey, there's this amazing set of tools out there, like get after it.
00:34:09
Speaker
Yeah. You know, use whatever you want, kind of that. We'll just check, we'll make sure the models are compliant and whatever. but like Yeah, see see what you like. We'd love to learn because we're all going to learn together.
00:34:20
Speaker
yeah and it turns out a very small number of engineers love to try all the things. And a very large number like, please tell me how you want me to do this. like yeah I don't want to figure all this out for myself. Someone else has clearly figured it out. right Just give me the tools.
00:34:36
Speaker
so, again, whether it's the hooks, the chunk sidecars, skills, just anybody can build skills inside their organization. Yeah. you know building and distributing those things to enable developers to get advantage of the tool without feeling like they're on the steep, you know coming from nothing learning curve. um Although I will say, and I say this quite a lot to people that like,
00:34:58
Speaker
feel like they're behind. um The learning curve mostly comprises of typing stuff into a box and seeing what happens. like ah There's never been a technology that's been so easy to try. yes
Measuring ROI of AI Tools in Software Development
00:35:10
Speaker
you know what i mean? And and so if if anyone's listening and feels like they're behind, just just start trying it. You'll be caught up in minutes. Don't worry about it.
00:35:18
Speaker
Yeah, and and i've I've gone through that same experience, right? On LinkedIn, there are all of these people posting about like different things that they're trying out, experimenting, running locally. And then obviously you get worried that, shit, am I too behind? And then you go to your chatbot, your subscription that you're paying for.
00:35:35
Speaker
And within like a few minutes, you're like, okay, okay, I have a good grasp, a good understanding of what these things are. And if I ever need to use it, I know where to start. I think, yeah, as you said, like it's it's really easy to to ramp up and learn about these things that chatbots and models themselves make it super easy. yeah So Rob, you also brought up like, I know we we started talking about this, but I wanted to, like, since you are the CTO, you lead an engineering team or organization at CircleCI. Like, how ah how do you guys use or have adopted AI internally?
00:36:07
Speaker
um and i also want to know about like, whenever a new model comes, right? Like, yeah, Opus 4.8 was great. But when Fable came, did you... ask everybody to just switch models and go with the best one? Or is there a pro and con analysis that somebody does and then shares those best practices? Like, can you walk us through all of that, please? Yeah.
00:36:27
Speaker
So, I mean, we I gave you a little bit. We started out with the like free for all. yeah And then we're shocked to learn that people weren't off trying everything. um And so then we put a little more direction around it. and We said, let's let's pick. I mean, again, with a little color of we're always...
00:36:44
Speaker
trying to make sure we're supporting the needs of our customers. So we said, all right, what are the products that are primarily dominant and being used right now? And this is maybe six, eight months ago. Let's focus in on cursor and cloud code. If you're like an IDE user, give cursor a shot for a minute.
00:37:00
Speaker
Yeah. If not use cloud code or try both, you can mix and match the models. I mean, we have obviously subscriptions to a lot of different things. um And then a smaller group, you know,
00:37:12
Speaker
as codex coming around, or let's go kick the tires on that. We had we had dabbled in AMP for a bit, but then they sort of like pivoted a few times and we we weren't sure if that was something we wanted to commit to. And so, you know, there there has been a little bit of that, but again, like for the masses,
00:37:27
Speaker
Here's how we think you should use it. Here's the channel that you can go to if you just want to see people stream of consciousness. I've tried this. I've tried this. Let's build a collection of skills in one spot that people can pull in sort of an internal marketplace sort of model. So a lot of different kind of sharing at that point.
00:37:43
Speaker
And then, yeah, when it when it comes to things like updates, I would say it's the same kind of distribution. Like,
00:37:55
Speaker
petat Well, there's two parts. So there's the distribution. A lot of people are like, I don't know, this is working for me. Like, are you sure you want me to change? Right? Like, cause even if it's, yeah it doesn't feel like much to point at a different model or whatever.
00:38:09
Speaker
um But it can be, i don't really like the results today. I felt like i was making good progress or I've built a harness and now my harness is not performing, you know, like those sorts of things. And then there's always the people that are like, oh, we got to try this out right away. Now, Fable Two funny things happened there.
The Shifting Roles Due to AI in Organizations
00:38:26
Speaker
ah One, which is consistent for us as well, and i think a lot of orgs now, is we're paying more attention to cost as we've had more success getting people to like really leverage tools. But we're like, cool, like we got to focus on ah ROI.
00:38:38
Speaker
Where are we getting real value? and And if you can do it with Sonnet, you know if we're talking to anthropic models, um or if you can do it with GPT-5.4 or GPT-Codex, then...
00:38:50
Speaker
use that instead of always pushing the frontier, right? So often the thing you're doing is not going to be noticeably different, but the price you're going to pay is twice as much, right? So let's not just jump to that. And so um that launch, the Fable launch, it's like, whatever, we all know what day it is based on my jokes about Finn, but like, was it kind of came and went so fast, but they launched it and reset, yeah i think everyone's default model to Fable.
00:39:18
Speaker
Mm-hmm. So it wasn't even an opt-in sort of thing from from our point of view, which might be fine. I think it was also like discounted or free or something for... And then two days later, was gone, right? And everyone was like, well, that was fun. and and so, yeah, everyone like back to business as usual sort of thing, um which does, I mean, that's that's an a very interesting dynamic, partly because of cost, now partly because people are like, wait a second, like this stuff can just disappear because someone else outside of our supply chain has decided it's unacceptable, right? So what does that mean for my...
00:39:54
Speaker
um I'll call it like business continuity is probably the best way to describe it, right? Like if my business is now built on delivering software with a set of tooling that I expect to be available,
00:40:07
Speaker
right? Like at some point, we're not even going to know how to reopen Vim and like start to, oh, I guess it's like a snow day, basically. like yeah There's no models available today. So I guess we'll just go like, let's have a design meeting and get some coffee, right?
00:40:20
Speaker
um And so I think that um that dynamic is really interesting. And so I'm seeing more like the routing layer, right? How are we like you, the user, individual user in the org are not making the decision about what's best from a model perspective, right? We're going to make that as a platform team or whoever kind of owns that. It might be IT, t might be platform, you know,
00:40:47
Speaker
developer productivity probably in most organizations, ah because we're on the hook for your delivery and our cost management, right? And it might be the case that Devstrol could handle this query for you. Like I could point at a fireworks or together or something that I'm hosting myself on Hugging Face, or this might be like a frontier query and I'm going to send it to, you know, the fables of the world.
00:41:12
Speaker
Um, so I got a little off of how we're doing it, but that's like, that's where like current thinking, right? but We witnessed this, we're seeing the price increase, which mostly we're super excited about, but we're having, making sure that everyone that's driving, you know, at they're, they're at the top end of our consumption. We're saying, Hey, tell me a little more about what you're working on. Like, is there something we could do to help you optimize your use of the tools? In a lot of cases there are.
00:41:35
Speaker
And then I think that's another thing actually. So you're seeing, model selection as a driver, partly because of resilience and business continuity, partly because of cost. yeah And then you're seeing, and I guess the routers or gateways are like a solution to that.
00:41:49
Speaker
And then a lot of tooling popping up around token optimization. And that could be, you know, i mentioned chunk side cars earlier. Like one of the things we notice is if we give feedback directly to the model as it's building, it drives to the outcome faster. So we don't use as many tokens. Yes, tokens. Right?
00:42:06
Speaker
People are doing like optimized MCP tools for code search and they're doing log compression. Like if you're going to put this stuff into the model, I'm going to strip out anything that's not a value in here so that you pass stuff.
00:42:19
Speaker
I mean, people are building these things for themselves and then open sourcing them or there's some businesses popping up around it. But I think there's a whole industry, right, in... AI optimization at this point. And like unsurprising, all we've made all these comparisons to cloud, right? Early days of cloud when it was just like,
00:42:37
Speaker
buy whatever you want. Wait, I could just keep clicking this button. I'll have more and more computers, right? And then someone got a bill and was like, what have we done? and there's still a massive industry in, you know, cloud cost management, fin ops, you know, whatever you want to call it.
00:42:51
Speaker
And that's going to roll straight over into um into AI consumption. And so, yeah, as I said, I got a little, we're doing that now. um And so we're like, I guess the balance I would take away from all that is,
00:43:07
Speaker
We're always checking to see if something novel is going to help us, but we're not pushing our whole team to the new novel thing. Okay. If you're using Sonnet 4.6 and it's getting results for you, like keep at it that's it. Yeah, keep at it because that's great for everybody.
00:43:22
Speaker
Okay, okay, nice. And like two more follow-up questions, right? One is how are you measuring a ROI? Like what's getting measured to
Staying Informed and Sharing Knowledge at CircleCI
00:43:30
Speaker
see if AI models are using ah ah AI coding tools is helping with your productivity or delivering better products. And the second is where do you stand on how or how do you interpret the whole consuming these models as a service versus spinning things up on your own infrastructure and hosting those open source models that Again, if you're talking about Sonnet or Opus family of models, right they are good enough and and have caught up and and maybe we don't need Fable in in all of these scenarios. How do you handle yeah those self-hosted versus consumption as a service type of decision?
00:44:04
Speaker
I'll say... I'll roll the second question into the first one. Like we're that we're dabbling and seeing what works. Like, and it's, again, I talk keep talking about the pioneers. I often use that term from like um pioneers, settlers, and plant town planners. Oh, yeah. an unworldly thing, right? So- So we have the pioneers who love like, hey wouldn't it be fun to spin up a model? Could we fine tune it? Like, could we do some reinforcement learning? You know, and that's like super valuable and educational for us in understanding what our customer's facing, how can we help them, et cetera.
00:44:36
Speaker
um But we're not we're not all in on one of those approaches at the moment. We're again, like, I think of it as if I can take control and route your queries, then I can make good decisions about how to route them. And then if I have that capability, then I'll worry about, hey, could I plug in a Kimmy, which is actually open weight, but pretty powerful, or like you know the...
00:45:01
Speaker
um devstrels, gemas, like smaller, kind of like, you know. Small language models. that kind of Yeah, exactly. Where in the middle here? Can I route specific tasks? Yeah. And then in our product also, we're looking at, you know, if we do so perform certain things, is it can we do something with like a classifier versus just handing everything to Opus and hoping for the best, right? And so, so there's all of that though is driven by the ROI question, right? If the return was so massive and so unquestionable, yeah that
00:45:33
Speaker
the investment didn't matter. we Why would we dabble? Like, why would we do any of this stuff? We'd just be like, let's keep coding, right? Just let's fable all of it if we had access to fable, right? Like, um and, I don't know I should say this. I'm a foreign national anyway, so I guess I'll never... ah Anyway, I digress. So um so like what you know it's driven by a sense of optimizing investment for return. Now, I think part of that, and this depends on what business you're in and where you are in the life cycle of your business, is the temporal...
00:46:09
Speaker
element, as in time-based element, not the reliability toolkit, but like the time-based element of return on investment. Yeah. Right. Like, yes, I am going to get more stuff built, but building stuff, like even if I can get that right. So like we have DX and, you know, we look at PR throughput and all these kinds of things and look at what people are But even if everything was brilliantly targeted and exactly what our customers wanted, which is obviously the case when we build software, it still takes time for that to generate revenue. Yep. Right. And at the end of the day, in order to pay the bill, I need to generate the cash. Yep. Right.
00:46:51
Speaker
And some organizations are so early or they just IPO'd for $75 billion dollars of net new cash or whatever, and then little bit on a different company. But like, It's very topical news today. so So like if I'm in that space, I'm like, whatever.
00:47:08
Speaker
just Or if I happen to be the owner of a foundation model, like just build, build, build. Don't even worry about it. But for many organizations, right, that are operating off of their profit, reinvesting that into growth, et cetera, et cetera, I can't assume that this money is coming into the bank as soon as I type that prompt. Yep.
00:47:26
Speaker
prompt And so there's always going to be some management there. And then that was all based on assuming that every time I type something, I build something brilliant for all of my customers, right?
00:47:40
Speaker
Then a lot of the ah ROI question is the classic ROI question of engineering that we've always struggled with, right? Like when we build this thing, isn is that going to turn into revenue? And I think where that The economics fundamentally shift on that is honestly that the cost of experimentation has gone down so much that we have a much better opportunity to figure out what customers want for a very low cost, right? Like if we were just running the classic machine that we always ran and like we've been saying this for weeks, but like in 2000 or whatever, we sat, we, I wasn't there, sat down and wrote the Agile Manifesto 2000, 2001, I think was. and um
00:48:27
Speaker
And then proceeded to build a whole industry of like consulting and textbooks and capital A frameworks or whatever, like capital S, most of them, right? If you're not doing it this way in Scrum and have this meeting on Tuesday mornings at nine zero o'clock, like you are not agile.
00:48:41
Speaker
Nothing about strict rules feels very agile, right? Yeah. And so we we built all of this around the notion that writing software is expensive and we want to be right yeah because we're going to invest all of this time and energy.
00:48:56
Speaker
But it turns out when you do that, you're back to waterfall and you're wrong. And the real goal is to build the quickest thing possible to validate your assumptions. And the fact that we can crank out that thing in like an hour right now,
00:49:09
Speaker
don't what it is, but let's just be flipping about it, means I can run a ton of experiments and drive down the risk that I'm going to invest a bunch of time and energy on the wrong path. right And that is a very difficult thing to measure, but you can feel it when it's happening. Do you know what i mean? so when we when I think about ah ROI, at least, I'm looking at it not just from the like linear compression level,
00:49:36
Speaker
of software creation time, yeah but rather my opportunity to eliminate or reduce, mitigate risk in my business, right? Which is pretty helpful at a time where that same force is creating a lot of risk in my business. Like everything's changing all the time.
00:49:51
Speaker
um And so then it gets down to very specific, like, what's that person doing? You know, and just like, hey, you're burning a lot of tokens.
00:50:02
Speaker
Yeah. Maybe you could get this job done with Sonnet or with Haiku even or whatever, you know, like Fable sounds
Podcast Wrap-up: Embracing Evolution in Software Development
00:50:10
Speaker
cool. I keep joking about Fable, but like Opus is great.
00:50:13
Speaker
You don't need to use it for everything. Right. List some basics or like maybe you could compress your context window a little bit because I think all this stuff is unnecessary and actually making it harder for the LM to do its job. Right. And so there's it's not like overall we feel pretty good about what we're getting.
00:50:29
Speaker
Then there's just outliers, right? Where it's a little education and training. And of course I wish like some gateway would just magically fix all these things, but let's be honest, at some point we have to use the tool as well. Yeah. um So yeah, I think we're like, I think so many things that are going on in my, in my head right now, like obviously there has to be a set of best practices that we all learn from it and then share publicly and obviously inside the organization, right. As you said, instead of having to share, uh, PBTs in a cloud conversation, ah cloud cover conversation, have ah have a MCP server that only extracts the things or a document, like, because it has to translate it into code and that and in, in,
00:51:06
Speaker
it increases the number of tokens it has to consume to analyze the same thing ah versus putting an MCP server. So obviously people are learning those lessons, but um I think there has to be more best practices, right? Every time instead of pointing it to the same code base, asking questions and having the model having to start from scratch, can it and build it in as part of skills.md file or something and and keep that information there. But um when you were talking about the the value, right, like will this help me drive more revenue? Like traditionally, if you look at it, right, i am I'm putting my product manager hat on. Like that has, that responsibility is also for like being part of one of the things a PM is responsible for. Like, is there a value ah in this? Like will customers buy it? Am I asking my engineers to focus on the right thing? Do do you expect
00:51:53
Speaker
bm's playing a role in this this decision ah for engineering, whether to not use models and ah or are to use them? Well, that question is so much bigger to me than the one you just asked, right? Which is like, what is anybody's role? Yeah. um And I think someone is going to have to make decisions about how to invest time and energy and what to build.
00:52:23
Speaker
right And I think a lot of people are dabbling, ourselves included, in shifts in organizational models. yeah right um We all know that ah increased number of nodes in a graph, you know drive I don't know, it's not technically exponentially, but like drives up the number of vertices, right? Rather quickly. Maybe it is expression this is like factorial. I don't know. I should probably look that up before I try to speak about it. It goes up a lot. as well as Use words that I know. goes up a lot, right? it goes up faster than the number of... so So more people on the team, right? More handoffs, more partitions in ownership means more communication, more overhead, more agreement, et cetera. And when you have... yeah If it takes...
00:53:21
Speaker
12 people, 10 people, I don't know how many people it takes to eat two pizzas, but if it takes that many people yeah to implement a thing, then having a tech lead and a PM and a designer and whatever, like whole group of people responsible for helping them deliver that thing makes a lot of sense, right? If it takes two people to deliver the thing and they're a little army of agents, i don't I don't want to go to one.
00:53:45
Speaker
I don't even know that I want to go down to two because people on their own will make very bad decisions, especially if they're just asking for Claude for advice or yeah you know chat GPT or whatever. This is going to tell you you're brilliant every time, right?
00:53:56
Speaker
and And so I think you want that people that group of people, but you want them kind of and little Little clumps, a few people working on something, and then the overhead of like all of the thinking happens outside of us shifts, right? Which is why, or or becomes so much bigger, such a bigger proportion, and which is which is why we're having so many conversations.
00:54:19
Speaker
There's so much rhetoric, you think about any way you want, about small teams, about product-minded engineers, right? Like the... Again, a capital A, Agile, capital P product, right? Like I am a product manager who sits here and thinks about what we should do.
00:54:37
Speaker
Starts to feel like a lot of overhead. They used to be able to stay ahead of like 12 people. Now you've got two people who are like, great, it's been 30 minutes. We did everything we talked about. What now, right? what next And so, and within that, right?
00:54:51
Speaker
Those engineers are making so, in that very compressed timeframe, they have to make so many product decisions. isn And so I think these things start to blend together. If you have a really good understanding of the customer and the problems that you're trying to solve for them, and you can, I'm going to totally dev, I am an engineer, just I want to say that very clearly. You can type in the magic box and make some code appear. like yeah You should be in a position to deliver and build in some useful, effective combination with some other people. and We talk about a lot about ah T-shaped people, which is a thing from, I forget his name, last name, Brown from IDEO back in the day. But like people who bring us strength, right? They're deep in one spot, but have some breadth.
00:55:38
Speaker
So you might be on a team, the person who's a little more design oriented. You might be on the team. I mean, then back in the day, you were the person who knew a little bit more about databases and everyone turned to you and we needed to write a new, hey, can you help me optimize this query? Right? Now you just ask cloud. But like, you might have a more design oriented person. You might have a more product oriented person who loves going and talking to customers and everyone else is like, ah could you just find out what they want? Right? Yeah.
00:56:01
Speaker
Although everyone should talk to customers. So You bring that together as a team, again, it can be a small team and it's highly effective. But if anyone is, I just do this one thing, then I think you're going to have utilization issues, right? That person is going to like, I guess I'll do this one thing for 20 teams because it's the only thing that I do. And I just don't, I don't know where that person's gonna going to fit in. Right. So, so it was like a super long answer to like the product thing. But I, I think,
00:56:29
Speaker
I don't spend a lot of time thinking about how we take what's happening and partition it out into the traditional roles that we have. I spend a lot of time thinking about what are the roles we're going to need.
00:56:40
Speaker
And they start to blend a little bit together, right? They're just less clearly defined, fewer bright lines. um And just someone in the team pushing to make sure that we get great outcomes, right? So someone has to be accountable, but I don't know that it's always going to be the person with the with the title. Yeah. Yeah. Yep. Makes sense. No, and Rob, I know we are like way over our time. There are so many other things that I do want to ask you, but I think for this discussion, we'll, we'll keep, ah we'll keep those safe results for the next discussion that we have. But I did want to get a couple of questions in one is with, with the the ecosystem evolving so quickly, right? Like how do you keep up? Right. I'm sure that if you have certain newsletters, certain people that you follow on medium, if you can share those, I think, our listeners would benefit from that as well. And then same with CircleCI, right? Like the journey that you were describing of the process or or the new way of doing things that you are implementing inside your organization.
00:57:39
Speaker
Do you guys publish like a tech blog where you highlight some of these challenges, things that you worked through so that other organizations can also learn from you? Yeah, I'm going to have disappointing answers to both of those, but they're interrelated. so try to be quick.
00:57:54
Speaker
Primarily, the way I keep up is a I work on this stuff, ah which is like creates a lot of division amongst leaders, but I feel like I can't be like what's the right word believable even.
00:58:08
Speaker
All of my expertise went out the window. like I understand how software is constructed, but I don't know how people build it today unless I'm working on it. And so it's a bit more of a pull model, meaning I try to do stuff and then I realize that matters to me and then I go search, right? ah Versus like like I don't follow a ton of big influencers. i there's like Stuff shows up in my feeds, but i'm I'm not that good at organizing it. Every time I try to follow someone,
00:58:33
Speaker
I realize don't time. So, so it's more just like I try to do it. I'm a learn by doing person. And, and then that forces me to ask questions. And those questions lead me to the people who are saying interesting things about it. Right. So that's number one. um And then ah in terms of us publishing, know, we're all so buried doing that. We haven't lifted our, head we we put out some stuff like we we have, I mean, ah I post stuff on LinkedIn, um which is one place to look. And then we put some stuff out on our blog.
00:59:05
Speaker
um But we've been trying to find a window to put a little bit more stuff out there because we do feel like we're learning stuff that people could could work from. yeah And I'd say at the moment, we're mostly trying to put out helpful product.
00:59:17
Speaker
So keep an eye like We're trying to put like even open source things out in the market just that people can use as we learn. We're like, this doesn't seem like a big product for us, but we would love for people to have it. Let's try to do a little bit more of that. um Yeah.
00:59:30
Speaker
Oh, that's awesome. We'll make sure that we link to your LinkedIn profile. And then maybe somebody from CircleCI who's listening to this, this is a weekend project. Like use the tools that you have and build ah a blog writing agent or something that that just creates content and you can review it once and publish it. I do, I do have a writing project actually that I use, uh, like in Claude co-work, that's just like bunch of samples of my former writing and, yeah you know, um, because the defaults don't sound like me. And I think people like everybody wants a lift.
01:00:03
Speaker
We all have a lot to do, but at the same time, when you're talking about stuff that really matters to me, like people don't want a bunch of fluff that was generated by an LLM. Right. So, so finding the right blend and really guiding it has been a thing that I've been trying to work on, but it's a tough one.
01:00:15
Speaker
It's yeah obviously I have my hand, but very firmly on that steering wheel still. Yeah. Gotcha. No, that's it. Thank you so much, Rob, for joining us for this episode of Kubernetes Bytes. would love to have you on again sometime in the future.
01:00:27
Speaker
Yeah, I'd love to. Thanks so much for having me. It's been a blast. Okay, that was a great interview. Hope you guys liked it as well. i really liked the part where we spoke about how to measure ROI on these AI agents and the AI spend, how developers can bring um some sort of deterministic behavior when using these AI coding tools by using the concept of hooks and stop hooks. I love how Rob delved into that detail as well. And then the fact that to to stay on top of these things, his recommendation is not following a specific podcast or a specific medium post, right? But just try things out, do things after work, over the weekend ah to actually to actually use these products and see the value for yourself rather than reading about it from somebody's experience. um We do have similar episodes lined up with practitioners from our ever-evolving ecosystem.
01:01:20
Speaker
So I would really appreciate if you guys can like, share, and subscribe to the podcast on Apple Podcasts, Spotify, even YouTube channel for that matter. Share it with your friends. We would highly, ah really appreciate that ah from our growing listener base. And with that pitch, it brings us to the end of another episode. I'm Bhavin, and thank you for listening to the Kubernetes Bytes podcast.
01:01:47
Speaker
Thank you for listening to the Kubernetes Bytes podcast.