Transcript
Speaker: You are listening to Kubernetes Bytes, a podcast bringing you the latest from the world of cloud native data management. My name is Ryan Wallner, and I'm joined by Babin Shah coming to you from Boston, Massachusetts.
Speaker: We'll be sharing our thoughts on recent cloud native news and talking to industry experts about their experiences and challenges managing the wealth of data in today's cloud native ecosystem.
Speaker: Good morning, good afternoon and good evening wherever you are. are coming to you from Boston, Massachusetts. Today is September 14th, 2026 and hope everyone is doing well and staying safe.
Speaker: I guess we are in the last week of summer, right? I know people like to switch to fall and and get their pumpkin spice lattes as soon as the football season kicks in but don't think I'm ready yet. I still want to enjoy that last week of summer officially. ah I'm happy that NFL is back though.
Speaker: And now we can spend the next few months showing our support to our favorite team. For me, that's Patriots. I don't know how this comment will fare come end of the season in February. We didn't get off to a great start in week one, but we'll see. um it's ah It's a good team. We have a good coach. Let's see what happens. um But as for the episode today, we have another great episode lined up for you guys. An interview with Maya Ayer, a hybrid cloud research software engineer at IBM. I'm excited about this conversation as we'll dive into agent identities and how you can introduce agents securely in your application ah using building blocks from the CNCF and ah how what is a Rosso CTL project from IBM and how it can help as well.
Speaker: So without further delay, let's get Maya on the pod. Hey Maya, welcome to the Kubernetes Bytes podcast. Thank you so much for being with us. do you do Can you please introduce yourself to our listeners and talk about what you do?
Speaker: Yeah, thank you for having me. ah My name is Maya. I am a research software engineer at IBM Research, and most of my work has kind of focused on identity and authentication for most of my career. But in the past year, it's been about agents. So how do you agents kind of um how have that has that kind of changed the way we think about ah distributed systems and and what new problems are arising?
Speaker: So, yeah. Oh, that's awesome. Okay. ah you You brought up the keyword, right? Like last year has been about all all about agents and I couldn't agree more. But before we like, again, go into the weeds and then all kinds of details, let let let me ask you a very basic question, right? Like, can you talk about the main building blocks of an agentic application, ah what are the different components, who talks to who. like I know it's very difficult to do it in an audio format. But yeah let let's try to paint a picture for our for our audio listeners, at least. ah yeah
Speaker: Yeah, sure. yeah So i guess I would maybe start by defining agent, which to me is any application that makes calls to a large language model and then may interpret the output of the model as some sort of action to take. So um i guess the canonical example is you go to a chat, like you have a user go to a chat interface, And maybe this interface is running an agent locally or is talking to some remote agent. And ah that can be done through various protocols. There's like A2A and ACP that are out there for communicating with agents. And the agent will be the thing that handles looping with the model, putting together the input, interpreting the output. It may go back to the user for more clarifications, or it may go...
Speaker: ahead and talk to other agents or other tools. Okay. Yeah. so I would say in general, agentic kind of centers around, of course, there's the user, but also the agents and the tools.
Speaker: Okay, perfect. So if I understood it, there's a human, right? we need Any application, it only makes sense if there's a human it's solving something that a human has to do. So you there's a human, then... Right now, we can't talk in those A2A or eight ACP protocols. it There has to be a UI. And then the from the UI, the agent can can take the user input, go and do its thing by talking to a local model or a model provider. And then um based on that back and forth, it then goes and performs certain actions on behalf of the user, whether it's trying to fetch some data that the user asked for or go and implement some changes. So it can be read only and read write. But the agent is performing that on on a specific tool, and maybe an external service. So that those these are the main components. Okay, okay makes sense. um So now, like just putting in an agent, right like I'm sure a lot of our listeners are in this boat where they are being asked by their execs and leadership teams that, hey, we need to introduce, we need to be more agentic, we need to introduce AI workflows, how do we bring in LLM?
Speaker: As they are going and doing their research, I want to first start by asking about anti-patterns because what not to do sometimes is more important than what to do. So can you talk about like a few anti-patterns when it comes to agent interacting with other agents or with tools or with models?
Speaker: Oh, yeah. Yeah. there is some There's so many ways to do the wrong thing. Yes. yeah ah I guess the main so coming at this from the security lens, yeah um I would say that like a lot of so so what makes agents powerful?
Speaker: is their ability to have hands and actually interact with tools. yep So that means that they need to be given access to some sort of tools. And so how does that get done? Well, I guess maybe taking a step back, there is lots of components involved already, right? Like there's applications, there's clients calling agents, agents calling agents and and tools. That means there's lots of requests happening.
Speaker: yeah And what's very important and the... A key principle in the distributed cy security space is um that every request should be authorized. yeah If there is a resource that doesn't do some sort of real authorization on a request, that resource is subject to misuse.
Speaker: So I would say this is this is a principle called zero trust. And I would say that a lot of anti-patterns kind of fall in the area over-permissioning or over-authorizing. So um yeah, I guess that's the underlying principle that I think if anyone gets anything from that. um Yeah, so I guess there's a couple of different anti-patterns that um kind of violate this. One of them is kind of like the just generally any use of static credentials. So for example, if I decide i want this agent to be able to interact with this database, yeah um ah maybe I create an API key for specifically this agent and give it all the permissions it needs to to do whatever it wants in the database.
Speaker: Why this might be problematic is from the database perspective, when they receive a request from the agent, that request is only going to have the API key, which represents only the identity of the agent.
Speaker: Now, if this agent is acting on behalf of some user, This is problematic because the tool call to the database doesn't understand that there's a user behind it. So if the user is malicious, um maybe I personally don't have access to the database to find out one of my coworkers salaries. But if I have access to the agents, I would be able to gain access to information I shouldn't be able to. so
Speaker: that's one issue is when only a piece of the context is kind of included. There's also an even more common anti-pattern, which is when i maybe pass along my own credentials to the agent.
Speaker: And then in this case, I won't be able to gain unauthorized access to tools that I can't. But it's very hard to implement this at scale. Like if an agent is connected to a bunch of tools, do I give all of my credentials to all of the tools at once? It seems scary and problematic. And another thing is like if the agent goes rogue, like if someone else has access to the agent or if the agent starts doing things It's completely out of left field. Maybe there's a prompt injection from like tool ah instructions. there's There's a bunch of ways this could happen.
Speaker: ah I'm going to be the one responsible for any tool calls that it will be using my credentials. yeah That is ah very scary to me. that I don't want to be blamed for any rogue agents. So i would say these are kind of the two is in general, if the authentication proof, like an API key,
Speaker: is something that doesn't reflect the entire context. Yeah. No, and I think it makes sense, right? And this does somehow translate back or connect back to how we think about zero trust, even for distributed architectures, doesn't have to include agents in them right now, right? Like ah yeah privilege escalation is like a very common type of thing that I don't know, developers and and architects and everybody ah need to keep in mind, right? So the first scenario that you mentioned where the agent has its own thing, but it doesn't really have the understanding of which user is asking it to perform a specific action. And then if you think about the tool that the agent is trying to access, it it just feels like, hey, this request came from agent, but there's no audit log capabilities to figure out whether it's a...
Speaker: an actual user, a malicious user, or even it doesn't have to be a malicious user, right? But if you have any sort of role-based access control for users in your organization, and if the agent has a super user and a viewer role can go and mess with it as well, and obviously that that's not great. So I completely agree and I see your point. And then obviously the second scenario, as you were describing you know it honestly, i thought, isn't that how I interact with a lot of tools right now? Like I give tokens to...
Speaker: tokens that I have generated. oh At least it's tokens. It's not my credentials directly that I pass on to an agent. And it goes and talks to Confluence and goes and talks to Jira. I'm hoping somewhere in this pipeline, either Jira is smart or the agent that we are using internally ah is smart to figure out that, hey, this is this is not Bhavin doing this action. This is Bhavin's XYZ agent, cursor agent, not Claude agent that's doing this. So I'm hoping there is some intelligence somebody has built in. But okay, that also makes sense. that There has to be a separate identity for the agent. Makes sense.
Speaker: Yeah. i So I would say that like the local case is is a li has has a couple of different concerns, right? Like if I'm running locally, maybe I'm okay giving the API keys because it's on my laptop. I'm responsible for whatever's running on my laptop. ah I guess a lot of these patterns kind of come when we're trying to bring this to...
Speaker: a deployment or or having agents deployed at scale. um If they're deployed on my organization's um cluster, i am the organization is responsible now for overseeing and governing, making sure, like taking responsibility, yes.
Speaker: Okay, makes sense. So now we know what these anti-patterns are and how they can lead to different weird security concerns. How do I solve for it, right? Like, can you talk about, um let's just stick to both of these scenarios that you have already described for our listeners. How do we solve for this anti-pattern?
Speaker: Yeah, so I guess as i said earlier, a lot of the issue is, um of these ah anti-patterns kind of arises from the fact that the credential that's being used at the tool does not have enough context or enough detail about like including both the user and the workload identity. So first is the question, how do we kind of glue those together? yeah And um so the work that I've been doing with my team kind of centers or has centered around um how do we
Speaker: take existing standards that have been worked on for years and years, and can they help solve the problems? And the thing that solves the problem of gluing the context together, of gluing like the user, the workload identity together, is actually this protocol called OAuth2.
Speaker: okay I'll refer to it OAuth2. It is sometimes OIDC, but OAuth2 is maybe the easier way to refer to it. It's essentially a delegation protocol.
Speaker: And the idea is there's this central authority that um people can log into. So you can kind of think of, I want to access my emails. i will go to the login page and provide my credentials. That's me authenticating.
Speaker: And the authority is going to be responsible for checking that I am the user, which is ah done by giving my credentials. And also that this is the browser that I want to use this browser application.
Speaker: to access like the backend data emails. So OAuth is kind of this protocol for being able to ah check both of these, authenticate both of these, and then provide an access token. And this access token is the key to everything. um It is assigned credential.
Speaker: So this means that it's verifiable by checking with the public keys checking the signature of the the access token. And then also it is expiring, meaning that it's not a static credential. So if I log in, it's only going to provide access to this browser for a set amount of time. Okay.
Speaker: That's useful because if the credential gets leaked, it can't be misused um indefinitely. And then finally, this access token is really a set of claims, almost like a dictionary. Okay. value pairs where there is a user id as well as the application ID.
Speaker: And because it has both of those informations, the resource, the database at the end will validate the access token, validate that it's not expired.
Speaker: And also we'll be able to have the info, once it decodes it, we'll have the information of who's the user, who's the application. and it can implement authorization based on this. this So this access token is really the glue to verifiable, short-lived identity.
Speaker: So is the OAuth2 provider huh ah Does it include this dictionary of key value pairs or is that a separate entity? right Because i was able to relate, like yeahp if I'm logging into Gmail or any other website, I do log in through either SSO so credentials or some sort of credential.
Speaker: it does have to go and do an authentication check. Sorry, an authorization, an authentication check. And then that, hey, this browser session is asking for this with these credentials. The application itself gets a token that it can use for any future communication. But then where does the dictionary fall in? like Is it part of OAuth 2 or it's a separate entity?
Speaker: Yes. So the token itself encodes the dictionary. Okay. It's like a set of claims, like a JWT, if you're familiar. This is yeah this is this the format of the access token and it's signed. So that's how it kind of takes care of um it takes care of gluing all the context together god the first step.
Speaker: And then is this something that Keycloak solves for? um Is Keycloak the answer here if we are at least talking about the Kubernetes ecosystem? Yes. so At least in CNCF, Keycloak is the implementation, the open source implementation OAuth as an identity provider. okay So this is actually what we've been kind of playing with because it's open source and and we like open source. Yes. Okay. Okay. Gotcha. And then we also hear these terms like spiffy and spire thrown around. Again, ah listeners would know that I don't say that I'm a security expert by any definition. So all of these are very basic questions from me. And yeah I use these interviews to learn more. But can you talk about what those are and how they fit into this picture as well?
Speaker: Yes. Yeah. So um OAuth is the glue. And we talked about, I mentioned how I would authenticate as a user by providing my username and password. Yep.
Speaker: Now, when in order for the workload to authenticate to the provider, often it would involve some sort of static credential in the past. And this is problem from a zero trust perspective. um because these static credentials can be leaked. yeah So this is where SPFY comes in. SPFY is a subpro another subproject under CNCF. okay And it defines a standard for workload identity and providing workload identity credentials.
Speaker: um From, I think, a high-level overview, there is now another authority, ah workload identity authority, so call it the SPFY provider. okay This um provider is an authority that signs credentials um that can be used for authentication, almost like the OAuth signs an access token. It's the same the same exact mechanism. It signs some sort of documents. okay And um this, when it provides it, it's only going to provide this credential to the workload based on
Speaker: attestation like an attestation process. okay So this means that ah the whatever identity is being used, it's probably it can be something like spiffy.example.org slash namespace slash demo or something like that. It will encode potentially some of the attributes of the workload. And in this way,
Speaker: um the credential that is provided is based only on strongly attested attributes. okay So yeah. Oh, so one follow-up question, right? So in our workflow so far, user logged in credential, the app got a token to ensure that this is not a long-lived token, right? Like that is being rotated. The application at this point has to go and talk to Spiffy and get something that's more short-lived. And what happens when the token that it got from OAuth expires? Like, how does it... Right. It's kind of a little bit reversed. So actually...
Speaker: The workload is going to talk to Spiffy to get a credential, a document. okay And this document is sent to the OAuth provider upon the authentication check.
Speaker: So in this way, the OAuth provider will be able to validate by talking to the SPFY provider the same like signature validation. Okay. okay Gotcha. So exactly so my OAuth provider and SPFY provider do need to talk to each other. They don't need to know that like each of them each of them exists. They can't operate independently. They they do need to I would say that the OAuth provider needs to be aware of Spiffy.
Speaker: yeah But Spiffy doesn't necessarily need to be aware OAuth. Spiffy is almost like the the root of trust. um the Yes, that is defined. And in this way, there's like this strong chain of trust that is created from OAuth to Spiffy, but the resource trusts um OAuth. And then what's Spire?
Speaker: and then what'spire Yes, Spire is the open source. It's a project under SPFY. It's the open source implementation of the SPFY. So it's ah ah implemented the SPFY provider and the attest attestation, um common attestation pieces.
Speaker: Yes. That's actually where I started was the SPFY community. So I'm very happy to talk about all that. Nice. So now we know all of these main building blocks. These, as you said, already exists in the CNCF ecosystem. ah But now let's let's talk about what you are building as part of your team, right? Like the research that you're doing, you and your team are doing with RossoCTL or RossoCuttle. What's the preferred way to say it? And then how are you putting all of these blocks together to solve for the anti-patterns or the agent identities?
Speaker: Yeah, yeah. um So I am in the camp of saying Rosso CTL. yeah ah Some people do call it Rosso Cuddle. I i guess either is valid, um but yes, it's Rosso CTL to me.
Speaker: Overall, this is a project started by my team, which kind of started around the the zero trust. Like how do we input put together the the ah different projects and solutions that have been kind of battle tested by the community for for many years. And how do we put them together in a single platform?
Speaker: And that's where it started. this is if you go to the main Ross CTL GitHub, um That's going to be the project where you can kind of play with it on your local kind cluster, or if you own an OpenShift cluster on that, it will deploy the components of Spire and Key Cloak, and ah you'd be able to kind of run demos on it.
Speaker: um From there, the project kind of evolved into ah how do we ah what what are the problems arising when we put these things together? So part of this was like, okay, how can what are the difficulties here?
Speaker: um And we saw that there were difficulties in terms of integrating the two technologies and ah That's stuff that's been upstreamed, which is which is really nice. um But another issue is that applications seem to both of these, kind of putting these together, applications seem to have requirements as far as understanding these standards. um So it needs to be able to speak OAuth.
Speaker: to be able to get tokens and talk to the OAuth provider. And it needs to be able to understand how to get spiffy credentials. And these are things that application developers just shouldn't be required to do, um especially if they don't do it properly, um which is possible. It could lead to like bypass of security controls, things like that which is problematic. So there's a couple directions RASO CTL is going. And the first step was really, how do we kind of wrap the agent in some sort of runtime so that it can do um integrate properly with the platform and currently do that.
Speaker: So that's where RASO Cortex kind of is is a subproject under that that has to do with that. Okay, so you you started by saying like, Draws a CTL is like an easy way to deploy all the building blocks on a Kine cluster or an OpenShift cluster.
Speaker: ah assumption is I can take that and run it in production as well because it is still using the the CNCF projects and and building blocks. like It made the deployment of all of these components easier.
Speaker: But then you're saying that it's not just that. right It's not just a fancier Helm chart to install all the components that's needed, but it actually helps remove overhead on the developer. right like when When they're building their applications, and they want to introduce identities or or agents in their application code, they don't have to worry about going and modifying application code. But also somehow, is it through a sidecar? Like somehow figures out like, hey, how how are these components supposed to talk to each other? Like how, ah what, can you talk a bit more about that, please?
Speaker: Yeah. So you you brought up a really good point that the, like, like The Rosso CTL, it starts out as like deploying the platform components. And it it is very important to note that it's not required but because we're so this solution is very standards-based. It's definitely not required that um i it's it's not required that people who want to play with this are required to use Key Cloak and Spire. There's a bunch of implementations out there. And if your organization is already using them, they should be able to. okay
Speaker: use them. um Now, yeah, the after the initial experimentation, it was just clear that all of the code agent code, all of the tool code had to be modified. And this was problematic because this is not the application developer's job. This should be taken care of at the platform. So yeah, Cortex is kind of this component that it does use kind of a sidecar model, which intercepts traffic in inbound and outbound traffic. And in this way, it when incoming traffic or or outgoing traffic needs to happen, the sidecar will be able to do the interaction with Spiffy to get the spiffy credential and do the interaction with OAuth to get the OAuth credential. It's almost like there's this pipeline that's injected upon inbound and outbound traffic. And kind of what the core text component is.
Speaker: Yeah. Gotcha. So then let's take an example. ah I have an application container and I installed this sidecar. Now, as part of this deployment, right like somebody does somebody from the platform team or the application team go and and manually enter, like put entries into Spiffy to create that dictionary? like if if s spffy If I remember Spiffy being the dictionary holder or something, right like where these microservices or these containers can talk to these containers. um
Speaker: the does the who Who does that? right like i'm I'm assuming it's not DrawsOS ETL because it's not the project's responsibility. It's on the customer or the application user. Yes, that's that's an amazing question. And this kind of gets into authorization. So everything that I've been talking about so far is identity. How do we get enough context that's verifiable to whatever whatever is being called so that they can know what's calling them? The authorization
Speaker: piece is now that we know who's calling, and they do what they want to do? And it turns out this problem of how you define authorization policy is extremely complex. And I would say still an open problem in in the ecosystem because there are so many places that you can implement um authorization policy. And each of those places is responsible for a different thing. So the application owner may really care about locking down ah who gets to call this agent and also what are the the tools that the agent can call.
Speaker: um This also might be something that at the gateway, the cluster level, um policy needs to be defined there. And I think what you were getting at was in Keycloak, in the identity provider, there's also delegation policy that needs to defined as far as which applications can act on which users' behalf.
Speaker: And the answer is there's there' is a lot of different authorization policies that are fragmented and there's a lot of potential for overlap, which means there's a lot of potential for confusion. ah So this is i I'm going to say this is an open problem. yeah It's really up to, i would say, the the managers of each of the components. so um Those that are managing like the OAuth, the identity provider services, they would be responsible for administrating on key cloaks, say. okay
Speaker: Versus people who are administering the cluster are responsible for writing the network policies there. and so on and so forth. Got it. Yeah, it's it's a very good question. because well and like Again, I know we are ah on the bleeding edge. like You are part of a research team that's going and solving for all of this, and you guys are contributing back to the community. So yeah, these are these are questions that come up, right? like I do work with a a team of devs that that are building a Kubernetes-based application. And I'm just putting my product manager hat on. Like, hey, if i go if I have to go and make this ask next quarter to my team, what level of effort is needed from them? So I'm i'm coming from that angle as well to to figure out like, hey, is this something easy? something Who has to do this? that That's the reason for some of those questions. But okay. okay Anything else that you would want to add um around Rosso Cortex or or Cortex in general?
Speaker: Yeah, I guess. So... So Cortex, I would say we've our research has kind of been evolving in the past half year, um because when when all of this was kind of talked about initially towards the end of last year, we were kind of dealing with a different breed of agents.
Speaker: um agents that were kind of defined in code that encoded prompts, you could write a container image and compile it, and it would kind of behave in this very simple way.
Speaker: But nowadays, agents are a little bit more stateful, if you're aware like the cloud code, the open code. um And because of that, I can take the same image with um open code and deploy two ah instances. And if I interact differently with each of them, their behavior will drift differently. So there's lots of interesting questions around this, that protects Cortex is one of the threads that we're focused on as far as like, how do we deal, like what what are the what are the things that all agents kind of would be useful to have which involves like authorization and the identity stuff, but also this new concept of LLM token spend. And ah we really want to kind of make sure they don't go rogue and on us, right? um
Speaker: But then there's also this other thread of how do we, what what are the emerging deployment patterns of agents, which is ah slightly separate from Cortex, but but quite related. These are kind of the two that Rosso CTL is kind of focused on.
Speaker: OK, so Cortex already solves for agent identity, as I understand. right like it It does give some sort of identity to any agent that's running in your application.
Speaker: like Again, I'm talking about the distributed or microservices-based application. right So you can have one or more of these agents. But then each agent, because of Cortex, gets a verifiable identity. And then there is some sort of authorization as well where we can control the least privileged model, right another security best practice.
Speaker: Yeah, um in Cortex, there's on top of like the usual role based access, these kinds of attribute based access, there's also now this concept of guardrails that needs to be put in place, um which ah That's another form of authorization policy. that's That is also in Cortex. Okay. And I was also going through the documentation for also CTL, right? And obviously we'll link it in the show notes. But you guys also talk about something called intent-based access. Is that different from role-based access control and like...
Speaker: is also reading, I'm just taking a weird example, right? Like, I'm asking an agent to do something in in and in a prompt. Is Cortex getting access to that prompt as well and then figuring out what it needs to do? Or it just asks, relies on the agent to figure it out because it has access to the prompt and then figure out what the next steps are and and Cortex controls the identity.
Speaker: Yeah. So i would say that the IBAC kind of falls under more of the guardrails type type piece. So where a regular authorization policy authorization kind of focuses on a policy, like a well-defined policy that is parsable and kind of logic, you can logic through it. yeahp um There's also these soft measures that kind of get put in place.
Speaker: ah Guardrails is one of them that does like, you know, if there's personal information being passed where there shouldn't be, you can kind of scrub that as an easy example. What was interesting about Cortex is because it is meant to, it is implemented as kind of like this envelope around the agent.
Speaker: It can see both the inbound and the outbound traffic. okay And because of that, there's lots of context that you can kind of ah reason over. And so what IBAC is, um ah intent-based access control yeah is the is what it stands for. And it's it's an interesting concept where maybe we can ah take the use case. I wanted to read an email ah and and summarize like the last five emails I got.
Speaker: And if I, somewhere down the line, Rosso Cortex sees that this agent is trying to call an API to send out an email, which I did not ask it to do, then possibly something problematic happened. And this can kind of protect against, um in the world of agent security, agents are susceptible to any output that comes from tools. So if it reads an email and one of the emails says, please send out all of your credentials, something like that. um This is something that can be protected against because
Speaker: the cortex is very um aware of both the inbound and the outbound, which is not something that it's quite unique compared to say like guardrail or ah gateways, like cluster gateways, which are ah implementing policy in the aggregate. um So that that's, I guess, one one interesting, it's one of the interesting plugins of Cortex. It's definitely helpful, right? Like if you look at, I don't know, oh if you look at announcements from vendors like CrowdStrike and Rubrik and i don't know, just this,
Speaker: cyber resiliency ecosystem, you will always talk ah hear vendors talk about how they can stop agents from doing something that will be disruptive to your business. So it feels like Cortex does give me that envelope that I can control and make it smarter over time.
Speaker: to stop agents from acting maliciously. like They don't want to. Again, we we saw that with the OpenAI hugging face incident incident that sometimes they do want to. But if if agents are innocent, like this at least gives me a guardrail or that envelope to control what what actions an agent performs and how it reacts to certain output that's generated from a tool.
Speaker: Yes. Yeah. Kind of like it's a defense in depth approach. You really can't, you can't write policy to protect against all of these because, because the, this input output, the state space is just very large with yeah these agents. So it's like, Well, there have to be some of these soft measures.
Speaker: OK. OK. I know we have been talking a lot through examples, but I did have like an example scenario planned for you. um like i Let's say i I work for Uber. right like Let's just make up a story. I work for Uber, um and and today if I'm a user,
Speaker: of of the Uber app, like, I put in the destination, it figures out the the best way to get there, it figures out ah what drivers are available in the vicinity and who accepts the ride and then gets me to the airport. right And now I'm putting my employee hat on as an Uber employee and I want to add a feature where the user, in addition to selecting the type of car they want, they can choose whether and whether they want the energy efficient route or not. I can select like they can select whether they want a scenic route. Like I'm i'm in a new city. I'm in no rush to get to the airport.
Speaker: um I want to get to get get to the destination through a scenic route. And I'm just making this as complex as ah as I possibly can. But basically I want the agent to know what user I am. ah Maybe I logged into Uber through my Gmail account, figure out what my search trends were, or if if it can figure out any additional details about me, and then figure out what that route would look like, and then follow the existing process of asking drivers who wants to go on on this route for this user. right But if I wanted to build a scenic route agent,
Speaker: how do I go from not having that functionality today to getting to that endpoint where this becomes a reality? like Again, we don't have to go and build it and do a design discussion right now, but what are the main steps that I need to think about if I'm a dev? i do one.
Speaker: Yeah, that's ah that's a really cool question. um So i guess I guess I would always say it like application first, figure out what the agent how the agent deployment pattern is um Definitely an agent is very useful for something as maybe it's not just even asking for a scenic route, but also taking personal preferences in mind. Yeah, agents are really excellent at that.
Speaker: And so I guess the first thing I would say is maybe start by choosing your agents or or whatever framework you want to use, and then choosing the tools that it could could potentially have access to. So one of them might be some internal Uber API for finding routes yeah between places.
Speaker: And then maybe another one would be the Google API for preferences or something like that. Is there like an MCP server we can fit in it? Like throughout this discussion, I don't think you or I have ever heard MCP, but like can you bring MCP as well? Yeah, I mean, MCP, it's it's an interesting project.
Speaker: MCP is the agent tool protocol, as I understand it. And it it it depends it mostly depends on whether ah an MCP is available. Okay, okay. Mostly mostly that. um MCP is very OAuth friendly, so in that like as a protocol, so ah probably all of these patterns do apply, whether or not there's an MCP server.
Speaker: And so once we define like the agent and the tools, ah whatever interface that is, then trying to build that with like a test user might be a good first step um towards figuring out how to kind of instrument all of it together. um i guess I would say that the the access story is interesting here because it's not just
Speaker: the Uber account that is required, but also the Google account and some sort of ah trust establishment there, which um i guess I guess it depends. I'm not sure if if Uber has a different um form of like SSO or something. Oh, I just made that up. Let's assume there is. yeah i can log in through my Gmail. Yeah. It's a lot easier because because they kind of are attached to the account. If not, then there's some extra complications around like federating the the two identity providers. But yeah, once once some that's kind of figured out, it's really a matter of this agent needs specifically, like if you can kind of explicitly allow list certain APIs,
Speaker: that might help users decide whether they want to use, um what whether they want to authorize like this particular agent with these tools because that means that the tools list is restricted. Okay. It's much better than ah giving it access to my full Google account, for example. Yeah, I would say that's where I would start.
Speaker: Okay, no, that makes sense, right? And so in that example, am I installing Rosso CTL on the cluster? Like, let's say, I don't know if Uber is my, I think they're microservices based, but they are microservices based around in, I don't know, OpenShift cluster. um do i need Once I got Rosso CTL up and running on the production cluster, do I need to configure Cortex just for this new component, like new agent that I'm building? Or do I need to use Cortex and package up my other parts of the application stack as well?
Speaker: Yeah, so um luckily I think Uber has been very, very active in the SPIFI community actually. So they might have the components necessary, at least from the platform perspective. yeah um So from a Rasa CTL perspective, I would say like whatever infrastructure, maybe we can assume that's already there and implemented in a standards compliant way.
Speaker: that means we can just use what's there. um Where I would say we need to kind of focus is how does this new application integrate with the rest of the um with the with the with those platform components. And so if if they don't have a solution for that already with their existing infrastructure, ah Cortex would be a good place to kind of wrap around the agent deployments and take in the inbound and and to handle the outbound, handle the gathering of credentials. that's That's where I would put it. And then there's lots of interesting things you can do there too, like implementing policy. This is already like taking up up a made-up scenario. Thank you so much for doing that, first of all. Like taking up a made-up scenario and then like helping users walk through like, hey, what do I think about? And then obviously this also covers anti-pattern. So I really appreciate you going on this scenic drive with me.
Speaker: I love it. Yeah. But okay. Uh, next question is, uh, like, I'm like, I think we're close to the end, but I want to ask you like, what's next for Rosso CTL? What are the next set of challenges you're working on? Um, and then if, if there is, how how can listeners help you? Right. If this sounds interesting to any one of them, like how do they reach out and and and contribute?
Speaker: Um, yeah, that's, um, the, it's very exciting work. I will say, i think, um, where Rasa CTL is kind of focused on is one, this Cortex component. It's really interesting considering what um what all agents what features all agents could use.
Speaker: um And we've actually also even stepped slightly, like if if a person wants to go and try this out, um the GitHub for Cortex actually has like a one line install. And based on this, you would be able to like and dont like ah run run your agents locally and and view some of the statistics on like token usage, on what are the network calls that are being made from your computer. And that would mean you'd be able to tell, oh like what what are these? ah
Speaker: ah you You can kind of vet the behavior of the agents. especially when the harness is kind of a black box sometimes. um If they're extract extracting information from you, you can kind of detect that and potentially block it. So that's kind of interesting.
Speaker: It's kind of this whole idea of what does the agent, what what do we want to wrap around the agents? And the other thing is ah kind of what I said earlier was that agents are kind of evolving and um where where this year agents evolving very stateful, um file system oriented, meaning that um we've learned that agents are really good at writing code. And if we give them a way to execute code, they become really, really capable. But it's also scary to have them execute code. So there's interesting things around
Speaker: how do we deploy agents kind of safely? And also is the agent, the concept of agents kind of evolving? um One thing that's interesting is maybe there's ways to separate the state out of the agents. um A lot of people are aware of the encapsulating, like run an agent within a sandbox. yeah It might also be the case that the sandbox is separate.
Speaker: So how do we, ah what does this work? What are the systems implications of this? And um yeah, kind of, I just just ah doing some experimentation around that. So yeah, I would say these are the kind of the two dimensions we're focused on yeah right now.
Speaker: And if anyone wanted to reach out, we, we love talking. We love talking about problems people are having or what they are worried about with when it comes to working with agents. We do have with the website, which i I can send you these links, I suppose. for sure. We'll include all of those in the show notes. There's the GitHub. And also some part of our team will be at KubeCon North America. So if anyone wants to come talk to us, there's a bunch of talks kind of happening. So... I know that's how we got in in touch. right like I know I've seen a couple of your sessions. So thank you so much for always representing at KubeCon. ah Will there be like a Rosso CTL booth, IBM booth, like that? where Or how do they find you at the show? Right? You just come to your session. Yeah, there's going to be a couple of talks during the co-located events that I will be giving. And then and there's also some planned talk during the main event on this work. So I can sense details on that. yes Sounds good. no and And apart from finding you at KubeCon and your team at KubeCon, how how do people get in touch with you? Do you mind sharing your socials or or anything else?
Speaker: Yeah, um my I would say on I'm happy to connect on LinkedIn and have discussions there. Perfect. Which is just search my name. and My name seems pretty easy to find. It was.
Speaker: yeah And then we can discuss more. and Awesome. No, Maya, thank you so much for your time today. This was, i don't know, a really educational discussion. And thank you for being patient with all the tangents and detours that we took. oh no no I'm sure listeners would love but love it as well. And whenever you want to come back, you have an open invitation. Like this is definitely an area that's not static, right? Like you guys will keep doing more and more things. So you have an open invitation to come back on the podcast and then talk about it some more.
Speaker: Okay. Yeah. Sounds great. Awesome. Thank you for time. Yeah, thank you. Thank you so much for listening to the episode. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app.
Speaker: Also, please consider giving us a rating or leaving a review or sharing it with your friends and colleagues, as that really helps us grow the podcast. You can find all past episodes or learn more about the show at kubernetesbytes.com.
Speaker: See you in the next episode.
Speaker: Thank you for listening to the Kubernetes Bytes podcast.
