Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Elliot Foreman & Andrew Delave: Math Beats AI in FinOps
44 Plays1 month ago
Elliot Foreman and Andrew DeLave from ProsperOps joined Yuliia and Dumky to discuss automated cloud cost optimization through commitment management. As Google go-to-market director and senior FinOps specialist, they explain how their platform manages over $4 billion in cloud spend by automating reserved instances, committed use discounts, and savings plans across AWS, Azure, and Google Cloud. The conversation covers the psychology behind commitment hesitation, break-even point mathematics for cloud discounts, workload volatility optimization, and why they avoid AI in favor of deterministic algorithms for financial decisions. They share insights on managing complex multi-cloud environments, the human vs automation debate in FinOps, and practical strategies for reducing cloud costs while mitigating commitment risks.
Recommended
Transcript
Introduction and Guest Welcome
00:00:01
Speaker
Hi everyone, it's Duncan Yulia from Straight Data Talk and today we are happy to be back in September. Yeah, it's going to be released in September and we're happy to have folks from ProsperOps, Elliot and Andrew.
Roles and Focus Areas at ProsperOps
00:00:16
Speaker
um So guys, please jump and introduce yourself. um I'm Elliot Foreman. I am ProsperOps's Google Go-To-Market director.
00:00:27
Speaker
So I do all things sales and Go-To-Market for our Google offering.
00:00:33
Speaker
My name Andrew DeLaeve. I'm a senior Phenops specialist here ProsperOps. um I also work largely on our Google products and been kind of working in the space for a few years now. So excited to get chat.
Automated Rate Optimization at ProsperOps
00:00:48
Speaker
Folks, tell us about ProsperOps. And for everyone else who doesn't know, what is ProsperOps? what is prosper Yeah, at high level, ProsProps has been around since about 2018.
00:01:01
Speaker
ah We focus around the rate optimization challenge with FinOps. So we are an automated rate optimization solution ah focusing on um you know reserved instance, committed use discount, and savings plan discount instruments.
00:01:16
Speaker
So our our tool is a... um optimization algorithm that's built out to really define what the optimal point of coverage is and you know determine you know what what to go place when it comes to these reserved instances, ah savings plans, a committed use discount. So if you're in the Google ecosystem, um we manage these things on an ongoing basis, looking at ah what's going on in the environment and you know what what you're using in the cloud.
00:01:44
Speaker
Sounds pretty complex. Yeah. So, so yeah, i'm I'm trying to figure out like, is there a, um, what would you say is is like your typical
Client Profile and Cloud Spend Management
00:01:54
Speaker
client? Do I have to imagine maybe, uh, like a company with like a, a big website that's trying to optimize their, the way that websites served, or is it also like companies with big data stacks or.
00:02:09
Speaker
Yeah, and we we have customers that are you know smaller on the ends of you know spending about 20, 30 grand a month on you know compute applications. So think and a vCPUs and RAM. And then we have customers that are even spending upwards of 50, 60 million ah ah month in their their cloud compute on AWS.
00:02:28
Speaker
So we have ah you know quite quite a range. It's not just the small and not just the big. um I would say it typically makes a little bit more sense when you have more complex environments. But if you don't want to manage CUDS and you you have a lot of changes in your environment, um you know a tool like us makes sense at any size.
00:02:47
Speaker
That's very interesting. I mean, the complexity side of things. Yeah. Yeah.
00:02:58
Speaker
So what is what is usually like um when you start with a client, right? So maybe to clarify, like like we're we're coming from a data perspective, right? And you can get very expensive in in the data world, right? there's There's a lot of compute and storage going on, but I think things can probably get more expensive if you bring it in the entire infrastructure of your your company into that as well. And so,
00:03:24
Speaker
Like when when your clients come to you, what is the kind of their usual process in in trying to optimize their spend in the cloud? Is there like big things that everyone starts with first or does it really differ per client?
Financial Optimization Strategies
00:03:40
Speaker
Yeah, so that's a good question. So I think they, know, the way we like to categorize kind of optimization in general is kind of aligned to the FinOps Foundation where we have Kind of on the left side, there's workload optimization, which is really around, you know, right-sizing resources, re-architecting services, deleting idle instances.
00:04:03
Speaker
It's attacking, um you know, making the usage itself more efficient and using less of the cloud. That's a great way to reduce your costs because, as you know, right, cloud spend is you get billed per hour of usage.
00:04:18
Speaker
um Now, the other kind of ah way to optimize is where Prosperops plays. It's on the financial universe. we We call it the rate side, but it's paying the least for the services you are using.
00:04:30
Speaker
um And on this financial side, there's a few different buckets. you know One of them is you know your enterprise agreements, your PPA agreements, which get you negotiated rates and negotiated discounts with the cloud provider.
00:04:42
Speaker
And the other coin is you know what ProsperOps does, which is managing the commitment instruments for specific services. um And you know these take different flavors depending on what cloud we're working with. So in AWS, it's RIs and savings plans.
00:04:55
Speaker
ah Same with Azure. And then in Google, it looks like committed use discounts. um Now to get back to your like initial question of you know where do customers start? um you know Traditionally, that the financial instruments are a little bit easier to start with because they're not going to have any sort of impact on your engineering or infrastructure.
00:05:14
Speaker
If I buy a cut, I'm not going to cause an outage to my prod application if I get the optimization wrong. um The main downside, though, is that a lot of times these commitment instruments, you only want to buy them after you once once you know you have a steady state workload.
Risks and Decision Paralysis in Cloud FinOps
00:05:31
Speaker
That's not going to change. um For example, when I buy a commitment, I'm committing to that usage for one or for three years. And so if I'm just migrating to the platform and I'm not sure what machine types I'm on or how much usage I'm going to have if I end up doing workload optimization,
00:05:47
Speaker
um If you're unsure on those things, it becomes a lot harder to buy these cuts. um And so we kind of end up in this, like we see a lot of customers end up in this cycle of, you know, kind of saying, and need to buy my cuts, but I want to make sure I'm optimized.
00:06:03
Speaker
but But that workload optimization a lot harder. It requires a lot of context. It requires engineering input. And so a lot of customers end up in this state where they have lots of recommendations on things they know they should do to reduce their costs.
00:06:15
Speaker
But enacting those changes on the workload side very hard. this is This is so interesting, if i if I can dive into this a little bit deeper, because it this this really hits home for me. So I do a lot of migrations of data platforms, as well as like greenfield setups of of data platforms.
00:06:32
Speaker
And every time i talk to my clients, they always say you know, I'm just going to wait with the the commitment because I have no idea what my workload is going to be, even though like,
00:06:45
Speaker
we can say from experience a little bit, like you're probably going to do around this or that, you know, give or take. I've always been wondering, I never did the the calculation on this, but I always thought that, you know, maybe it would have been better to just commit upfront, even if it's more than, you know, you're actually going to do because all this on demand stuff is usually pretty expensive.
00:07:09
Speaker
Yeah. do you Do you see that with with clients? Yeah, absolutely. There's um some really interesting math behind this, actually.
Risk Mitigation and Scaling Strategies
00:07:19
Speaker
It's it's called like a break-even point, but it's a point at which if you buy a discount, right, you're paying for that discount whether you use it or not, if you buy a commitment.
00:07:28
Speaker
Now, there's a point at which if that commitment goes unutilized and you end up wasting it, as long as you've generated enough savings to offset the wasted portion of that commitment, you're going to break even on your investment.
00:07:43
Speaker
um And so, you know, that break even point is really determined by um one, how much you're utilizing the commitment. ah But then secondly, the discount rate that you're getting on that instrument.
00:07:56
Speaker
If you are saving more, you can afford to have more waste. If the discount gets a worse discount, you can only take a little bit of that waste before you end up yeah before the the cost ends up outweighing the savings.
00:08:08
Speaker
um So that's kind of the the the the concept we look at. It's it's really a break-even point or optimal point. um There's actually strategies you can do to kind of mitigate the risk of that.
00:08:22
Speaker
A lot of times we see customers, again, kind of like in this paralysis phase of analysis paralysis where you end up going back and forth on what I should or shouldn't do. And the nice thing about commitments, it's very easy to start conservatively.
00:08:35
Speaker
um If I purchase a commitment today um and then I add a little bit more tomorrow and a little bit more tomorrow, I can slowly grow coverage over time. And that affords, you know, that you're kind of building in this native flexibility with on-demand usage. And so if on-demand usage goes down at any point, I can stop purchasing new commitments to prevent over-commitment in the future.
00:08:58
Speaker
um And so you don't need to necessarily attack it all at once. And that's what we traditionally see customers try to do is they try to buy everything based on a current state, alt, front, and a batch purchase model.
00:09:09
Speaker
When in reality, you can do it much more conservatively and grow into that usage over time. but Maybe this would be getting too much into into psychology of of how people buy things. But do you have any thoughts on on like why is this is happening? Because I see this so much. like Yeah. Like teens are afraid to commit to something. Even like you said, like you you can commit to just a little bit for for a year and it would still save you a lot of money.
Automation in Financial Decision-Making
00:09:38
Speaker
But there's something in the in the psychology of when it comes, maybe when it comes to finances or budget, I don't know. that causes this this pattern to happen right there you're like i need to know all the details before i can actually make this commitment exactly and nobody wants to be responsible for waste this is really just the short answer of it if that thing goes unutilized they don't want to be held responsible for for purchasing it and causing their company ah you know what money at the end of the day um
00:10:09
Speaker
And so, you know, the the problem is, though, there's also an opportunity cost to inaction. Every day you go leaving that on-demand spend, you're paying essentially list price for a service that you could be saving 50, 60, 70 percent on.
00:10:24
Speaker
And so um the the the kind of real driving factor, it is largely, I think, a little bit of psychology. And One of the coolest parts of my job and what I get to do on a day-to-day basis is I work with hundreds of customers where I have these conversations talking about financial commitments.
00:10:41
Speaker
And at ProsperOps, we're a huge believer in using automation, right? Let's use automation to provide real-world forecasts using data to make this decision process easier.
00:10:54
Speaker
We can model out different scenarios, whether they be conservative or aggressive, and show you the exact dollar amounts that we can save if you decide to move forwards with option A or option B.
00:11:05
Speaker
um And it's no longer a human that's liable for that decision-making. Because we use automation, it's essentially a very advanced series of algorithms that determine the logic of how we should go be covering a customer.
00:11:18
Speaker
And so it removes some of that like you know human component of it away when you can make it more about the objective outcomes that you're providing. so I have it's a two question. Very interesting topic, though, I think. yeah don Yeah, I have two questions. So let me let me ask, please.
00:11:34
Speaker
Yeah. Question, folks. ah We are also in FinOps mar ah um um industry a bit, right? In different angle, but how do you navigate this conversation when it comes to...
00:11:50
Speaker
Okay, we're gonna try ProsperOps and we will see what is the best commitments for us. um Why would we need to commit with you for the next year or two? Like, okay, you predicted we apply as the savings. Why would we need you?
00:12:08
Speaker
it's you know we we Why do we have to pay subscription or whatever? Yeah, that's ah it's a great question. and yeah he i'm italy like its It's really around like the the ah ROI of using a tool in the first
Benefits of ProsperOps Solutions
00:12:19
Speaker
place, right?
00:12:19
Speaker
um And this this is not just a unique problem to cross props. This is really any FinOps vendor in the space. to To get somebody to want to buy or acquire your tool, you have to offer them something that is either too time-consuming, too difficult, ah or is just not possible to do themselves, right?
00:12:38
Speaker
Or it's not a priority, right? There's lots of different reasons that you may consider using a tool. Now, our three kind of pillars of our value proposition are one, we're gonna generate better savings than you know teams can do themselves.
00:12:54
Speaker
um we have ah We manage over $4 billion dollars in annualized cloud spend today. I think we've objectively proven we can do that extremely well. ah The second part though is is a kind of a less intuitive one, but it's actually the idea of commitment risk.
00:13:10
Speaker
um the The second thing that Prosperops is trying to do, and it's arguably the most important, is mitigate the risk of commitments. You can only generate really good savings with PUDs if you have flexibility, right? We have to be able to make these commitments elastic like cloud usage is.
00:13:31
Speaker
We have to be able to scale up into usage as it grows, and we have to be able to provide downward relief for commitment coverage as usage goes down. And it's by doing those two things that we can keep the commitments we buy at extremely good coverage and extremely high utilization, which ultimately generates better savings back to customers.
00:13:50
Speaker
um And so I always like, it you know, customers don't really think about it, but it's by mitigating the risk of the commitments that you can get aggressive. um And we have some really advanced strategies around purchasing and managing the cuds that help us do that.
00:14:06
Speaker
The third component is really just the automation. um You know, as as fun as commitments are to talk about, most companies don't want did better things to do than managing commitments themselves.
00:14:18
Speaker
um FitOps is a big space. There's lots of much more complex things in this universe, like workload optimization, which are not just objective math. They require human input. They require engineering time to go and act.
00:14:31
Speaker
um You guys are probably very familiar with this on the BigQuery side. That's a very complicated area that requires lots of kind of hands-on analysis. um And so, you know, if we can take this component of of of optimization off their plate, specifically CUDS, which are less reliant on that engineering context and are a great candidate for automation because it's very objective,
00:14:55
Speaker
If we can take that off their plate and provide them time back to go focus on other things, that's an excellent value of the automation. um and And really, i i like I kind of describe the automation as like the glue that holds this whole thing together.
00:15:10
Speaker
When you manage commitments at the scale that we do, um for most customers that we work with, we're taking hundreds, if not thousands of portfolio actions every single year, um whether that's across AWS, Azure, or Google.
00:15:24
Speaker
And so that's just a scale and, you know, kind of nuance and granularity that human teams do not have the bandwidth to support. But when you offload that work to an algorithm and to automation and programmatic system, it becomes much more scalable, um which is really what enables kind of our system to work at the scale that it does.
00:15:45
Speaker
Okay. My second question, Duncan. Yeah, I have one other thing to to add here, if you don't don't mind. i think from the the business standpoint, like what we're we're really providing is just reliability and predictability. I mean, we we are ah software at the end of the day, so we're not a ah human that can make mistakes.
00:16:04
Speaker
We're not a human that needs to take breaks. We're not a human that could go leave the company at any point in time. or you know require you know additional you know oversight into making sure that the the actions are taken care of properly.
00:16:17
Speaker
i mean, when it comes to bringing on people, um you know you you have this kind of like risk and you know challenges where you don't really fully know what the human could go do And you know there's there's caveats. Sometimes it could be beneficial. Sometimes it could be risky.
00:16:30
Speaker
We have a very predictable, reliable tool that will give it a guaranteed outcome um to our our specifications because we're using logic-based math and determining optimal points of coverage and looking at your environment all the time.
00:16:44
Speaker
This is, ah if I can dive into this a little bit more, Elliot, would you say that, Julia, I'll let you get to your question. um Like there was a time when, if you looked on LinkedIn, like everyone was recommending to hire a thin ops person.
The Role of FinOps Beyond Automation
00:17:03
Speaker
Would you say there's still a need for that, like for a specific role, or can you totally outsource that as a company to let's say ProsperObs? Yeah, that there's certainly ah a need for FinOps people. I mean, there's a lot of cool things that you can do go do with FinOps around looking at, um you know, products like, you know, what are they called? Unit economics, where it's like you look at the the product that you're selling and the cloud costs that underline that product from an infrastructure standpoint of like compute network costs, storage costs, and then you you align that to clients and then to determine those kinds of things. And then, know, as you make like engineering changes, like there's a lot of decisions that, you know, could good influence ah or that an a FinOps person could help influence. They'll be quite impactful.
00:17:48
Speaker
But from a ah rate optimization standpoint, like that, that's probably best done with math. Like if you're, if you're just looking at, you know, some um very objective data points and making very predefined decisions um probably probably best to have a system that, you know,
00:18:04
Speaker
lives up to 99%, you know, expectations there versus something that that could make a mistake, you know, in a calculation of a, you know, a very, very, very defined logic problem.
00:18:15
Speaker
So, but it it goes back to kind of like that the human versus software or an AI. i think that, you know, there's some things that, you know, make sense, some things that don't make as much sense.
00:18:29
Speaker
So I have a question and it's a little bit of provocative, let's put it this way.
Transparency and Risk Management at ProsperOps
00:18:35
Speaker
I am known for those inglorious questions, but it comes from the experience folks.
00:18:40
Speaker
and I mean, like, do your point, Elliot, it's a SaaS, it's software. Well, you know, we we're all coming from this SaaS world. We perfectly know what is software and it also makes mistakes.
00:18:57
Speaker
Maybe not today. And the algorithm today is sophisticated advice, but there is false positives. There is, you know, sometimes it doesn't perform, there is bugs, whatever.
00:19:10
Speaker
So um and with you mentioned this this insane figure, $4 billion dollars that you reduced to manage for your clients. Do you want to share the case when things didn't go as you expected?
00:19:25
Speaker
Yeah, and how did you mitigate that? Because ah this is actually tells more about you as organizations because like it happened to us at MassEd. I can tell about it. Yeah, the reason we go sideways, we reacted on that in three days, but there was a little bit tension between us and the client.
00:19:48
Speaker
Yeah, ah totally. And let's I think this just goes back to like and how do you want to operate as a company? Yes. um we We really pride ourselves on you know trying to do what's right for for our customers. you know And sometimes, um let's just take an example, right? When we're buying these commitments, we are committing to something for, again, one or for three years.
00:20:10
Speaker
Just for our question, Andrew, want to clarify, you guys, as a Prosperob buys the commitments or the client buys on your advice? but We have i e are actually an automated execution engine.
00:20:22
Speaker
So we will buy on behalf of a client. So we put you in their accounts. I want to be very clear. Prosperops is not doing, not violating any terms of service to cloud providers. We're not moving discount instruments between customer tenants. You know, that's that's a practice that is that's actually gone at this moment. i idea I didn't know that was a violation, to be frank.
00:20:42
Speaker
ah would I wasn't pushing to disclose any kind of dirty secrets here. Sorry, I didn't know that was violation. Like, why not? Okay. It's all good. But yeah, we what what we do, we are essentially acting as an automated buyer for a specific customer tenant.
00:20:59
Speaker
And, you know, going back to kind of your your question, Yulia, about like, you know how do we handle mistakes? I think one is we try to prevent them as much as we we possibly can. um and And obviously, like every customer we talk to has a different preference for kind of the risk they want to take on in exchange for the savings they're trying to achieve.
00:21:17
Speaker
um And every customer falls somewhere on that spectrum, right? It's what determines if you're willing to cover up to 90% or only 70%. It's what determines if you buy one-year cuts or you buy three-year cuts.
00:21:30
Speaker
It's what determines if you buy resource-based cuts or flex cuts. Every one of those is some kind of decision between risk and savings. um And so what we tried to do is um align our system so that customers are have ah full control over the parameters that govern our system and govern that risk tolerance.
00:21:51
Speaker
right We have settings that can kind of determine if a customer wants to get more aggressive in their coverage or if they want to be a little bit more conservative. They can govern what type of discount instruments we're using for the for the default kind of primary coverage rate.
00:22:05
Speaker
And we'll we'll work through that with them. um And that's really just about transparency. yeah When we have a decision making process that's about tradeoffs, we have to be transparent about what those tradeoffs are and what the risks are with that.
00:22:18
Speaker
um Now we talk about kind of mistakes, right? Sometimes we see a ah decrease in savings because there's things that are outside of our control. Sometimes a a customer will agree to, you know, we' we'll deploy these commitments in this way.
00:22:30
Speaker
And then the usage gets slashed by 50%, you know, within a week. um That's not something we can easily react to, but it is a little bit, you know it's going to have an impact to our performance. um There's other things, right, that may be, you know, something our system didn't account for correctly.
00:22:46
Speaker
And in those types of cases, right, we try to work with the customer, notify them immediately of a ah mistake, and then we'll do our best, you know, with their customer experience or customer success reps to mitigate that mistake and, and you know, ah true it up, whether that be, you know, a credit or whatever that may be moving forwards.
00:23:02
Speaker
So. make so That sense. That's, you know, um those who never try, do everything right. But I know, i know the struggle, exactly. It's it's pretty difficult, especially with changing the world. Yeah.
00:23:19
Speaker
yeah and maybe Maybe to get into those changing workloads a little bit,
Analyzing Workload Patterns for Optimization
00:23:23
Speaker
right? So was I think we talked a little bit about the ah the steps of like, you know, getting these commitments first, like rethinking your contracts, rethinking how your team works with on the FinOps side, but I would like to also get into a little bit more like technical details of how to like actually do these optimizations. And so one thing i was wondering about if that's, which is something I always see from from the data side, and if that is something that you can account for is that on the data side, there's always this difference between like batch loads and,
00:24:05
Speaker
more like on-demand queries. right So recently, i did a simple calculation for a um a client. They were on Snowflake. They were like turning on their Snowflake in a warehouse all the time.
00:24:20
Speaker
ah because of like small analytical queries. And then once a day, they had to turn it on for their big batch workload. And then turned out when they switched to BigQuery, where you know there's ah ah the on-demand is just much cheaper than it would be for Snowflake. They were saving, like I don't know, 3,000 euros a yeah ah month or something.
00:24:44
Speaker
And so i was wondering, like, if you look at that, like, maybe your algorithm looks at that and sees like these very differentiating workloads, is that something that you can optimize for? Or like how how do you deal with those kinds situations?
00:25:01
Speaker
Yeah, this is um is probably one of the hardest things to actually manage when it comes to commitments, whether it be on BigQuery or compute resources or SQL or whatever you're you're managing a commitment instrument for, right?
00:25:13
Speaker
um This really goes back to the the concept of like workload volatility and cyclicality. um If you have batch jobs that are only going to be you know running for maybe a couple hours a day, um you're paying for a commitment whether you are covering into that workload or not. So you're only going to have a brief portion of the day where that commitment actually goes utilized, and which makes the the math pretty complex of kind of like, okay, again, whereas where am when am I breaking even on my investment with these types of workloads?
00:25:44
Speaker
Now, I can speak from my own experience on this. When we talk about like compete workloads as an example, um there tends to be a very stable portion of usage. um Every customer has some type of usage that is always odd.
00:25:57
Speaker
ah They have you know servers they run or they have ah just core infrastructure that does not scale or least scale to an extreme extent. And then on top of that, they have those sort of dynamic workloads, which you know maybe are leveraging Kubernetes. Maybe they have um scheduling enabled and and they're you know they have periodic bursts where they're either on or off.
00:26:21
Speaker
um And that that kind of creates these hills and troughs of usage, which is a ah really interesting concept. Because that's really where like the the hard part begins. because It's really easy to cover something that's always odd. You can just cover 100% of it.
00:26:34
Speaker
It's a lot harder to cover something that's scaling up and down, especially as it's changing week over week, day over day. um And so this goes um into kind of an interesting concept with commitments, which is the idea of an optimal coverage point.
00:26:51
Speaker
um If we cover you just up to the stable portion of a workload, great. We're going to have 100% utilization of that commitment, but we're actually leaving some coverage on the table because we're not covering into the volatile portions.
00:27:05
Speaker
um Or you can get aggressive and you can cover 100% coverage, but your utilization is probably going to be too low to where you're generating too much waste. um And so part of what our system is doing in the background for compute workloads is we're looking at um looking at a graph of kind of ah different points of coverage and seeing when are we able to generate the maximum amount of savings and we're actually you know breaking even on that investment.
00:27:34
Speaker
um And that is largely determined, going back to your question, it's largely determined by the kind of volatility of a workload. um If you have workloads that are predictable and you know maybe they're on for five of the seven days of a week, that's pretty easy to determine an optimal point of coverage for.
00:27:51
Speaker
If you have something that's only on maybe for an hour or two a day, that's something that's probably not going to be consistent enough to really justify committing to. um And we can provide a very objective answer to this when you run the math and you can actually see where that coverage target will change depending on the volatility of different workloads.
00:28:11
Speaker
so
00:28:14
Speaker
that's super That's super interesting. And and so would it that make sense? Like, maybe not from a operational business operational perspective, but but so there's this, ah like Databricks says this, Netflix says this, they always say like, oh, you can just pick the the biggest warehouse for your workloads because it will run your job faster.
00:28:36
Speaker
and now I'm thinking like, what if I just squeeze that job a little bit more? If I don't care, like if it just runs from midnight until like, six in the morning, but on a very like small machine, maybe I can get a better discount with, with commitments on that. Is that like the kind of logic that would work, uh, would work here?
00:28:55
Speaker
Yeah, it's, it's certainly part of it. There's, there's so many different components that go into like maximizing your savings rate. Um, we, we have a metric, we call it the effective savings rate, but it's like looking at the percentage you saved off the list cost for a service.
00:29:10
Speaker
When you consider all the savings you've generated on it. um And that's a really nice metric for rate optimization because it gives you kind of a ah benchmark that you can use either against yourself or against your peers to see, okay, am I managing my commitments effectively compared to other companies in the space ah this of similar size?
00:29:30
Speaker
um But I guess your your kind of question is interesting because it's talking almost about like price performance. um For every resource you run, you have an option to deploy it on differing machine types.
00:29:42
Speaker
um And those machine types, you know, offer a performance to a price ratio, yeah ah which, you know, really really depends. um And I see firsthand a lot of customers end up in a very interesting scenario where um maybe, like, let's just take an example.
00:30:01
Speaker
C4A as a machine type right now on Google is extremely price performance, along with N4s. They're the newest machine types, and so therefore you would expect them to have the best price performance ratio.
00:30:12
Speaker
However, ah if you have a negotiated agreement with Google or enterprise agreement, some customers may have negotiated rates that favor you staying on N2, even if it is a less price-performing machine type.
00:30:24
Speaker
And your negotiated rates with Google might actually skew that to the point where ah maybe you're better off just staying on N2 for the next year or two, at least until that commitment expires. And so these are like very real conversations I have almost every week kind of talking through these decision points.
00:30:42
Speaker
And you need to have some kind of, you know, whether you're using core mark or another tool, you have to have some kind of sense for what the performance is you're getting for the negotiated rate that you have to pay for a service.
00:30:56
Speaker
You have so many variables. Like, dude, look like it's way too much. And I can see that but the product, like people cannot comprehend, you know, more than five things simultaneously. And here we are talking about maybe depending on the scale, like 20, 50 and more.
00:31:15
Speaker
um But my question, folks, because um to this point, and we are half an hour at the recording, no one mentioned Yeah. but what What is wrong? Are you guys out of bubble? Did you see there is, you know, Gemini, ChatGPT? Like, what is wrong? Like, don't and your customer threaten you like, oh, we can we can actually do it ourselves. And, you know, there is ChatGPT. Like, how do you, what are your feelings about it?
00:31:45
Speaker
Yeah. um I guess maybe I'll start and then I'll hand it off to Elliot. but and We talked about the idea of commitments and risk. um yeah <unk> We're not actually large users of AI, at least in the sense of like you know generative AI.
00:32:01
Speaker
Do you want to cut it out now? What's that? Do you want me to cut it out? No, that's the um I mean, that this is actually kind of an interesting discussion point because um When we talk about commitments, you know, if you're kind of like, let's imagine you're vibe coding a system that's going to manage millions of dollars of financial commitments.
Objective Logic and AI in Financial Commitments
00:32:26
Speaker
You want to be very sure in the logic that is governing that system. And ah to Elliot's earlier point, we use conditional logic because it's objective. um We want to make sure we understand the parameters which decisions are made in.
00:32:40
Speaker
If we do not understand that, we're putting our customers at risk, right? And so that's something we take very seriously. And so for that reason, we're very conservative in our approach and use of AI in that sense.
00:32:51
Speaker
um That being said, though, there's lots of, you know, future possibilities. I'm not saying no to anything in the future, but Elliot, I'll maybe hand it off to you if you have more to highlight there. Yeah, I have kind of a little bit of a comic ah anecdote here. I mean, in the generative AI is really cool, but you know there there is this like random aspect to generative AI that's similar to like a slot machine.
00:33:14
Speaker
And I'd say when it comes to to managing... like financial commitments, you probably don't want to be playing slots with, ah you know, what you're committing to. So um it's, it's, you know, realistically, like when we, we define our systems, it's, you know, very objective logic-based math um because we want the outcomes to be very precise, exact, and, you know, to the specification.
00:33:37
Speaker
you know if I want something random, um like a slot machine, I'll use generative AI. But generally speaking, i don't really want that for something that's planned and, you know, from a financial instrument standpoint.
00:33:51
Speaker
So that's our thoughts on AI right now. I get a question on that. Should it be more helpful to model and predict Well, I know it's not the Germanistic. Yeah, I know. it's I absolutely know what I'm saying right now. Just, you know, giving you heads up.
00:34:08
Speaker
Yeah. Shouldn't be easier to predict the performance with AI? I don't know. Maybe you can massage it, fine tune it somehow, the necessary functions that you already have in your overview. It shouldn't be easier to predict which outcome is the best for the client.
00:34:27
Speaker
Yeah. And I think ah to to that point, it's it's an interesting use case. I think we're talking about like forecasting future usage based on previous patterns. Yeah. That's a very, you know, applied use of AI that I think, you know, does make sense.
00:34:41
Speaker
um Now, and this this is, you know, i'm kind of getting into the technical details of our product, but we try to be as reactive as possible to the current state of the environment and less reliant on forecasted usage.
00:34:53
Speaker
um For example, we don't want to be buying a commitment before that usage exists. right We want to be making sure that it's based on the current state of the environment, which we're getting directly from the cloud providers billing exports.
00:35:06
Speaker
um And so we just have to be cautious right with kind of how we leverage that forecasted data because we want to make sure that we are still providing good outcomes you know what for what our customer is doing today, um if that if that makes sense to you.
00:35:21
Speaker
No, that's totally, I mean, it's so basically like a values of the company and organization and your mission. So yeah, totally makes sense to me. It's just I envision so many startups to pop out, ah you know, AI automated management of Kubernetes ABCD.
00:35:41
Speaker
Yeah. Yeah. And I'll, I'll, I'll add, you know, it's like, you know, the, the ability to to forecast and kind of like plan, plan out, like a lot of people, if they're, you know, forecasting like ah a growth of a product, um, like they'll, they'll, they'll know that in the next couple of months, they're going to have a bunch of users entering their platform and they'll be able to react.
00:36:01
Speaker
it's It's really hard to go predict that, you know, one day you're going to spike of like, you know, 20X the amount of servers but for this period, um you know, where, you know, a machine it could be wrong. And it's like, you know, i yeah a lot of people get worked up when the the the weather reports are wrong. And we have more data points on weather and AI modeling this stuff out than, you know, anything else on earth here. And it's like, we we still can't predict when the weather like is going to, you know, when the when the rains are going to come down. And it's like,
00:36:32
Speaker
Forecasting like cloud costs is is kind of kind of similar here where it's it's easier to go predict based on talking to a person and having a regressive or pre-planned set of lookbacks to go make the modifications appropriately.
00:36:48
Speaker
So, um you know but back to the whole concept of mistakes have costs here and, you know, the less mistakes we can make are typically better. You know it's better for us to be a little bit less risky and not make a mistake than be more risky and make a mistake in most people's eyes.
00:37:07
Speaker
I like the approach. I just want to give you a... ah kudos for this qualitative and quantitative approach where you say, okay, we have metrics and you're dealing data and the usage, but also we would like to contact and talk to the clients about their future plans and strategies. What are the goals?
00:37:26
Speaker
Maybe we can suggest and it came up with a better strategy for that. Yeah. Yeah, and it's also is very refreshing to to hear a company that's not necessarily jumping on that AI hype train, but actually sticking to you know consistent deterministic rules as well as interaction with the client. So yeah, I really appreciate that.
Cross-Department Collaboration for Cloud Optimization
00:37:46
Speaker
and And so regarding that interaction with um with the client, where where do you usually see Maybe I'm asking this question little bit from a perspective of how could I or the teams that I work with kind of leverage this?
00:38:02
Speaker
Is this usually located in a data team or really on the like financial team? Or like how could a data team, for example, leverage this type of data? especially i'm um I'm saying this especially as data teams sometimes struggle with their own ah ROI, right?
00:38:22
Speaker
Yeah. Domki, just guys are serving to software teams mostly, no data teams, right? No, it ah so no so i get i get that it's like, it goes to software teams, but like The FinOps function, I would say, a um is part of of the financial department. right So obviously, the financial department has an interest interest interest in decreasing finances.
00:38:52
Speaker
What I'm saying is that a lot of i see a lot of data teams that are interested. They're bringing in finance data. They're bringing in operational data from the company. They try to be like a single point of ah truth, basically, for everything that's happening within a company.
00:39:07
Speaker
and So obviously, like I can see that the software and and like the development teams of a company would want this kind of tooling, but I can also see an opportunity for data teams where this could be really interesting um you know to get a ah little bit better ah ROI on there on their on their data, basically.
00:39:31
Speaker
But yeah, I'm also curious to see ah here if you guys see any kind of like infighting in in between teams at the clients you work with. Yeah, I hope it's not infighting, but we definitely hear it.
00:39:44
Speaker
Yeah, you have conversations, right? there is For a lot of companies, and this obviously varies so much depending on the size of a company, maturity, etc. but I'm just going to give one kind of rough example.
00:39:57
Speaker
um If you're like a medium sized company, you probably have a centralized FinOps practice, right? You have ah either a guy or a team of people that kind of oversee the overall cloud spend in an environment.
00:40:09
Speaker
um And traditionally, because commitments in Google, at least for CUDS, this is a little bit different with like BigQuery slot commitments. But for CUDS specifically, they apply across the whole scope of resources on a billing account.
00:40:23
Speaker
And so they're a really good thing to centralize management of because you kind of have to, by nature, share them out across multiple different projects, business units, et cetera. um And so for commitments, we tend to work with those kinds of FinOps personas.
00:40:37
Speaker
That being said, some, some companies, right, that ends up falling to a data engineer that has that responsibility to manage that. um Sometimes it's a finance persona. Sometimes it's a SRE or DevOps engineer.
00:40:50
Speaker
I've seen, you know, every sort of mix of of people that end up kind of taking on that day-to-day responsibility. Um, Now, there's definitely an interesting kind of interplay, I'd say, between the kind of different teams and the the people that manage it.
00:41:04
Speaker
um For everything we're we're talking about, it's the idea of context into the future. um If you want to get the best savings, you have to have a pretty good idea of the context of what's about to happen.
00:41:16
Speaker
ah Part of our goal at CrossFrops is we want to mitigate that requirement for context as much as we can through our purchasing strategies. But that said, some context is still required. And that's why when we meet with teams, we want to go over a kind of the structure of what we want to go do moving forwards when we talk about the plan and portfolio composition.
00:41:35
Speaker
um But with that kind of like interplay, a lot of times people just have to validate what the future state's going to look like. and We want to understand, are we going to still be in these regions? Are we still going to be you know using application A in the next couple of years?
00:41:50
Speaker
And just getting that kind of rough alignment is usually where we see that kind of cross-collaboration between FinOps and like direct engineering resources, whether they be on the data side or infrastructure side.
00:42:01
Speaker
Yeah.
00:42:04
Speaker
it's not Do you want to share how much money you saved ah for one of your clients? like um Please don't don't name the client. just yeah It's interesting how much money you saved for one client the most.
00:42:19
Speaker
Yeah, I think um like can I can say across all of our clients, the the number to date, the one I like the most is that it's, think we're above $2.5 billion dollars of incremental savings we've generated.
00:42:33
Speaker
a net of our fees. This this is is kind of a long answer, but it's a really cool one because ah the two and a half billion is incremental. So for every customer we work with, we baseline their current performance before they started using cross props.
00:42:47
Speaker
And so that's stating that after they started using, we added an additional two and a half billion because a lot of customers probably already have commitments before they started, right? So we have to baseline their performance to see, okay, what do we actually add above and beyond what they were doing?
00:43:02
Speaker
The second part that's cool is it's net of our fees. um Obviously, there's you know we're we're not a free service. There is a cost to... so Non-profit. that there' There's a cost to using our service. and We try to quantify that when we when we run an analysis and talk to customers of what percentage of the savings we'll take.
00:43:23
Speaker
The majority of the savings will go back to the customer. and Ideally, right that's above and beyond what they were doing prior to using our tool. But The $2.5 billion being net of our fees is cool because you know even after we've gotten paid, we're still saving that much money in addition to to where customers started off with, which I think is pretty cool. but do Do you ever get bad looks from from the big cloud providers?
00:43:45
Speaker
they They must see this kind of as a bit of a loss as well, right? Well, the funny funny thing there is like you know the that the smarter account managers at the different cloud providers actually see you know what we do as
Strategic Savings with Cloud Providers
00:44:00
Speaker
beneficial.
00:44:00
Speaker
um And the the reason there is that you basically have a commoditized product that is compute. And and that that product is like you know it's available at all three clouds. It's really not that differentiated. like Everybody has the same Intel processors, AMD processors, and and RAM setups.
00:44:17
Speaker
Now, what what is differentiated with these cloud providers is like whatever AI tooling, some of the APIs, some of the specific databases they might might have available or some of the other like widgets and functions.
00:44:28
Speaker
Those things are a little bit more sticky for the cloud provider. So we're we're actually you know working with Google and AWS and Azure around some sort sort of initiatives around moving commoditized spend savings towards a AI project or something that's a little bit more strategic to those organizations where they're re-allocating and getting their customers quote-unquote efficient spend on their core applications. And then they're able to go reallocate budget towards other things.
00:44:58
Speaker
And so the net gain is that you know the customer gets additional functionality, cloud provider gets additional stickiness in that the environment, and the customer feels like they they saved and got a new new product.
00:45:09
Speaker
So I'm sure Andrew has more more thoughts there as well. So you kind of get them get them to save to spend more. yeah but position i think this is a super interesting question.
00:45:23
Speaker
yeah I think about this one a lot, right? It's like, How does this play out? And what we hear constantly from customers that are in migrations is the reason they're leaving is because they didn't feel like they were getting a strong ah ROI on their cloud. um right they They either thought they were overpaying for resources or were not getting the support they needed or the know reliability or functionality wasn't there.
00:45:49
Speaker
um And when you hone in specifically on cost, what we see across our customers is that when we deploy our service and you get the rate side, at least when you when you knock off one of the tickets, when you get the rate side optimized, it makes the customer stickier to that cloud provider.
00:46:06
Speaker
um When you feel that you're generating great savings and that you're gritting getting a great value for the services you are using, you're less likely to go shop around for better deals from other cloud providers. um it's very It's a counterintuitive concept, right? You think the more you save, like, okay, great. like that you I should probably be looking to go move my workloads. But in reality, the data we have supports that the exact opposite occurs, which I think is pretty pretty fascinating.
00:46:38
Speaker
I have last question box before we break power.
Challenges in Cloud FinOps
00:46:41
Speaker
Why do you think the cloud phenotypes is actually this difficult?
00:46:49
Speaker
I think the random nature of what you could go do in the cloud and the ever-changing variables. I mean, that the cost of a compute ah resource is different each day slightly, and it can change at any time.
00:47:04
Speaker
The needs of a cloud workload could could change at any time. And there's hundreds of different things that you could go do in your environment. And and every every customer's discounts are slightly different.
00:47:17
Speaker
um So they're they might the amount to variables upon variables upon variables makes it a very complex challenge um that is is hard hard to come up with great answers on.
00:47:30
Speaker
I'll take it over to you, and Andrew. that's ah That's a fun question. i think like you know there's a lot of like the the answers you'll see online of like CapEx versus OpEx and the the challenges with that.
00:47:41
Speaker
i think the the real simple answer for me is just the number of decisions that you have to make in a cloud model um because you're kind of handing off the authority to purchase or make resource changes to direct engineers.
00:47:55
Speaker
And that just, you know, multiplies the number of decisions that impact costs when you give somebody that authority to do that. um And that's like how cloud designed, right? It's designed to make people feel enabled to to get access to the services and infrastructure that they need.
00:48:12
Speaker
um But at the same time, that can complicate the cost structure of how you pay for those resources. um in In general, you know, we we went from an inelastic model where you paid for everything up front and you used it and you were stuck with that capacity.
00:48:26
Speaker
Now we've adjusted to a model where you have infinite capacity, but your costs have moved to a very variable model. um And so it's it's just a it's a trade-off, right? um And our goal is like, okay, if we're going to have elastic usage, let's have elastic discounts so that at least we can scale up and down with that usage and have more predictable savings on it.
00:48:47
Speaker
So... Yeah, this is this is super interesting. And so you the way you explain it is like reminds me or it like triggers this analogy for me of you give people, your engineers, entrance to like this giant infinite marketplace and you give them a budget and just say, like go make some food today. And then you hope that all works out well and they can make the right choices. But obviously, yeah, you need some guidance guidance in that.
00:49:18
Speaker
Yeah. And it's, it's kind of, you know, like everybody kind of, it was new for everybody when, when this transition first started occurring. And, you know, we've seen, I think maturity in, in how people handle this. There's a more structure in standardization around best practices than there ever has been.
00:49:34
Speaker
um You know, people are looking at automation as solutions for parts of these problems. um But I don't think it's ever really truly going to be solved.
00:49:45
Speaker
um It's always going to be challenging. The key part is just is your business in a position where you can adapt quickly and um adopt the best solutions that are available at that point in time.
00:49:58
Speaker
I don't know. and that That's so beautifully put. ah Folks, i i feel like we can spend an hour talking to you extremely knowledgeable, extremely experienced.
00:50:11
Speaker
So it was pleasure talking to you. I don't know how you guys feel ah listening to it, but I already want to have a demo from Andrew ah and deep dive into the product. and I don't think I necessarily need it but I feel like I need the demo.
Conclusion and Call to Action
00:50:27
Speaker
ah Yeah, so um just tell us how people can you know find you and possibly follow you. um Yeah. Well, we do have a website, prosperops.com. And if you want to get ah ah a free analysis of what our model could go do in your environment, we do do have that.
00:50:47
Speaker
You just basically have to go set up a minimal privilege access into your environment and we can model out what we could go do and all for free. cool. Sounds exciting.
00:50:58
Speaker
Yeah. Folks, thank you so much for stopping by. Yeah. Thanks so much, guys. Thanks a lot. Appreciate Thank you very much.