Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
What is Serverless?
1.4k Plays1 year ago
In this Episode, Bhavin and Ryan tackle the topic of serverless computing. Bhavin and Ryan try to help answer the basics of what serverless is, isnt and why the term can be so confusing.
Show Links
- https://www.redhat.com/en/topics/cloud-native-apps/what-is-serverless
- https://www.cloudflare.com/learning/serverless/what-is-serverless/
- https://www.readysetcloud.io/blog/allen.helton/lets-stop-talking-about-serverless-cold-starts/
- https://keda.sh/
- https://www.redhat.com/en/topics/microservices/what-is-knative
- https://www.prisma.io/dataguide/serverless/serverless-comparison
- https://workers.cloudflare.com/
- https://blog.back4app.com/cloudflare-vs-heroku-which-is-the-best-option/
- https://www.bmc.com/blogs/serverless-faas/
- https://world.hey.com/dhh/don-t-be-fooled-by-serverless-776cd730
Recommended
Transcript
Introduction to Kubernetes Bites
00:00:03
Speaker
You are listening to Kubernetes Bites, a podcast bringing you the latest from the world of cloud native data management. My name is Ryan Walner and I'm joined by Bob and Shaw coming to you from Boston, Massachusetts.
Discussion of Cloud Native News and Interviews
00:00:14
Speaker
We'll be sharing our thoughts on recent cloud native news and talking to industry experts about their experiences and challenges managing the wealth of data in today's cloud native ecosystem.
00:00:26
Speaker
Good morning, good afternoon, and good evening wherever you are. We're coming to you from Boston, Massachusetts.
Personal Updates from Hosts
00:00:32
Speaker
Today is March 3rd, 2023. Hope everyone is doing well and staying safe. Let's dive into it. Bavin, it's Ben.
00:00:42
Speaker
a couple weeks as per usual. Although I've talked to you before then anyway. What's going on with you? What are you up to? I don't know, man. I'm just heads down into work. So Pure Storage has sales kickoff next week. So it's all getting ready for it and working on my presentations and labs and things like that. So keeping busy.
00:01:02
Speaker
And as you can see, I've been lazy enough. I still need a haircut, even though I fly out in two days. And then I know we keep talking about like the blind that I need to put on that window. You bought it, did you not, right? I had it for I think four weeks at this point, but it's just about like drilling a couple of holes in that window pane and then putting those blinds on. But yeah, I have been lazy. How about you, Ryan? What's going on?
Ryan's Travel Plans: New York to Maine
00:01:28
Speaker
It's all good. I've had a blind up. You can't see my window.
00:01:32
Speaker
basically since I moved in here. It's a great blind, I will admit. Did it myself, quite proud of it. If I seem slow down, I'm recovering from being sick. Got the coof, shout out to Tim there for the third time. This time I couldn't taste, which was a weird sensation. People often talked about that, never happened to me until this time.
00:01:59
Speaker
for a few times it was basically like I had allergies, I barely even noticed this time. It was strange, I had vertigo and it's been a rough week. So we're covering, catching up, those kind of things. I also did, I changed one of my summer plans, which I'm always looking forward to. I was supposed to go out on this big trip to the Midwest and I decided after planning it, that eight days in my car alone,
00:02:29
Speaker
was not something I wanted to do. I was very okay with it for quite a long time. Even if I drove like even longer, it'd be like three days in the car for an event that is three nights, four days. So it's like half the trip in the car. And then I was convincing someone to go with me on this trip.
00:02:50
Speaker
And I actually had them pretty hooked. And then they convinced me like, hey, we have this other thing in our backyards in the Northeast. And it winds up that we're going to do this 10 day, 1400 mile trip from New York to Maine over.
00:03:09
Speaker
over the summer. And we're doing that instead of me driving out there. And this time I get to hang out with my buddy, you get to do it. You know, get to be sort of in it the entire time, closer to home.
Weather Impact on Travel Plans
00:03:20
Speaker
I think it's overall a better choice. Driving eight days on your own, not a fun plan.
00:03:28
Speaker
I do wish to do that one day though. And maybe like with the whole family and like a camper or something like that. Like the typical thing that, you know, we'll do, you know, I feel like I, there's so much to see in the U S in, especially in the Midwest. I have spent very little time out there. So I think, uh, during, during 2020, I used to follow on Twitter. I still am, but, uh, he, he rented an RV or something like that and basically drove from North Carolina.
00:03:56
Speaker
Through all the national parks went to Washington and then came back Washington state not DC because that yeah not a fun trip across the country hitting all the different national parks and did a loop back so
00:04:09
Speaker
Yeah, that's a lot of people. Yeah, a lot of people rented and did those in 2020. I remember I looked to do it in a short period of time in 2020, or even maybe it was 21. And like everything was rented out because I already did it. Yeah. If you really want to get a kick, I think if you do a YouTube search, I think the Daily Show with Trevor Noah did a clip about how
00:04:29
Speaker
Novice people rented RVs and didn't know how to operate them on freeways. Tried to take exits, like the ramps that they usually do in their car, and like, boom, everything was toppling over. So, yeah. Well, I mean, they will rent you and just send you on your way. As long as you have insurance. Yeah, you ever rent a giant U-Haul truck and you're in this thing like, I shouldn't be driving this thing.
00:04:55
Speaker
that's how I feel like you feel anyway anyway yeah so that's I'm looking forward to that and yeah other than that I'm looking for this weekend got some snow coming but
00:05:08
Speaker
gonna chill, I think, at home. I was gonna head to New York and visit some family, but with nine or so inches of snow potentially, probably not. Well, I'm glad you're doing better. Yeah, thank you. Yeah, I am too. Especially it was break here, so I couldn't fully enjoy it. Anyway.
00:05:26
Speaker
Gotcha. Small potatoes.
Exploring Serverless: Definition and Implications
00:05:28
Speaker
The show for this this week is is Bob and I, you know, you haven't had a show with just Bob and I in quite some time. We like to do these every now and then mix it in. And today is going to be about what is serverless.
00:05:42
Speaker
Going back to school. Going back to school, we like to kind of dive into topics, do sort of a mile wide, foot deep episode on what certain topics are. We actually are doing this. On purpose, we have a couple of guests lined up that are going to touch on some of these topics so it feels appropriate. But before we dive into that, we're going to do a little bit of news. We have a few things going on. Bhavan, why don't you
Akamai's Acquisition of ONDAT
00:06:04
Speaker
kick us off?
00:06:04
Speaker
Yeah, sure. This week, over the last couple of weeks, we saw acquisitions and pivots from startups. So just talking about a couple of things. I think one, one of the vendors in our ecosystem, our friends at OnDart got acquired by Akamai. And I know, and I think this news just broke out yesterday. So I know when we met the senior principal engineer at
00:06:28
Speaker
KubeCon, I forgot his name, that works for Linode and how they want to build out the whole Kubernetes ecosystem and these Edge deployments. Sorry? Stephen. Oh yeah, Stephen. Yes, thank you. And how they wanted to build these managed services out and at these Edge locations and CDNs and offered it as a service. I think on that fits right in like as that storage layer.
00:06:50
Speaker
good for the people at ONDAT that now have a new, bigger home and can have a bigger impact with the footprint that Akamai has already. Absolutely. I know ONDAT was one of those players when we first were in this ecosystem with Portworx before we ever
00:07:08
Speaker
you know, we're up here and things like that. And we were, you know, always close and in touch with what they were doing. And now it's just exciting to see, you know, all these players that entered five or six years ago, coming out on top. Obviously, not everybody can. So it's really good to see everyone over on that, join
ContainIQ's Business Model Pivot
00:07:29
Speaker
a bigger family.
00:07:30
Speaker
And yeah, excited to see what I think Akamai is doing a lot of cool things, especially like you said with Linode and what they're doing with Kubernetes. I think I read an article recently that was like, it compared all the Kubernetes services out there. And Linode's was like the fastest to like bootstrap Kubernetes cluster. So a little, a little. Good job for that, I guess.
00:07:50
Speaker
And to your point, there were so many vendors. I don't think there are any more independent Kubernetes storage vendors out there. There are. Yeah, there definitely are. We talked about a few recently, but I feel like they came in after. Oh, after the first wave, I guess. Yeah. I think the first wave on that is now Akamai. Robin is now Rakuten. Maya data got acquired by Datacore. Portworx by Pure Storage. So the first wave is kind of gone. Now we have the newer vendors, I guess.
00:08:21
Speaker
Yeah, yeah, it's good stuff.
00:08:23
Speaker
And then second, like after talking about acquisition, just wanted to highlight a pivot from one of the startups. And we had these people or the CTO for ContainIQ on the podcast to talk about EBPF and give us a one-on-one overview of what EBPF is. They have decided to do a pivot and start a new content generation company or things like that. Again, I don't clearly understand what they are pivoting to, but it's just that I don't know what happened, but ContainIQ is
00:08:51
Speaker
has a new future which is not public yet so they did like obviously transition the customers off to some somebody else are we waiting for those news but you went through it so if you are looking for. I have the one thing i really like about them is how they created such a huge base of.
00:09:07
Speaker
Kubernetes 101 articles, right? Like they had these native searches going on your website and they had a lot of useful information. So we definitely missed that. But again, congrats to, I guess, the co-founders on a successful pivot and now they can continue doing what they love.
Kepler and Power Consumption Estimation in Kubernetes
00:09:24
Speaker
Absolutely. And then the final article was around Red Hat and how it now open source the project or not open source but donated the project Kepler to CNCF. So Kepler has been around for a while and what it does is it uses eBPF again tying it back to eBPF I guess.
00:09:42
Speaker
and then uses eBPF layer CPU performance counters and some machine learning models to estimate the power consumptions for your workloads that are running on that Kubernetes cluster. So it shows you a time series data and you can actually take these metrics and create awesome Grafana dashboards and then show how much power your container spots, namespaces or different compute nodes in the Kubernetes cluster are consuming. And Kepler can also help you like take smarter
00:10:11
Speaker
orchestration or provisioning decisions. If you wanted to provision workloads on nodes that are not as carbon footprint intensive, you can do that using Kepler and the Kubernetes scheduler. Interesting project that I came across, I just wanted to share it with the ecosystem. If one of the organizational priorities you have today is making sure you track your carbon footprint, Kepler can be the tool that helps you with that. Cool, absolutely. That's it for news for me.
IBM and Red Hat Integration
00:10:40
Speaker
Cool. Uh, I had just one here, um, which was, you know, we've been following the story of, of sort of Red Hat and IBM kind of coming together that, that sort of happened by the end of 2022. It was okay. And, um, you know, they've been putting a lot of effort into sort of.
00:10:59
Speaker
their Open Data Foundation and their Ceph products and really kind of pushing that whole thing. This article is really about how sort of the spectrum line, IBM spectrum is now renamed to IBM storage, but it's not just a rename. They're doing a lot of stuff underneath that. As far as I understanding it, they're sort of simplifying what they're offering underneath that name, part of which is sort of that
00:11:24
Speaker
data AI hybrid, which does include Ceph and what's in the Red Hat portfolio. But then it also includes the Fusion products from IBM. And I think it's just a way for like, obviously, Red Hat and IBM are both pretty big companies. So I think this is sort of a natural thing we're going to see as they figure out their portfolio, things are going to change. I don't hate the name, I don't hate simplifications.
00:11:51
Speaker
especially when you have a pretty complex portfolio with Red Hats and IBM. So exciting to see. I know we're always a big fan of ODF and what OpenShift and Red Hats been doing there. So cool to see how they're starting to integrate it all. But if you're interested, go take a look at what that means and if you want to go read about it.
00:12:13
Speaker
And I think I like your point, right? Like as long as it simplifies things, that's perfect. Like there was a lot of confusion in how ODF and Spectrum and all of these different storage portfolios work together. So I like this too.
00:12:25
Speaker
Yeah. Yeah. I think, you know, in a complex multi-layered landscape, like we are always living in, simplification is an underrated thing to do when you're living in a world of abstractions. Speaking of abstractions, let's talk about serverless. That's an awesome segue, dude.
00:12:47
Speaker
First, let's talk about the name serverless and what it actually means. Do you want to kick us off there? Yeah, sure. So serverless computing, I think when the word was first introduced, there was a lot of pushback around, like, there are actually servers there.
00:13:06
Speaker
I think whenever you talk about serverless, that's the first disclaimer that you have to give to the listeners. Like, okay, yes, everybody knows there are servers there. It's just about the management as you, as you rightly pointed out, like it's about the abstraction. Like, am I the one who is responsible for managing those servers?
00:13:22
Speaker
If I'm not, then it's serverless for me. Like I am just worried about how my applications are running or how do I deploy my applications. I don't really care about what the infrastructure stack looks like. So I think that's like a good overview for me for what serverless is in just like a one line. Yeah, I mean, should we call it managementless?
00:13:42
Speaker
Really, every article you read will say something along the lines of this is what serverless is. But lo and behold, you know, things are still running on the server somewhere. You just do not have to be aware of it. And the way I kind of like think about it is, you know, you almost have to go because we're living in this Kubernetes cloud native world. And this is what this podcast is really about. If we take a step back,
00:14:04
Speaker
and think about how servers were pets. Servers were provisioned, your application was put on, and you had to manage it to think about it. This is really an extension of now, you can send your application off to run somewhere and not have to worry about the rest of it. I feel like it's often conflated because we're living in this Kubernetes world of like, we deal with infrastructure a lot. You're like, well, you can't just run without servers.
00:14:32
Speaker
It's a valid point. Whoever came up with this name, great buzzword, terrible, terrible buzzword. Maybe there's a top 10 worst buzzwords of all time. This would be towards the top in my list, personally. I hate the term, but it is used a lot. We have to be comfortable with what it means by that. It just really means that you're not managing the servers that the thing is running on. There's a lot of more to that detail.
00:15:02
Speaker
When we're going into serverless and we talk about that real servers are still being used, but developers don't need to be aware of them. Let's talk about some of the like, why would you use this? What are the benefits? Yeah, for sure. I think we already covered like there are no servers to provision or manage part, but one of the key benefits of serverless brings to the table is you never pay for idle resources, right? Your application can truly scale down to zero.
00:15:26
Speaker
If there are no API calls being made to that piece of application code, it can actually go down to zero and then it can scale up as more user traffic is being generated and more resources are being needed by that application. So it scales with usage, which basically means that you'd never pay for idle resource consumption. So it allows you to just have your application code submitted.
00:15:50
Speaker
And then whenever your web app or your users want to interact with it through, let's say, an HTTP request, that's when the application code is spun up on a container or on whatever a micro VM or whatever the cloud provider who's offering the service wants to run your application code. And so it allows you to scale with users.
00:16:09
Speaker
Yeah, and there are some caveats to that. In research and hearing about how people are actually using it, some of the benefits are, if you're using it at such an enormous scale, some of those benefits can start to wean in the sense that if you're running enough serverless functions, you're effectively are using an entire server, right?
00:16:30
Speaker
Whereas the initial benefit of you're just running what you need, which I do like the analogy of if you're paying for a cell phone plan and only use some of your data, that whole concept versus just paying for what you are using, the idea behind the services is you just send off your thing and it's paying for the CPU cycles and memory it's using at the time of its stanchion.
00:16:54
Speaker
But yeah, I have heard horror stories of it's another way for Cloud to make money and all that kind of things. I can't speak from personal experience. If you can, we'd love to hear from you, but I know there are some caveats to it in the sense that you still want to make sure it's right for what you're doing. So taking how the future will look and how you're planning on using it at what scale,
00:17:24
Speaker
You could get into that. But generally, if you're using it as intended, I think at a reasonable scale, you're going to definitely find that sort of cost because you're sort of just renting space instead of an entire server.
00:17:39
Speaker
It also helps you reduce your operational overhead. I think in my previous job, I was responsible for a lab that I built for other teammates to use. And since it was I who provisioned it and was responsible for managing it, there were a lot of
00:17:55
Speaker
things that included like updates. So starting from the BIOS and XCC level on the servers itself, the OS level, applying patches whenever needed, then the virtualization tier. Again, there was a lot of work involved, right? And obviously, like, if you just want to run your app, you don't want to think about all of these.
00:18:11
Speaker
Given some of these challenges were already solved by cloud computing, but then if you're running EC2 instances, maybe you have to patch those EC2 instances as well. So this is taking that level of abstraction one layer ahead or above and just allowing you to focus on your application code.
00:18:27
Speaker
Yeah. And I think we have the bullet point here, simplified backend code. You'll probably see that in a bunch of articles. And really, that just means that instead of writing all the code to instantiate an entire server to serve maybe a particular function in the backend code, you might be able to just write that function and hand it off to the
00:18:45
Speaker
serverless platform. We'll talk more about platforms and like what you get out of them. But you hand it off and it'll just serve it from an API endpoint, right? So you don't have to actually build all that around whatever your code is doing, whatever that bit of compute that you need is doing. It kind of handles a lot of that for you. So like technically, you're reducing the amount of stuff you have to care about and write yourself.
00:19:08
Speaker
That being said, the natural question I think is there is sort of this question around lock-in in the sense that I don't think there's a standard sort of interface for serverless functions or serverless code that works across cloud providers or anything like that.
00:19:28
Speaker
It doesn't, right? But I think that's what we'll talk about in the later half of the podcast, where projects like Knative, like, so if you're running Kubernetes, projects like Knative gives you that standardized interface to run serverless functions anywhere you want. But yeah, if you're talking about things like AWS Lambda, or Azure Functions, or Google Cloud Functions, there isn't that real uniform layer that can allow you to take off Lambda and then run it on Azure Functions, as you said.
00:19:53
Speaker
Yeah, yeah, exactly. And I think the last bullet I had here was just like the overall efficiency, right? If you're only focusing on stuff that matters, technically, your devs can focus on what matters, etc. All those fancy sounding things. I'll say that, you know, serverless is often sort of touted as like, oh, well, you know, just throw serverless at it, you know?
00:20:17
Speaker
Again, we're a proponent of using the right tool for the right job. I think we talked about that many times on this podcast, especially in the context of Kubernetes or even in the Wasm episode, I believe. So it really begs to do the research of like, okay, here's what serverless is. Here's what it's supposed to give you. Does it fit what I'm trying to do? And hopefully we'll have some guests on that can talk to that exact point. Wink, wink, nudge, nudge.
00:20:44
Speaker
And so forth. So we have a few drawbacks here. I can kick it off with the first one, which is the concept of sort of cold and warm start. There's a lot of articles out there that talk about cold starts, warm starts.
00:21:02
Speaker
how serverless has this inherent problem of cold start. There's people out there that say, hey, let's stop talking about this because it's just like it's an edge case and it's used as a scapegoat for why not to use serverless. Then there's a lot of people out there that like it's mostly been solved, etc. But the idea as a cold start is in serverless, you don't own your servers, you don't own your compute. So you're not putting your application out there and it's not sitting there waiting.
00:21:31
Speaker
for a request to come in for it to be used all the time, necessarily. After a certain point in time, there's an initial spin-up time. If that serverless endpoint hasn't been used for a while, it may go dormant because it's essentially rented space. So if it is going to be used after some time, it has to become ready and then be used. So there's a lag basically.
00:21:58
Speaker
And certain platforms are better at instantiating than others. I'm sure there's differences between those platforms. But the idea of a cold start is that's what you're getting to. You know, that's a potential issue versus a warm start is the things ready for use and it will respond right away. Yeah. And so put this in terms of like.
00:22:17
Speaker
people that are using virtual machines today, right? Like your VM is always running and that's why whenever a request comes in, you have your application ready to respond. But then let's say if you had cold start problems in VMs, obviously VMs have a much larger footprint, but the time it takes for the VMs to actually boot up and then the application to be up and running is the cold start issue. That's the timeline that like you need your underlying
00:22:41
Speaker
orchestration layer to make sure that your application is ready to serve code and that's the same issue. It completely translates to how serverless works or how functions work.
00:22:50
Speaker
Yeah. And, and I think it's, it's become more of a prominent topic in service because you never have the ability to just always run the thing. I mean, there, there is, you know, a certain level of, um, you know, expectation for how that thing is available depending on the platform, but you never really own it yourself outright. So, um, you know, I think again, this just comes back to what are you trying to accomplish? Right.
00:23:15
Speaker
Cold Start may be really beneficial to you because you can scale down to zero. If your primary objective is potentially saving costs and cutting costs and using serverless in a very specific way, maybe Cold Start is something you just deal with. You monitor, you figure out which ones are the problems. But if you're going into it blind and then this happens to you, I think figure it out again, understanding that it's a thing.
00:23:45
Speaker
I think talking about use cases, right? I think, let's use an example. So like, let's say I'm building a website which has the ability for you to upload your own photos and we'll give you different resolutions. So your website front end shouldn't be serverless because if your website is taking too much time to load, your users might go to a different site or churn away from your application. But you don't need the backend components where once you upload that file, what happens is always online. So once a user accesses your application,
00:24:15
Speaker
uploads that image, that's when your function or your serverless function can be online, take that, and they'll do all the backend processing. So that can be a possibility where you save on costs. I think one of the examples that I really loved in the early days, I think this was like four or five years back. I don't know if you remember this, Ryan, but A Cloud Guru, right? There used to be a training platform for AWS users. So I did all three of my AWS certs through A Cloud Guru.
00:24:40
Speaker
They were a completely serverless hosting platform, so obviously they didn't need their videos and the media players to be available all the time. Only when you log in and you hit play, you select a course and you hit play, that's when they spun up resources. It was a really interesting use case because they showed that they can save like five or six million dollars a year just by using serverless functions instead of having EC2 instances be ready all the time for that use case.
00:25:04
Speaker
Yeah, and I think web is definitely where you see this a lot. Maybe e-commerce, right? Those kind of things where, you know, there's certain, you know, sort of widgets or functions. When I say widget, I feel like an old person. No offense. You use the word widget a lot. But I like cringed a little bit when I said it. Anyway, you know, there's certain things that aren't used a lot on your website. Those are perfect examples. I think I see a lot of
00:25:32
Speaker
of the examples in web most of the time. The other thing is, obviously, you don't own a lot of the infrastructure, any of the infrastructure, so to speak. So you are giving your trust to that vendor who owns the security of all those things, which, again, isn't always a bad thing. I think it depends on who you are. I feel like we come to this middle ground a lot.
00:25:55
Speaker
on the show, but maybe just because there are a lot of use cases. It depends if you're working in the government or if you're working for a bank or if you're a startup. There's obviously different requirements that you have, one of which you may be more or less comfortable handing over things like security or having the amount of input, those kinds of things.
00:26:16
Speaker
But if you are just another organization that's using the cloud and using serverless, I think AWS has many more resources and smart people working for them and they can spend that money to make sure the layers are secure. Then you with a team of three people can. So like shared security model obviously helps organizations who don't want to take in all of that additional work on their own.
00:26:38
Speaker
Exactly. So, I mean, I hopefully, I think we've done enough from a high level to kind of explain what Surrealis is, what it isn't. And now we're going to dive, I think, into some of the offerings.
00:26:54
Speaker
When we get into some of the offerings, maybe we'll tease out some details about how that specific thing is used versus others. And we are going to talk, like I said, about Knative serverless versus functions as a service. You've heard us say function a lot because it does serve us up that way. So we will touch on that. So yeah, let's kick it off. Why don't you go into some of the offerings that you put on here?
00:27:16
Speaker
Yeah, for sure. So let's start by talking about I think one of the most popular ones, right? AWS Lambda. So Lambda has been around for a while. And I know people who have attended like green, when AWS summits that are local to specific cities, you will have seen the serverless espresso demonstration where whenever you ask for a coffee, like
00:27:34
Speaker
Executes a bunch of functions and then eventually tells the barista what to make again. It's just a cool demo But lambda has been around for a while. They do have that cold start problem where whenever Your function is triggered. It has to create a new execution environment. It has to download the code or your image and
00:27:52
Speaker
and then have the runtime instantiated and then make sure that the initialization code is all ready to go. And after that is when your function is actually run on the execution environment. So there is an environment lifecycle and there is a cold start piece of it. But again, I don't want to talk about all the different benefits of Lambda. We kind of covered that with serverless.
00:28:15
Speaker
But I just wanted to cover a couple of new things that were announced recently. So if you had learned about Lambda a while back and you were just like, still cold start, I don't want to use it right now because it doesn't fit my use case. There are a couple of things that can change your mind. So one of those things is SnapStart. So it's called Lambda SnapStart. I don't know why I'm having issues saying that word, but
00:28:37
Speaker
What it does is whenever your serverless function gets triggered for the first time, it goes through that cold start process and then it takes a snapshot of that execution environment. So it's ready for that warm start. So whenever a second request comes in, it checks whether there is an already running execution environment.
00:28:58
Speaker
If it's not, it sees whether there is a snapshot that's available of that execution environment. And if there is, it basically goes ahead and starts running your code. So it really reduces the amount of time from a couple of seconds to like less than 100 milliseconds is what AWS says with the snap start functionality. And then if, again, there isn't a snapshot available, it will take a snapshot and then go through the cold start process again. But if you, so let's say,
00:29:25
Speaker
you had 20 different streams of requests coming in for your function. The first one, without Snapstart functionality, all of those will go through the cold start issue, like it will go through the same steps. But with Snapstart, only the first one will, and then for everything else, you'll have a snapshot of your execution environment ready to go, and you can start running your code. So it drastically reduces the amount of time that is spent on spinning things up.
00:29:49
Speaker
That was one of the things that was announced. I think the second thing was around a Lambda function URL. So what this means is usually if you're doing Lambda, you have to have different AWS components. Like maybe you have an API gateway resource running in front of your Lambda function. So the API gateway is able to take in requests, but
00:30:08
Speaker
Now with Lambda function URLs, what you can do is if you have HTTP requests, there is a standard URL that AWS has created that you can use to access your functions. So if I go through my nodes in
00:30:23
Speaker
specific time, okay, which I can't in time. Okay, yeah, it takes the URL ID, a lambda dash URL, the region ID on AWS. So it basically gives you an endpoint directly to your function. So you don't need to create and pay for those additional AWS resources just to handle HTTP requests. So again, that's a cool feature, a cool way to make sure that the infrastructure provisioning is even less than what it used to be before. So those were a couple of things that I wanted to highlight when it comes to AWS Lambda.
00:30:53
Speaker
Yeah, absolutely. And then I think the next thing I want to talk about was moving cloud providers, moving to Microsoft Azure. Again, Azure has a similar capability like Lambda with Azure Functions and Google Cloud has the same thing with Google Cloud Functions. But I wanted to start moving the discussion a bit to Kubernetes and how, again, this is Kubernetes by its right. So let's see how we can get some of these serverless capabilities where I'm not responsible for the infrastructure.
00:31:22
Speaker
or in the case of Kubernetes, I don't want to write those manifest files or those YAML files. Can I just take my container image and have you run for that for me? That's what I think the first step for Azure Container Apps is. Again, it runs on AKS as that Kubernetes layer, but you are not responsible for creating that AKS cluster. You're not responsible for defining your deployment manifests and your service manifests and your storage manifests. You just give Azure Container Apps the container image that you want to run.
00:31:50
Speaker
and it dynamically creates the entire infrastructure for you. But again, going over the point of control that Ryan mentioned in the previous section where you won't have access to the AKS cluster, you won't be able to modify the service manifests, you won't be able to modify your deployment manifests. It's just taking your simple container image and then running it on a Kubernetes cluster. So you're still getting the benefits of Kubernetes, the scaling capabilities based on KDA, the event-driven capabilities, all of those still exist.
00:32:15
Speaker
but you're not responsible for managing and maintaining the Kubernetes clusters. I think that's one of the things that sets apart the Azure service, right? Like you're getting the serverless functionality, but you are able to bring in those same container images you had to run on Kubernetes without actually managing the Kubernetes layer.
00:32:32
Speaker
And this is just one step. I don't think Microsoft recommends you to use this for all of your applications. This is just if you're getting started or if you have a really small code, a really small application that doesn't warrant its own Kubernetes cluster. You can use this function or use this feature and provide that capability to your end user.
00:32:50
Speaker
Absolutely, and something is interesting about Azure and sort of their serverless concepts is, you know, as we mentioned before, it really just means that you're not caring about the physical server and Azure actually has the concept of a serverless tier for databases.
00:33:06
Speaker
as well. So if you look at Azure SQL Database Serverless, it almost feels like it conflicts with a lot of what we've already been talking about, but the idea is still the same. You're not managing the compute and your database only runs when it needs to. So again, this means that the database will actually pause itself when it's dormant or not being used and warm back up when it's
00:33:33
Speaker
kind of instantiated, you know, an event comes in that it needs to be accessed. Again, they call this, right, the serverless compute tier. It's really just another compute tier that you don't have to manage the compute. That's all that serverless is just thrown in there. Again, this is one of the examples I just kind of hate the idea of, like, because first I hear serverless database, I'm like, oh, how would that work, you know?
00:34:03
Speaker
And they do kind of go into a scenario. If you look it up, they go into scenarios that's suited for this, meaning if it's unpredictable usage and it has periods of inactivity.
00:34:15
Speaker
then it might be fine for this database to be like, oh, it needs to wake up and I'll serve you whatever data you need and then go back to sleep. And then that actually is a good use case for saving the cost instead of just the alternative, which is you boot up the SQL server. It runs all the time. And yeah, maybe it's only accessed on Wednesday nights every three months. I don't know. But the idea is that wouldn't make sense. So again, this is another example of the term serverless.
00:34:44
Speaker
use in a different way than say what we're talking about and sort of smaller sort of functions and stuff that we're about. And I'm pretty sure or even more or even compared to something like land. And I think I'm pretty sure we'll get a lot of comments on this. Like AWS has the same thing like AWS database managed services has that serverless tier where I think they call it Fargate or something like that. But yeah, we have I haven't done much research into the database serverless
00:35:11
Speaker
things. This episode is just focused on the compute layer, I guess. But yeah, there are similar offerings from cloud vendors. Yeah, I mean, it's the same. I mean, we talk about databases on Kubernetes a lot here on this podcast. It's like if you ran a database on Kubernetes and you scaled it down when it wasn't being used and you had some kind of API gateway that when you got a request and the database wasn't up, you'd spin it up, you'd serve the request and then you'd spin it back down. Like it's really all attacking. Give a native section.
00:35:40
Speaker
Anyway, yeah. But again, just be aware of that when you see the term serverless because it is super loaded. And, you know, one of the 10 reasons that Ryan doesn't like it. And, okay, so moving on, you know, I have Cloud Fill Air in here, right? It's a slightly different use case. Cloud Fill Air is another managed offering, you know, not as huge as, say, like an AWS, but you can, you know, install an application and then basically say publish this thing.
00:36:09
Speaker
and it will go ahead and publish it on a worker and it will run your code on those Cloudflare workers. They claim no cold starts because of how they're built on a CDN and things like that. I have no experience, so I'm not going to comment on it. But again, this thing that comes up a lot, you'll see a lot, is that term of cold starts because it is, I think, maybe a fear when people get into it. There's a whole bunch of other serverless platforms and offerings
00:36:39
Speaker
out there. Netlify also has the Netlify functions. And so if you're into that, we'll put a link to Netlify functions and is a good segue into the difference between serverless and functions as a service.
00:36:56
Speaker
Do you want to kick us off? No, go for it. I'm confused with this. I'll appreciate your breakdown. Yeah, absolutely. So serverless, again, is the overarching term. And I want to say serverless functions is sort of a subset of how we think about serverless. Again, serverless just basically means you don't own or care about managing the server, again, as we see with serverless compute tiers with databases and things like that.
00:37:29
Speaker
That's still an entire application. It's still an entire database. It's just managing when the thing is active and kind of serving that thing. Serverless functions takes that a step further. It's a further abstraction of a serverless compute as a whole application versus running part of an application. Think about, like you said before,
00:37:55
Speaker
a web application that maybe serves images or stores images or lets you search for those images. This is basically taking an application and only part of the application and deploying that thing as a function.
00:38:13
Speaker
If you're ever written code, you're writing in Python or Go, there's the concept of a function. You're writing some piece of code that does some particular task very well. You can actually put that into a functions as a service platform and have that sit behind the API call. Basically, you could have a whole bunch of things that make up
00:38:38
Speaker
sort of the guts of what an application does, have them behind APIs, and just have a very lightweight shim of a front end that kind of calls these things. So your functions are quite literally sitting back there. But in concept, there are little tiny pieces of compute that still run somewhere. And so they're functions as a service. So it's a, I would say, a subset of serverless because it is a
00:39:02
Speaker
a piece of compute running on a non-managed piece of hardware compute server somewhere. So it is a serverless. So you'll often hear the term serverless functions or functions as a service. And really, that's what it means, is that you're breaking up an application
00:39:19
Speaker
It's a newer concept than even serverless, I would say. It's really aimed at more of the developer than it is a bigger term at the CIO level. Lambda actually is a good example. Lambda is used as a function platform. Azure has the Azure Functions specifically, Google Cloud Functions, Oracle Cloud Functions. They're all out there.
00:39:47
Speaker
It's basically application logic, but it's all stateless compute in the sense that it runs somewhere else. It's more in line with event-driven architecture. We had this on here as a bullet point.
00:40:01
Speaker
But basically, event-driven means that say you have a message queue or an HTTP gateway and you get a request or a message comes in over your queue, that is an event that you can react to and that event can trigger a function.
00:40:16
Speaker
And it goes runs, it goes and calls like some function as a service somewhere. So hopefully that, I mean, that's how I understand it. And hopefully that makes sense for you. It does. Right. I think when you were talking about how the application is broken down, I was able to draw parallels to how we talk about microservices applications running on Kubernetes. Right.
00:40:35
Speaker
Like you have a part which does one thing and one thing really well. And then you have the service objects that allow you to talk between these things. But then in case of Kubernetes, you always have these things running. Whereas I think when you break it down into functions, you have these in front of and behind an API gate, when only triggered when it's needed, when that component of that application was needed. So thank you.
00:40:56
Speaker
Yeah, absolutely. I would say in the ADBS realm of things, the Fargate maps better to the general server list term, and ABS Lambda maps better to the functions term. It's not exact, but I would say if you are familiar with the ABS environment, then you're probably more familiar with it.
00:41:17
Speaker
Yeah, anyway, so there's a whole bunch of links, show notes that we'll put there. And let's move on to Knative. Yeah, and Knative has been around for a while, right? Like, again, we heard Knative in like 2017, 18, I think, if I remember correctly, but I looked at Knative when it was still starting up. And okay, that's another option of running serverless. I thought my understanding at that point was it's just
00:41:45
Speaker
people trying to use buzzwords and come up with a new open source project. But spending the last week learning more about the Knative project that they have made a lot of progress actually helping you run serverless on Kubernetes. I think one of the things that you asked at the beginning of this podcast, like
00:42:01
Speaker
the vendor lock-in component where you are tied to a specific vendor and we hinted at Knative, but Knative basically allows you to run your own serverless platform for your applications on any Kubernetes cluster. So it provides you that same level of orchestration capabilities that you need to make your application vendor agnostic or cloud agnostic.
00:42:21
Speaker
I think looking at the Knative project itself, it is broken down into three major themes, themes being serving, eventing, and functions. Again, I think from the research that I've done, I'll spend a lot of time on serving and then maybe a bit on eventing as well. But Knative serving is that serverless capability, right? So it defines what is it, like Knative serving.
00:42:43
Speaker
defines a set of objects or custom resources or CRDs or custom resource definitions that are used to define and control how serverless workloads behave on your Kubernetes cluster. Again, this can run on any Kubernetes cluster as we have killed that point to death, but it provides the same set of CRDs and resources for you to use. The first one being a service CRD.
00:43:04
Speaker
This is not Kubernetes service. It's just a K-native service, which basically manages the whole lifecycle of your workload. So you define a K-native service and as part of that definition, you specify things like configuration and revisions. So configuration is your application configuration and revisions
00:43:24
Speaker
I help you define different levels or different versions of your application itself so you might have we want to be three and then you have a different crd color out and using the serving crd you can specify okay. You can do traffic routing traffic traffic routing you can define in the route object that twenty percent of my traffic needs to go to revision v one.
00:43:48
Speaker
50% goes through version V2, and then the remaining 30% has to go to V3. And again, revisions are immutable, right? So if you make any change to a revision through the configuration object,
00:43:59
Speaker
That creates a new revision. So revisions actually give you that immutable thing. So you can test out very specific versions of your code, see how users react to it. And then if you wanted to kill a revision, you can and then just promote or send all your traffic to a specific revision. All of that is built in. All of these capabilities are built in and can be defined with a simple YAML file.
00:44:20
Speaker
So if you break down the architecture, right? So you have a Knative service. Okay. I'm going to use my hands a lot. So if you're listening to the audio podcast, maybe check out our YouTube channel. Shameless plug, right? But you have a Knative service box at the top. And then let's imagine there is a route box on the left side of it. And then the configuration box on the right side of it.
00:44:40
Speaker
below the Knative service. So you have user traffic coming in from the route section. And then as we said, the route helps you to find traffic routing. So you can send specific versions of revisions, specific traffic to specific revision versions. And then each revision, and this is how it ties back to Kubernetes resources, each revision deploys a Kubernetes deployment object.
00:45:02
Speaker
So now I guess we are in the familiar territory of Kubernetes deployments and Kubernetes objects. So when you have a revision, it points to a deployment, and that's how you can have multiple revisions mapping to multiple deployments. And each deployment will have application pods. Again, this will depend on how much traffic you are expecting. So it can scale to zero, or it can scale to say 100 if you have a lot of traffic coming in for that specific revision.
00:45:25
Speaker
Since it's a deployment, it should have a pod. So you will have a pod with a couple of containers. One container is your user application, your application code. And then they have a special component called QProxy, which is basically responsible for keeping a track of the number of requests that are coming into the pod. So it basically works with another Knative component called the autoscaler.
00:45:46
Speaker
So auto scaler is responsible for looking at this Q proxy metrics. And if it's reaching a point where it needs to spin up more parts in that deployment object or increase that replica account for that deployment object, it will automatically do that. So without the user getting involved, the auto scaler component looks at those metrics, spins up more parts if you're getting new traffic. So that's how the auto scaling component works in Knative. But I think one of the things that we discussed when we were talking about serverless is the actual scale to zero.
00:46:15
Speaker
So let's talk about how scale to zero works. So scale to zero is actually achieved because like, okay, if you don't have any requests and your application is like you don't have any pods or deployment objects, how does auto scaler know when to spin it up? Because right now auto scaler only looks at the metrics already in your pod to scale up replicas.
00:46:34
Speaker
So they can it has another component called activator activator what does it do it looks for so if you don't have any deployment objects activator is the object that looks at those revisions and then if a specific revision is getting a lot of requests it will talk to water scalar and ask it to spin up those deployment objects that spin up those application parts of this is how everything works a scale to zero application can be dead.
00:46:57
Speaker
If you get new user requests in, Activator knows about that. It spins up at least one instance, and then from that point on, Autoscaler takes it over and then maybe spins it up to, let's say, 50 pods for the amount of traffic that you're expecting. So hopefully that helped kind of show the flow that Knative uses to serve user traffic between different versions using the configuration object and the scale to zero and scale to many capabilities. Hopefully that helped.
00:47:25
Speaker
Yeah, it does. And I think like if you're interested in what Knative and sort of Kata can do and help you accomplish, it is quite an interesting topic and does connect with the overall serverless topic because, you know, even if you aren't sort of using it by itself, you can connect it to other services that we mentioned, like Azure Functions and whatnot. So.
00:47:48
Speaker
Yeah, I think I do need to talk about, do want to talk about Keira a bit as well. I know we covered Knative. So Keira, I think this is, this is one of my, I guess, to use a weird term, but this is one of the beefs that I have with Keira, is even the Cuban talk that Keira did was named serverless.
00:48:05
Speaker
And it doesn't have anything to do with serverless. KDA stands for Kubernetes event-driven auto scaling. So Ryan, the example that you use, right? Like you're monitoring your messaging queue for events or HTTP queues. KDA does that for auto scaling decisions. So how is it different than HPA or horizontal pod auto scaling? Horizontal pod auto scaling just looks at your CPU and memory metrics. So how much?
00:48:31
Speaker
CPU and memory your part is actually consuming if they're getting getting overworked it will go ahead and spin up those additional parts that are needed this is how it works but kada the communities event driven auto scaling project actually runs on a community cluster it uses hpa as that orchestration there but it allows you to log into different.
00:48:49
Speaker
sources so it will monitor let's say your MongoDB cluster or it will monitor your Kafka cluster it will look for those events and entries being generated and to anticipate the traffic coming in it will go ahead and spin up those parts so it helps you scale not just on your CPU and memory and what's happening right now
00:49:06
Speaker
but it can help you anticipate additional user traffic and spin up your application components that are application pods. And I think looking at their documentation, they support like 55 different event sources, including things like Prometheus and RabbitMQ and SQS or PostgreSQL. Again, a lot of things, but yeah, it basically looks at things that are happening and then gets your pod ready to accept that user traffic. They do have a couple of different objects. One is called scaled object. Again, these are the resources that will spin up your pods and spin up more resources.
00:49:35
Speaker
But i want to talk about skilled job so if you have long running. Long running jobs right if you have a task in your application that takes an hour. If the metric is gone like the number of new requests that are coming in is gone i might also scale down your application you want to obviously want that when your application is actively processing some data.
00:49:56
Speaker
That's where a scaled job comes in, and it helps you handle those long-running executions. So if you have, instead of spinning up pods, it spins up Kubernetes jobs, and then it helps you handle those long-running executions. So that's a quick pitch on KEDA, and hopefully that is a project that you can check out and helps you in your day-to-day life. But yeah, those were the serverless projects that I think we wanted to discuss today.
00:50:20
Speaker
Yeah, absolutely. I mean, those are some really good projects and I think, can you use them together or are they sort of like competing a little bit in terms of what their purposes are?
00:50:31
Speaker
I think competing, right? Because even Knative has its own autoscaler called KPA, Knative Pod Autoscaler, not Horizontal Pod Autoscaler. So HPA, Horizontal Pod Autoscaler, can never really scale down to zero. It will always be one and more. KPA and KDA can actually help you scale down to zero and then use the HPA mechanism to scale your application up when needed.
00:50:53
Speaker
Got it, got it. Very cool, very cool stuff. And of course, we will link to both of those projects, and if you're interested in how to get started, as well as everything else we talked about. This brings us to the part in our show where we ask Chat and GPT a question about the topic we're talking about today. And today, we asked a question. We asked Chat and GPT to come up with a question, and then we asked it to answer itself. And the question was, if serverless computing were a superhero,
00:51:22
Speaker
Who would it be and why? You may have your own answer to this. But what chat GPT says is serverless computing or a superhero, it would be the flash. Just as the flash can move at incredible speeds without wasting energy, serverless computing allows developers to run their code without having to worry about managing and scaling servers, thus minimizing the energy consumption and cost.
00:51:47
Speaker
Additionally, the Flash's superhuman speed enables him to quickly respond to any situation, just as serverless computing can instantly respond to changes in demand and scale resources accordingly. Overall, both serverless computing and the Flash have a similar goal, to quickly and efficiently get the job done with minimal waste.
00:52:06
Speaker
I think I have an answer for this. Like when we were doing the podcast, I was like, okay, if it was a superhero, I think it will be Superman or maybe for that reason, like any other superhero. But the reason I say that as soon as there is a crime happening, Superman actually has to go inside a phone booth, get changed and then respond to the crime. It's got a cold start problem.
00:52:31
Speaker
nice. Nice. Unless he's already dressed. Yeah, he's already going down. I want start ready to go on a snapshot. Sitting down and doing the Clark Kent work, then yeah, maybe it needs some cold start time.
00:52:46
Speaker
Awesome, awesome. Oh man, all right. Well, hopefully this was a useful episode to you. I know Bhavan and I had fun putting it together and sort of digging into some of the topics and technologies that we often hear and the buzzword serverless in general. Hopefully it clears things up, gives you some things to look into. If you have feedback or questions, please reach out to us in any form that you feel is necessary, DM, email, anything like that.
00:53:13
Speaker
And we have some cool guests coming up in the show in the future. But yeah, that brings us to the end of today's episode. I'm Ryan. Thanks for listening to another episode of Kubernetes Bites. Thank you for listening to the Kubernetes Bites podcast.