Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Kubernetes in Cloud Native Healthcare
1.5k Plays1 year ago
In this episode of Kubernetes Bytes, Ryan and Bhavin sit down with Dan Arcari - a cloud and backend technologist at Zus Health to talk about how they use a combination of Kubernetes, ECS and Serverless to run their home grown and off-the-shelf applications. Dan was employee #1 and has been involved in building the whole stack for Zus health, so the discussion focuses on everything from HIPAA and SOC2 to how you should chose the right tool for the right job. Listen in to learn more about the practitioner approach to Kubernetes in the healthcare ecosystem.
Are you ready to earn extra income from sharing your expert opinion? Head over to userinterviews.com/hello to sign up and participate today!
Show Notes:
- https://community.cncf.io/events/details/cncf-cncf-online-programs-presents-cncf-on-demand-webinar-kubernetes-in-2023/
- https://kubernetes.io/blog/2023/03/10/image-registry-redirect/
- https://learn.microsoft.com/en-us/azure/aks/use-pod-sandboxing
- https://www.meetup.com/data-on-kubernetes-community/
- https://cast.ai/press-release/cast-ai-receives-20m-in-new-funding-led-by-early-stage-vc-creandum/
- https://siliconangle.com/2023/03/07/kubernetes-traffic-replication-startup-speedscale-raises-9m
- https://kubernetes.io/blog/2023/03/17/upcoming-changes-in-kubernetes-v1-27/
- https://www.infoq.com/news/2023/03/aks-edge-essentials-ga
Dan Arcari - https://www.linkedin.com/in/dan-arcari-45a90314/
Recommended
Transcript
Introduction to Kubernetes Bites Podcast
00:00:03
Speaker
You are listening to Kubernetes Bites, a podcast bringing you the latest from the world of cloud native data management. My name is Ryan Walner and I'm joined by Bob and Shaw coming to you from Boston, Massachusetts. We'll be sharing our thoughts on recent cloud native news and talking to industry experts about their experiences and challenges managing the wealth of data in today's cloud native ecosystem.
Seasonal Greetings and Boston Weather
00:00:30
Speaker
Good morning, good afternoon, and good evening. Wherever you are, we're coming to you from Boston, Massachusetts. Today is March 20th, 2023. Hope everyone is doing well and staying safe. Let's dive into it. Speaking of March 20th, happy spring. Happy spring. The weather is finally turning around. Last week was nor'easter here, or depending on where you were in Massachusetts, it was rain. 30 inches of snow.
00:00:56
Speaker
Could've been either way, but yeah. What's up, Boston? What's up, Bobbin? I called you Boston. What's up, Boston? What's up? No, Boston has been good, like, as you said.
00:01:08
Speaker
First day of spring, temperature's looking better. And then we had St. Patty's Day weekend last weekend. So again, I didn't do the traditional parade thing. I think I was just lazy enough to motivate myself to go there or even convince my friends to join me. So I think we ended up going in the other direction, like finding a brewery in Wuburn and then drinking there. I feel like if you don't go to the parade, you might as well go the opposite direction than anybody else.
00:01:32
Speaker
Not that I've been there either, which I feel like is a bit of a Bostonian sin a little bit. Maybe, maybe it just maybe we're just not parade people. Probably an implant. I don't know. Yeah, same bad as day is a good time. We mostly hang out with a couple of neighbor friends of ours, family friends, both Friday and Saturday. It was a lot of fun. We just kind of
00:01:56
Speaker
enjoyed some company and staying put. I was in New York past few weekends and things like that. So it's always nice to stay put and hang out a bit.
Family Story: Snowboarding Lessons
00:02:05
Speaker
Did a bit of snowboarding on Sunday. Well, I can't say that. I took my daughter for a snowboarding lesson. Oh, nice. I watched who sit, which was awesome to see a little four and a half year old, you know, just ripping and shredding it up.
00:02:17
Speaker
for an hour and a half. She did good. I didn't have to go because I was more interested in watching her. I think you had shared like a picture of her snowboarding, like maybe the first time she had done it. Yeah, that was with that was with us, like just like running up the mountain with her and bring her down. But this was like an official lesson. We weren't with her, you know, big step up. She took a lift, aka the magic carpet for the first time and was
00:02:46
Speaker
head over heels about going up an automated magic carpet thing, which is highlight of her weekend, I'm sure. That's so cool, man. Four and a half years, I can't even imagine. I don't think I was doing anything so outdoorsy at four and a half years old. I was just playing in, so we have apartment buildings in Mumbai, so there was a playground. We were just hanging out there with friends. That's it. That's the extent of my craziness.
00:03:13
Speaker
I know. I figured, you know, if I started earlier and just kind of, you know, get her used to it and that kind of thing, I started when I was like 12. So and I really enjoy it. And so I have this like vision of just like, go on snowboarding with her. You know, we'll see if it gets there. She's got to like it at the end of the day. You know, I can only introduce it to her. At least that's the way we do things around here.
Guest Introduction: Dan Arkarion from Healthcare
00:03:32
Speaker
So.
00:03:32
Speaker
Anyway, it's overall a good weekend and we have a fun guest that's actually going to be a little bit of a change where we've never had someone, I don't think we've had someone from sort of the healthcare industry on the podcast yet.
00:03:48
Speaker
We're going to bring Dan Arkarion and we'll tell you more about him in a second, but we're really excited about that. Before we do that, let's jump into a bit of news. Bavan, why don't you kick us off? Yeah,
Kubernetes Startups and Funding Discussion
00:03:59
Speaker
sure. For news, I have a couple of funding rounds that I wanted to talk about. I know the startup market looks really slow and with the Silicon Valley bank,
00:04:10
Speaker
Yeah, that wasn't a fun weekend, that's for sure. I know, and everybody was in panic mode, so all the people, all of our friends in the startup ecosystem, man, I don't know how you guys survived that weekend, but I'm sure, I'm glad that we are at least past that first hurdle. Yeah, some might say we're not. We haven't seen the fallout, right? But I don't know.
00:04:33
Speaker
I wonder what it will do to the acquisition market, if anything, right? So maybe you'll know more than I do. No, this is just, I think the rounds that actually get announced publicly right now have been under the works for months, right? So this is just things that they decided to disclose, even in this environment. So a couple of Kubernetes focused rounds, cast.ai, they raised, so they are a cost management or cost optimization startup,
00:05:01
Speaker
that I've started using the term AI in the description, so they are powered by AI. This is the market for today's money with anything with AI, right? Yeah, if you're not into marketing at all, it doesn't have to make sense. Just put it in the name.
00:05:18
Speaker
Yeah, so they ended up raising or closing on a $20 million investment round. They didn't say that this was series B, but looking at the crunch base history, their seed was in, I think, 2020, when they founded the company.
00:05:34
Speaker
Series A was in 2021, and then this is Series B in 2023. I think they have raised a total of $38 million over these three rounds. But again, their value prop is similar to what we have done with CubeQuest. I know we had people from CubeQuest on the podcast where you can connect your communities, clusters, and then organizations can see recommendations and use these automation techniques to reduce the costs, reduce the amount of money that they're spending on these different cloud platforms.
00:06:04
Speaker
another player in the ecosystem and again, cost management and now with the AI buzzword is
Microsoft's AKS Edge Essentials
00:06:10
Speaker
a really hot space in 2023, right? So congratulations to everyone at Cast AI. And then the second startup that raised some money was SpeedScale.
00:06:20
Speaker
So I know they are a Community Spites alumni. I know we had the CTO Matt LeRae on the pod. They raised a new funding round, bringing their total funding or total money raised to $9 million, which includes the pre-seed and seed funding. So they're still in the really early stages trying to find product market fit and trying to solve a very specific use case for customers. So if you don't know what SpeedScale does, right, SpeedScale helps organizations test their apps.
00:06:46
Speaker
using a copy of the production data. So like taking that snapshot of all the load that your application will go through in production, taking a snapshot of it and making sure you do that, do run your apps through the same data or same scale before you actually push your apps to production. So this looks like a seed funding round of 6.8 million, which brings your total funding raise to $9 million. So congratulations, Matt and your co-founders. Congratulations, Matt. Yeah, that was, I feel like,
00:07:16
Speaker
That was early days with SpeedScale, but throwing it back there. Yeah. I know. I think when we had Matt on the pod, it was like six months after they had graduated from Y Combinator. That's right. And we actually got into touch with them from Container IQ. But yeah, again.
00:07:31
Speaker
New funding round is always good news, especially in this environment, so great work guys. And then the final news article that I had was more around that edge space. I know Ryan, you and I have found these articles over the past few episodes, but Microsoft enters this ecosystem with something called as AKS or Azure Community Service Edge Essentials.
00:07:50
Speaker
This is now a generally available service where they provide a cncf conforming communities and k3s distribution, which is fully supported and managed by microsoft so like you can you can like they provide you with the powershell scripts and commandlets to set up these remote communities clusters these can be single or multi node communities clusters and they're automatic that their deployment is fully automated and they can be done on any.
00:08:14
Speaker
Windows PC class device so it can be running Windows 10 Windows 11 IOT Enterprise Enterprise Pro any of these different flavors on a Windows PC class device and you can run these edge essentials Kubernetes clusters so that's something like if you're a developer if you have these edge devices which are not
00:08:35
Speaker
the ruggedized server hardware that you might have. If you just have some basic Azure devices like a laptop running in the back of a closet, you can now run Kubernetes on top of it and manage it through the Azure portal. Very cool. I do like what Azure is doing with their cloud services and really staying modern. I was just writing a blog post about
00:08:58
Speaker
sort of the WASM ecosystem. And I know we had a podcast about that more recently. And they have sort of a preview for
00:09:08
Speaker
running the RunWazzy node pulse, which was mentioned on our show. And just seeing all the things they're doing here, and actually one of the pieces of news that I have here was a preview for POD sandboxing, which is basically, and we did cover POD sandboxing, but I can say that terribly, POD sandboxing, say that 10 times fast, is actually CataCote, no, not CataCote, it's...
00:09:35
Speaker
Cata containers, thank you. Too many terms in this world. Yeah, so we covered that on the security. Just imagine being a consumer or you would have already seen all this stuff. Exactly, exactly. We did cover those sort of concepts in the security high-level episode and what Cata containers do for you and sort of isolate things and kind of
00:09:57
Speaker
use the kernel and things like that so you know just between that I think now is a preview with their AKS service and just seeing everything they're doing here it's really cool to see how they're staying sort of on top of it so that was definitely one of them I'll put the link in there for the pod sandboxing another one is
Data on Kubernetes Meetup Promotion
00:10:19
Speaker
The Data on Kubernetes community, we're huge fans. We've done some talks, participated in this community. If you're at KubeCon Amsterdam, instead of their typical Data on Kubernetes day, which you typically get as part of like the co
00:10:37
Speaker
events prior to KubeCon. They're doing a meetup instead on April 18th at one of the offices in sort of the district area of Amsterdam at one of a company who basically runs data on Kubernetes.
00:10:55
Speaker
We'll put the link in there, but basically, they'll cover everything about what do you okay is they'll have some community talks, which is a little different style of the day, there will be the day again in
00:11:10
Speaker
North America, Chicago, in the fall. So a little different, but we'll put the link to that meetup. Not a huge amount of space. I think it's in the 100, 120 people, which is actually pretty big for a meetup, always considered. So definitely go check that out if you're into the...
00:11:26
Speaker
sort of this community about data databases on Kubernetes that's an awesome community and really happy they're going to be over there in Amsterdam. No, like the one thing that surprised me was I saw on Twitter today, like just today that Bart is leaving the Data on Kubernetes community and I was like, man, the community is going to feel a lot different, like the amount of work that Bart has put in.
00:11:47
Speaker
Building and managing and making sure it's a fair and equal community man that that guy did some awesome work so thank you but if you're listening to this if you're not you should be come on but
00:11:57
Speaker
We'll reach out to him, get him on here and talk about his experience. It might be fun. But yeah, he's done a lot for that community, so I'm sure he'll be greatly missed. The other one we had here, if you haven't been paying attention to the Kubernetes community and ecosystem, or if you haven't gone to Kubernetes.io, if you're at all, you'll see a giant banner at top for the last few weeks.
Kubernetes Image Registry Migration
00:12:22
Speaker
And I know we've actually mentioned this on this show,
00:12:25
Speaker
quite a while ago in the episodes about the container registry kas.gcr.io is being basically phased out. And there was a lot of concern about what this was going to do, what they were going to replace it with. The latest news, this is what this update is covering, is that this kas.gcr.io will be redirected to registry.ks.io.
00:12:52
Speaker
And there's a whole bunch of information on what you need to know about image polls, what it does for that.
00:12:59
Speaker
whether you have things sort of blacklisted, whitelisted, DNS wise. Anyway, the point being is that this could cause a lot of pain and they're doing a lot in the community to try to make it less painful. This is one of the things, the redirect is supposed to be less painful. So go read about it. I was reading about it, right? Like the redirect is perfect, but just keep in mind, right? It is valid only from March 20th. Like today is when they'll actually switch over.
00:13:26
Speaker
I think they end the redirect on 3rd of April or something like that. So after that, the older registry is completely gone. So they are giving you kind of a grace period for you to make sure that you are moving everything that needs to be moved. So again, go to the link that Ryan will put in the show notes and we'll make sure that like, or you guys make sure that you have your basis covered. Yes, absolutely. It has the potential to be, um, yeah, pretty, pretty hurtful if you still use some of the old registry. So don't,
00:13:55
Speaker
Um, and go read it. So we'll put it in there and be the best advocates we can in this space. Um, the last one I have is, you know, we've had, uh, Brendan Burns on the show. Um, and he's, uh, I believe already done a CNCF webinar. Um, it's called Kubernetes in 2023. I love, I actually love how broad this topic is. Um, and they really keep it at a level, which, uh, relates really well. I do it with the folks from we works, which are also a fan of, and, um, they kind of dive into sort of.
00:14:24
Speaker
a lot of things in the ecosystem, the community, you know, what it means to do things, you know, GitOps-wise, because they're talking with WeWork. So definitely go check that out. I think it's a really good video.
00:14:37
Speaker
to go learn there. And I believe that is all my news and all your news. Yes. Let's dive
Healthcare Industry Insights with Dan Arkarion
00:14:47
Speaker
into our our guest then. So our guest is Dan Arkari. He works for Zeus Health, actually, local Bostonian at one point. Well, the company is is anyway, he travels here every now and then. So it's good to have someone local on the show. But he's sort of a cloud and backend technologist. But
00:15:05
Speaker
has done all sorts of things for Zeus and a lot in the healthcare ecosystem. So you'll learn more about that when he comes on the show. So without further ado, let's get Dan on the show.
00:15:20
Speaker
We would like to thank our very first sponsors for the Kubernetes Bytes podcast, User Interviews. You might be thinking, who or what is User Interviews? User Interviews connects researchers with high quality participants to provide real feedback on real products. Participants can earn some extra bucks for their feedback, which allows them to have an impact on how the products are actually being built.
00:15:45
Speaker
User interviews is free to sign up and most studies are less than an hour and pay over $60. Some studies pay several hundred dollars for a one-on-one interview. And as we all know, who doesn't want some extra beer money for the weekend? You can share your opinions with relevant companies who are building products like Adobe, Spotify, Amazon, and many more.
00:16:08
Speaker
Are you ready to earn some extra income for sharing your expert opinion? Head over to userinterviews.com slash hello to sign up and participate today. That's userinterviews.com slash hello and tell them the Kubernetes Bites podcast sent you by using the promo code kubernetesbites.
00:16:29
Speaker
Dan, welcome to the show. Welcome to Kubernetes Bites. Bob and I are happy to have you here. Give our listeners a little bit of an introduction of who you are and what you do. Yeah, thanks. It's great to be here. I appreciate the opportunity. So I'm Dan Arcari. I'm currently manager of a small security and infrastructure team at Zeus Health, a health care startup focused on the health care data interchange. We've been going about two years. I was, I think, employee number one.
00:16:58
Speaker
Darn close to it. I'd have to go check. Not a founder, but number one after that. Prior to that, I worked at Athena Health, a much larger player in the space. And then if you want to delve deep back into history, at one point I worked in banking and also on slot machine firmware, which could be its own whole office. I want to know if I have any ins with those slot machines now. You got to tell me which ones to go to. Everybody asks.
00:17:29
Speaker
He didn't say no, just this. Should we just drive to Everett? Like I know we have an Encore there, right?
00:17:40
Speaker
Nice. So I think then we really, as Ryan said, we're really excited to have you on the podcast to talk about that practitioner's approach of how you manage your infrastructure and how you use containers and Kubernetes maybe. So before we dive into what it looks like today, as you said, you are employee number one. So you were responsible for building this from the ground up. So can you talk about where you started? What did it look like?
00:18:07
Speaker
And then we'll talk about how we got to where we are right now.
Zeus Health's Deployment Strategy
00:18:11
Speaker
Sure. I think we had a pretty standard approach. AWS, we run entirely on AWS. I should add that. So there's no on-premise, there's no colo facilities. So it makes the footprint pretty small. There's a few other SaaS services, but primarily it's a AWS only deployment.
00:18:28
Speaker
And for the large part, this is like most applications, a bit passing setup, right? There's no hardware. There's no IOT. So starting off, we have a multi account approach. That's pretty typical these days. You run things through control tower and organizations. We could dive into that a little bit at some point. There's some interesting nuances there.
00:18:46
Speaker
And for the large part, we deploy services, or at least we did in the beginning on ECS with one team using EKS. And I think over the past two years, that's still largely the case. And we'll get into sort of tuning down the EKS a little bit in that story. Teams are also moving over heavily into Lambda and step functions, where I think those have really matured as what used to be maybe interesting options a couple of years ago to, why wouldn't we use that now? It seems to be almost the fault.
00:19:16
Speaker
Gotcha. No, that's interesting, right? Like you basically covered ECS, which used to be AWS version of container orchestration to EKS to serverless and step function. So like you basically have a breadth of all the different types of application deployment that one can have. So like what kind of applications do you run on top? Like are these native in grown applications built on containers following microservices principles or what does your application stack look like?
00:19:43
Speaker
Yeah, these are all microservices. So they're primarily homegrown, some combination of Node and Go. There's one licensed product, which is something that's provided as a container to us that we operate. That's Java-based. So that's going to stay on ECS for a while. You can't Lambdaify that or turn that into serverless. Yeah, the rest are homegrown.
00:20:07
Speaker
And then there was some other, the Kubernetes things also had some scheduled jobs and some batch processes that were running. Those have been migrated for scalability reasons over to, again, the serverless stack and Lambda. Not that you couldn't have done it in Kubernetes necessarily, but the team felt more comfortable writing those in Lambda. I think people were largely more familiar
00:20:31
Speaker
AWS native services, then diving into Kubernetes and then writing on it. So I think you do find this in some environments where if people are not already familiar with Kubernetes, it's at least a perceived lift to take that on in addition to the engineering work to actually build out the service.
00:20:50
Speaker
Some of that is possibly a hangover. If you go back, I don't know, seven years or so, seven or eight years when you had to operate a Kubernetes cluster and then run stuff on it, that's a lot of work. Operating and Ryan knows this well. Standing up a cluster, especially on on-prem, on your own metal, is a big task. Now, of course, you wouldn't do that in most cases today. You would use a cloud offering, you would just use EKS, which takes most of that load away. But even so, I think there's a perception of
00:21:17
Speaker
learning that control plane separately, that people feel comfortable in ECS. And so they stick with that or now, you know, moving over to serverless.
00:21:28
Speaker
Yeah. So that's a good point, right? I mean, we we've touched on several different things already and it's in the first five minutes, right? You have some ECS in there, which is, you know, you mentioned is sort of a product that's that's given to you. That's a really good reason to kind of keep things there. I know you mentioned there's some Kubernetes. And personally, I know we had a conversation in the past that, you know, some Kubernetes was there, but it didn't really fit the bill. And you've mentioned serverless a few times. Could you kind of dive into
00:21:57
Speaker
why you've moved towards serverless from Kubernetes. And I know there's some things that may still exist in Kubernetes and what fits well there, right? Like kind of like talk to that.
00:22:10
Speaker
Yeah, I'll say it's not just the Kubernetes tasks that have moved over to serverless, but other things too that might have been on ECS, that when we look at the kind of scalability we need, it's not clear that a container really offers us anything.
Benefits of Serverless Technology
00:22:24
Speaker
When you're starting from ground zero and you have no code, and you don't have to worry about what might have been there before, and maybe that was packaged as a container, or maybe you had all that code bundled in one place before, when you don't have any of that, when you're starting from scratch, these days, the sophistication of
00:22:39
Speaker
Lambda and step functions and related technologies does make you ask, why don't we just do that? You know, it's one of these. I like to have sort of background mantras when I think about running a software company or working in one and one of them is what are what are other people doing? Why are they doing it? And how's it going? And so whenever you approach these things these days, especially when you don't have some unique aspect to it, like we have a hardware transponder or there's a train involved somewhere, but it's just data.
00:23:08
Speaker
it does seem that serverless is one of the simpler things you can do because you get infinite scalability, basically. So that can displace not only just some
00:23:18
Speaker
Kubernetes workloads, but ECS and other things too, when you don't need to have it in containers. So I would say that if I thought I had to run a massively scaled container fleet at some company for whatever reason, I would probably stick with Kubernetes. I like the control plane. I like the separation of infrastructure from application operation that affords, and I would make a deliberate effort to ensure that people at the company are trained up on it if they're not. But when you don't have that,
00:23:47
Speaker
you know, challenge or operational requirement. In a lot of cases these days, I really do. It does seem like Lambda is, for a lot of cases, an easy way to go. So it's not really a Kubernetes issue. It's just, why don't we just do that, right? Okay. So it sounds...
00:24:02
Speaker
It sounds like then your team uses a combination of all, but then at some point you guys evaluated ECS versus EKS. Were there any specific findings that made you lean more towards one or the other? I know you said knowledge. Knowledge gap was definitely one point. People were more comfortable with ECS and EKS. Are there any other ones that you want to highlight?
00:24:25
Speaker
Yeah, I think you'll find that I have no data to support this. So, you know, I could be wrong and we should, we should look, but I would think on average people are more familiar with ECS or comfortable with the EKS unless they've operated with Kubernetes before ECS is AWS's own service. I think there might be a presumption that what should run cause it's what they wrote. Um, and it has some pluses and, and you know,
00:24:49
Speaker
gotcha areas too, or pitfalls. It's not perfect. But it's there by default, it's supported by cloud information, it's supported by Terraform, and it just works. So people think,
00:25:00
Speaker
Okay, we'll just use that. When you talk about some of the organizational decisions, so this is something that happens even at a 50-person startup all the way to a 6,000-person company. These questions of who's deciding what we're doing, right? Are we all going to deploy services in the same way? Are we going to invest in our own? These things always get code names like it's Hydra.
00:25:23
Speaker
It's technical. Everyone's going to deploy using this thing that we developed that always has a good name associated with it. But then you're investing time in that technology that then blocks people behind it. So you have to be careful about
00:25:36
Speaker
are you all actually trying to deploy things the same way? And so sometimes you just don't make a decision, which is what I think we let our teams do in the first place where we said, okay, your team is going to be responsible for operating your service. This is sort of the AWS model that they practice, obviously at much larger scale, but you get paid at two in the morning, you answer the support calls, you write the service, you do the bugs, you do the whole thing.
00:25:56
Speaker
and we let our teams do that in the beginning. You want to run on ECS? Okay. We're not here to tell you, we're here to run, you know, my team is here to deploy networking infrastructure, to run our SOC2 audit critically as a major function of the team, but we're not necessarily here to tell you how to run every aspect of your deployment. And left alone, most people gravitated toward ECS, and then a couple used EKS, and then now there's serverless and popular.
00:26:21
Speaker
Okay. That's, that's an interesting thought, I guess. Yeah. So like for the compliance checks, again, you said SOC 2, obviously you guys are in healthcare. So HIPAA might be a framework that you have to comply with. Using, it looks like using a native service like ECS does have some benefits or an open source based managed service that maybe Amazon offers. And maybe it's easier to use or run through all those compliance checks.
00:26:49
Speaker
I think you could satisfy, and HIPAA, if you actually read it, it's actually tougher in some ways to comply with than SOC 2, which is more prescriptive. HIPAA has a lot of general prescriptions about how to operate your company. SOC 2 is much more specific.
00:27:05
Speaker
I think you could be compliant with either. It's especially in SOC, you're primarily focused. There's a few critical things around data encryption and privacy. But in a lot of cases, what you're really focused on is operating in a manner that matches your policies. So there can't be sort of arbitrary process. If you say we're doing this, this is how we review our access to critical systems quarterly. Okay, you have to have that in the policy and you have to follow it and you have to show that you did that.
00:27:31
Speaker
And then you have 100 or so controls like that. So I think you could do it with either service really. I wouldn't be too worried about either ECS or EKS or anything else being deficient for an audit.
00:27:43
Speaker
Okay. No, thank you. Thank you for clarifying that because for some reason I was under that impression. So talking about serverless, right? Yeah, there was, I guess I could maybe add, there was one particularly difficult thing to resolve with AWS. This took until just last year to resolve. It was, was whether EKS on Fargate was on their HIPAA eligible, you know, and you go to the AWS HIPAA eligible list, which is now most of it, hundreds of services, used to be about 10.
00:28:09
Speaker
EKS Fargate was specifically excluded, whereas ECS Fargate was explicitly listed. And they never really had a good answer until we kind of worked through with their legal team. And eventually, whatever they had to prove out was validated, and now you can do that. That was a little bit of a reservation for a while. But that's legally cleared up now.
00:28:29
Speaker
Was that also, you know, I know you had mentioned that, you know, there's been a lot of changes in HIPAA and AWS recently as well, that being one of them, but also something about dedicated instances. So are you finding that, you know, AWS and cloud in general is really kind of, you know, moving sort of with healthcare now and kind of unblocking a lot of sort of things that you can use to accelerate what you want to be doing, you know, at Zeus and,
00:28:59
Speaker
It was definitely the case, this was 2017 or 2018. I'd have to check where you had to use dedicated instances if you were running HIPAA or healthcare stacks, PHI stacks in AWS. That was a requirement. I'd have to go check. It's 2017 or 2018. So four or five years ago.
00:29:17
Speaker
When that was removed as a restriction, that opened up things like API gateway and hundreds, you know, it really opened up their full suite of services that because that's all run on, you know, their ephemeral hardware behind the scenes. When they didn't have that dedicated requirement, you could really open it up. So now you see a lot of
00:29:37
Speaker
I'm not sure what the overall migration is. You might've seen more lift and shift before. Now you have, well, why don't you just build it on AWS to start with? You don't need a co-location center. You don't need to run this on your own stacks. Just develop it in AWS. But you know, it would be interesting to see data on, uh, on that sort of native development. Yeah. Uh, go ahead.
00:29:59
Speaker
No, I was going to switch gears to serverless. So if you want to add something to that, go for it. Okay. One last point, right before we move to serverless. I think it was interesting, like requiring dedicated instances doesn't really help with the whole value prop of cloud that it's elastic or it helps you save on cost because you're not doing a lot of things on your own or managing hardware. Dedicated infrastructure. I'm glad that AWS moved away from that and you guys were able to use shared infrastructure.
00:30:25
Speaker
Yeah, it would save on capacity planning for sure, which at a large scale is, you know, having worked in a place where there was both a large cloud footprint and on-prem, that was still a significant area of investment, right? Having to figure out how to account for that future traffic growth. So it does, it would have saved you that, but yeah, then you're having to purchase, you know, reserve capacity and savings plans and all these things that go along with, why don't we just not have it run when we don't need it to.
00:30:52
Speaker
You know, if it's a, you know, just, or even a scalable node group, right? Just, there's nothing happening at two in the morning. Just, okay, we don't need to. Yeah. What's more secure than something not running.
00:31:04
Speaker
That's a valid point, and a good segue, I feel like, to serverless as well, because the epitome of shared compute, that being services, and maybe that's a good place to start. What are the benefits of serverless for you? And I know you had mentioned, if you can run it there, that's the go-to these days for you. So yeah, let's start with the benefits. I think one is just conceptual.
00:31:34
Speaker
Often, in software, we think of it as a technical field, but if you really stop and think, ultimately, up in the top five most important things are communication and clarity, actual technical concerns are somewhere between 9th and 17th probably. When you think about code, you're really thinking about, well, who's going to have to read this later? Something like using lambdas or these tightly packaged
00:32:00
Speaker
groups of compute or compute modules, it lets you express what you're trying to do. What are we actually doing here, right? In a way where you don't get bound up, we have to build an image and then we have to have security for the Docker base layer. And we have to test the whole image and ship it and push it around. And we're going to run, you know, an API service on top of the thing in the container. Like those are all fine ways to develop an application, but you don't have to think about that. What's the code? What are you actually doing? And so just as a
00:32:27
Speaker
Because again, the people part of software development is so important and often the hardest part to get right. When you don't have to think about that stuff, just where's the lambda that does X? Oh, here it is. And it's 50 lines of code. And that's all there is other than some, maybe some terraform to deploy the whole thing, right? And all that. So I would say first and foremost is just that conceptual simplicity that you think about and reason about what you're doing.
00:32:50
Speaker
Oftentimes in a way that's easier than a container. Maybe not all the time. The second, of course, is just infinite scalability, right? When you don't even have to worry about auto scaling groups and scaling policies and all those things. You just say, hey, this will run, but it needs to.
00:33:05
Speaker
It won't otherwise. That's a pretty nice model and it works. There are some, you know, and you can go look for these curves of where cost-wise, depending on the amount of memory and time you're invoking, right? There are some, it's not that all use cases necessarily work in Lambda. But as long as you're under that curve, that's a very nice approach. So you have as much scalability as you need without really having to do a whole lot. That's a nice win.
00:33:32
Speaker
Yeah, but I think when we did our intro to serverless or serverless 101 episode last week or last couple of weeks, one of the things that keeps coming up is the cold start
Cold Start Challenges in Serverless
00:33:43
Speaker
problem, right? Like how do you handle that inside your organization?
00:33:47
Speaker
cold start for development practices or? No, for the Lambda functions to actually be deployed, like whenever they are invoked, like it takes some seconds to maybe a minute. I don't know what the time frame is for the actual application to be online. Is that okay in the use case that you have or how do you handle that?
00:34:03
Speaker
You know, that's not something that I've had, I've run into too much dealing with it directly. So one of the, one of the aspects of being a, you know, running audits means I don't get that right hands on code too much anymore. So I would, I would defer that to our dev teams and ask them what we're having to do to.
00:34:21
Speaker
you know, to account for that. So I don't, I have not heard of any major hurdles there, but it's, it could also be that I'm provisioning sock evidence and maybe I haven't run into it myself. That works for me. Yeah. I know when we were kind of digging into it, we, we sort of found people were on both sides of that fence where it was like, you know, that's a, that's something you just accept. And it doesn't even happen all the time. Right. It just happens when something
00:34:44
Speaker
isn't prepped and ready to go for some time, it's not used for a while, and it might be an issue versus not. So I feel like there's sort of a split of folks in the community out there, and those using serverless and are happy with it, I find are just like, yeah, it happens sometimes, but it's something you accept, right?
00:35:02
Speaker
You know, I do want to dig in. You mentioned the term step function. And we didn't we didn't actually cover that a lot last week. And I'd love to kind of dig into sort of conceptually where where lambda and step function and, you know, function as a service kind of all kind of mixed together. What's the difference? What's what's, you know, used for what?
00:35:28
Speaker
Yeah, stuff functions is just a way to compose an application made of many lambdas into a state machine, basically. So without getting into, you know, taking the whole thing apart, there's basically JSON or maybe YAML2 file that controls, okay, here's the lambdas that do things as part of the state machine. And here's the data that can be passed between them. And here's the reasons why the state would change. And you pay AWS every time the state change. And you pay AWS every time the lambda gets invoked.
00:35:54
Speaker
And so you can compose state machine-like applications together as one machine, basically, versus just going and deploying a whole bunch of Lambdas and then knowing.
00:36:09
Speaker
One thing to look out for in programming, surprise dependencies, or the code just happens to know that you have to call this thing without it being injected or discovered somehow. It lets you get around that problem a little bit by saying, well, these are composed as part of a holistic application, but they're just separate functions versus deploying, say, 50 separate lambdas that happen to know how to talk to each other. Then you have to keep a confluence document somewhere that boxes the lines on it, which then nobody looks at it after it was first made.
00:36:38
Speaker
That makes sense and I know I feel like we might have not mentioned on this show yet but I know some things were being rewritten for Lambda and I know in many cases you said some teams just you know they choose ECS they choose Kubernetes or EKS or they choose Lambda but in some cases it sounds like you're
00:36:57
Speaker
Either making things kind of work with Lambda or choosing them. What is sort of rewriting something for Lambda that necessarily wasn't there look like if maybe it's on EKS and your team wants to move it or a team wants to move it to Lambda?
00:37:14
Speaker
Yeah, so previously there was some batch processing done where data was being sourced from external partners, and you had to sort of poll periodically and figure out is there data there, right? And when there is, you pick it up and then process it, do what you have to do, and then move it along to the system. In this case, that can be streamlined by not having to worry about the size of data to be discovered from the other endpoint, just knowing that
00:37:39
Speaker
When it's picked up, so you still have a polling mechanism because we don't control the other end of it. But for however much data you get back from the partner, you can kick off one of these step-based pipelines to say, here, do what you need to do, and it will end up in the right place. And we don't need to worry about how big is some cluster or how much compute has to be allocated, as long as each of those little steps is done in the way where you don't take, you know,
00:38:01
Speaker
50 gigs of memory and enormous cost to do it, you break it down into dead sized chunks. It lets you just say, okay, we're just going to offload that and however many of them need to run are running and we don't have to worry about things like actually executing batch jobs ourselves to process the data. It will just run when it needs to.
00:38:21
Speaker
I think that's an interesting approach. We have spoken a lot about all the different technologies, right? But I wanted to talk about talent and how, when you hire them, is choosing something like serverless the right decision when it comes to onboarding new people?
Learning Curve for AWS Technologies
00:38:37
Speaker
Do you feel that in the industry right now, we have a lot of developers that already know how to work with these modern technologies? Or what does the ramp up time look like for people that are new to the team that have to work on these technologies?
00:38:51
Speaker
Yeah, that's an interesting question. This is casually from a fairly small set of data points. I would say that
00:38:59
Speaker
Because they're AWS primitives, there's less ramp up for Lambda than there actually is for many container services. And I would include ECS in that. ECS has a fair amount of nuance associated with it to get it right. If you want to do anything more than just, here's a container file, just go run it and we'll forget about it. If you really care about blue, green and scaling,
00:39:25
Speaker
There's a lot of hot updating things where you don't have down time. That's not necessarily dead simple in ECS either. I would say that serverless or at least Lambda, it's a big field, but let's at least say Lambda probably has a lower ramp up cost than operating containers more generally. Kubernetes, I think you tend to find people that have used it heavily in production before and they're experts on it and quite competent.
00:39:51
Speaker
or people that have heard about it and are interested, but don't have a lot of deep experience with it. And then you have to consider, you know, whenever you bring a tool into a company, it's how do we get everybody on board? We don't want to have this be with like three people that know about it. And you have to always go ask them if you can, let's level up everybody. We did that with Terraform. I think in the beginning, Terraform sometimes can be something where people, people like it, but they're just a little questions about how do we actually do this? How does it compare to cloud formation? You always want to try and level up
00:40:20
Speaker
get everybody rowing in the same direction. So, um, there's some nuance to serverless, but I think overall, once you, especially once you have a template to say, here, here's a way to deploy, you know, it bundles it into a zip file and you can go push it out there. Um, there's a learning curve, but I think it's, it's actually barely shallow. Um, interesting to measure though.
00:40:44
Speaker
Yeah, I know, like, again, the reason for my question is, like, we did an episode around platform engineering, and that was one of the tasks, right? Because recently, the reason larger organizations are building these internal development platforms is because they don't expect the developers to know every technology, that they give them options that's available. So I just wanted to know how it is when, when we're talking about all the different kinds of services, but still in that AWS ecosystem.
00:41:08
Speaker
Yeah, that's always a critical question. Something in economics, which is a hobby of mine, is the amount of the degree of specialization is based on the size of the market.
Managing Cloud Expenses
00:41:18
Speaker
If you have a mom and pop restaurant, grandma's in the back cooking everything, just making the sauce, baking the cookies, the whole thing. When you have
00:41:27
Speaker
a restaurant that's serving a couple hundred people a day or more, and you have 20 people working in the kitchen. There's one guy doing sauce. That's it. He makes the sauce. And you can see this in dev teams too. If you have a four person dev team at a startup, you can't go hire a DevOps person to just say, okay, you're going to go do all the build stuff and make sure they check on security. And you know, everybody kind of has to pitch in because you can't afford to have a quarter of your budget in one particular skill. Yeah.
00:41:56
Speaker
If you have this prototypical two-pizza team and you're eight to 12 people on a service, then you could, and your revenue is sufficient to support that. Sure. On each team, you can say, look, security is a cross-cutting concern. There is a centralized InfoSec team, but there's also going to be somebody who's maybe half their job on your team is to look out for security and someone else whose specialty is build systems and CI and deployments and so on. It really depends on the size of your organization.
00:42:23
Speaker
Yeah, I really like that analogy. And I know it's one that we've we've briefly touched on before. And along those lines, you know, we did, I think we talked about in the past, which I'd love to double click on here is just because I've heard you say just because you have the money doesn't mean you should. And, you know, that's somewhat related to so the size and the marketer to the specialization. But, you know, is there a lesson learned there that you've been through that maybe you can apply here that
00:42:54
Speaker
Yeah, you want to look at, I guess on some level, it's just basic financial accountability, but I've seen cases where there's a massive spend on cloud at the beginning of a project, including a third party licenses and all sorts of tooling and you will need some of it. But you want to be careful about, you know, have you spent a million dollars before any one penny has been earned? And I, you know, my
00:43:21
Speaker
sort of somewhat known for wacky analogies, but it's sort of like I'm going to deciding you're going to get into rock climbing and you go buy $2,000 of the best shoes and fancy ropes and a belayer and like before you set foot on the face, you should start with someone's leftover equipment and see if it's going to work.
00:43:36
Speaker
And the cloud can be that way too you can spend fifty thousand dollars on code scanning tools and you can spend twenty thousand dollars or more on your audit tools and you can spend another thirty thousand dollars on a container security that you can keep spending twenty to fifty thousand. GitHub will want you to buy their advanced security. Everybody has a plus version of whatever they're doing before you know it you're spending three hundred thousand dollars a year on.
00:43:57
Speaker
license upgrades and you have to at some point stop and think what can we get away with that's still secure that meets our requirements and then see if you really need it you always want to let your you know find the pole that touches the bottom of the lake right like where are we do we need to spend a million right away and with AWS you can actually you know I remember at Athena health we were actually I felt fairly early on and the the approach to saying we're going to have a lot of accounts
00:44:25
Speaker
possibly hundreds, if I recall correctly, each segmented into a specific purpose. And it felt like the tools actually evolved to support that. So I don't think we were the only ones deciding that. Things like Transit Gateway showed up, Control Tower, and organizations, and all these things that govern that federated account footprint. But then if you actually look under the covers, there's a cost to that because Control Tower will turn on Config for you by default and just start tracking in Config.
00:44:55
Speaker
if you don't pay attention to it, it tracks every resource change. So if you have your dev accounts and your test accounts, and then your production accounts, every single time you deploy a container, you're paying a small couple pabs of a penny for that. So you actually do want to be careful about even the assumed best architecture defaults in AWS, like config and control tower. They're great, but there's a cost to them, and you should be aware of that when you get started. And maybe you don't need 30 accounts right away, maybe two is fine. So something to be aware of.
00:45:25
Speaker
when you get started with AWS until revenue is enough to securely justify all those licenses. Yes, it's an interesting problem because I feel like on one hand, we often dive into these new architectures. I feel like we saw a lot of that spin happening when people were diving into cloud, but on top of that, it was Kubernetes and containers.
00:45:48
Speaker
And I often wonder, is it the shiny new toy and having enough money? Or is it we often work in this industry by solving pain points? And if you've been on the side of the pain point, you often want to not have that pain. But there's also some real benefit to your point about using used gears, experiencing a little bit of pain.
00:46:12
Speaker
to use a tool to solve a problem versus to get a tool in case you have a problem, right? Which is not really money at all. It's more of a mindset and I think maybe goes back directly to your kind of communication and organizational conversation as well. Yeah, I think we all know if you had a dollar for every time somebody said we're going to put an abstraction wrapper around the database in case we want to change out the database. You never will.
00:46:36
Speaker
You won't. And if you do, it's because your database requirements have probably changed sufficiently that you're going to redesign the whole application anyway. You're going to remove the thing from a relational setup too. You're not just going to switch from Oracle to Postgres, and that's going to be that simple. You will have done so as part of a broader change to move from relational to say something like Dynamo, right?
00:46:59
Speaker
It's back to that, we all know about the premature optimization being the root of all evil. I think that does largely hold true in computing. There's not always great first principles to stand on all the time. But if you can just wait and know that you need something, I do think you're most of the time in a better off place than pre-deciding that you have to have all
Licensing Costs and Needs Evaluation
00:47:18
Speaker
these things. And then you're in a weird unknown world of then you have to go continually check those to justify that or else they just stay there forever.
00:47:27
Speaker
Yeah, absolutely. Cool. Well, I mean, I think time flies. We're about a half an hour in and I want to make sure we have time for our fun news segment. And then we do want to, before we dive into that, if there's anywhere people can either get ahold of you or if you have anything to share on GitHub or a website or someone wants to just ask you a question, is there any place that you prefer people get in touch with you or anything like that?
00:47:55
Speaker
Sure, you can find me on LinkedIn, easily searchable. I also maintain a blog called Bite Wise, B-Y-T-E-W-I-S-E, which has very occasional comments. I do try to tie in those economic concepts into software engineering practices. So folks can check out Bite Wise. Otherwise, yeah, just find me on LinkedIn. And is Zeus hiring? Yeah.
00:48:19
Speaker
Zoo South currently, I don't believe there's any positions at the moment. We did just have a second successful financing round. And I think if you want to reach out to us, we're on LinkedIn or just find us at healthcare conferences. And I think if you find us, you know, again, on LinkedIn, our product people will let you know where to find us at healthcare conferences because they're always in attendance. Yep.
00:48:43
Speaker
Absolutely. All right. On to the fun segment, Dan. I know we had prepped you a little bit that we were going to ask you a question. And here's the question and then we can hand it over to you if you want to answer it or let chat GPT answer it for you. And the question is, if you had to explain cloud computing in healthcare to a five year old, how would you do it? And what kind of metaphor or analogy would you use?
00:49:08
Speaker
Well, having three kids, I'm pretty sure I have explained this before, but I'm actually curious on on chat GPT's answers. So can we can we see what it? Yeah, absolutely. Absolutely. And I like to ask that, too. Yeah, absolutely. You can compare it. All right. So this is this is how chat GPT would explain cloud computing in health care to a five year old. It starts like this. Have you ever played a game on your tablet or phone that saves your progress so you can pick up where you left off later?
00:49:37
Speaker
Well, imagine if doctors and nurses could do the same thing with their patient's health information. They could save it in a special place called the cloud, in quotes, as I'm putting it there, if you can't see my hands, which is like a giant invisible backpack that can hold lots of important things.
00:49:53
Speaker
That way, if a doctor in a different place needs to see the information, they can just go look up in the cloud instead of having to ask the patient or family to bring it in. It's like magic, and it helps doctors take better care of their patients.
00:50:09
Speaker
That's how chat GPT would go about it with a five-year-old. I dig that the backpack part, I feel like, was hitting on something there. I think that's a sales effort in there, too, to be clear. It's not just an explanation, but... Yeah, yeah. Clearly pulling from many different sources there. I mean, the backpack... I can't remember whatever... Yeah, whatever cloud service GPT runs on, I can't remember the sponsor. Azure. On Azure, yeah. Yeah. There you go. I like that.
00:50:37
Speaker
Yeah, so that's pretty fun. I don't know if you had thought about it at all, but we always like to ask those kind of questions. My simplistic answer for the cloud is always, just imagine all the computers it takes to run the software. In this case, imagine all the nurses and doctors are doing all day. Put those computers in someone else's building so they don't have to worry about it. And there you go. It's somebody else's computer. Yeah. Yeah.
00:51:08
Speaker
Awesome. Well, this has been super fun, Dan. And I think, you know, we touched on a lot of different topics, but I personally learned a lot. I think, you know, our listeners will as well and they may have follow up questions for you and that's all good. So again, I just want to thank you for coming on the show and being a part of Kubernetes Bites. Thank you both. I really appreciated it. Thanks, Dan.
00:51:33
Speaker
Nice. That was fun, Bob. And I know we mentioned it earlier that we haven't had anyone really in the healthcare space. So it's really fun to have someone who not only has that serverless experience, but you know, has kind of really had experience in Kubernetes and why they
00:51:49
Speaker
Use it don't use it still use serverless. I thought it was a really interesting point of view and and also a lot of the the pieces about the non-technological Technological pieces of the puzzle there. So yeah, let me hear your takeaways about that conversation
00:52:03
Speaker
No, I think you started in the right direction, like importance of finding the right tool for the job. Not everything had to be Kubernetes. They started with some applications, some of the commercial applications that they are buying from off the shelf can only be run on ECS.
00:52:21
Speaker
They're not messing with it, not trying to move it to serverless. They have some teams inside the organization that are using Kubernetes and EKS. But then, as Dan said, some of them, they have a go-to serverless as the default option, if you can. Policy, they're trying to guide people, but not forcing them to choose on one path or the other. I think that helps.
00:52:43
Speaker
Everybody has a different skill set different comfort levels if you're ready to take on the additional responsibilities or cost that comes on. I'm using something like these yes or yes go and feel free to do that but if you want something that's more supported by the organization. How is the solution so i think that that helps.
00:53:00
Speaker
Like, again, it didn't come as a surprise that organizations do this, but it just helps us reinforce the point that, okay, you always need to make sure that you're not adopting technologies because they are really cool. And that's the next big thing, but you're adopting technologies because it helps solve a specific problem for you.
00:53:18
Speaker
Yeah, and to the point of feeling the pain first is not always a terrible thing. Although if you listen to the sales folks, they definitely will tell you otherwise. As I think Dan made very clear in his example, for me, I like the conversations around the economies of scale related to adopting new technology, especially the metaphor around
00:53:41
Speaker
one guy making sauce really well versus sort of a cook doing everything in the kitchen, so to speak. Probably not paraphrasing, of course, to what Dan said. But it's a good lesson and sort of thing to pick up on. And on top of that, sort of the organization as a whole and the communication, which came up a lot in the platform engineering podcast as well.
00:54:06
Speaker
being sort of one and two in the priority list, right, versus technology, as Dan put it, I think somewhere between ninth and 13th or something like that. Just to kind of push on the point of, you know, if you get those other things incorrect or you really struggle with them, tech's not going to fix it.
00:54:24
Speaker
I did also like, to your point, using the right tool for the right job, ECS being sort of delivered or he mentioned a product being delivered that way. That's a perfect reason for it to stay that way. I know we've talked in the past, there's some cases where you don't want to migrate, you don't want to move things off of or choose new tooling and the fact that you don't own it and don't run it is a perfectly good reason. So that's a really interesting one to me and just the adoption of
00:54:54
Speaker
of serverless and what it can do for sort of the scalability, I think is a really interesting concept of definitely the case here being such a new company, being able to start with a fresh set of architectural eyes, so to speak, and kind of choosing that path mixed with some of the others, as you mentioned. Awesome. That's it.
00:55:18
Speaker
Yeah, good. Well, I hope everyone enjoyed that one. I think it was really enjoyable for me as well. As Bhavan mentioned, you know, mystery for next week's episode, Bhavan will be taking some well deserved time off.
00:55:34
Speaker
So, you know, enjoy that. That's fine. So it'll be me or me and someone next week. Yeah, this is a surprise even for me, guys. Ryan hasn't told me what he has planned. So even on my video, I will make sure that I tune in and check out the next episode. No pressure, Ryan. There you go. There you go.
00:55:54
Speaker
Cool. Well, you know, as always, if you can review the podcast, let us know what you like, don't like, send us encouraging messages. If you don't, we won't listen to the other one. I'm just kidding. All are welcome. And please go have a listen. And please feel free to suggest anything else. And with that, it brings us to the end of today's episode. I'm Ryan. I'm Bobbin. Thanks for joining another episode of Cobra Denny's Bites.
00:56:23
Speaker
Thank you for listening to the Kubernetes Bites Podcast.