Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Kubernetes Storage in AWS
395 Plays3 years ago
In this episode, Ryan and Bhavin interview Luay Alem, an enterprise solution architect at Portworx. Luay is a Kubernetes enthusiast, working with Cloud Native and Open Source technologies helping customers with their digital transformation journey. The discussion dives into challenges associated with running Kubernetes on AWS, the different storage options available, and how customers are leveraging Portworx for their Data Management, Data Migration, and Disaster Recovery needs.
Show Links:
1. Pokemon Go uses Kubernetes: https://thenewstack.io/how-pokemon-go-creator-builds-on-kubernetes-for-developers/
2. Creating the enterprise app store - using GitOps and Kubernetes: https://youtu.be/ke-cKKzqriM
3. Improve Core-to-Edge Mobility and Resiliency for Cloud-Native Applications: https://community.cncf.io/events/details/cncf-cncf-online-programs-presents-cloud-native-live-improve-core-to-edge-mobility-and-resiliency-for-cloud-native-applications/?utm_source=hs_email&utm_medium=email&_hsenc=p2ANqtz--UstWsorHoSJybBAmT9r_f8z61VwnHr0ELrHqmFbUmktlg8skl62FSwF98-y5pXr-R80PS
4. Lacework funding round: https://www.lacework.com/press-release/lacework-closes-seriesD-funding-round/
5. Datacore buys Mayadata: https://blocksandfiles.com/2021/11/18/boom-datacore-arrives-in-kubernetes-storage-space-by-buying-mayadata/
6. Inspecting container filesystems: https://blog.px.dev/container-filesystems/
7. Portworx is hiring: https://www.linkedin.com/posts/bhavin04890_kubernetes-vexpert-vexpertmodernapps-activity-6861314066154041344-WzEU
Recommended
Transcript
Introduction and Podcast Overview
00:00:03
Speaker
You are listening to Kubernetes Bites, a podcast bringing you the latest from the world of cloud native data management. My name is Ryan Walner and I'm joined by Bob and Shaw coming to you from Boston, Massachusetts. We'll be sharing our thoughts on recent cloud native news and talking to industry experts about their experiences and challenges managing the wealth of data in today's cloud native ecosystem.
00:00:27
Speaker
Good morning, good afternoon, and good evening wherever you are. We're coming to you from Boston, Massachusetts. Today is November 19th, 2021. I hope everyone is doing well and staying safe.
Upcoming Events and Guest Introduction
00:00:44
Speaker
Let's dive into it, Bobbin. What is up? How have you been?
00:00:49
Speaker
I'm doing great. Like I'm ready for the Thanksgiving week. Uh, it's, I'm excited for it being like a slow week. And then week after that we have AWS re-invent, which is again, a show that I'm traveling to. So I'm excited. Like this, I'm assuming this is a bigger show. I saw AWS already sold out all of its in-person tickets. Maybe we are expecting 15,000 plus people. So this should be fun. Yup. How about you? Fun fact, I have never been to a re-invent.
00:01:17
Speaker
Oh, yeah. Yeah. Uh, this year didn't work out just because traveling two weeks in a row would be a little tough, but, uh, I, I really do plan on getting out there to one, uh, maybe next year, maybe next year. 2018, I think, uh, and it had like 42,000 people. It was just so crazy. Like there were queues.
00:01:38
Speaker
For everything like from the shuttles to getting into sessions, people were missing sessions to get in line for the next session. If you didn't have sessions that you already reserved your spot, you couldn't get in. You just wasted the whole day.
00:01:52
Speaker
Yeah, that doesn't sound enjoyable, honestly. But hey, I still want to experience it. Okay, moving on from that, what have I been up to? Let's see, I had a wild house this past weekend, we had a lot of family up, and it was so much fun. This week really been, you know, been super busy trying to get ready and
00:02:17
Speaker
before I know a lot of people taking off for the holidays next week. I'm looking forward to that. I'm going to travel down to see family to New York from, from good old Boston. So, um, that's fine. Now we're just, well, I should say we are driving, uh, and, and it's not a very far drive and we don't go through any major cities. So, um, while traffic still is always tough that Wednesday,
00:02:47
Speaker
It's usually not too bad. Yeah. Let's see. We have a guest on the show today, Louis Allam. Actually, he works for Portworx, disclaimer, and he's a solutions architect and Kubernetes enthusiast. He works with the cloud native and open source technologies, specifically helping customers in their transformation.
00:03:14
Speaker
to start using these technologies. And he does a great job. I'm super excited to learn about what he does day to day, you know, how he interacts with his customers and really around this topic of, you know, using storage and Kubernetes on Amazon Web Services.
00:03:30
Speaker
And I agree, like, as you said, he works for Portworx, so some of these use cases and customer stories will be from Portworx
Kubernetes Storage and Real-World Applications
00:03:37
Speaker
customers. But I think we felt that it's important to also discuss how Kubernetes and Kubernetes storage is actually being used in the real world. So I'm excited for this one too.
00:03:48
Speaker
Yeah, absolutely. That practitioner's approach is super key, especially because, you know, we don't, you know, personally, we don't interact with customers every day. So it's really cool to see and hear, um, before we dive into that and get Louis on the show, um, let's talk about some cloud native news. How about, uh, why don't you go first?
00:04:05
Speaker
Yeah, so like the past couple of weeks have been kind of slow. No major conferences. I'm just catching up on some of the talks that were done at KubeCon. I'm catching up on like GitOpsCon sessions. I saw a keynote from Alexis, Alexis, their CEO, and that was really cool about creating an enterprise app store and using GitOps and Kubernetes for that.
00:04:29
Speaker
He had a really cool comparison of how apps run on phones, and you can stop them at any time, delete them at any time, and you can install it easily, switch between phones, and how you can use GitOps and Kubernetes for a similar workflow. Like, okay, you can install your containerized apps on Kubernetes running anywhere. You can start, stop, or delete them as needed. You can also use continuous delivery to auto-update and install your patches, even have some drift detection in place
00:04:58
Speaker
and then just in the customized apps and certified section where you can trust apps that are uploaded on the marketplace on your phone. Similarly, you can build an internal store where you can still have custom packaged apps, but then they are certified. You can have security checks to make sure whatever you're running is secure. So that was a really cool session. I recommend we'll have a link for it in the show notes, but I recommend people watching it. It's just a quick 10 minute talk.
00:05:29
Speaker
What else?
Scalability and Production Use Cases
00:05:31
Speaker
Again, along similar lines, I saw a new stack video of how Pokemon Go or Niantic, the bigger company, uses Kubernetes internally, and how they have been using Kubernetes for the past five to six years. Pokemon Go was actually running on Kubernetes, so when we all went crazy and started using Pokemon Go, I was one of them.
00:05:51
Speaker
I paid for six months and then got really bored. But all that scale was handled by Kubernetes under the covers. So that's also an interesting talk about how Kubernetes is used for these gaming apps and how you can build a platform on top of Kubernetes.
00:06:07
Speaker
Yeah, I remember Pokemon Go on Kubernetes being one of the biggest sort of scale and production use cases for Kubernetes as people were starting to learn more about it. And I think seeing a use case like that, that was the first one that people were like, oh, you know, you can do real stuff. And it was it was a really big deal. I remember that I'd personally never played it. But I know that a lot of I think it was all built on
00:06:36
Speaker
Google cloud right I believe I think so yeah okay and let's see what else do you have any other news time I just have not news topics like just some funding rounds and acquisitions that's something that I like to keep an eye on if you follow the Kubernetes storage space data code picked up Maya data
Industry News and Acquisitions
00:06:59
Speaker
for an undisclosed amount. Looking at like so now they they obviously my data code was contributing to open EBS now with the acquisition of my data they'll be obviously supporting that project. I was looking through blocks and files article and I found there was a slide section at the bottom where
00:07:18
Speaker
Datacore actually had a minority stake in MyData last year. Like they bought some shares and now they basically went from minority stake to a full-fledged acquisition. So things happening and then like a crazy funding round like LaceWorks is another startup in the cloud native security space and they just raised 1.3 billion dollars of funding
00:07:41
Speaker
valuing them at $8.3 billion. So like not valued at $1.3 billion, not being a unicorn. They raised that much money in a single round. It's just crazy. Like I'm still not able to comprehend that. I'm not able to like keep my mind around it, but yeah, it's just crazy.
00:07:57
Speaker
Yeah, that's a lot of dough, that's for sure. Really good to see though that the investments are still coming in the space. I mean, it being security focused, it doesn't surprise me. I don't think it surprises you either, but really cool stuff.
00:08:13
Speaker
Yeah, I like the whole data core story, you know, friends, then now family. And I didn't, I wasn't familiar with data core, to be honest, but it, it, it seems like a technology that sort of is an abstraction in itself, right? It can aggregate other types of storage into, you know, its own type of storage. If you have a bunch of sand, it can turn it into a NAS or whatever on top of it. I may be actually wrong about that, but it is an abstraction.
00:08:40
Speaker
from what I can tell and just having now the Maya data, which is basically another abstraction. I think it fits right in their model. I think it makes sense. I'm excited to see where that goes.
00:08:55
Speaker
Yeah, I worked with data core a bit at my previous job at Lenovo. They were like a software defined storage solution running on our x86 boxes where they can leverage like abstract, like aggregate the physical drives and abstract a storage layer, block layer for applications or virtual machines, blue consume. So pretty cool.
00:09:14
Speaker
Nice.
Community Articles and Job Announcements
00:09:16
Speaker
Yeah, in terms of articles, I really liked this week. There was one from the CNCF community about edge mobility from the core and the edge. Actually, the talk was, I think,
00:09:31
Speaker
A member from Trilio which talked about sort of the mobility of applications and you know the need for security from the edge to the core and it's really a migration use case and a backup and restore use case but a really good talk. I definitely recommend it if you're trying to understand sort of that mobility concept that we talked about.
00:09:50
Speaker
probably a lot here on a lot of these episodes. And the last thing before we dive in and get Louis on the call, a little bit of a shameless plug, I'm hiring. So if you're interested in working for a really cool company in tech marketing. A really cool manager if I might.
00:10:08
Speaker
Oh, the brown nose there. The roles we have here, we both have some on our team as well in Solutions. So please, if you're listening and you want to get in this space, please reach out. We'd love to hear from you. So without further ado, let's get Louis on the call.
Louis Allam's Experience with Kubernetes
00:10:32
Speaker
All right. Welcome to the show, Louis. It's great to have you here on Kubernetes Bites.
00:10:37
Speaker
I want to learn so much from you today. But first, let's learn a little bit about you. Tell us a little bit about yourself and your background, why you're here, what you're doing. Hey, guys. Thanks for having me, first of all. My name is Loy Alam. I'm part of the Portworx Solutions Architect team. I've been at Portworx for close to three years right now, about two and a half.
00:11:06
Speaker
So my background, actually, my entire career been around the storage. I started about 25 years ago working at EMC. And then I moved along five years ago when I started working on Kubernetes. So some exciting stuff with Kubernetes. And thanks to Portworx, I had the opportunity to join the team
00:11:35
Speaker
About two and a half years ago, I became a member. So right now working with customers, pretty much helping them with their digital transformation journey, right? So that's my premier role. Awesome. Lots of stories.
00:11:55
Speaker
Yeah, OG EMC before Dell picked it up. I had a few years at EMC as well, not as many as yourself, but yeah, it's great to see the storage industry evolving. It's good to see. So, you know, today's topic is really, we want to try to focus on AWS and what kind of storage options are there and which one of those play well with
00:12:20
Speaker
with Kubernetes, I know you said you work with customers. And one of the things that we want to tell our listeners about is really what options are there for using storage in AWS, right? And there's obviously different ways to use Kubernetes. There's EKS, there's running Kubernetes sort of on your own. And then how you connect to storage obviously varies. So maybe tell us a little bit about your experience using Fortworks or other options that you've seen in the field.
00:12:51
Speaker
Yeah, definitely. So as you guys know, there's a fair amount of customers we have as well running on AWS. AWS does offer the 3D driver for CSI that they use, commonly some customers use. Also some EFS that we see some people were using as well, right?
00:13:20
Speaker
And we have all our customers basically use Portworx as their storage provider, right? And that actually due to the challenges that, you know, there is using the traditional, you know, CSI or storage on AWS, basically the high availability across AZs. So, and that's the challenge we have from some customers, right?
00:13:51
Speaker
Yeah, that's a great point, right? Like making sure that your EKS deployments are resilient to AC failures. What are some of the other challenges that you see? Like how about backup or disaster recovery or any of those things?
Case Study: Portworx for Data Replication
00:14:05
Speaker
Yeah, definitely. So give you an example of the challenges that the customer, I actually work with one of our customers on.
00:14:16
Speaker
They do have policies in place due to compliance issues that the data has to be available across multiple AZs. So when they deploy their application, they were using actually our sync to sync the data because they have their pods deployed. They're using three AZs. They're up in the US East.
00:14:43
Speaker
So they have one A, one B, and one C. They're EC2 instances, EKS instances across these three ACs. So for them, what they were doing, they were using Rsync to sync the data between US1A and 1B.
00:14:59
Speaker
And that is actually, that does not scale. And that's not the way you want to do when, you know, at the enterprise level, when you have hundreds of pods, right? It becomes a lot of work to do that. And it's not the way of doing things in a cloud native way. So we work with them and we actually use Portworx. And since Portworx is topology aware,
00:15:29
Speaker
So Portworx automatically knows that we have three availability zones that the cluster is deployed in. So when you create your volume, Portworx will automatically place your replicas across the availability zones. So when, you know, and sync, and, you know, synchronously replicate the data across all AZs. So in pod, you know, let's say you're running, for example,
00:15:57
Speaker
elastic, right? And this is what the customers run. And, you know, they're running elastic on US 1A that, you know, no terminates for whatever reason or fails for other reasons. The data is simultaneously available from the replicas running in the other AGs. So the pod actually gets served from these other nodes, right? That have the data.
00:16:26
Speaker
So that one customer, we worked with them and we helped them actually migrate over from using EFS, actually they were using, to use Portworx, right? So they migrated, they have both EFS and EBS, they were using. So we helped them migrate over. So that's one customer use case, also some other challenges they have.
00:16:56
Speaker
They do have for compliance, they need to have a region recovery. So they're in Ohio region, so they also needed to fail over to Virginia, right? And you kind of do that. You need a software that is DR capable, right? And we worked with them and we introduced Portworx Asynchronous DR.
00:17:26
Speaker
which we implemented and they were very happy with it, with the results. So basically automatically the cluster will fail over from Ohio to North Virginia. They do run that, they have a policy that they need to exercise that every three months. So every three months, this exact process exercised and all applications are brought up on North Virginia.
00:17:53
Speaker
They run them there for a short period of time and then they fill them back to Ohio. That's awesome. Testing the ER plans is the best way to make sure they actually work. Yeah, and I imagine that's probably hard to do with our sink.
Challenges in Data Backup and Migration
00:18:10
Speaker
I've seen Rsync used in certain situations, even I think recently I was kind of investigating the OpenShift migration tool for containers. And even there, there's an option to try to do basically an Rsync thing, which I think is only okay if you're going to do like a one time move of maybe a few things, right?
00:18:32
Speaker
The moment you say, I need to do this all the time, or I need to do this across a thousand different applications, I imagine even with Rsync, if you were successful at being able to script it and manage it yourself, that it's still a very particular runbook that you have to keep in mind and maintain. I take you one month to test every three months. I bet.
00:19:02
Speaker
Yeah, they do. Actually, I felt, you know, the guy was like, really super cool guy that was running this. So he was like, he was really relieved when we just met. He was super happy was the first one was like, in the meeting every like we were meeting with them.
00:19:21
Speaker
weekly. So we're doing work and then we go test it, implement it on their own. And then he was like super excited. He's the first one to the meeting and he's like, Hey, let's get this going. Uh, you know, um, I, I could feel his pain based on, you know, as you said, our sink is not really the tool. Yeah.
00:19:40
Speaker
for your price scale. Yeah, it sounds like whatever storage you're running in AWS, it sounds like trying to manage the efficiencies and reduce the complexities, whatever tools you use. In this case, it's Portworx, but it's really going to help you scale as you grow your business and grow the types of applications you're running on AWS.
00:20:03
Speaker
I don't have much experience with EFS. I know it's newer. It's interesting to see that you have already seen customers use it and even use it and try to move away from it. I'm curious, how often do you see the use of EFS? I know it's very different than EBS being file shares. One might think
00:20:26
Speaker
Hey, I can get around the AZ issue because I can connect this file share across multiple AZs, but are there challenges there? Definitely, there are challenges with it. Everything comes with the price. If you're running this highly performant application, this database, it's not going to perform well when you're using DFS for sure. That's definitely
00:20:59
Speaker
Yeah, well, it's the biggest challenge, right? If it's not performant enough, you might not get, like, the end user experience might suck for the application that you're running. Right, correct, correct. And so basically, you know, I had one customer that I worked with, they basically used it, but they did not really care about performance at all. So the performance was not an issue for them when they used that
00:21:25
Speaker
But that's one solution, right? So what they did, we end up, when they migrated to Portworx, we do have similar, you know, we offer to the FSL or shared v4 volumes, right? And that gives you similar, you know, one-to-many access.
00:21:47
Speaker
I started using Portworx for shared volumes and then for a demo that I was working on, I had to go back to EFS. It took me a while to actually make everything work. I started with an article that was, I think, older. It wasn't using dynamic provisioning, so I had to manually create the file system. For the node group that was part of the EKS cluster, create mount points for that file system and then
00:22:14
Speaker
I also try manually provisioning a PV and then attaching it to a PVC, so that was not fun. Then Ryan pointed me to an article that, okay, this has been simplified a bit, so there is dynamic provisioning, but there are still steps involved in like, okay, you need to either have scripts using EKS, CTL or AWS CLI.
00:22:31
Speaker
to create those file system endpoints, to get those identifiers, use it in your storage class definition, and then provision shared persistent volumes. For migrating to Portworx, it's really easy. This is what we have. When you're talking about migrating to Portworx, we do have actually the tools in place as well.
00:22:59
Speaker
That customer we worked with when we did the, who worked on the DR as well. We have the tools in place. You can migrate the workloads from any story provider to Portworx. So that we did not have to run his Rsync neither to copy the data.
00:23:19
Speaker
from the EBS volume to the Portworx volumes, right? So we have the tools in place at Portworx. So basically, we just converted these applications on the cluster, migrated them locally to use Portworx using our KDMP tool. So yeah, that was like really smooth transaction with that migration as well. Nice.
00:23:50
Speaker
One thing I'm curious of being in AWS, I know we're focusing on storage here, but do you find customers are choosing to use EKS more often than deploying their own Kubernetes infrastructure on AWS, or what breakdown do you see there?
AWS Kubernetes Preferences and Migration Strategies
00:24:09
Speaker
Actually, most of the customers that I have been working with on AWS, they're mostly running
00:24:19
Speaker
They either run in the EKS or they deploy with cops, right? So basically it's a majority is EKS for sure. The only other customers that I worked with that run only AWS that they're running, they run different distro, for example, OpenShift, right? If they're not using EKS or something, they're running different distro.
00:24:52
Speaker
Now, and for, you know, all fairness, when you go to the cloud, you look at managed services, right? You don't want to end up managing the whole infrastructure on AWS itself. You don't want to, so you're not going to go just to spin up the, you know, some, you see two instances, managing yourself. Do you worry about the upgrades and just deploy, you know, off the tree, cook or netties on them, manage it yourself.
00:25:14
Speaker
but pretty much she cares.
00:25:21
Speaker
So the customers, they do go with EKS for the ease of use. And that's number one reason for going to the cloud because they don't want to manage anything. So in your experience with customers, have you come across customers who have migrated from on-prem to the cloud or migrated from virtual machines to using containers with EKS?
00:25:45
Speaker
Yeah, so that's a good question. Actually, we do have customers who did migrate from on-prem to EKS, and I'm actually working with one right now just migrating, moving their workload from on-prem to the cloud. Basically, it's going to be about like 70% of their workload is going to be on AWS. They've been running on EKS for a while.
00:26:15
Speaker
So they don't want to increase with that and run more on EKS. And the beauty of it, basically Portworx offers you the tool to migrate your workloads from one cluster to another, right? So basically, and they use that as a type of blue-green deployment, right? So they spin up the EKS cluster and they start moving the namespaces one at a time, verify with their end users,
00:26:45
Speaker
that their application is running. Basically, they migrate it to EKS. They spin it down on the prem for a week before they terminate everything for them. They give the users the namespace on EKS, let them run for a week. They get the final OK from them. Everything is smooth. They terminate the namespace on the prem permanently, and they continue running on EKS. So we do have customers migrating, and we also do have customers
00:27:13
Speaker
they use the EKS clusters just for basically a DR scenario, right? So with the DR, so what they do in case of a DR on-prem, the cluster, the data center is down, they activate a synchronous Portworx DR on the EKS cluster and that spins up their workload automatically.
00:27:40
Speaker
and PowerWorks does have the API hooks in place. So basically with the way they have it, they automated the whole process for them. So all where they do have a script that runs when they detect a failure for the on-prem site. And that will automatically scale up their deployments on EKS and their application runs from there.
00:28:06
Speaker
Nice. Those are like a couple of interesting use cases, like using blue-green deployment. Again, it sounds obvious, but to actually implement it and have a process around it, actually working with customers to make sure that they don't see any performance hit or they don't see any downtime. That's actually a sound plan. Yeah.
00:28:25
Speaker
solution is definitely you know hard to come by probably easier these days I just was flashing back to a previous employer I worked at where we had mesosphere clusters and on-prem and in the cloud and we were
00:28:40
Speaker
doing similar migrations. We had Ceph on-prem and AWS Storage. There was no port works at the time. We just basically said, well, migrate all your stuff, but then you got to load all your data back in. That was just the crux of it. We had Kafka streaming some stuff.
00:28:56
Speaker
It was such a different world in terms of integration, even four or five years ago, to see customers and hear customers doing this with ease. I'm sure there's some thought process, obviously, and design work that goes into making sure it works well, but it's really good stuff. Louis, what type of applications are you seeing being used most
00:29:22
Speaker
in these environments? Is it mostly databases? Is it stateless as well? It's mixed, to be honest with you. Basically, because I work with the story, so all my customers that I work with, they do pretty much persistent data.
00:29:44
Speaker
They do, I see a mixed of regularly applications. I see a lot of Jenkins, for example, they run the pipelines, I see elastic, they'll have customers running elastic in there. We do our customers running databases, Mongo, Redis as well, Postgres. So it's really mixed. They're not really one more than the other. So it is pretty much...
00:30:13
Speaker
It's a mix there. And, you know, again, each company, they do have different requirements, right?
Modernizing Databases for Cloud Environments
00:30:20
Speaker
So like, for example, what customers that I work with, they really all what they're running there is Mongo, Redis, and some Postgres. They don't have others.
00:30:31
Speaker
I do have others, they all what they're running there is like just basically Twistlock and some other tools, right? Or Bitbucket, right? I guess some other, you know, different customers, they're just running their L clusters here. So it's really mixed.
00:30:47
Speaker
Now, are you finding these customers are going sort of all in once they start to run a database or something like that and it goes fairly well, or are they still split across the chasm of like, some of my stuff is still not in Kubernetes, but we're putting data in Kubernetes, like where on that journey do most of them line? Actually, so yeah, you know, that's really an interesting question because we do, actually we have both right now, so we do have some customers
00:31:16
Speaker
that they stole, you know, half and a half, for example, right? And we have customers, they are already all the way in. And some customers, the one with the half and a half, they're basically moving, you know, in that direction, right? Some, you know, and you can understand that digital transformation, you know, there are some applications that you really, it's hard to break to chunks, right? You know, if you have that,
00:31:46
Speaker
Oracle, you know, yeah, because if you have, you know, it's hard because I was actually talking with one, um, customer out West. They, uh, they have this Oracle, uh, you know, database is like you're talking about, uh, it was close to, uh, one, you know, it's,
00:32:11
Speaker
It's really humongous, one better bite. It's like really huge. How are you going to break that? How are you going to break that? It's hard, right? You need to break that to chunks and that's a lot of work. Yeah, I know.
00:32:32
Speaker
I know in the past I've also seen companies take the tact of, well, okay, we have this sort of on-prem stack, which is fine, but we want to modernize.
00:32:42
Speaker
but we're not just going to put Oracle in a container. You could, I've seen it, but a lot of the times they want to break that data into silos or smaller silos and try to transform it. Some of them stream it off to Kafka and transform it and load it into Neo4j and Cassandra and all this stuff. That's a huge undertaking, especially when you're talking about databases that size. Right. This is exactly like we talked with them about breaking
00:33:11
Speaker
these into silos and just the idea of the guy with the look on his face is like, you know, we are not touching this.
00:33:26
Speaker
So you said you would still work with storage guys inside organizations for this migration, right?
Shifting Storage Responsibilities
00:33:34
Speaker
Once they modernize two containers in EKS, are they still responsible for managing storage? Or how does it handle? Do platform operators come in? Yeah. That is a really good question because there's the mixture right there. Because when you go to the cloud,
00:33:55
Speaker
you know, from on-prem, some of them, they're still getting away from, you know, needing to deal with like the storage team, you know, policing their storage usage. It's like you get this, submit the request to get the disk added or stuff. So a lot of teams, what we see for the on-prem, they do like the
00:34:15
Speaker
they're going or out of using local drives that they come with the servers, right? So this way they don't have the need to go to the storage teams like, Hey, we need, you know, to carve us like one terabyte volume attached, you know, to this note, they don't need to do that. Right. But, but so with the cloud, it's actually a lot of these guys like this specific team and working with that, they do, uh,
00:34:43
Speaker
they are migrating to EKS. They currently, they're on-prem, they're using their VMware, right? So every time they need a volume, they need to submit a ticket and it goes through the whole process. They need to wait 24 hours to get a volume, you know, attached policy, right? Companies politics, they got to go through. It's like different groups, totally different team, different managers, everything. When they go to EKS,
00:35:10
Speaker
They don't have to do that. It's like they, you know, because they are the cloud operator team that owns this whole project themselves. So the volume is there for them. It's just easy access. They provision it. So it's definitely, there is a relief for these guys not having to deal with the internal, you know, politics, right?
00:35:38
Speaker
Thank you. So for people who are getting started or want to move to AWS or want to move to EKS, what are some of the tips and tricks that you can share, some just one-on-one level things? Yeah. To be honest with you, a lot of customers we work with, it's like you really want to plan your capacity or your workload upfront on what you got to do.
00:36:04
Speaker
understand what applications you're going to deploy and the need what your end users require from you, what else LA you have, you know, with them in place, you're doing backups, you're not doing backups, right? And this side, so that these really capacity planning is really important to understand. You don't want to you don't want to under provision
00:36:29
Speaker
you know, but at the same time, you don't want to go crazy and over provision, you know, pay for some crazy instances you're not going to use, right. So you do need to understand and plan, you know, plan your upgrades, plan your your node, your node groups, right, your age, how you're going to deploy it, you're going to go, you know, you need to understand the architecture if you're going disaggregate versus doing, you know, you're going to aggregate this setup. So
00:36:57
Speaker
you're going to use the outer scaling or you're going to be static with the cluster. And beauty of, you know, running on EKS, it gives you all these flexibilities, right? So that's the beauty of it. So you can have, you know, believe it or not, there's a lot of, you know, a lot of restrictions from companies right now, even when you run on AWS, like some companies, they do restrict them to only use like two AZs, for example.
00:37:26
Speaker
They cannot go use three. So there are restrictions in some places. They don't give them all these accessibility. So it's not like for you and I, we log in, we can deploy anywhere, you know, when we want to do it because it's our own accounts, right? For these companies, it's, you know, their policies, their security groups, their regulations that they have to bind to.
00:37:53
Speaker
So all that you get to understand. You get to understand the security policies from your company in place. Like a lot of companies, they have mandatory AMI refreshes every 90 days or every 60 days, right? So all this, you get a planet upfront before really just go spinning up. It's really easy just to do Google EKS CDL and just create the cluster, right?
00:38:23
Speaker
Exactly. But the fact is, you really need to understand all these regulations and plan on them. And you bring a good point, Juan, with Portworx, actually, when you do the EKS upgrade, it's really super cool about it. We do understand
00:38:47
Speaker
the volume placements with it. So when you do the upgrade, basically we use something called Cloud Drives that we attach to the nodes when you create the cluster. So when you bring in the new cluster, Portworx will automatically disturb the node.
00:39:10
Speaker
that going away when you do the AMI refreshes gets attached to the new node automatically, right? So it's everything, the entire AMI refresh when you're doing it, it's fully online. You don't
00:39:25
Speaker
see a node going, yeah, it's totally online operation. Like I haven't done the upgrade myself. So like, okay. Yeah, it's totally. Yeah. Yeah. Yeah. So it's totally, it's totally, you know, online. There is no, no downtime for it. And that's, you know, that's, you know, remember, this is something, uh, yeah, if you're using, uh,
00:39:51
Speaker
the EBS, you know, that CSI volumes, that node goes down, you know, another node comes up. But so, so you do, you would experience these three starts and pause and, you know, that maintenance when they get a take. When using Portworx, DCMI refreshes, we do with the customers, it's online operation. And their application continues running. Now, granted all the companies when they do touch any cluster, they do have to submit
00:40:20
Speaker
a management request for maintenance, but really adjust for a procedure more of than taking a maintenance window to do upgrades or maintenance on the cluster.
00:40:34
Speaker
Right. It's just more like covering their tracks of like, Hey, we're going to be doing something in case things go bad. We don't expect to, but in case they do, don't blame us. Exactly. It's, it's, it's formality. It's formality of the process. Right. Well, uh, Louis, you know, I think this has been really insightful to get the practitioner's approach, um, and sort of insight into how, uh, you know, folks that you're working with are using storage in AWS.
Closing Remarks and Future Topics
00:41:03
Speaker
I know cloud can be complex, it can be super easy. Like you said, think beyond day one, it sounds like is a big part of this and really think through your application needs and customer needs. I think there's, as we know on this podcast, there's a ton of storage options out there. Find the best fit, really kind of touch each one, whether you're using EFS or EBS or a storage
00:41:31
Speaker
solution like Portworx or others in the industry. I think it's been really insightful and I just want to thank you for being on the show, Lue, before we wrap it up here. You have an open invitation. Next six months, we'll get you back on and share some more customer stories. Yeah, definitely. I think the upgrades could be a whole podcast in itself. I was refraining from asking so many questions there. I think it's a really cool topic.
00:42:01
Speaker
Luwe, we'll let you go. And again, thank you very much. Thank you guys for having me. It was a pleasure. Thank you. All right. Well, that was really nice having Luwe on. I know we work with him, also a East Coaster, Massachusetts person. I don't think we brought that up, but that's pretty rare here. You know, it was really good to get his perspective from, you know, working with
00:42:28
Speaker
various customers, what he sees, how they're using portworks and everything. I think some of those takeaways we were just talking about are really important, especially the plan, plan, plan. I think that's what's sticking my head through that whole conversation is you really have to think through what happens after day one. I don't know about you, Bobbin.
00:42:50
Speaker
Yeah, like agreed like the blue-green deployment that he said of how customers are migrating from on-prem to the public cloud using EKS one app at a time or one namespace at a time and making sure that there are no outages. That was really cool. Also the fact that our customers are actually conducting DR tests for containerized applications running on EKS and like doing that every three months.
00:43:16
Speaker
with automation it's just like okay it's good all of this sounded like really good in theory but actually having this talking to people who are working with customers and seeing that actually being a reality was really cool.
00:43:28
Speaker
Yeah, I think getting that real world approach, just like you were talking about with Pokemon Go, seeing that happening in real life and putting a real use case behind it, seeing it work at scale is always important. AWS is obviously a huge place where people run
00:43:49
Speaker
cloud native applications, and we barely touch the surface, I think, when it comes to some of the complexities and choices you have working with storage. I mean, we talked about EFS and EBS, but we didn't even go into other services like RDS or all those things that you can connect to your Kubernetes ecosystem of services and platforms.
00:44:10
Speaker
We will have more episodes on AWS. If you have a specific topic in mind, whether it's something to do with AWS or not, please let us know. We'd love to have some new ideas as we go through this. And we will be talking about what Kubernetes objects use storage next week, or two weeks from now, I should say.
00:44:34
Speaker
Um, and specifically we want to kind of break down what is a Kubernetes object and which one of those you storage and how they do. I think maybe this is not the most obvious, uh, to, to some, and I think it's going to be really interesting topic. What do you think? Yeah, I'm excited. Like we might have a surprise that we might be in the same room to record this. So the world is changing. We might be in the same room is so exciting. Um, anyway, until next time, everyone, please take care and stay safe.
00:45:05
Speaker
Thank you for listening to the Kubernetes Bites Podcast.