Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Cloud Native Storage and Traditional Storage: What's the difference?
628 Plays3 years ago
In this episode of Kubernetes Bytes, the hosts explore the topic of Cloud Native Storage vs Traditional Storage. If you are curious what Cloud Native Storage is, how it may be different from traditional SAN and NAS or are interested in why the abstractions introduced by containers, docker and Kubernetes are driving change in how applications consume storage, then this episode is for you.
Show Links
- https://www.openfaas.com/blog/faas-storage-cassandra-astra
- https://thenewstack.io/ibm-red-hat-bring-load-aware-resource-management-to-kubernetes/
- https://containerjournal.com/features/the-challenges-of-kubernetes-storage/
- https://sdtimes.com/security/sd-times-open-source-project-of-the-week-kubescape/?utm_source=pocket_mylist
- https://www.suse.com/news/SUSE-Rancher-2-6-Brings-Enterprise-Customers-Improved-Interoperability/
- https://github.com/longhorn/longhorn/releases/tag/v1.2.0
- https://www.idevnews.com/stories/7472/Platform9-Releases-Managed-KubeVirt-Solution-to-Unify-VMs-and-Kubernetes-Stacks
- https://thenewstack.io/on-premises-vs-cloud-native-storage/
- https://cloudian.com/blog/understanding-cloud-native-storage/
- https://a.rever.co/challenges/168
Recommended
Transcript
Introduction and Context
00:00:03
Speaker
You are listening to Kubernetes Bites, a podcast bringing you the latest from the world of cloud native data management. My name is Ryan Walner and I'm joined by Bob and Shaw coming to you from Boston, Massachusetts. We'll be sharing our thoughts on recent cloud native news and talking to industry experts about their experiences and challenges managing the wealth of data in today's cloud native ecosystem.
00:00:26
Speaker
Good morning, good afternoon, and good evening wherever you are. We are coming to you from Boston, Massachusetts. Today is September 1st. We've made it to September 1st, Bobbin. And I hope everyone is staying safe. Let's get into it.
Hosts' Personal Updates
00:00:40
Speaker
So what have you been up to lately?
00:00:41
Speaker
I didn't do anything interesting this weekend. Last weekend I did go to an arcade slash escape room place here in Massachusetts. It was fun. It was for a friend's birthday party. But then I've been heads down into Tanzu stuff. You should see a couple of blog posts coming out soon from me.
00:00:59
Speaker
Did you escape the escape room? Did you succeed? No, it wasn't a traditional escape room. They had individual rooms defined. They don't give you any starting points or things that you have to do. You have to figure it out as you go in. And then they don't give you any hints. If you fail, you exit out and try again. And you keep doing that until you pass.
00:01:22
Speaker
So it wasn't a traditional escape room but the arcade piece mixed in with the different escape rooms which had mental challenges and physical challenges made it a lot of fun. Nice, nice. I've only done one I think once in Wooster and don't do it with someone that you're really competitive with. Let's just put it that way. You got to work together now.
00:01:47
Speaker
Uh, cool, cool. Yeah. I, I took a really fun trip this weekend. I took one of my motorcycles up on a 700 mile round trip to almost up to the Northern tip of New York. Um, so basically went up the beautiful state of New York and across Massachusetts, but, uh, it's been on my sort of list of things to do, uh, as far as like doing a more, a longer trip.
00:02:13
Speaker
I brought the rain gear and everything had to use it you know was dedicated to the to the whole trip but it was a lot of no joke no yeah. And the great part is you can use this app which allows you to basically donate every mile to childhood cancer.
00:02:32
Speaker
So it was a really good opportunity for me to just say, I'm doing this anyway. Let me turn this on. And they give you like a certain amount of sense. They're trying to raise a million dollars for the month of August. I think they hit it. Um, anyway, I'll put a link in. Thanks for those who might be interested in donating your motorcycle miles. I'll put a link in the show notes. Um, anyway, yeah, that's, that's what I was up to over the weekend. And yeah.
Understanding Serverless Storage
00:02:57
Speaker
Okay, so let's talk about like what's going on in the industry today. Do you have any anything that stood out to you? Yeah, you know, in one of the cube weekly breakdowns, there was an article on serverless storage, which I just have to I couldn't resist putting in here because serverless storage is a term. I mean, I don't like a lot of terms, we have too many terms in this industry, but serverless storage is one of my favorites lately.
00:03:21
Speaker
And it's a really good article actually that talks about the concept, I believe Cassandra and data stacks with their Astra product, which is really around sort of decoupling the need to manage servers, but providing you a persistent sort of application. So Astra provides you sort of a consumption based pricing decoupled from the need to manage servers and allows you to still scale with demand, which is a really cool concept.
00:03:48
Speaker
Um, and they call this early storage, but I think what I liked about this article is they basically said, yes, we're using the term, but I'm going to explain to you why we're using the term because it's kind of nonsense. Right. Um, I mean.
00:04:03
Speaker
Storage has gravity. Storage has got to land somewhere. Serverless storage is one of those terms that I'm not fully on board with yet. We'll see how it goes into the future. Maybe I'll go to the dark side. But it actually is a great article explaining for those new to that term. I'll put it in the show notes. And I do have a plug, a request from anyone listening. If you do have experience with serverless storage,
00:04:32
Speaker
or you have sort of an idea of some use cases and all these things. We're sort of naive here. We're definitely getting to know that, but we'd love to talk to you on the show. So please let us know.
00:04:45
Speaker
Yeah, that would be great.
Exploring Kubernetes Scheduling
00:04:47
Speaker
One thing that stood out to me was an interesting blog article from IBM and their research teams have been working on improving Kubernetes as a scheduler. So they have a couple of open source projects. Again, one of them I think is shipped in OpenShift 4.8 and one will be expected next cycle.
00:05:06
Speaker
But one for the first project is called trimaran, which basically helps communities schedule boards by not just looking at available capacity, but also existing utilization. So it doesn't like you can have a five node cluster. And if three of your worker nodes are pegged,
00:05:26
Speaker
You don't want Kubernetes to keep scheduling pods because those nodes have resources available. You want something that helps you spread out the traffic across or spread out the application across all the parts. That's where I think Trimeron helps you. Then the second interesting thing was the vertical pod autoscaler. We have been familiar with the horizontal pod autoscaler where you add more pods if you're seeing or observing an increase in traffic or application demand.
00:05:53
Speaker
Virtual Pod Autoscaler helps you right-size your applications. If your developers are not behaving and they are putting in crazy resource requests and limits as part of your application YAML files, VBA basically looks at historic data of how your application is doing, how much resources they are consuming, and then it corrects them. It might also delete pods and ask a deployment or a stateful set or whatever is deploying the application pod.
00:06:22
Speaker
to deploy another version and then using something called as the modifying the admission webhook and changing the resource request and limit. It's pretty interesting. I think VPA is part of OpenShift 4.8 and trimaran will be... So you can use it today then? Yes, VPA can be used today.
00:06:40
Speaker
So essentially it's doing some sort of like a predictive analytics almost, um, by saying, Oh, you know, here's, here's how much CPU you've used over this amount of time. So I'm appreciating that you're going to need more. So let me do something about it. Yeah. It's not like I didn't find that it's predicting as much. It's looking at historical and like, if you have, uh, if individual pod is asking for like two gigs of memory, but you are only using like.
00:07:03
Speaker
20 megs, they'll make sure they right size your application in that way and not predict. Maybe that's something that Red Hat has in store for us in the future. It's cool technology. I mean, you definitely, you said it might replace your pod. So like, do you have to design your application a certain way? Probably. Yeah. If you're using the cloud native principles correctly, you might like to build an application that's resilient. You know, you need at least two copies of a pod as part of your deployment. Yeah.
00:07:30
Speaker
or having the ability to turn it off because I only have my giant application in one container, right? Cool, cool.
Challenges in Cloud Data Management
00:07:38
Speaker
The only other article I think I put in here was an article from ContainerJournal.com, which was really around the challenges of community storage from a high level, but I think it really did a great job showcasing what is different about data in this cloud-native world in Kubernetes.
00:07:59
Speaker
And it really talks a lot about the fact that data has a lot of gravity. We know this. It's hard to move lots of data. We talk about petabytes of data. But the need for controlling your application plus your data in today's world is still absolutely reality. So how do you do this? And the article talks about some of the trade-offs where we know it's easier to move compute closer to where your data is.
00:08:26
Speaker
And also where understanding where the data is being generated. So we talk about hopefully a future topic on edge, right? A lot of data is produced at the edge. And so how do we in the Kubernetes world, right, bring that data to where it needs to be, or bring the compute to the data, right? And that's, that's, I think it's a really good article when it comes to understanding sort of
00:08:50
Speaker
the needs around observability, but also coupled with the fact that we absolutely still need to think of the application and the data as a single unit. So we'll put that in the show notes as well. Like Kubernetes at the edge is definitely a topic we need to deep dive on in the future episodes.
00:09:07
Speaker
we'll add it to our backlog. Second plug, if anybody wants to talk about it, let us know. Yes, definitely.
Securing Kubernetes Clusters
00:09:14
Speaker
So one of the things that I wanted to go back to, like in our initial episodes, we had highlighted the NSA Kubernetes security hardening guide that NSA had published. There is an open source tool called CubeScape or CubeScape.
00:09:28
Speaker
based on how you choose to pronounce it. But it gives you a tool that you can use. It has tests listed for all the different vulnerabilities that were described in the NSA report. So like non-root containers or immutable container file system, privileged containers, all of those. You can run a check against a cluster and make sure that your Kubernetes cluster is secure. So you're not paying a vendor. It's an open source tool. So why not go for it?
00:09:52
Speaker
So what can you actually accomplish with this tool? So it has like YAML files for each of the use cases that we just discussed. In addition to like just privileged containers, it also talks about like, how do you harden your control plane and the best practices there if you have the Kubernetes dashboard exposed, because that's a pretty common thing that people end up doing.
00:10:12
Speaker
if you have privilege escalation involved in your cluster. So all of those things, it checks your clusters for all of those different use cases and then gives you a report that, okay, you need to fix XYZ things if you want to comply to the standard. So, yeah. Yeah. These are really cool too. I mean, this reminds me of, you know, there was entire companies built around tools that scanned your container images. There still are, right? This is a big, a big problem to solve. It sounds almost like a scanner for your cluster.
00:10:39
Speaker
Yes. Yup. That's one way to put it, I guess. Everybody start running it right now. Nice.
SUSE Rancher 2.6 Launch
00:10:47
Speaker
And just going around with what's happening in our ecosystem, Rancher 2.6 is out from SUSE. So this is their first major release after, I think, they got acquired by SUSE. And it introduces new dashboards and new UIs making lives easier for administrators. But then it also introduced full lifecycle management capabilities for
00:11:09
Speaker
your Azure Kubernetes Service or AKS clusters and Google Kubernetes Engine or GKE clusters. That functionality existed for EKS in the previous release. They just added two more managed Kubernetes clusters to your Rancher interface. Continuing that support for multi-cloud, it sounds like. Yep, definitely. Yeah, part of that release was Longhorn 2.6, right?
00:11:30
Speaker
Oh, yeah, I forgot about that. Yes. Good point. Which I think, you know, it being the podcast around cloud native storage, Longhorn 2.6, I think, you know, some of the highlights are encrypted volumes, encrypted backups, and some other things. So really cool tech if you are sort of new to the cloud native storage space too as well.
00:11:50
Speaker
Yep, that's one of the topics that we'll talk about. That's the primary topic that we'll talk about in today's podcast. So we'll definitely look at that.
Rafay's Funding and GitOps Model
00:11:59
Speaker
And then just keeping along with the trend, like last time we spoke about how venture firms are putting more and more money in community startups. One of the communities operations management platform startup, that's a weird marketing term, but
00:12:11
Speaker
Rafe basically raised $25 million for CDJ as part of their next funding round. They have a cool GitOps pipeline model where you can have your pipelines for not just your applications but also the infrastructure or the Kubernetes clusters that are your infrastructure into a single pipeline and deploy everything as a complete unit.
00:12:32
Speaker
And then they have additional like blueprinting functionality around how you can create or gather logs and gather telemetry data and metrics from your individual communities cluster that it's managing.
00:12:45
Speaker
You know, I didn't realize this, uh, before when you're going through all these topics, but there's sort of a multi-cloud theme here, you know, with Federation and a new startup and, uh, Ranch coming out with more, I mean, it's, it's obvious. I think people are starting to use Kubernetes in more, more places and more clouds. Um, so really interesting to keep an eye on. True. All right. Is that the end of our news? I think it is. I think it is. Yeah.
00:13:07
Speaker
We'll put all those links in the show notes if you're interested in any one of them so you can dig in.
Cloud-Native vs Traditional Storage
00:13:13
Speaker
Today's topic, without further ado, is cloud native storage versus traditional storage. This is really a topic focused on understanding where we've come through the years when it comes to
00:13:28
Speaker
you know, having storage for our applications and what it's evolved into and how it differs really, really from traditional storage and what we're using today in Kubernetes. So I guess we start with, you know, where were we, uh, in, in throughout history with, and we're not going to start talking about tape.
00:13:46
Speaker
We're not going to start talking about tape, like I promised you. But let's talk about traditional SAN and NAS systems. These were really designed for racks of servers, bare metal applications running directly on those servers. You have complex networking set up to these servers. And there's a huge sort of reliance on the storage administrator.
00:14:15
Speaker
Yeah, as like my first job out of school, I joined that app and I was a tech marketing engineer on their flex part team. And I remember like we used to publish design guides and deployment guides for each specific application and how you can best run it on top of a flex part. So that just not just included compute side with Cisco, but also included all the storage parameters or knobs that you have to turn to get the best performance. So.
00:14:42
Speaker
going from like a 100 page document to deploy an application with traditional cyan or NAS systems. We have definitely evolved and as our applications are modernized, we have also modernized the storage system.
00:14:54
Speaker
Yeah, and modernize, I mean, we're talking about the complexity there, but we haven't necessarily reduced complexity in cloud native storage, which I think we can talk about going forward. But I think there's something to be said there around, you know, one of my one of my first jobs at EMC. And I spent a lot of time in the labs physically racking, stacking and plugging things in and, you know, configuring networking and switches and, and I haven't done that in ages. I mean, a lot of things
00:15:19
Speaker
are software defined these days. So I think that's an evolution that we'll definitely mention here. But I think it's important to understand that when it comes to storage supporting different applications, we've gone from bare metal and created an abstraction to virtual machines. And with that abstraction, we typically still use traditional SAN and NAS because you carve a loan out to go directly to a VM in that case.
00:15:47
Speaker
But we'll talk more about how that continued path towards abstraction plays into cloud native storage. I think there's a middle ground here when you talk about a whole bunch of hardware networking and storage. We can talk about how OpenStack uses Cinder for plugging in systems to the SAN. But there's a part of this when you lump it all together, which is sort of HCI.
00:16:10
Speaker
Yeah, so again, given the complexity of managing these complex systems and converged infrastructure solutions, one of the logical evolutions was hyperconverged infrastructure. So by reducing the complexity that comes from networking and configuring storage, by bringing in everything into a single sheet metal box. So you have your compute. And then you also have local drives. And then when you have multiple of these systems,
00:16:39
Speaker
you aggregate them into a single data store. If you're talking about VMware, you have a single B San data store. If you're talking about Nutanix, you have a single Nutanix storage pool, and that's what's presented to your virtual machine. So all the complexity that was involved in configuring fiber channel or iSCSI and configuring those lines and mapping them using the correct WWPNs and WWNNs, you reduce all of that by just having a data store which will be used to provision
00:17:08
Speaker
your virtual machines and provision your virtual disks that are participating, providing storage for your applications. So that was a clear evolution in trying to reduce complexity through automation and ease of management.
00:17:21
Speaker
Yeah, I think something you said there around it traditionally serves still VM workloads, right? Even when you kind of pack these things together into a really powerful system, you're still sort of working with that dynamic of storage provision purposefully to a specific, another virtualized piece of hardware.
00:17:42
Speaker
And I think this is where, you know, we have this shift to containers, you know, since the early 2000s, and obviously, you know, even more now that you know, Kubernetes is sort of the de facto mainstream. So, how did we start
00:17:59
Speaker
moving traditional storage, which is the SAN and the NAS type systems for containers to start using. Because obviously, the natural sort of evolution, which we covered a little bit of how containers started using these types of systems in the last podcast episode. But we started seeing plugins and connectors, right? Yeah.
00:18:23
Speaker
And definitely, in addition to how storage has evolved, if you're an organization who is modernizing their application from virtual machines to containers, you need to think about a couple of things. There are going to be technical requirements because of the way containers are deployed and Kubernetes orchestrates them. And again, things that we covered in our last episode around how that ecosystem has matured. But there are also a set of organizational requirements.
00:18:53
Speaker
If I'm a developer building containers or deploying my applications using Kubernetes, I don't want to wait a couple of weeks to open tickets and have a LAN or a volume being provisioned on a back-end storage system by a storage administrator. I need to have a good workflow where if I am asking for storage, it is provisioned for me rather than having to jump through hoops and going and talking to different teams inside my organization. So different set of requirements when we are dealing with different set of other modern applications.
00:19:23
Speaker
Yeah, and those requirements we've seen changed over the years too, right? Even when dealing with traditional SAN and NAS, there's sort of the DevOps movement sort of predated almost the Kubernetes and container movement, where we were seeing a high level of automation software come out, like Chef and Puppet, where you saw these libraries coming out to automate all these
00:19:47
Speaker
sort of tasks that Storage Aspens typically had to do to connect, you know, a lawn from a sand to a VM, you know, you know, scanning the iSCSI bus and connecting it and doing all these things that really showed the I think the need and the itch for, you know, making these operations streamline for the way that the industries were moving, right. And that I think ties into sort of the way organizations were starting to
00:20:17
Speaker
really think about how do we innovate fast as cloud started really taking a front and center sort of approach in those organizations as well.
00:20:26
Speaker
Yeah, like if we talk to any developers, right, the thought of them pushing anything to production even a decade ago was crazy, like, oh, moving anything to production takes a lot of testing and QA and unit testing and integration testing. And then on one fine day, people decide, oh, we'll push our application to production. That has completely changed. I was talking to one of my friends while we were picking up some bagels in the morning.
00:20:51
Speaker
And he said that, oh, I have an issue in a production environment because of some code he pushed last night. So again, this is showing you the agility that each developer is now empowered to not just build their applications, but also push it to production as needed, and then making sure the infrastructure, regardless of the environment, supports those requirements. So I can develop something locally and run it on my test dev clusters, and then
00:21:17
Speaker
if I have the same Kubernetes cluster running in production, I can just move my applications there and promote them to the production tier. And subsequently break production, but that's a different topic. Like the only reason I found out that he pushed something into production because his slack was going crazy and his production was down for a specific feature. So yeah, definitely it comes with more responsibilities as well.
00:21:42
Speaker
Yeah, 100%. And I think, you know, in the storage space, right, when we talk about IT administration, storage administration, you know, all these abstractions that we're talking about from traditional storage and compute to moving towards VMs and driving that, you know, that storage admin is still a
Adapting Traditional Storage for Containers
00:22:01
Speaker
vital part of it. But then now we're moving towards containers. And that's yet another abstraction of the VM. So now we're abstracted from
00:22:08
Speaker
bare metal to the VM or striking the VM into these containers that carve everything up and it yet demands more, right? And so I think there's sort of this evolution of storage that's needed. And that's why we're seeing this transformation of
00:22:24
Speaker
of the sand system and naz systems as something needs to either change or we need to be able to use these things in a way that's not going to hinder us or develop something new so i think that's where it comes into okay if i already have a sand and if i already have a naz and i'm moving forward with my application developer.
00:22:45
Speaker
developers and we have a DevOps movement and we're taking on these abstractions, we need to be able to plug in my new orchestration for containers into these SAN and NAS systems. It almost sounds like you need a connector between the two, right?
00:23:02
Speaker
You do, you need a connector, you need a plugin. I mean, I think it's anonymous there in terms of what it's doing, but essentially you're connecting a container system, whether that's orchestrated or Kubernetes to your SAN. And I think there's a number of challenges there.
00:23:16
Speaker
Yeah, and to start with challenges, it starts from the installation process. So I might have a storage system or SAN or NAS system that I've been using for a few years now. But to adopt it and present storage to my Kubernetes orchestration system, I have to make sure that I have a Kubernetes cluster with worker nodes. I have to install the correct NFS and ISCSI utilities on those worker nodes. Going back to the storage array, I have to make sure that I configure
00:23:45
Speaker
whatever multi-tenant name that they have, configure it. If it's an NFS backed storage, I have to configure export policies, make sure that all my worker nodes have like the IPs that they are using are listed in the export policy. So eventually when my developer asks for a volume.
00:24:02
Speaker
and a persistent volume backed by an NFS volume is provision. It is able to be mounted on the worker nodes so my apps can use it. Even though the connector sounds easy and because of the orchestration frontend, it feels like my volumes are getting dynamically provisioned, it's a really tedious process on getting things up and running. Making that connection between
00:24:24
Speaker
something that's modern like Kubernetes and a traditional SAN or NAS system is really difficult. It involves a lot of hand-holding and administrative overhead. Yeah. There's a lot of different vendors that do take that approach. Personally, I worked for a company called ClusterHQ, which was literally a product around plugins.
00:24:49
Speaker
It was essentially a giant library of different plugins for various vendors that could do all these provisioning and attaching for Docker containers. It was pretty early in the movement towards containers, but that's a prime example of we had a lot of logic in all those drivers, those plugins that understood
00:25:12
Speaker
Specifically how that's you know specific san or naz did sort of scanning and attaching and failure modes and and and those kind of things and interoperability that's a lot yeah a lot to manage and it's too much but today we see a lot of different connector approaches from you know trident from net app
00:25:33
Speaker
to Amazon EBS has its own plugin in Kubernetes like other clouds Azure disk and GCE persistent disk to other large vendors like HP and Nutanix.
00:25:43
Speaker
Yeah, like at my previous job at Lenovo, we had an HCI offering with Nutanix and VMware. And again, those systems were built for virtual machines. But then because of this connected model, they could present their storage using volume plugins. So Nutanix has a Kubernetes volume plugin. And VMware definitely has the cloud native storage plugin, which allows developers to consume storage from that one single data store by provisioning these virtual disks and mounting them onto the worker nodes.
00:26:12
Speaker
So that's something that, again, every vendor has an implementation for Kubernetes, but it just differs on how they've actually configured or developed that implementation.
00:26:23
Speaker
Yeah, and I think the important thing to note here is like for many use cases, it can be absolutely good enough and what you need, right? It feels a little bit like a middle ground, like we're adapting traditional SAN and NAS to work with Kubernetes rather than making something that was built for Kubernetes. And I think that's a distinction. When we get closer to this, what is cloud native storage topic is, you know, some things feel adapted and some things feel purpose built, right?
00:26:51
Speaker
Yeah, definitely so like you can you can buy a thing off the shelf that was built for like I can buy a DSLR camera mirrorless camera and get the best performance of out of it like so going back to my Acadia trip right I was sitting on on a sand on the sand beach there and looking at the night sky if I had a DSLR which was purpose built
00:27:12
Speaker
I could have captured the Milky Way, but then I tried pulling out my iPhone, which is again, which is great at what it does, but it's still adapted like to for the camera functionality that was not its main purpose. I could see a black sky that said it didn't capture any stars. So it's all about like whether it's built for the thing that it's supposed to do or it's just trying to fill a gap.
00:27:33
Speaker
Yeah, a hundred percent. I mean, I could have duct taped my old school video camera onto my motorcycle for my trip. But, you know, GoPro makes a really good small camera built for that job. I totally understand that. I think it's I think it's a vital distinction when we talk about cloud native versus traditional and not to say that like one is necessarily better or the other. It just serves a different purpose. Right. And you have to understand what your needs are from that organization.
00:28:00
Speaker
You know what what might be some other you know concerns when you're thinking about something that's adapted versus purpose built.
00:28:08
Speaker
If you're talking about adapted, just having the brand new or the latest and greatest feature set requires updates at not just the connector level. My plugin needs to be updated, but I also need to make sure that my back-end storage is capable of providing those functionalities. If I need NVMe storage and I have a traditional or a hard drive-based storage, I'm not going to magically get a faster storage tier if I'm using an adapter storage.
00:28:37
Speaker
If I want a better deduplication, I need to update my plugin and I need to upgrade my back-end storage system as well. So it's just delays that are caused because of this multi-tier functionality or multi-tier architecture and just not getting the latest and greatest features that purpose-built Windows can provide you.
Defining Cloud-Native Storage
00:28:58
Speaker
Yeah, and I think this is it's probably a good lead in to open the topic of what exactly is cloud native storage. And for that matter, what's cloud native, right? And I and the first thing that always comes to mind for me is a it's purpose built, we covered that already. But what does that mean, right? So I have a cloud native infrastructure, and this is likely Kubernetes. I think
00:29:21
Speaker
having this storage management software be able to be dynamically deployed, I think deployment installation is a huge part of it, meaning that I should be able to install my cloud native storage as any other application on the Kubernetes if I wanted to.
00:29:39
Speaker
Meaning that deployment and management is a huge part of this in the sense that it should feel Kubernetes native or at least container native. When I mean that, it should either come to me as a container, be deployed as a container, or be managed as another container application. Wouldn't you say?
00:29:58
Speaker
Yep, and then having or meeting those requirements definitely adds its old set of features, right? Like OK, let's say I started with a three node cluster. I need more storage. What do I do? I can just add more worker nodes and since these these cloud native storage systems are deployed as demon sets, it automatically gets deploys a new pod on that new node provisions back in storage for it and then increases my storage capacity. So that's another benefit that you get. You get that agility when you're dealing with cloud native storage.
00:30:28
Speaker
go ahead and manually provision things and include those in your cluster. A lot of it is automated. This is just like the ease of use point that you mentioned earlier. Yeah, absolutely. And there's, you know, cloud native is a loaded term in general. So I think to me, it really means taking advantage of sort of these modern technologies and containers and Kubernetes. But also when you slap the storage on it, there's the concept of, okay, what do we care about? We care about the application a lot more in Kubernetes.
00:30:56
Speaker
And so I think that's a distinction with cloud native storage that, you know, traditional storage, typically the focus from the storage admin's perspective is all about, you know, setup and configuration and then provisioning a LUN and taking a snapshot of a LUN and making sure data is offloaded to here and data, data volume, volume, LUN, LUN.
00:31:14
Speaker
There's nothing as you don't see the application structure come in nearly as much as most of these sort of container focus cloud native storage. They take the whole view of the application. Snapshots have to be application centric, but also when we move data around, we also consider all the pieces that Kubernetes holds about the application.
00:31:34
Speaker
True, like application is the king, right? One of the examples that again anecdotes, I was talking to another friend of mine who's a developer and he uses AWS to run their infrastructure, their production workloads. And I asked him, okay, what do you do for storage? And he had to think for a bit and he said, maybe EBS
00:31:54
Speaker
Or maybe EFS because he wasn't sure like he focused on the application. He knew what his application needed and he just specified the comments. He didn't really care about the back end storage. So any cloud native storage system definitely needs to have that focus and be more application centric.
00:32:10
Speaker
Yeah, and in Kubernetes, we know that many teams are provisioned either an entire Kubernetes cluster and or a namespace and they're completely self-service, right? They create their applications based on YAML or Helm or something like that. And that just ties in one single string that defines like, here's the storage you're allowed to have. So it should just feel completely seamless, right, to the end user.
00:32:34
Speaker
One of the things that we have to clarify is even though Amazon EBS is cloud storage, it's not really cloud native storage. It was built for virtual machines. It was built for EC2 instances. Even though you can use EBS for your EKS clusters, it still falls under that connector category where it's not able to expand as needed or provide that multi-zone functionality that exists in cloud native storage vendors.
00:33:02
Speaker
Yeah, so the lesson learned there is that because it has the word cloud in it doesn't make it cloud native, right? Yeah, I mean, we've obviously seen that with EBS. It can be slow. It doesn't allow you to detach and reattach really easy class availability zones.
00:33:18
Speaker
even though it's all very easy to use in cloud native. So that's a really good point. So what types of cloud native storage are out there today? So again, there have been so many different terms. There is CAS, container attached storage, there's CNS, that's container native storage, that's KNS, that's Kubernetes native storage.
00:33:41
Speaker
You definitely need to know what these different terms mean because it's like finding out later in your life that iPhone and iPod literally meant internet phone or internet port. So I stood for internet. Like I've been using iPhone for so long, but I don't think I knew it when I bought my first iPhone. So you need to understand these different terms. I'm not sure if I knew that today. Like I might've learned something new today. Nice. That's one key takeaway right there.
00:34:05
Speaker
Yeah, I think CAS, if you're in the storage industry, you've heard CAS with content addressable storage. And that's not what we're talking about at all. This is container attached storage, which is really the concept of your container software being deployed as a container natively onto a container orchestration solution. So I believe OpenEBS, Kubera is the product now, I think.
00:34:32
Speaker
you know really push this term in the industry and it's used a lot. I would say CNS cloud native storage and container native storage are pretty much the same as synonymous. I think more so we see it used as cloud native storage today. And that's completely different from VMware CNS which is their CSI plugin.
00:34:52
Speaker
Yeah. I think they, they probably, you know, bandwagon the cloud native storage like, like a lot of us are doing today. Um, and I, and I'd say Kubernetes native storage is probably the least common, but, uh, you hear it and essentially it's CNS, right? Um, we can, if you're talking really only underneath the Kubernetes, uh, storage space. So, you know, some of the, some of the projects and products out there, um, you know, which ones are we talking about?
00:35:19
Speaker
So definitely we can start with Portworx because that's where both of us work. But then we have other vendors in the ecosystem as well with StorageOS and OpenEBS, as you mentioned. There's robin.io. There's even Longhorn that we referred to earlier in the podcast. So there are a bunch of vendors in the ecosystem. And then again, you have to compare features to decide which one fits the best for the use case that you have.
00:35:43
Speaker
Yeah, and I think at the core here, the ones you just mentioned, they sort of fall directly under that CNS label because for the most part, they've been purpose built for containers. And I think the distinction there is that when you look at cloud native storage vendors today, or even look up articles that talk about them, you'll see others in there too, such as Rook,
00:36:07
Speaker
and Ceph or essentially which is the product project under OpenShift container storage and even NetApp Trident or Linbit and even VMware vSAN. And I think the distinction there is that while they weren't necessarily purpose built for Kubernetes, they've been adapted enough to be put under that category and essentially provide enough of the features where they're sort of considered in that space.
00:36:35
Speaker
Yeah, but one of the requirements that you mentioned earlier when we were talking about cloud native storage is you need or you should have a storage solution that's deployed using the same orchestration system as well. So a system that's not just built for containers, but which is also built using containers that runs on the same Kubernetes cluster.
00:36:55
Speaker
alongside their applications and providing different storage capabilities. Yeah. In full circle, that's driven by the abstractions, right? Again, once again, we're seeing a lot of these changes driven by the abstractions that are necessary within IT in general from compute to VMs to containers. And when you're built for that abstraction, you can deploy anywhere. There's a components cluster, right? And that's one of the benefits for
00:37:20
Speaker
Kubernetes native storage, I'll put it. One of the things that again, I was listening to a different podcast a couple of months back and it was actually a company called Flexport and they handle like a real world shipping. So if you are shipping or if you're ordering something on Amazon and your stuff gets shipped from China on those huge container ships, we all know the
00:37:43
Speaker
the metaphors or the comparisons between containers and Kubernetes and containers in the real world shipping industry. But then one of the interesting thing that his company is doing was, you don't really know what's packed inside the container. So if I packed a bunch of say, glassware from China and then on the other side, I'm not ready with the correct crane or whatever it's needed to unload that container, that's going to cause delays and that's going to cause me
00:38:12
Speaker
difficulties, but having the same storage, same cloud native storage or container or Kubernetes native storage at both ends. So if I'm, again, going back to the multi-cloud world, right? If I'm running something on-prem, I'm moving my application as containers, that's all good. But then if I don't have a storage system that's running in the same cloud or that's running in the same
00:38:36
Speaker
Ecosystem and providing me the same set of feature set I might not be able to port or migrate my applications and end up in trouble or end up with delay so I need a system that's consistent across all the different cloud vendors are across all the different communities clusters. I don't know if that helps but it just sounded interesting to me and i was able to type back.
00:38:55
Speaker
No, I think it actually plays a part in it, right? Well, I think we're going to have a topic up here on the podcast in a few weeks that's all about how to run and scale Kubernetes cluster as a team of one. And I think part of that, I won't spoil it, but part of that is really around having all those standard APIs, not just the fact that your infrastructure and things can scale, but that you have a seamless and consistent way to interact with it.
00:39:21
Speaker
And so I think that's kind of like, you know, also kind of hits the head nail on the head there with what you were saying.
00:39:28
Speaker
What do you think? Did we do the topic of cloud native storage versus traditional storage justice?
Storage Evolution Recap
00:39:33
Speaker
Should we wrap it up? Yeah, let's wrap it up. I think we did what we could in the 30 minute time frame. Let's wait for some feedback from the users. So what are the three key takeaways or as I like to call them, like TLDL, like too long, didn't listen. Again, I've been known to like zone out when I'm listening to podcasts, but let's wrap it up. Yeah.
00:39:55
Speaker
Sure. I think for me, it's definitely that we've come a long way in the enterprise that follows those abstractions. So traditional SAN and NAS built for bare metal servers and VMs. We saw the abstractions towards VMs. And so we naturally had more automation. We've had more cloud involvement. And so we needed more of that automation.
00:40:21
Speaker
And then we saw another abstraction, which is VMs into containers. And that new abstraction demanded a new set of problems, which was higher scale, lower container lifecycles, which meant that storage needed to act differently and provide those different things.
00:40:40
Speaker
So that evolved into how do we adapt our existing technology or existing SAN and NAS systems into something we can use and how do we create something new, which is Cloud Native Storage. I think if you were to understand that takeaway, that's the evolution of it. Then beyond that is what do you get from these Cloud Native Storage vendors, which is the app-centric view of storage and application world dynamically deployed across
00:41:10
Speaker
sort of any infrastructure when you have containers and Kubernetes. And that's a huge sort of efficiency gain for storage if you're coming from traditionally managing a SAN.
00:41:22
Speaker
Yeah, if you are an organization thinking about containers or Kubernetes, you need to understand the difference between a connector based on a cloud native storage, understand the requirements that you have from your application, look at the infrastructure that you already own, and then make the best decision for your applications or for you as a business and move forward with Kubernetes.
00:41:44
Speaker
Awesome.
Closing and Upcoming Episode Teaser
00:41:45
Speaker
Well, I think that wraps it up. A quick reminder here that wherever you listen to our podcasts, please, if you can either send a message or review us, absolutely do that. We'd love to get your feedback. Also, I believe the next episode, if you're interested, is data management on various Kubernetes platforms. So we'll be talking about the specifics of data management and persistence on things like Rancher and OpenShift and Anthos and EKS.
00:42:12
Speaker
Stay tuned for that. Isn't that our first episode where we might have a guest? We should be. Our guest is a hard maybe right now, but yeah, we're going to start having some guests on here so we won't just be listening to Bobbin and I the entire time. Awesome. Perfect. That's a wrap. That's a wrap. Until next time, everyone, take care and stay safe. Thank you for listening to the Kubernetes Bites Podcast.