Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Kubernetes Security Posture Management with Mondoo
482 Plays1 year ago
In this episode, Ryan and Bhavin talk to Christoph Hartmann - the CTO and co-founder of Mondoo about all things Kubernetes Security. The discussion starts by talking about Kubernetes Security Posture Management (KSPM) and then dives into Software Bill Of Materials (SBOM) and SBOM Attestation. They also talk about the open-source project "cnspec" and how it can help organizations scan their entire infrastructure, including VMs, containers, container registries, code repositories, etc for vulnerabilities.
News:
- https://www.weave.works/blog/weaveworks-donates-project-kured-to-the-cncf
- https://sysdig.com/blog/top-15-kubectl-plugins-for-security-engineers/
- https://techcrunch.com/2023/01/25/dell-has-acquired-cloud-orchestration-startup-cloudify-sources-tell-us-for-around-100m/
- https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/cncf-hosted-co-located-events/about/
- https://techcrunch.com/2023/01/26/mirantis-acquires-shipa/
- https://www.harness.io/blog/harness-acquires-propelo
Show Links:
1. cnspec Project Pagehttps://cnspec.io
2. cnspec on Github: https://github.com/mondoohq/cnspec
3. Hacking Kubernetes https://www.youtube.com/watch?v=9onasWkaeuE&t=3s
4. Mondoo https://mondoo.com/
Recommended
Transcript
Introduction to Kubernetes Bites Podcast
00:00:03
Speaker
You are listening to Kubernetes Bites, a podcast bringing you the latest from the world of cloud native data management. My name is Ryan Walner and I'm joined by Bob and Shaw coming to you from Boston, Massachusetts. We'll be sharing our thoughts on recent cloud native news and talking to industry experts about their experiences and challenges managing the wealth of data in today's cloud native ecosystem.
Casual Chat: Boston Weather and Personal Hobbies
00:00:28
Speaker
Good morning, good afternoon, and good evening wherever you are. We're coming to you from Boston, Massachusetts. Today is January 27th, 2023. Hope everyone is doing well and staying safe. Bobbin, let's dive into it. How you doing?
00:00:45
Speaker
I'm doing good. It's snowed a couple of times since we last spoke. Finally. Yeah, but I had to shovel. So I don't know that it wasn't as bad like it wasn't a big storm. So at least in Arlington. This is like the most mild Boston winter I've ever been. Oh yeah, for sure. I couldn't be happier, you know.
00:01:04
Speaker
I was contemplating in November or October, should I buy a snowblower or should I not? Yeah, good thing you didn't. That would have been just sitting in the garage. Now what? Now it'll dump two feet of snow several times in March or something weird for you. Don't worry. Mother Nature has her ways to get back a Bostonian New Englander.
00:01:26
Speaker
Anyway, what else have you been up to? No, I'm just excited with the football season. Like I know we're coming to an end. Last week was divisionals. This week is conference championships. Those games have been awesome. That's my weekend wins always. I forgot to watch any of it. It started already? Yeah.
00:01:41
Speaker
You should really watch the games this week and some good games this weekend. But yeah, enough about me. Talk about your noodle tour. What was going on there? Noodle tour, yes. Well, as you know, a few weeks ago, I won't say which day, but I celebrated a birthday. I'm one year older. Very specific.
00:02:04
Speaker
But like we do on our conferences, you, Tim and I, whoever we can drag along, go to a good noodle spot in whatever city the conference is in. It's always a lot of fun. If you don't do that, I recommend it, especially if you like noodles. Go check it out. Ask us for a list if you'd like some. We'll give you our list. But for my birthday, I picked a few spots in Worcester, Massachusetts.
00:02:31
Speaker
that I have not been to, and there have been in my list. One was a really good Vietnamese place called Saigon House, and another one was Char Su.
00:02:43
Speaker
were Japanese. And yeah, just went to I mean, I envisioned that I could eat more than just two places. But after I got done with the second place, I was like, Nope, Nope, definitely could have done more than two places. And I ate modestly at both because I wanted to be able to eat both. Although the sake at the second place was very good. You got to pick your own cup and everything. It was
00:03:06
Speaker
It was delicious. Nice. That sounds like a body well spent. Okay. Yeah. One of the, one of the better ones can't complain.
00:03:17
Speaker
Anyway, so have any fun plans or you want to dive into the news?
WeWorks Donates Cured to CNCF
00:03:23
Speaker
Yeah, let's do news. My plans for the weekend is just watching NFL football. So I'd say that's pretty good. Cool. Why don't you start us off? Yeah. So a few things, right? We have a couple of articles that I wanted to talk about and three different acquisition news.
00:03:38
Speaker
Let's start with the articles. One of the things that came across was WeWorks donated their third project to CNCF now. The project is called Cured with a K and this is the third after they donated Cortex and Flux already.
Essential kubectl Plugins for Security Engineers
00:03:52
Speaker
Cured was originally I think started back in 2017 but it's not a fancy tool but it's
00:03:58
Speaker
more of like a reboot daemon, right? That performs safe automatic node reboot. So what it does is it continuously checks for the reboot sentinel file in slash where done reboot required. And if that exists, it works with the API server, locks it in so that only one node of your cluster reboots at a time. It also cordon's and drains worker nodes before the reboot and uncordon's them after. So like it
00:04:20
Speaker
makes it a safe reboot instead of just pulling the plug and restarting the node and maybe losing some data or your applications going down. That was a good tool that's available and now part of the CNCF ecosystem. Awesome.
00:04:34
Speaker
Yeah. And then the second link, I found it really helpful and very relevant to the topic today. Top 15 kubectl plugins for security engineers. Again, they are good tools. I won't list all 15. But a few that caught my eyes were like stern. So stern basically is a plugin that will help you do tail dash F. So if you if you do a kubectl
00:04:56
Speaker
Yeah, if you do a kubectl stern app name, it basically gives you the logs of that app, which basically matches the name of your pod. So that's a cool tool. Policy Advisor, it suggests you pod security policies and OPA policies for your Kubernetes cluster. So once you run that, it will identify things where you need to have policies in place and help you apply those. Cube Login,
00:05:20
Speaker
I didn't know this was a need, but this looks cool. It allows you to allow you to authenticate again for your CLI sessions using your OIDC provider. So like if you have an OIDC provider from CLI, you don't really have that integration. So if you install this plugin and if you run it for the first time, it will open up a browser session where you can log in and then it authenticates your CLI session as well. So that was a cool tool.
00:05:42
Speaker
And then a final one that like out of the 15 that I wanted to talk about was case sniff. It's for capturing and analyzing network traffic. I know we have used Wireshark in the past a lot. So this I think gives a similar flavor of tool for the Kubernetes cluster. So read through the list, we'll have the link in the show notes, some cool looking plugins for kubectl. Wireshark, TCP dump, basically into kubectl.
00:06:10
Speaker
Love it. Yup. And then talking
Industry Acquisitions: MyRantis, Harness, and Dell
00:06:13
Speaker
about acquisitions, right? I'll start with a couple of ones that I want to talk about and then I'll hand it off to you. The first one being my Rantis acquired a startup called Shipa. So Shipa used to be in the cloud native delivery and application delivery ecosystem. So now they and they had raised around
00:06:31
Speaker
$3.75 million of seed funding in 2020. And the acquisition amount, again, wasn't disclosed at an exact amount, but it is assumed to be in between $10 million and $30 million. So I would say pretty good return. And I think from reading a couple of articles, MyRantis is planning on integrating Shippa with its Lens platform to accelerate the cloud native application delivery pipelines and also integrate it with their MyRantis Kubernetes Engine service that they have.
00:06:59
Speaker
So a good exit for Shiba and some good, cool technology for MyRant is to integrate into its own products.
00:07:08
Speaker
The second one that I had for today was a startup, again, not a big company, but a cloud-native startup called Harness, acquiring another cloud-native startup called Propello. So this, the Propello had, like, was a smaller startup that had raised, I think, $16 million across their Angel Seed and Series A round. So it was not a big thing. I think the number of employees, if I looked up, was like 51 or 53.
00:07:34
Speaker
Not a big company, but it adds a capability to the harness software and harness platform as the eighth module. They like to call it the harness software engineering insight module, but it basically helps organizations. One of the customers listed on their website being Broadcom to.
00:07:50
Speaker
make their developers more efficient, right? So obviously, with 2023, the way things are, we obviously want to make sure all of our resources are more productive. But this, I think, basically fits into Hardness's ecosystem really well and helps them go or move up from, I think they were a $3.7 billion valuation company to even higher now in the future. But yeah, that's a quick list from me.
00:08:14
Speaker
Yeah, absolutely. One to add to the list, right, is Cloudify is acquired by Dell. So, basically, if you're familiar with Cloudify, the infrastructure as code, automation, those kinds of things, it makes a lot of sense for Dell. I mean, I worked there, I won't say what I know, but, you know, what, what Dell's doing and especially instead of their Apex and cloud journey.
00:08:37
Speaker
for their customers. I think it's a great acquisition. We'll know more where it surfaces over time, obviously, and the amount was around 100 million.
KOLO Events Preview for KubeCon EU
00:08:48
Speaker
The exact number publicly and really exciting stuff over there.
00:08:53
Speaker
There, another Israeli company, actually, Dell and EMC, I feel like they do a lot of acquisitions from them, to be honest. And then the only other one I had here for my news is the KOLO events were officially announced for KubeCon EU. So if you're not familiar with what the KOLO events are, if you're going to KubeCon EU, these are events that typically take place a day or two before the official KubeCon conference and focus
00:09:21
Speaker
more into specific communities, right? Whether it's CICD or security, Wasm, Edge, all those things. There's usually a specific day for the thing you're interested in. And I highly recommend if you can get there early to attend one of these, the organizers often, you know, it's a closer knit type of community. The day is a very well spent kind of diving into the different technologies. If you're a technologist or just there to kind of consume and learn
00:09:51
Speaker
Definitely go check these out. There's usually more that get added that aren't officially part of the CNCF. Companies and vendors add some as well. I don't think all those are announced yet, but these are from the CNCF. Check it out. We'll put the link in the show notes.
00:10:09
Speaker
And usually they have so many day zero events running simultaneously. So like you have to actually pick and choose like, okay, what am I most interested in? But I don't want to lose out on the other thing. Yeah. But don't worry. Like even if you attend your most interesting thing, everything else gets posted on YouTube. Like I know for the service mesh episode that we did Ryan, I basically went and like watched 75% of the sessions that were part of the day zero service mesh con event at KubeCon Detroit. So like,
00:10:34
Speaker
Content will be available so they don't have a phone around it, but it is really valuable to attend in person if you can. Yeah, absolutely. And we were part of DOK Day, which was a day zero one, and the entire nine hour, 10 hour day was on YouTube, all in one video. So that was an interesting way. But you can always go back and just literally go through and have another day within your day. Yep.
Interview with Christoph Hartmann on Kubernetes Security
00:10:58
Speaker
Our guest today is Christoph Hartmann, CTO and co-founder of Mandu. He's here to talk about all things Kubernetes and security and security posture management. So without further ado, let's get him on the show. Well, Christoph, welcome to the show. Welcome to Kubernetes Bites. I know it's been quite a time coming to get you on the show. Why don't you introduce yourself and let everyone know what you're up to. Thank you. Thank you, Ryan. Thank you, Robin, for having me.
00:11:26
Speaker
at this podcast super excited. I'm Chris, co-founder and CTO at Mondo. And my key mission is to make the work a little bit more secure and giving people the tooling to essentially make this happen.
00:11:40
Speaker
My career really started, I think it's nearly a decade ago, at Deutsche Telekom, which is parent of the mobile. And back in the days, we really started in how do we build out cloud infrastructure with OpenStack back in the days, like this was the thing. And it really drove the old security models over the
00:12:04
Speaker
owner because it really hasn't worked right and so the question was like how do you secure such a critical infrastructure like tablet communication in a really highly ephemeral world and it was really really early and what we've built and Dominic the other co-founder at
00:12:21
Speaker
at Mondo. We together worked there and we built out a project called DevSec.io and that was the first server hardening framework built all around Chef, Puppet, and Ansible. Back in the days, people still use those things to secure infrastructure. And it's really important to figure out how can security be rolled out automatically. And we had a huge learning of that and that learning was that
00:12:51
Speaker
It's not really about fixing the problem. It's really helping people understand what the problem is in the first place. And then automation plays a key role to do this continuously. But it's all about giving the knowledge to people so that they know what to do and they can make better decisions. Based on that experience, Dominic and I started the company called VolcanoSec.
00:13:16
Speaker
Volcanosake built a very popular open source tool, InSpec, which was the first policy code engine that we made really, really successful. Chef software acquired our company and we made this essentially the biggest part of Chef to
00:13:37
Speaker
like get deals because like a large enterprise is used at Fortune 500. If you look it up in the internet, like MITRE, the research organization of the US government used it, NSA used it. So it's essentially, we have seen a lot of compliance requirements in the world, like used by large corporations, but also governments and how they actually want to go in this fully automated paths.
00:14:05
Speaker
And that really led to the point where we really learned quick how to do automation for security. But as time flies by, Dom and I left, essentially, Chef, because it was not missing some of the new attack like Kubernetes. And we always believed in Kubernetes as a thing, and we should build a security product around that.
00:14:30
Speaker
And so, yeah, that was essentially the cornerstone for starting Mondo, like really building a security product around automation, around security, but built for platform engineers and security engineers.
00:14:44
Speaker
Okay, and that's awesome, right? I think helping organizations build these secure environments or like, I want to ask my next question, improve your security posture is a really great thing to do for the community, right? So I know Ryan and I did a Community Security 101 podcast, it's one of our
00:15:04
Speaker
I think top three episodes at this point. It is. So then that was just you and me, right? As we spoke about in that podcast, it was a mile wide and just an inch deep, maybe. So that's why we're like, okay, let's get get let's get Chris on the board. And let's start talking about and going into some details around Kubernetes security. So let's start, right, Chris? Let's start by talking about what is Kubernetes security posture management or KSPM for short, I know,
00:15:28
Speaker
People have been familiar with CSPM and cloud security posture management. How have things evolved and what does this mean for organizations? Yeah, so first of all, I think we should talk a little bit about security posture management in general and then dive into Kubernetes specifics. So security posture management essentially really helps you identifying what is the risk in your infrastructure. And if you want to break it down, it is essentially like two things.
00:15:56
Speaker
It's what is, where are your biggest vulnerabilities? So essentially it's everything patched. And the other problem is like, is everything configured properly? So finding misconfigurations. So, so this is like the, the short version of what it is, of course, like their nuances, but I think this, this is security posture management. And if you apply this to Kubernetes,
00:16:17
Speaker
it quickly becomes complicated because in Kubernetes land, Kubernetes is a really amazing technology to horizontally scale your infrastructure. And that naturally involves a lot of different layers of technology to make that happen. And if we just look at the most frequently used cloud providers that
00:16:41
Speaker
and it's like Amazon has EKS and it's also like the most frequently used Kubernetes environment, the same continuous with GKE and AKS, right? And so now you're not having like a Kubernetes security posture management for the cluster only, you also have it for the cloud. So it really starts with the cloud account, then goes to the cluster. And even though in Kubernetes, like you luckily don't have to deal with the management node,
00:17:10
Speaker
But we still, the nodes need to be taken care of and we see some really good improvements where cloud providers go to really container OS or AWS is using bottle rock, hopefully more and more going forward. So to secure those nodes. And then it comes to cluster misconfigurations, workloads, but also applications. And the challenging thing like using it in clouds is
00:17:39
Speaker
if you use IM stuff, essentially, if you misconfigure, it can happen that from your part with a wrong mapped service account, you can break out Kubernetes completely, and you have too much access to the cloud. So that's why it's so important to not just see security posture as an isolated piece, but really add all technology layers that you essentially use. And that's really key for
00:18:06
Speaker
or like when we talk with customers and users, because we want to explain them, hey, there are all those layers, you need to understand them, and then you can prioritize the issues properly.
00:18:20
Speaker
Yeah, as great as Kubernetes is, to your point, it does add a whole other level of complexity. I don't think we've said to someone that it adds complexity and they've disagreed with that. But I can imagine from the security standpoint, that's all, like you said, a whole other layer on top of CS. Maybe it should be CKSPM.
00:18:42
Speaker
Yeah, we call this XSPM, Extensible Security Posture Management, for the reason that essentially nowadays you can't really restrict this to any technology anymore. A couple of years ago, it was really like you use a tool for X and a tool for Y.
00:19:01
Speaker
But what we believe in is that engineers should use the right technology for the right job they're trying to do. And that doesn't mean that you reduce it to a few different tasks, like if serverless is great, like your serverless is container grade. Sometimes you still want to have a VM with GPU attached directly. So there's always a good reason. And I think this is what we also need to acknowledge as the industry. There's not one right thing. There are many. And we need to make sure that
00:19:32
Speaker
The tooling that we use in our ecosystem is essentially supporting users in a way that they can do their job. And it's not limiting them and their technology choice to do the work.
Addressing Kubernetes Misconfigurations
00:19:43
Speaker
Security tools are traditionally doing a really bad job at it. We need to say this. They normally are perceived as the blocking piece of things. If you have platform engineers and you should... We have done many therapy sessions with them. That was always the biggest thing. Security is blocking me. It's not helping me. I can't integrate this in my pipeline. It's pain, pain, pain. I think this is
00:20:12
Speaker
This is what the whole security industry needs to change.
00:20:16
Speaker
Oh, yeah. I imagine the amount of times a developer has run into a security thing and said, well, how do we disable it so I can get past it, right? You know, like, oh, SELinux is getting in your way. Disable it, right? You know, like, I know that happens quite a bit. And I, you know, to what you said before around automating, you know, security, I know that security has often come up as sort of a tertiary or secondary thing to functionality.
00:20:43
Speaker
And fundamentally, I think, as an industry, and the more we see all these, you know, you know, cybersecurity issues in the industry or in the world, right, it has to become more prevalent. And it starts with, you know, from the get go, we need to think about this stuff. And the best way to do that is to take it out of people's hands and automate it. So I'm all for that. You know, speaking of which, you know, people in general are, you know,
00:21:09
Speaker
Foulable they make mistakes a misconfigure things And that's that's fine. We're human but so I'm curious what types of misconfigurations you come across to Maybe in your day-to-day or with your customers or that, you know, you generally see as a community and in this industry
00:21:28
Speaker
Yeah, very, very good question. I mean, just saying this up front, there are like more nuanced things. And I just like highlight the biggest issues here. Sure, yeah. Which are the biggest problems that we see with our users. In Kubernetes, definitely biggest problem is like service accounts. Like service accounts mounted into parts where they shouldn't. They have default permissions. Like if you run workloads that don't need any access to Kubernetes API, like why?
00:21:57
Speaker
So that's definitely something it's an easy win, but it's also helping you to prevent accidental pass-through attacks. And so it's simple check, but still very, very often. Privileged containers are still too often being used. It's sometimes even
00:22:21
Speaker
security tools like force you to do that. And like, I recommend you like storage tools do too. And we're very familiar. So essentially, like you should always like try to avoid this as much as possible, like to the minimal amount. If you do that, you need to have double watch at those things. Because like there could be a good good reason for it. Right. But like,
00:22:43
Speaker
In most cases, for most workloads, you most likely don't need that. And it's really important to understand that because once you have a privileged container, you can, and we do demos around this quite often, you can have a malicious container. It could be a standard container just with a vulnerability. You break out of it. We demonstrate this with the latest patch like EKS. You break out of a container.
00:23:10
Speaker
come to the node and take over the whole node, and then also take over the whole cloud account. So this is possible with simple misconfiguration, but the good thing is Kubernetes has the right primitives to essentially secure that. It's not that the system is inherently insecure, it's more like it's wrongly configured, and the security features that are built in are just not enabled, and that's why this can happen.
00:23:36
Speaker
The next biggest issue is all about IAM permissions, RBAC. This is too much permissions. It's always a problem. You always want to go with least privilege. It's really hard to do it right. A blanket policy. You check the box, right?
00:23:52
Speaker
Yeah, but it's really hard. Let's be real. You need to test your application properly. You need to make sure that you cover all the use cases. What we recommend is really build a fully automated application pipeline.
00:24:08
Speaker
like frameworks like Salsa help you, guiding you through that, but essentially like help you to not do like click ops instead of like automated like get offs. And that helps you also to identify those things. You can fix things easier. You can test things easier. So I think there are many benefits in doing the automation. It helps you also with the authentication authorization issues.
00:24:32
Speaker
So since these are like common misconfigurations, they're also like low hanging fruits. Are there easy ways to like identify these in your infrastructure stack or in your applications and then easy ways to fix this or how do we go about it?
00:24:47
Speaker
No, it's totally easy. In Kubernetes, you have the security context. It's just manifest annotations, essentially, or make sure those annotations are not there. So that's why I'm saying it's not that Kubernetes inherently is insecure, that you should be worried about using Kubernetes. I think Kubernetes has really, really good primitives. The problem is that the primitives are
00:25:11
Speaker
deactivated or just not properly activated. And this is what we call about this configuration. I don't believe that the tools itself we use are insecure. And we see a lot of things happen in the industry, like GY is our fire corrector. This gets more and more secure, but it needs to be enabled. It needs to be used. And that's the tricky point. Yeah. Yeah. I imagine Iam and Arback you mentioned before,
00:25:40
Speaker
have similar problems in the sense that a lot of times people will get an IM policy that works, but not really understand or realize they're probably having too much permission than they actually need to get the thing working. I know I've been guilty of this myself. I'm like, I checked that box, I'm thinking about security, I've built an IM policy, and my application's happy, but really, I just have too much going on. I can actually dial
00:26:04
Speaker
And I think having the understanding of how do you find out where that sweet spot is, right? Is a challenge. It's a challenge that it also depends on the attack factors. Like if you run your own application to know what your own application is, you have a better understanding of what you need. But if you run like untrusted, like external code,
00:26:24
Speaker
Um, then like an arbitrary get pipeline, like sure. Um, so then you need to make extra care,
Shared Security Models in Managed Kubernetes
00:26:33
Speaker
right? Like you need to make sure that those, those, um, things that run it have are really protected. Otherwise it's a supply chain attacks all over, but it's really, really easy to mostly protect those things, but it needs to be surfaced.
00:26:48
Speaker
Yeah. Gotcha. So I know we spoke about the managed control plane, right? Offerings, we spoke about EKS, AKS, GKE, where users don't get access to those control planes. So it's more of a shared security model as AWS likes to call it, right? Where you are only responsible for your work nodes and
00:27:06
Speaker
your applications. But even with the shared security models, are there any other things that I can make that I can do to protect my cluster overall? Like, how do I make sure that I'm running my applications, assuming the applications are really well built in a secure environment? Like how to are there are there like three things or something like that, right? Like an easy formula to make sure my base cluster is all good to go.
00:27:30
Speaker
So first of all, if you use managed Kubernetes, you need to secure your cloud account. That's the base primitive. You don't think about this immediately because you think you run Kubernetes, but it really starts with the cloud account, narrowing it down, narrowing IAM permissions, make sure this and preferably run one cloud account per cluster if you can, if it makes sense just to isolate them properly.
00:27:59
Speaker
two-factor authentication, all the stuff that applies. It's really the basics that come in. Permissions, permissions, permissions is definitely the biggest one. And then on Kubernetes level, I think it doesn't really matter if you use it on-prem or in the cloud. The master node is mostly protected, but the nodes, you still need to check.
00:28:20
Speaker
workload protection needs to be enabled as well. So coming back to misconfigurations, we just talked about simple manifests. I think it's just to understand those layers, cloud account, then Kubernetes, and then application workloads.
00:28:36
Speaker
Okay. Thank you.
Best Practices for Securing Kubernetes Workloads
00:28:38
Speaker
Absolutely. So, you know, I think let's shift towards, um, the application a little bit more, you know, I know Bhavan and I, you know, covered a little bit of workload security. Last time we, we talked about this and we talked about things like image scanning and patching and, um, but we'd love to hear your opinions on sort of best practices in terms of, you know, the actual workload that's running inside as well. And.
00:29:03
Speaker
And I know I've seen some predictions and read some articles around like the year of the S bomb and Kubernetes and things like that. So yeah, I'd love to get your take on that.
00:29:12
Speaker
Yeah, I think I'm not necessarily a fan of like those like fancy keywords. I really always try to talk with users about like, okay, like what's the biggest problem first? And then let's see how we solve it technically. From my perspective, when we talk about like, when any security, we can generally split this in like management plane, like wherever you have the cluster and
00:29:33
Speaker
cloud account which is one way of securing and now we're focusing on the part that continuously deploy into Kubernetes via hopefully pipelines. And the first step to essentially to secure your things is to know what you are running.
00:29:50
Speaker
So it sounds strange, but essentially, it's just knowing is half of the way. So you essentially need to know what kind of workloads you're running. What kind of containers are you running? Are you running self-built containers? Are you running external containers? How many vulnerabilities have my running things? How many misconfigurations do I have in my Kubernetes manifest understanding this?
00:30:16
Speaker
makes you much better because then you can prioritize, you can hand off work, you can teach and train people so that helps you much. And the next level is then really using mostly industry best practice guidelines. CIS or NSA, they offer
00:30:36
Speaker
really good guidance. They're sometimes overreaching. So still, it needs to be considered based on your requirement because a bank obviously have different standards than a startup. So understanding that there's not one way, but those levels normally go with
00:30:56
Speaker
This is the one thing that you have to do and I think it's like a little bit more nuanced, like they come with good guidance and then like talk all those things through and tools help you to guide you through the way.
00:31:12
Speaker
I think my next question is around, as you said, right, like you always have to make sure that whatever you're running is from a trusted source is from the container image, for example, is properly signed. And I know the word provenance has gotten a lot of
00:31:27
Speaker
discussion, like it has been the topic of discussion recently. And that's how we ended up with the software bill of materials or S-BOMs. But the new thing that I think I wanted to get your take on was S-BOM at this station, like why is that required? I know we have the VEX standard, like how do all of those things fit in, right? Like if I'm a company that's producing an application or pushing an application to production or a vendor that's producing a product,
00:31:54
Speaker
How do I make sure that my users trust it, right? How does that attestation process work? Yeah. First of all, we need to understand the different attack vectors. So we have attack vectors in our software when we build software. And like this is normally like use like software dependencies checks and use like different tools to check vulnerabilities. Then you have the pipeline and you can make supply chain attacks. And then hopefully the software that you have defined in your Git like is being built in the pipeline.
00:32:20
Speaker
And then we have the software deployed in our runtime, and then things happen there as well. So when we think about...
00:32:30
Speaker
Again, it's all about like vulnerabilities on software level, but also like package level. So get runtime and build time. And then the, the next level is like, how do you make sure that the stuff you're running is actually the stuff you you've been built from? Right. So, so that's, that's the biggest problem. So if you use tools like cosign, for example, are really good, like for image signing.
00:32:53
Speaker
They don't solve your vulnerability problem. They don't solve your misconfiguration problem. What they do, though, is they say, like, hey, Ryan, you've built this container. Why is this container wrong? So I can go to Ryan and say, hey, something is off. Why is it off? And we can discuss it. So it prevents men in the middle attacks. And you can, first of all,
00:33:15
Speaker
only prove that this is the thing that you've built in. So there are still proof issues here because you essentially just signed the container.
00:33:25
Speaker
All the stuff that went into the container like this, there's still part, that's why a Celsa framework is important to make the whole chain like provable. Um, station helps you. It's to help you in preventing men in the middle attack, but you need to understand it's not preventing vulnerability attacks. It's not preventing this configuration attacks. It's just like proof of work, like who has done it.
00:33:47
Speaker
And basically, it's on the user, right? Do I trust Ryan? Like, can I use his image? And Ryan is shaking his head, so, okay, we can't trust Ryan. Okay, so that makes sense. And Chris, you used the term Salsa a lot, right? I know that stands for supply chain levels of software artifacts. I know because it's written down in my notes. Like, what's that framework? Like, that's the new thing, right, that we have been hearing in the community. And how does it help improve the security posture?
00:34:15
Speaker
Yeah, 100%. So let's assume like we have our Node.js app and then we have a fully patched thing, like all the JavaScript dependencies are 100% patched, everything is green. So that means we are really good in our security level and we want to
00:34:31
Speaker
We think, whenever we deploy it into production, everything is green, right? The problem, though, is what happens if the build pipeline is being detected? And you're essentially during the build process, the attacker injects a new vulnerability or a new source code dependency.
00:34:46
Speaker
And this is the key problem. So if you have manual work, you can't prove that the thing that you essentially have in your Git is really used to being built. And so that's why you want to have a framework like Salsa that guides you through that process and first make sure you have a fully automated pipeline step one. So you go from nothing automated,
00:35:08
Speaker
People like developers just deploy this container from my workstation, boom, done. We have done this many times. I think we as engineers, we have probably done it quite often in the past just to get our work done, but this is not a good approach. This is not good for companies who want to build up trust.
00:35:30
Speaker
Instead, you really want to use Git pipelines to have everything automated. The whole build process is automated. And then in each step, you make sure that really the dependencies that you have in Git is being used. And so that helps you to essentially say, OK, this binary is being built from this source code. And then you can guarantee, OK, the green Git is actually also the green binary, right?
00:35:57
Speaker
or package. And that's why this framework is so important. It has source code integrity, but also build integrity. And that means the stuff that you're building and releasing out to people is the stuff that you're saying is in Git. And that's the important part. Because otherwise you have to disconnect between the green state in your Git and the packages you provide.
Understanding the Salsa Framework
00:36:20
Speaker
Yeah, I like the connection here with Git, because I know, Bob, and we've done a few episodes around concepts around GitOps, right? And it always comes up that GitOps helps in overall security posture.
00:36:32
Speaker
And so connecting it to this, I think, and it's clear on why those types of models and pipelines help your security posture. I like that connection. And I don't know, what would you say? Is our trend towards things like GitOps helping your day-to-day, I guess, with customers as well?
00:36:55
Speaker
We use views, like everything I want to do is like fully automated built because it also makes reliable build process. You normally like have advanced, like faster build processes. And it also, it doesn't solve the vulnerability part per se, but you have everything tracked and good. So even if you have an attacker later on, like you can always prove, Hey, this attacker like manipulated get here and then
00:37:23
Speaker
It's in the git history. So it's also coming back to proof of things, what happened, right? So you can always have that. And then you like the next levels, you have forced essentially like peer review for approvals, like nobody can like push directly there. Then mechanisms you can then improve on the source code level to make this more, more difficult.
00:37:44
Speaker
And it's all about making things more difficult. Can you prevent all the different attacks? Probably not, but you can make it so that it's a lot of work. And attackers always take the easiest way to get in. So think about a house. You have a fully secured house, all the fancy new alarm stuff, and then you leave the door open, right? So it doesn't help you. And attackers, they don't
00:38:13
Speaker
like switch off their alarm if your door is open. So that's why it's so important to build, essentially, understanding the security posture and know where you are, like on which level in pipeline and in production. And that really helps you to know where you need to navigate and where to pick stuff first.
00:38:35
Speaker
No, that makes total sense, right? Like I think I was thinking about the same. Yeah, home analogy that you use. Another thing is like, if a line has to feed, like it has just to conquer the slowest hiner, right? Like you, you don't want to make sure you have to make sure that your organization is not that slow as I know, like you're not the most vulnerable because as you said, attackers will go for the slowest or the weakest target there.
Introduction to CNSpec by Mondo
00:38:57
Speaker
Yeah, I like the there was one that came up in my head about, you know, the GPUs and iPads that have been coming up, like filled with cement, like the middle attacks. For some reason that came up in my head as well. Anyway, great analogies. But I think something you said that was key there is, you know, make it more difficult for the attacker. But at the same time, you're also trying to make it less difficult for a developer to
00:39:22
Speaker
use it, you know, right? So, because if you make it more difficult, but more difficult for a developer, they're not going to use it. I think that's a challenge as well. And maybe something you're tackling already. Okay. And then I think while doing some research for this episode, right? And I came up across a tool that you guys maintain called CNSpec and how that's available as an open source tool for organizations to consume. Can we talk about how, what CNSpec is built for? Like what, what challenges does it solve for? And then how can people use it?
00:39:51
Speaker
Yeah, so CNSpec is our take on how to quickly assess your security posture. It essentially helps you to continuously monitor and misconfiguration across all the clouds. It helps you to scan VMs, containers, Kubernetes workloads. It helps you to identify identity misconfigurations. And it's also built for shift left. And the way to think about this is
00:40:19
Speaker
It's extensible on multiple ways. We have providers that allows us to do AWS, GCP, we have GitHub integration, we have operating system, container, and remote. Different providers, similar to how Terraform works. It's really the same thing. Think about Terraform for security.
00:40:42
Speaker
And then we have the resource packs that allows us to write resources on top of those providers and then you can build policies on top. And this is a highly customizable framework. We call it the only extensible security framework where you can
00:41:02
Speaker
You're not stuck into a specific technology, and that was really important for us. When we started Mondo, we wanted to build something that is holistically for users, where they can essentially focus on the problem. And the problem is, where is my biggest risk? My problem is not how does Docker work, or how does Container work, or how does the Container Registry work.
00:41:22
Speaker
I just want to know, like, where's my risk, right? And so, abstracting that problem is actually hard. Like, a lot of people, like, always told us, hey, guys, like, this is not going to work. Like, you can't do that. And, and they're, they, they, we've proved them wrong, because, like, obviously- One of the best motivators right there, right? Yes, yes. And, and I think he, like, we need abstract, giving, giving users to say something like, scan,
00:41:48
Speaker
SSH scan container image scan AWS and essentially runs policies with checks that are totally dynamic all the checks that we ship and we have open source policies available and we have CS policies available we have NSA policies available so you can just like out of the box use those policies run them and see quickly where your security posture is and it's really an open framework
00:42:14
Speaker
like similar to Terraform, but for security. So it's really extensible. We encourage people to like contribute, tell us what's missing because we really want to help users to solve the problem of, Hey, where's my security poster? No matter like what kind of tech they're using.
00:42:31
Speaker
we have customers asking for AIX support and we have customers asking like going in the different direction where they all use like it's for for IAM permissions, S3 buckets and so on and that's all working and so that makes it like super easy for for teams to first start with an out-of-the-box policy but then also go off and say look
00:42:53
Speaker
It's in my environment, there are specific requirements, so I want to customize it. And that's why they convert their own policies and override things. And that's the key. This is why it's also open source because we believe this should be your own right to secure yourself.
Community Contributions to CNSpec
00:43:11
Speaker
We want to make this available to everyone. And that's why we really open sourced it to make it available to everyone.
00:43:21
Speaker
And I really like the breadth, right? Like it can scan everything from like a VMware host to a VMware virtual machine to like your Terraform plans to Kubernetes clusters through Docker registry. Like regardless of where you are on that modernization journey, like this can help you scan everything.
00:43:38
Speaker
Yeah, that includes vulnerabilities and misconfiguration. And what you said is really important. We believe organizations are not just in one tech, like organizations are always transforming. And so they have something in VMware and they have something in cloud and they have something in containers and something bare metal. And I think it's
00:44:01
Speaker
Back to my original story, it's really not about telling people this is all right or this is wrong. They normally know why they're doing it. And so we don't want to judge it. We just say, look, we look at this, we help you get what's the risk there, and then you can make your decisions. So it's really built for humans to make better decisions.
00:44:23
Speaker
Yeah, and you mentioned earlier the idea of sort of mixed workloads. And I really think we're seeing a trend towards that anyway. And I see that as sort of a foreseeable future, right? These mixed VM container serverless workloads and picking the right tool for the job is also very much embedded in that thought process. So having security tools that also work like this.
00:44:43
Speaker
And I could see, you know, we're already seeing orchestration tools that kind of get into this, right? Let me do both. Let me do all three of those. So I think very important and a good lead in, I think to our final question here is like, how do people get involved? Obviously they can go and check out this project and see if it works for them and contribute, but is there other resources that you'd love to share and or you mentioned a cool EKS demo where you broke out and controlled the cloud.
00:45:09
Speaker
Do you have that available? I know I'm interested in that, but anything you have, that'd be great. Yeah, we can definitely share this. I can send it to you so you can get this to the podcast. And then, yeah, we always, first, we want to get more user feedback. Essentially, we want to have a really useful tool.
00:45:31
Speaker
available. So when we have open source policies, we have open source resources. So I think the easiest one is to look at the policies, make sure like we do a double checking and triple checking for those things, but sometimes you miss things. So I think it's easy to contribute. If you have an application where you say this is not covered, you can contribute this to this shared policies, make this available to the community.
00:45:57
Speaker
because we believe that building up this community around the security content is just helpful for the community itself.
Reflections on Christoph's Insights and Community Importance
00:46:05
Speaker
We want to help people to be secure and that means the content needs to be available, it needs to be easy to share those content. And it starts on the
00:46:16
Speaker
the queries, the checks, the policies, but then also goes down to more integrations. So even if you're not a go coder, feel free to just suggest features. It's always helpful to see that. So I think that's always my
00:46:35
Speaker
what we can ask for like really try try use it give us feedback we are really nice humans so you can go to our github discussions like start start a discussion if you're unsure um and and yeah we are definitely looking forward to this yeah you being good humans definitely is something that we can attest i guess okay thank you thank you it's super exciting to review with you too
00:47:05
Speaker
Well, great, Chris. It was really a pleasure to have you on the show. And I know I learned a lot. So hopefully everyone else listening did as well. And we'd love to have you back one day. Thank you, Paulman. Thank you, Ryan. Thank you so much. It's super exciting.
00:47:18
Speaker
All right, Bobbin. Well, I think that conversation was great. I think I feel like I say that every time with our guests, I probably do, but I, you know, especially in security space, I always feel like I'm learning a lot. And Christophe was just probably full of even more information that we could spend hours and hours more with him. But, you know, if you had any takeaways from that conversation, what would you take away from it? Yeah. Like, I really liked the succinct way in which he described the cognitive security posture management, like the bite size, right? Like,
00:47:47
Speaker
Make sure you don't have any configuration, like misconfigurations, and then make sure you are keeping all your components patched. That's an awesome description. Just want to keep a quick reminder of what security posture management is. This makes sense. But in addition to that, right?
00:48:04
Speaker
trying to understand what some of these terms mean. I know in the ecosystem, if you're not working with security on a day-to-day basis, some of these terms can feel a lot overwhelming. Things like S-Bombs. I know S-Bombs has been around for 15 months at this point.
00:48:20
Speaker
Keeping up with S-BOM and how the standard is evolving, what is S-BOM attestation, what's the VEX standard, how does emit signing work, all of these things are getting pretty important. It's not enough to just have a list of all your different application components in an S-BOM. For me to trust that the application that you are giving me is secure, I want you to make sure that you sign it or you attest to it.
00:48:45
Speaker
So that's a good standard and the way it's evolving. And then just for this episode, right? Like when I was doing some research, I came across the Salsa framework and like, okay, what is it? Like it has a cool name, but then how is it helping organizations? So I know Chris broke it down into what it does for organizations, but there are different levels, right? Level zero means you're not doing anything, but then there are levels one through four and it basically gives you a standard baseline. So I know Chris mentioned that healthcare companies or finance companies don't have to be at the same security level.
00:49:15
Speaker
Startups don't have to be at the same level, but having these different levels gives the community gives the different organizations a level to standardize to like, okay, I'm Santa standard level to certify it or things like that. So having these in place and then having somebody like Chris actually interpret them for us was really helpful for me.
00:49:36
Speaker
Yeah, absolutely. Honestly, maybe you want to go get some pico de gallo. But technology wise, yeah, absolutely was very enlightening. And I like the fact that, you know, Chris said, he doesn't love those big terms, right? Yeah. And it's more the way he kind of goes about working with customers is like, let's identify where you're at, and then go from there to make, you know, to improve and make it more difficult.
00:50:00
Speaker
for any amount of actors to get in there. I like that approach of sort of practicality rather than just like, you know, CIO objective using big terms and saying, we have to do this, right? And even if that's the case, it's, well, let's fit the level for your application. For me, the takeaway that I think it most rang true was the fact that
00:50:24
Speaker
Kubernetes and these cloud sort of accounts that we're using every day, they have the right primitives. We're not using them correctly. Or I shouldn't say, I shouldn't generalize to all, but a lot of what he's seeing is misconfigurations. And so the concept around automation to take people's hands quite literally out of
00:50:46
Speaker
the pipeline and have it automated and doing it in a way that's using it correctly. Just because RBAC is there, IM is there, those tools are hard to get perfectly right for your application. I remember being asked and still are asked, what's the right IM policy for this specific application? Well, it takes some time to figure that out. I just want to, as my takeaway,
00:51:11
Speaker
I think it's very important to just reevaluate what you have. What tools do you already have that maybe you're not using or maybe you could double check, um, you know, how your company is kind of enabling it. Is it enabled? Right. I know Chris said that at one point as they're turned off, right. Um, I think that is well, so, uh, you know, I think all worth, uh, diving into and, and there's a lot of concepts in security. So, um, you know, there's, there's a ton more we could talk about here and I want to do call out if anybody
00:51:39
Speaker
is working on a specific problem around security or improving your security posture or using some of these things that we talked about today. Come talk to us. We'd love to have you on the show. I think security is something we as an industry have to take a lot more seriously. I think the cloud native community is seeing that, cloud community in general is seeing that. And so all four are doing some more interviews and content on this space.
00:52:05
Speaker
With that, I know there's a couple things we wanted to do before we sign off today. One is we wanted to shout out to Amir Dixit. I hope I'm saying that correctly. He sent over a really kind appreciation email and just thanking us for some of the
00:52:22
Speaker
content we did on, I think it was GitOps. Service mesh. Service mesh, sorry. And he gave a really good example of how he was working in the space that he is and a very specific sort of idea of how it's enabled him to adapt and take on MTLS and some of the things we talked about in the show. So we really appreciate getting those messages and emails and Amir,
00:52:47
Speaker
Thank you. If you want some stickers, send us your information and we'll send you out some Kubernetes Bites stickers and hopefully you can enjoy those. So I think that was really the end of today. So without further ado, that brings us to the end of today's episode. I'm Ryan. I'm Robin. And thanks for joining another episode of Kubernetes Bites. Thank you for listening to the Kubernetes Bites podcast.