Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
GitOps, DevSecOps & Kubernetes w/ GitLab
876 Plays1 year ago
In this episode of Kubernetes Bytes, Ryan and Bhavin talk Cesar Saavedra of Gitlab about the importance of GitOps, DevSecOps and more with Kubernetes. They also cover what Gitlab has to offer in this particular space and what new features they have been working on.
- EKS/Container Marketplace as EKS add-ons https://aws.amazon.com/blogs/aws/new-aws-marketplace-for-containers-now-supports-direct-deployment-to-amazon-eks-clusters/
- New – Announcing Amazon EFS Elastic Throughput
- This new throughput mode is designed to provide your applications with as much throughput as they need with pay-as-you-use pricing.
- Trivy now scans Amazon Machine Images (AMIs) - https://blog.aquasec.com/trivy-now-scans-amazon-machine-images-amis
- multicloud with crunchy postgres https://www.crunchydata.com/blog/multi-cloud-strategies-with-crunchy-postgres-for-kubernetes
- Get ready for 1.26 https://www.armosec.io/blog/kubernetes-1-26-everything-you-should-know/
- Get ready for 1.26 (diff vendor) https://sysdig.com/blog/kubernetes-1-26-whats-new/
- Argo has graduated - Dec 6 - https://www.cncf.io/announcements/2022/12/06/the-cloud-native-computing-foundation-announces-argo-has-graduated/
- Flux graduates from CNCF incubator - Nov 30 https://www.cncf.io/announcements/2022/11/30/flux-graduates-from-cncf-incubator/
GITLAB INFO
@cesar_saavedr
GitLab Snapshots (YouTube videos)
Recommended
Transcript
Introduction to Kubernetes Bites
00:00:03
Speaker
You are listening to Kubernetes Bites, a podcast bringing you the latest from the world of cloud native data management.
Meet the Hosts and Podcast Focus
00:00:09
Speaker
My name is Ryan Walner and I'm joined by Bob and Shaw coming to you from Boston, Massachusetts. We'll be sharing our thoughts on recent cloud native news and talking to industry experts about their experiences and challenges managing the wealth of data in today's cloud native ecosystem.
00:00:29
Speaker
Good morning, good afternoon, and good evening wherever you are. We're coming to you from Boston, Massachusetts. Today is December 7th, 2022. I hope everyone is doing well and staying safe.
Highlights from reInvent
00:00:41
Speaker
Let's dive into it. Bhavan, I hope you're recovering from re-invent. How you doing?
00:00:46
Speaker
I'm doing good. I know we had a busy, busy week last week. Lots of people at re-invent. I was so glad that it was back in its full attendance, I guess. Like 50,000 people were still like the normal for re-invent. So I'm guessing next year we hit the 75K mark.
00:01:02
Speaker
But I had a lot of fun at the show. I know it's a lot of work, standing at the booth and talking to people over and over again about the same topics. But I like that. That's the best part of my job. So I had a fun time. How about you? Yeah. I learned a couple of things this year. One,
00:01:19
Speaker
is that re-invent seems huge and there's people everywhere, but CES is like triple the size. Yep. Never going, I think is what I decided to see. And these are really apt because there's just sensory overload all the time. If you're into that thing, so be it. And then two, did you see all the cowboys at the end of the week? Yeah. So I learned through my Uber driver that there is like this huge rodeo conference that comes in at the tail end of re-invent.
00:01:46
Speaker
So if you were there and were wondering why people were walking around in cowboy boots and cowboy hats, dressed like they were straight from the plains of America, they probably were. That's what I was getting at. I was noticing that too, and I'm glad my Uber driver explained that to me. I thought it was just like a Vegas thing, the way they dressed to come to Vegas.
00:02:10
Speaker
Hey, you know, there's always something interesting. So Vegas has a busy December, like re-invent with all the tech folks, rodeos with all the cowboys, and then CES with all the, I don't know, mix of both, I guess. Yeah. Well, there's Gartner conference. CES this week. Yeah. Yeah. Yeah. So anyway, gosh, you know, it was a good show though. I had a lot of great conversations.
00:02:32
Speaker
I don't love Vegas, but hey, you know, not my thing. Cup of tea. Other people love it. I'm sure. And, uh, anyway, so, um, we have a really cool topic today. We're going to be talking about, uh, DevSecOps, GitOps and Kubernetes with the folks from GitLab. Uh, we have Czar.
00:02:48
Speaker
Saavedra on the show, and we're going to dive in and introduce him just shortly. But before we do that, let's get into our news.
EKS Marketplace and EFS Updates
00:02:59
Speaker
Since we're just talking about reInvent, let's start there, right? So some of the news that came out of reInvent that's sort of relevant to our listenership is the EKS container marketplace. Sorry, I
00:03:12
Speaker
I should say, the container marketplace that now allows for EKS add-ons was announced. So if you had a specific sort of project that can be added to your EKS cluster, you can easily search for that and basically add it, whether that's something like core DNS, kubeproxy, kubecost, I saw one that come out. There's a bunch of them, right? So this is something we are doing with our Kubernetes clusters regardless. We're adding on these features. I know.
00:03:41
Speaker
I know Microkates has this idea of sort of an add-on. You can just like, you know, Microkates enable this thing. Interesting. And so this reminds me of this in sort of a cloud-based way. I really like that concept. So yeah, go check that out with that link in there. The other thing was EFS Elastic Throughput. You can now basically pay for the performance you want is how I'd sum it up.
00:04:07
Speaker
So EFS, I know if you've used it, mixed bag about people's experiences on what it's used for, how it's used for its performance and things like that. I think this is sort of response to, hey, we can do more. You just have to pay for it, which makes a lot of sense coming from cloud. I'll just say cloud. I don't have to pinpoint AWS on this.
00:04:27
Speaker
Um, and, um, you had one here, you put here. Yeah. So, uh, uh, I had like, I didn't put down like an AWS announcement, although I, I think
Trivi AMI Scanning and AWS Innovations
00:04:38
Speaker
looking back at it, right, the AWS application composer thing that they announced, uh, was really cool. It was part of Werner's keynote and he used, uh, if you have, if you have been to any of the AWS summits or the events over the past couple of years, you have seen that coffee booth where.
00:04:53
Speaker
It's serverless based coffee. So you just submit a request, it shows you how the event event thing works and it gives you a coffee at the end. He showed what the architecture looks like on stage and then how you can now compose that entire application by adding different AWS services as components and makes it
00:05:10
Speaker
easy to look at visually and which can be shared between developers, things like that. So that was really cool, something that caught my eye. But the announcement that I had in our notes was more around Trivi, the open source project that we usually refer to. It can now support scanning your Amazon machine images as well. So if you have AMIs that you want to test without actually spinning up instances from those AMIs to see if the base image or base AMI is good.
00:05:36
Speaker
Trivi can now support that as well. So you use the Trivi VM command, point it to that AMI ID, and it will do the scanning for you. So if you're using AMIs, want to make sure that they are secure, you can still use an open source tool for it.
00:05:49
Speaker
Awesome. Awesome. Um, cool. I had another one here about multi-cloud and crunchy Postgres. First of all, I know I've used this, um, in some demos and I've worked with crunchy for, I love the name, just crunchy. It makes me think of snacks and I love snacks. So, um, kudos to your naming. I guess it's probably nothing to do with snacks, but, um, we love
Cross-Data Center Strategies with Crunchy Postgres
00:06:13
Speaker
you too. We love you too.
00:06:16
Speaker
But they do have an article here released earlier in November, but around cross data center streaming replication out of the box. So this is really around the multi-cloud disaster recovery movement between Postgres clusters reaching cloud providers. I know a lot of us worry about things like AZ failure or vendor lock-in. I think as we talk about multi-cloud, hybrid cloud, and how to support these types of workflows more
00:06:43
Speaker
and more. This is definitely an interesting tech. So go check that out. We really liked the project over there at Country Coast Postgres. And then, believe it or not, Kubernetes 1.26 is the last release of 2022 that is slated for release.
New Kubernetes 1.26 Features
00:07:03
Speaker
So we have a couple links here that we'll dive into the what's new. We're covering it a little early. It's not actually out yet, right?
00:07:11
Speaker
No, it's not. But I think it's due in a couple of days. I know it got delayed because of some go bug. But I know we are recording it on the 7th. But I think it might come out on the 8th or 9th. So like, almost in a couple of days. Yeah. Perfect timing then, I guess. Yeah. A couple of things I wanted to note in the list of many features and kind of fixes that are in here is the non graceful node shutdown for stateful set pods is now beta.
00:07:37
Speaker
data. So this is all about the case where stateful applications go down and don't exactly fully terminate. So the release of the PV and things like that can cause downtime and outage. This kind of cleans things up forcefully if you're aware of what's going on. So very useful thing in the stateful and persistence
00:07:59
Speaker
world here. The second one is the removal of the entry CSI integration for OpenStack, the cinder volume type I'm talking about here, which I just, you know, was talking to a couple of colleagues about a resurgence of OpenStack, if you believe it or not. What? Yeah. Yeah.
00:08:14
Speaker
I'll put the link in here too. But this could be, I don't know, according to what's sort of happening with, you know, just the macro economics these days, but that's a whole nother podcast. So I've successfully avoided the whole open stack.
00:08:31
Speaker
So I was like a VMware guy for years, and then I completely missed OpenStack. And then as soon as Kubernetes came on, I was like, okay, I can't miss this. I jumped on it. And I was glad that I didn't go through the whole OpenStack pain that you guys did. It was a great community. It's still a great project. I still love what it's all about. And it's actually just, if you're just into sort of tech and if you like Python, I was like a Python developer, so it just made a ton of sense.
00:08:57
Speaker
Again, a whole other podcast. And then you had one listed here as part of this digs announcement. But go ahead.
Argo and Flux Graduation
00:09:05
Speaker
Yeah, so it was another what's new in 1.26. The volume snapshot API that we have in our CSI standard. Initially, before this release, if you wanted to create a new volume from a snapshot, it had to be in the same namespace. But now with 1.26, as an alpha feature, you can take a snapshot that's running in a different namespace on the same cluster.
00:09:26
Speaker
and create a new volume in a completely different namespace as well. So cross namespace volume creation from a snapshot is now going to be an alpha feature. That was just a quick heads up that applied to the storage ecosystem. Great.
00:09:39
Speaker
And then a couple of graduations for CNCF. So I really had to go back and because both of these blogs have different titles, like Argo has graduated and Flux has graduated from the CNCF incubator. I was like, do they mean the same thing? So I had to like go and check the different phases that CNCF has. But yes, both of these projects that are around the GitOps and the topic that we have for today have now graduated from being a CNCF incubator project to a CNCF graduated project, which
00:10:07
Speaker
I think brings the list to 19 different projects that CNCF maintains now in the graduated state. I know both of these projects are super popular. Argo has been used or is actively used in production by over 350 organizations when I looked at that blog post. And then Flux, for example, has had a 400% growth in the last 12 months. So this is a really important topic for people in 2022. And I'm pretty sure that trend is just going to continue and grow in 2023.
00:10:36
Speaker
That leads us perfectly, well done, into our topic today.
Cesar Saavedra on DevSecOps and GitOps
00:10:41
Speaker
As if we planned it, Robin. Around, again, DevSecOps, GitOps, Kubernetes. And we're going to talk with Cesar Saavedra, who's a senior technical marketing manager at GitLab and an expert in his field. So actually, without further ado, let's get him on the show.
00:11:00
Speaker
All right. Well, welcome to Kubernetes Bites. It's so glad to have you here. I know we're excited for today's conversations. Why don't you give our listeners a brief introduction of who you are and what you do. Hello, Ryan. Thank you for the invitation. My name is Cesar Saavedra, and I'm a senior technical marketing manager with GitLab. And my focus areas are CD and GitOps. And I do a little bit of Dora metrics as well.
00:11:29
Speaker
Awesome. Yeah. Great to have you. I know like Ryan and I did a GitOps 101 episode and that has by far been, has been our most popular episodes of this. Like we had to get an expert on to like dive into more detail. So I'm like, we are so glad to have you on the show Cesar. Thank you, Babin. It's a pleasure to be here.
00:11:48
Speaker
I think so. I'll start off with the questions, right? So GitLab, the first thing that comes to my mind is CICD. So can we start by there and give us a thousand foot overview of what a CICD pipeline is, how it works, and then if I'm a developer listening to this, right, who's still starting new, how do I get code that I'm writing on my ID on my laptop into a production cluster running in Kubernetes? So like talk about the entire workflow, maybe.
00:12:15
Speaker
Very good.
GitLab's DevOps Platform
00:12:16
Speaker
Yes, we are known for SCM and CI-CD. I think those are the top areas. However, we describe ourselves as a comprehensive software innovation platform. We provide all the essential DevOps tools in one single platform. And you can go from idea to production
00:12:36
Speaker
and within GitLab. Basically, we help teams improve things like cycle time. We have many stories of customers that have gone from weeks to minutes by adopting GitLab. We reduce development costs, speed time to market.
00:12:55
Speaker
and also deliver more secure and compliant applications. In addition to that, because we're a single platform, we provide a single data model, also with a single UI, all in a single application. And not only does this help developers become more productive,
00:13:14
Speaker
But it also allows us to correlate data because all the data that we collect end to end through the DevOps lifecycle, we're able to correlate it and surface insights of the entire lifecycle and put that in the fingertips of our users in the form of dashboards, for example, not sure if you heard of
00:13:35
Speaker
or value streams analytics, we can provide those in dashboards. And we can do that easily because everything is in the same data model.
00:13:46
Speaker
So the second part of your question was, how do I get my code from the moment I'm writing all the way to a Kubernetes cluster? So I think it's a good moment to talk about what the GitLab flow is. So GitLab flow basically creates a seamless approach to software development by integrating a Git workflow.
00:14:15
Speaker
with an issue tracking system is really a simpler alternative to Gitflow. And it combines what we call a feature-driven development and feature branches with issue tracking.
00:14:30
Speaker
Within GitLab Flow, all features and fixes go to the main branch while enabling production and stable branches. It includes also a set of best practices and also guidelines to ensure software development teams follow a smooth process to ship features collaboratively. So how does GitLab Flow work?
00:14:59
Speaker
So for example, with Git Flow, we just talked about Git Flow, developers created a develop branch and then they make that the default. Whereas with GitLab Flow, it works with the main branch right away. So GitLab Flow incorporates a pre-production branch to make bug fixes, for example, before merging those changes back to the main branch and before going to production.
00:15:28
Speaker
Teams can add as many pre-production branches as needed. For example, you can go from main to test or from test to acceptance or from acceptance to production. Your teams can practice what we call feature branching while also maintaining a separate production branch. So whatever you, for example, see the main branch is ready to be deployed,
00:15:56
Speaker
users can merge into it, I'm sorry, into the production branch and also release. Let's see, what else can I cover by GitLabflow?
00:16:09
Speaker
It's often used with release branches. So for example, if a team requires a public API, that may need to maintain different versions for it. With GitLab Flow, the team can make version one branch and a version two branch. And those two can be maintained individually or separately.
00:16:32
Speaker
which can be helpful if the team identifies a bug, for example, during code reviews, and then that goes back to version one. So there are many benefits when you're using GitLabFlow. It offers a simple, transparent way to work with Git. And also using GitLabFlow, developers can collaborate and maintain several versions of the software in different environments.
00:17:01
Speaker
It also decreases the overhead of releasing, things like tagging and merging, which is a common challenge encountered with other types of Git workflows. And all in all, it creates an easier way to deploy code. Got it. Yeah, I think overall, from a developer standpoint, it sounds like there's a lot that goes into it when you're either
00:17:26
Speaker
doing bug fixes or developing new features. Overall, it sounds like Gitflow enables the ease of use in this aspect. If you're a developer working on a project that's ultimately looking to get your code into Kubernetes or an application that's running in Kubernetes,
00:17:44
Speaker
A lot of the tools I know we'll talk a little bit about sort of the integrations with Kubernetes in a little bit. But overall, sort of being able to go through, you know, creation of new code to being able to do that in a feature branch or in a bug fix and do that in a proper way through that CI CD pipeline to get that in Kubernetes.
Exploring GitLab's Review Apps
00:18:02
Speaker
Sounds like Gitflow and overall GitLab enables a lot of that, you know, in an automated fashion. Is that correct?
00:18:09
Speaker
Yes, that's right. So there is a good depiction. If you search over your Google for the term GitLab flow, you will see a nice picture of what it what it consists, you know, an example of what it is. And it all starts to create an issue, which is basically this description of a problem. And now we're talking about the CI portion of the pipeline, right? So so once you create the merge request, which is where actually the main collaboration takes place among the different stakeholders.
00:18:38
Speaker
Then as soon as you commit the changes, then a pipeline is fired up. Then the CI steps take place and whether it's security scans, if you want to run things like dependency scanning or code quality scans and things like that, they are run there automatically.
00:18:59
Speaker
And then we also have the option of what we call review apps and review apps within GitLab is a feature that basically when you're running Kubernetes in this case.
00:19:12
Speaker
GitLab will spin up an ephemeral environment for you. It'll be the review environment, and then it'll build and deploy the application that is in the feature branch to this environment. At that moment, you have a running application with the changes that have not been merged yet. That running application can be accessed by all the stakeholders, and that's where a lot of the collaboration takes place again.
00:19:42
Speaker
They can review the app, discuss among themselves, and then do that iteratively within the MR at the merge request. Also, the results of the review pipeline, as well as all the jobs and steps that take place.
00:20:00
Speaker
And the output that they generate are displayed in a widget inside the merge request. So all the information is easily accessible within the same location, within the MR. So that if there's a conversation going on, they can refer all the way down to the code, or even aligning the code. Or if you run a test and it generated errors, then that output will be also part of the MR, so you can discuss that all in the single place.
00:20:29
Speaker
Okay, I think one thing that I picked up from your answer was that as part of the review process, GitLab will automatically develop or deploy FML Kubernetes clusters. How much pre-work is required? Do I just give it my AWS credentials, for example, and it will spin up an EKS cluster to test it with? Or does it need pre-created or pre-deployed Kubernetes clusters that it will use for this review phase?
00:20:53
Speaker
When I described review applications and how the ephemeral environment is brought up, the assumption is that there's a connection to Kubernetes already. That's what you're asking. Before I jump into that, I'll tell you that once the review is finished and the merge happens, all those artifacts and resources that have been created in Kubernetes are cleaned up for you automatically by GitLab.
00:21:17
Speaker
and another pipeline is the merge pipeline is then run. So how do you connect from GitLab to a Kubernetes cluster? So that's
00:21:29
Speaker
We can talk about now the GitLab agent for Kubernetes, which is the approach that we use for securely connecting a GitLab instance to a Kubernetes cluster. Now the agent supports Vanilla Kubernetes, all the major distributions of Kubernetes, EKS, AKS, GKE, OpenShift, et cetera. And the way you
00:21:56
Speaker
The way you connect to it is you would do it first, you go into the GitLab UI, you go into the Kubernetes menu option, Kubernetes clusters, and then you would create, well, that's at the moment, this is where you will connect it. Before you do that, you need to within Git, GitLab, you need to create a configuration for the agent.
00:22:19
Speaker
Okay. So that's basically a manifest file that it has a specific syntax. It's all in the documentation. But basically, you are putting in that manifest file things like the ID of the agent. You can set up a lot of properties like you know, what kind of level of of logging you want, where in the in the specific case of GitOps, there is a
00:22:45
Speaker
there's a location where you have to specify a specific directory, for example, that basically you're telling the agent, I want you to observe that directory. And if any manifests are dropped in there,
00:23:01
Speaker
you need to take action on them. So for example, if you want to create an nginx instance or a pod, then you will drop an nginx.yaml file there. And then the agent, which is running as an in-cluster agent, it's observing that directory and it sees that file. And then at that moment, it will go ahead and instantiate a pod in the cluster, creating that nginx instance.
00:23:28
Speaker
This is, yeah, so this tool, this GitOps agent, I want to take a step back just briefly. And maybe for our listeners that might be asking like, what is GitOps in general? And what's the importance to Kubernetes? Maybe we can talk about sort of the overall concept of GitOps, maybe some of the differences.
Understanding GitOps
00:23:46
Speaker
I know there's a couple of different approaches that we discussed in the past, which is push and pull. I want to dive into that a little bit. And I think that'll help the context a little bit.
00:23:54
Speaker
So in the past, I don't know if you've heard of the terms click ops or script ops, you basically have a bunch of scripts that would configure your different infrastructure components, whether it's a router, a load balancer, or a server, or whatnot.
00:24:11
Speaker
or click ops, meaning that every infrastructure vendor would have its own UI for you to basically configure that specific component in your data center, whether it's a load balancer or anything else, or a rack of CPUs, for example, of servers.
00:24:31
Speaker
So that was one of the pains that GitHub solves because that process is manually slowed, and it's also stored in the operator's or administrator's machine. So it's not easy. First of all, it's not something that you can easily share or others to leverage or use. So that knowledge is basically stored in that person's head. So now at the same time,
00:25:00
Speaker
Git has become the SCM tool most used by developers. And if it's very well with the 12-factor app methodology, for example, for building software-as-a-service apps, things like microservices and non-jail development methodologies.
00:25:17
Speaker
In addition, DevOps is now being used more and more by organizations as a highly collaborative, cross-discipline approach that focuses on the optimization of value delivery, cost and risk, et cetera. And DevOps offers the promise of improved speed of development by embracing this collaboration and tears down the traditional silos between development and operation teams.
00:25:46
Speaker
So, we have Git, we have the concept of ClickOps and ScriptOps, and we have DevOps. And the last component is that came into this
00:26:01
Speaker
into the market and into organization is a very important component called Kubernetes. It's taken over the market as a container orchestration technology of choice for developing cloud-native applications. Just as DevOps is effective at improving speed of development,
00:26:23
Speaker
GitOps is the application of DevOps best practices and principles to the automation and management of infrastructure. So you can think of GitOps as a step in the evolution of DevOps. So how can we apply what developers have gotten so well at doing in DevOps? How can we apply those concepts and those best practices?
00:26:45
Speaker
because we still need to do the same thing, but now for operations. And DevOps got the developers super productive. They were able to do things much faster than before, but then now we have to do the same thing with the operations teams. So that's where GitOps comes in. Now, when we talk about the best practices that are transferable or applicable to
00:27:11
Speaker
two operations that are applicable to the GitOps concept that we're discussing. Number one is codification. So this is the idea of having your infrastructure components defined or configured in code, right? And a file basically.
00:27:33
Speaker
Yeah, like creating Git as that single source of truth, right? Like this is where everything is stored. Right, right. So it is basically declarative code that describes the desired state of that infrastructure component and storing Git, as you mentioned, Babin.
00:27:50
Speaker
And let's see, the second thing is collaboration, right? So in the collaboration, as I mentioned before, is in the form of merge requests or other tools call it pull requests. And they are the gate of these changes. And then you have the concept of your main branch, which is your product branch.
00:28:09
Speaker
And then you have also the concepts of reviewing that and collaborating among different people and also approving changes to infrastructure. So that's the collaboration component. And the last thing is automation. So you have to be able to support the reconciliation loop. And that's the concept by which when someone logs into a Kubernetes cluster directly, that makes a change.
00:28:38
Speaker
Right now, you've changed the configuration of that cluster and that it may not match its configuration in Git.
00:28:46
Speaker
So now you have to reconcile that. You have to first detect it and then reconcile it. And that needs to be automated. Also, when the infrastructure is out of sync, that means when somebody updates Git with a new change to an infrastructure component, there needs to be an update immediately on the infrastructure. And all that needs to be automated.
00:29:13
Speaker
So, you know, so again, codification, collaboration and automation. And also, above or on top of a cross cutting concern, I should say, is, you know, you also need observability and security, right? Sure. Yeah, absolutely.
00:29:31
Speaker
So in order for those changes to be made actually on the infrastructure on Kubernetes, this concept of sort of pushing that change versus an agent pulling, I know that was sort of covered briefly there, but maybe go into maybe how that works a little bit.
00:29:49
Speaker
Very good. Yes. That was the second part of what I said I was going to say. I didn't say it. All right. So let's discuss pull-based and push-based approaches. So I mentioned an in-cluster agent. So the GitLab agent for Kubernetes is a lightweight agent that you deploy to your cluster.
00:30:11
Speaker
Now the way you configure that, I mentioned earlier a configuration file or a manifest that you have to create, in which you have to spell out the configuration of the agent, including its name. And then what you do through the GitLab UI is you basically go into the Kubernetes clusters window,
00:30:31
Speaker
And then you select that agent for which you have a configuration file. It's a pop-down menu. And then when you select that, you will get a pop-up with a command that you need to run on a terminal window.
00:30:52
Speaker
and the command uses a secret, a token that will allow you to basically load that and install it on your cluster securely. When you run that command, it starts out the pod and it also connects your GitLab instance to that Kubernetes cluster. That's how those two are connected securely. Now, the in-cluster agent
00:31:22
Speaker
can communicate via gRPC or web secure web sockets. That's what we support so far.
00:31:30
Speaker
And the agent, there is a component in the cluster. There's another component in GitLab called CAS, K-A-S. It's a service. Think of it as a service. And that service is constantly communicating with the in-cluster agent. And so when there's a change in Git, for example, and remember I mentioned that the directory where the agent is observing? Yep. So let's say you drop or you make a change to a YAML file there.
00:31:59
Speaker
That GitLab instance service will inform the in-cluster agent about it, and then the in-cluster agent will bring the changes up and install them or update the cluster accordingly. So that's how the changes happen. Also, the in-cluster agent can detect drift and bring the cluster configuration up to date to what is supposed to be in Git.
00:32:26
Speaker
Now the agent supports, so that's what we call pool because the agent is actually pooling. As changes happen in Git, the agent is observing them, observing this directory. Any changes that happen there, the agent is bringing them over and updating the cluster accordingly. It's pooling from Git. Think of it as pooling from Git over to the cluster. Now also the agent allows you to access
00:32:55
Speaker
the cluster securely from inside a pipeline. So imagine you have a pipeline with stages and jobs, and when you run a job, you have the option of having a script section in the job or script lines that you can execute.
00:33:11
Speaker
And in that script section, you can actually use kubectl commands to access that specific cluster that you're connected to. Now, when you're accessing that cluster from the CICD pipeline, you have the option of pushing changes from the CICD pipeline to the cluster.
00:33:34
Speaker
So the concept of push-based approach to GitOps is a concept of pushing updates from a pipeline to the cluster. Now, and the agent supports both, okay? But one thing I want to say, I know this is Kubernetes podcast, but one thing I want to say also is that this push-based approach works also without the agent. So let's say you don't have Kubernetes.
00:34:02
Speaker
then you can use the push-based approach to update or keep your infrastructure in sync with whatever is in it. Makes sense. I mean, this is the Kubernetes podcast, but I think a lot of us who are practitioners or in this sort of market understand that our infrastructure is going to be a mixed bag of both containers and Kubernetes and VMs. Those things aren't going anywhere, so that's fair game. Don't worry about it.
00:34:29
Speaker
Okay, so this is our one question. I think I love the drift prevention or drift detection feature, but I also wanted to bring in the security angle. How are you recommending or guiding customers on the route of shifting left?
Integrating Security in Development Lifecycle
00:34:45
Speaker
I know we spoke about DevOps and GitOps and ScriptOps, but we also have a new term, DevSecOps. How do we help organizations move towards more secure application development lifecycle than just
00:34:59
Speaker
making sure that the changes are getting pushed. So can we talk about or bring in the security angle as well? Yes, for sure. So GitLab supports a variety of security scanners out of the box that you as a developer can leverage.
00:35:20
Speaker
And the idea is that you want to run tests as early as possible in the development lifecycle, because as you know, it's better to catch an issue while you're developing the code than catching it in production, right? Because then you may actually have an unforeseen outage.
00:35:39
Speaker
and which is definitely not a good thing for your customers and for the business. So that's the idea of shifting left if you heard of that term. You want to fail fast and move all your security as far left as possible in your CI CD pipeline.
00:35:58
Speaker
Yeah. So we have scanners, as I mentioned before. We have SAS and DAS scanners, so static application security and dynamic application security. We support a bunch of different languages and frameworks to get a list of everything that we support. It's only the documentation. But just to give you some examples of the things that we support,
00:36:23
Speaker
Let me find here my nodes here. For example, we support a generic test for your unit tests that will basically run all the unit tests in your code and it'll
00:36:39
Speaker
save the output of that and append it to the MR so you can see the output of the tests right in the MR. We support another scanner for code quality that's basically doing code checks on the code of your project.
00:36:56
Speaker
We support things like secret detection. It detects credential and secrets exposure. We support dependency scanning. This is the one that analyzes for project dependencies and security issues. We support license compliance scanners, which basically scans the license dependencies to see if they are
00:37:17
Speaker
incompatible or compatible with a set policy. We support auto container scanning and this identify security issues in your containers. Review apps, which I mentioned before, the ability to bring up, real-level bring up an ephemeral environment, deploy the application, you can test it with the stakeholders and collaborate and fix any issues.
00:37:43
Speaker
We support dynamic application security testing. We support browser performance testing that measures the browser performance of a web page, for example. We even support code intelligence. You know, this is code navigation features common in interactive development environments and OIDEs. Now, when it comes to
00:38:06
Speaker
runtime and or infrastructure specifically. We support operational container scanning, which is basically that scanner scans container images in your cluster, security vulnerabilities. We support infrastructure as code scanning, which scans all your IAC or infrastructure as code configuration files for non vulnerabilities. This is
00:38:32
Speaker
We support Terraform, Ansible, AWS CloudFormation, Kubernetes.
00:38:37
Speaker
Wow. Okay. So a quick question, right? I know this is a long list, but for all of these different kinds of scanning, like I know you said container scanning and image scanning, do you work, like, is all of this functionality built into GitLab or do you integrate with open source projects like an AquaSec Trivi or an Armor CubeScape, I guess, Armor CubeScape or those projects out there in the ecosystem? Right. So we include
00:39:04
Speaker
Everything I mentioned comes with GitLab. Now, everything we do in GitLab is open source, including GitLab itself. But we do leverage open source projects for some of these.
00:39:19
Speaker
I mean, I can, JUnit is one of them, for example, for unit testing. Okay. If you have JUnit, you can use that. But for each of them, we use different open source projects. This was like a long list. I was like, okay, let me just ask this, right? This is everything in GitLab or integrations. Okay, thank you. Thank you for answering that. You can even create your own scanner if you want to. Go ahead. Oh, nice. Yeah, that's a whole other level, I feel like.
00:39:47
Speaker
But probably useful, right, if you have a very specific use case. And we definitely point folks who are listening at where to find this information, where to create your own, where all those that are included as well. So I want to kind of go back to the concept of DevSecOps. Because I feel like just there, we talked about the concept of the CICD pipelines, the testing, sort of integration with Kubernetes. But it sounds like, to me, generally, the idea is kind of
00:40:15
Speaker
Like we said, shifting left, but also getting these conversations and awareness, the observability of the output of these scans built into the MRs, like you said. Visible for the developer is what I'm seeing is that we're making this security information based on the automation approaches available for the developer to start to be aware of the outcomes of
00:40:42
Speaker
essentially their actions, the code they're writing, right? Is that a decent way of explaining kind of what this observability, kind of visibility of security into Dev is sort of like? Right. So the main, if you can think, I mean, if we think about the main difference between DevOps and DevSecOps is that DevOps, you know, brings Dev and Ops together, but it
00:41:07
Speaker
it kind of implicitly mentions or describes security. Whereas DevSecOps explicitly introduces security to the DevOps processes, right? So there is that slight difference there. And to achieve this, the goal of DevSecOps is to reduce security risk, right? Principally for software, and in this case, infrastructure.
00:41:37
Speaker
and what really matters from a security standpoint and also provide automated ways to fix those issues. Besides detecting a vulnerability, GitLab provides recommendations on how to fix it and even training. If you have a training
00:41:59
Speaker
software of your choice that shows you how to fix specific things, you can even link that to GitLab. Makes sense. Now, I'm curious with your experience in this field, do you find that there's a lot of organizations who have adopted DevOps practices with things like GitOps and that are now thinking about how to embed security? Or do you find that people are thinking from
00:42:24
Speaker
you know, the get go how to involve security when they're sort of taking that DevOps journey. So from my experience, I mean, security, if you look at all the surveys that go back all the way to, I don't know, even I would say even 20 years ago, surveys that, you know, foresee level people. Sure. Security always comes out as number one or number two concerns or areas of interest.
00:42:54
Speaker
However, when it comes to development, it's always been sort of an afterthought because developers are really asked to deliver new functionality, right, and fixes. So security kind of always came towards the end as an afterthought, I should say. And so, DevSecos tries to correct that. This shifting left tries to correct that.
00:43:21
Speaker
But from experience, again, I think organizations are still working on that. I don't think the majority of organizations are running security tests or shifting left security testing.
00:43:42
Speaker
You know, I, you know, I don't want to guess on a percentage, but no worries. Yeah. But yeah, it's still happening. And I think it's something that, you know, you know, this concept of DevSecos is trying to address.
00:43:55
Speaker
Yeah, absolutely. And I think, you know, Bob and I through doing the show and also, you know, being at shows all through 2022, since we're sort of getting towards the end here, we've definitely seen security be sort of, like you said, one of the top concerns with, you know, in general, I think, not only in the Kubernetes community, but
00:44:14
Speaker
highly, you know, of importance in the cloud native community, which is great, I think. And if we're going to, you know, adopt new practices, I'm glad it's security, to be honest, right? So let's all be fair. All of our data is out there and something or other is going to be running on Kubernetes that we're interacting with.
00:44:32
Speaker
Cool. So actually, since we're talking about your experiments in the field and working with certain organizations, do you have any customer examples? We don't have to name names, but big or small, how they've implemented either DevOps, DevSecOps, GitOps, and talk about that experience maybe in the next
00:44:52
Speaker
Yes, for sure. But before I describe that, I'd like to say one more thing about including security into the pipeline. The idea of DevSecOps is also to automate all this so that when you are a developer,
00:45:10
Speaker
you know, security is being taken care of, but it's almost, it's getting out of your way, right? So that you can focus on innovating. And GitLab provides some mechanisms to do that besides, you know, all the scanners that I mentioned and the different types of security testing that I mentioned.
00:45:29
Speaker
for example, in the form of compliance frameworks and pipelines. So you can create a framework. For example, you want specific scans to be run for every project within your group. You can set up a framework and then you can set up a pipeline that will always be run and it needs to pass before the pipeline for that specific project runs.
00:45:53
Speaker
Right. So you can have a security team that is designing that and creating that, not the developers. And then when the developers run the pipelines behind the scenes, there's this compliance pipeline running for them. Also, you can use, we have deployment approvals now so that you can determine if you need extra approvals from the security team, for example, to be able to deploy to production, then you can have that set up so that
00:46:22
Speaker
you know, someone from the security team will have to approve something before it goes to production. And also we have policies related to scan execution and scan results. All right, so let's move on to your question, which is,
00:46:39
Speaker
examples of customers that are using DevSecOps and GitOps specifically.
Customer Success with GitOps
00:46:47
Speaker
So I won't mention the names because I'm not sure if they are public or not. They may be public. So there is this company that develops innovative software technologies focused on microservices and cloud native development. And they're pains or drivers
00:47:07
Speaker
to move on to GitOps was they were pushing Kubernetes cluster definitions manually, and they were just becoming more and more cumbersome. They were doing direct updates to Kubernetes clusters, and sometimes it would cause unexpected changes to the application behavior, and they would have to roll back whatever changes they had applied to the cluster.
00:47:34
Speaker
Also, they needed to use multiple clouds. The problem was that they had to configure each separately, which became more complex to add to the complexity. What they ended up doing was
00:47:53
Speaker
They implemented both the push and pull-based approaches to GitOps. I remember I mentioned the infrastructure as code scanning. They're using that and they provide the scan results in the MR so that stakeholders can see those.
00:48:13
Speaker
They use our integration with Terraform to install their Grafana stack on their clusters, which is made up of Prometheus, Loki, and Grafana, and also Jaeger for tracing.
00:48:32
Speaker
this integration for creating GKE, AKS, and EKS communities clusters from their pipelines as well as using Helm. Besides GitOps, they use GitLab for documentation, running static pages as well as the GitLab built-in registries for container and Docker images, NPM, and Terraform modules. You can store your Terraform modules within.
00:49:00
Speaker
within Gail up as well as your Docker and container images. Wow. I can't imagine without all of the tooling that we discussed in this episode. How complicated that all of that work would have been. Exactly. So the results that they saw, at the end, some of the benefits that they saw, this whole solution, it helped onboard new developers faster.
00:49:25
Speaker
so that they could start to get to work and innovate faster. Hence, they include the developer satisfaction. They improved the auditability of changes because everything that was captured in version control, not just the application, but also the infrastructure.
00:49:46
Speaker
And they also significantly improve the development efficiency across the organization as there is more sharing of reusable code between projects. So why did they pick GitLab and nobody else? So this is the feedback that we got from them.
00:50:05
Speaker
They chose GitLab because it's a single platform for application and infrastructure automation. The fact that GitLab supports multiple clouds, the fact that we supported Go, which is one of the many languages that we support, and the fact that we are open source and we're not proprietary.
00:50:25
Speaker
Yeah, I want to ask one question. I will make one comment at least based on that, which is, you mentioned the concept of manual changes to infrastructure. And I feel like, you know, when, when digging into GitOps, this is definitely one of those
00:50:41
Speaker
You know, sort of freebies, I feel like that comes when you're using GitOps, that not a lot of people recognize as a security thing, which is, you know, when you adopt GitOps, whether it's push or pull, you know, ideally in a Kubernetes environment, I like the idea of pull, but, you know, someone who might have admin permissions, right? Who maybe is used to going in and making like a change to a YAML file or an actual node or something like that.
00:51:08
Speaker
Those changes may really cause a lot of issues, and with GitOps, it actually gets reverted, right? I know. I like that GitLab has the drift detection. I was like, no, you can't do that. I would classify that almost as a security feature as well. I don't know about you, Cesar, if it kind of comes up in that.
00:51:28
Speaker
No, no, for sure. So you mentioned that you prefer the pool-based, which is the agent base, the in-cluster agent that is bringing changes from Git. I was thinking while you were talking about that, I was thinking the agent also supports the push base when you are pushing changes from the pipeline. And if you want to refine the
00:51:55
Speaker
the security of who can run what from the pipeline to the cluster, GitLab allows you to impersonate service accounts. For example, you may have created on your cluster and for whom you may have just given minor access to some resources in your cluster.
00:52:17
Speaker
And when Gil up in person is that person then or service account, you can only do whatever you decided to give that person the permission. So it's pretty granular. Yeah, that is that is super nice. I mean, so I don't think we're going to go into those other two use cases just because of the time we're at here. But I think we've covered a lot of ground today and really focused on sort of the benefits of of DevOps and GitOps and DevSecOps. I know it feels like it flew by, but I think we have a lot in here.
00:52:47
Speaker
But I think what we want to end with is for those who are listening who might be wondering where they can find out more or get started using these types of methodologies, where do they actually go to kind of find more information out?
Learning More About GitLab
00:53:05
Speaker
So there's the documentation. Number one, all of this is documented. Number two, if you go to our YouTube channel, there's a GitLab YouTube channel. There are tons of videos there, but I would recommend going to, if you're the technical type person, go to a playlist called
00:53:29
Speaker
snapshots, GitLab snapshots. And those are snippets, technical snippets of different things that you can do with GitLab for CI, CD, GitOps, SCM plan, et cetera. And if you have any ideas for more videos, please let me know. Shoot me a message, or you can find me on Twitter or LinkedIn.
00:53:54
Speaker
and we'll create a new video, short video about whatever topic you would like to see. The third thing is there's also GitLab Learn. There is a few learning paths in there. You can go and learn how to use different areas or different aspects of GitLab. And the blog site, the GitLab blog site, you can actually do a search and find, if you search for GitOps, you will find a big assortment
00:54:24
Speaker
a technical and non-technical blog posts about how to do GitOps with GitLab. Makes sense. Well, I will make sure we include all that information and the links that will take folks to those areas, as well as your Twitter handle. Actually, why don't you say it for folks? Yeah, so my Twitter handle, it says C-E-S-A-R underscore
00:54:53
Speaker
My last name is Saavedra without the A at the end. So my last name is S as in Sam, A-A, V as in Victor, E, D as in David, R, and that's it. Got it. There's just R at the end. And in case you didn't get that, we will play the integer note.
00:55:11
Speaker
No worries. Well, Cesar, it's been, I think, a great conversation around the concept of GitOps and DevOps and what GitLab is up to, what you are personally up to. I know I learned a lot today. So, you know, thanks for coming on the show and maybe we could do it again in the future. Yeah, for sure. Thank you so much for the invitation. And remember, we release on the 22nd day of every month. So we're constantly putting our features, new features in the hands of our community and customers.
00:55:41
Speaker
So, and also you can check the direction page for all of the feet, basically all the areas of the product. That's all public information. One of our values is transparency. So pretty much everything is public. So if you want to see where we're going with GitOps and DevSecOps, feel free to go to our directions page. Gotcha. That's a good day to remember. 20 seconds, man. Like, yeah. Every 20 seconds we put on a new release.
00:56:05
Speaker
Love it. All right, Cesar. Well, take care. You too. Thank you so much for the invite. Thank you. It was a pleasure talking with you. Well, Bobbin, I think that was a very enlightening conversation. I know I say that a lot in many episodes, but I really enjoyed our conversation with Cesar. He's clearly an expert in what he does and sort of how he works with customers. But let's go into our takeaways. What did you think about it?
00:56:30
Speaker
Yeah, it was a great episode, right? I know we ran a bit longer than we had anticipated, but that's what happens when you're having fun or you're talking about interesting topics. I think the three key takeaways that I got from the discussion with Cesar was the benefits of GitOps to operators specifically, so codification, collaboration and automation. I think that's a good way to put it in just a three keyword way of describing what the benefits are. It enables you to
00:56:57
Speaker
Git as that single source of truth enables more collaboration because everybody knows what's going on and then automation is because of the push and pull methodologies and the things that we spoke about in the actual episode, how it enables continuous deployment as well. I really like a couple of GitLab specific features.
00:57:13
Speaker
I know we have been talking about GitOps as an overall trend. The ephemeral clusters that it uses to test your code as part of the build pipelines or as part of the CI pipelines, that was really cool. And then the drift detection, right? Like with the agents that you have running on your communities cluster, it's always helpful that you notify or generate an alert when somebody, we can just blame an administrator who wants to go in and make manual changes.
00:57:38
Speaker
do your deployment objects or do your Kubernetes objects that are running on your cluster. So GitLab will obviously generate an alert and actually fix it for you. So that's a good thing to know when what you're trying to do is GitOps where you don't make any manual changes. So those were my three key takeaways.
00:57:53
Speaker
Absolutely. For me, the one I'm going to hone in on is that idea of shifting left on the lines of drift detection. I particularly liked as a developer, having those visual cues of maybe a lot of the security automation and scanners that SSR was talking about.
00:58:11
Speaker
sort of brought back into the development workflow. I know, you know, if you've worked with Git or GitHub or any GitLab, you know, you've worked with sort of the, you know, feature branch, you push, you let your automation run, but to have the sort of workflows directly there to give you feedback on sort of the scanners and maybe what it picked up from a security point of view, you know, not every developer is going to be an expert in security.
00:58:37
Speaker
Nor should they. I mean, obviously, I think we're shifting that way to allow developers to think that way. But it's hard to keep up in general with a single technology, a whole field. So I really like that concept. So again, for anyone who's listening, we will make sure all the links and places that Cesar was talking about, as well as his Twitter handle, will be in the show notes.
00:59:04
Speaker
And you can find out where to find out more. And we want to remind our listeners that, you know, that idea of sending us sort of a topic you're interested in, you know, again, if you want to record a little bit of about what you're working on or sort of a passion project, maybe,
00:59:22
Speaker
and send it to us. We're still interested in putting this sort of community episode together, so we really encourage you to do so. And, you know, next episode, which is in a couple of weeks, is going to be our last of 2022, believe it or not. I know this has been a busy, busy season.
00:59:38
Speaker
Yeah, season two has been very long compared to season one. We started halfway through the year with season one, so it makes sense. But next episode will be our last, and we're going to take a brief break and then kick it off with a bunch of great guests starting in January. So tune in to the last few episodes. And with that, that is the end of today's episode. So I'm Ryan. I'm Paul. And thanks for joining another episode of Kubernetes Bites.
01:00:07
Speaker
Thank you for listening to the Kubernetes Bites Podcast.