Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
Choose the right Infrastructure as Code
7 Plays2 years ago
In this episode we look at four different methods you can use to create Infrastructure as Code, and how to choose one that meets your needs. We will look at Cloud Formation, Chalice, Serverless Application Model (SAM) and the Cloud Development Kit (CDK). Each of these options has there own benefits and limitations, and each one fits its own use case. Choosing the one that is right for your project can make a big difference in the long run.
Recommended
Transcript
Introduction & Personal Disclaimer
00:00:00
Speaker
Hello basement programmers and welcome to the basementprogrammer.com podcast. My name is Tom Moore and I'm a developer advocate working with Amazon Web Services. The opinions expressed in this podcast are my own and should not be assumed to be the opinions of my employer or any other organizations I might be associated with. Also basementprogrammer.com is not affiliated with Amazon in any way.
Understanding Infrastructure as Code
00:00:23
Speaker
In today's podcast, I'm going to cover a question that comes up quite often. How do you differentiate between the different ways to create infrastructure as code? The idea of infrastructure as code has been around for quite some time. As virtualization became the standard way to spin up IT assets, we developed a need to be able to make the process more automated and more repeatable. If you use a script or a template to spin up identical virtual machines, then you have less risk of somebody accidentally missing a step or misconfiguring things.
00:00:54
Speaker
Add to this the ability to have your machines pre-configured with standard software and configurations and we really have a winning combination for IT administrators.
00:01:04
Speaker
Operating in the cloud enhances both the need and the opportunity for automation to create repeatable infrastructure deployments. This is because we have resources available on demand and we don't have to do forward planning to make sure we have the available infrastructure.
Advantages of Cloud Over Local Setups
00:01:20
Speaker
One of the great benefits of operating in the cloud is the ability to create entire environments that are either exact replicas of production environments or environments that are designed the same but have scaled down resources.
00:01:34
Speaker
Consider this, when I was a consultant and I was contracted out to a customer, we would spec out a production environment for the systems that we wanted to eventually run. The environments would be designed around the needs of the system when it was running in production. Let's say we were building out a web application. We might have two web servers running behind a local load balancer. We might have a database running in a high availability configuration. And we might have certain security constraints for the operating environment.
00:02:02
Speaker
However, when it came out to building the test environment, there often wasn't enough spare resources to match the production environment. So we got one web server and we got one database instance. And oftentimes the security constraints were loosened up. As we work our way back to the development environment, most of the time development was being done on our laptops or on a local VM where everything was installed on a single machine. And of course we had local administrative access.
00:02:30
Speaker
Unfortunately, a lot of errors can occur because of the environments being different.
AWS CloudFormation: Capabilities & Best Practices
00:02:35
Speaker
Are we using packages and libraries that are available locally, but not in the target environment? Are we depending on everything being installed on the same machine? Are we storing session data in local memory? Do we have a file that didn't get added into source control? These are some of the issues that created the it works on my machine problem.
00:02:56
Speaker
When we start moving into the cloud, we gain the ability to create an exact replica of the production environment for all of our environments, dev, test, pre-production, DR. They can all be set up exactly the same using the same infrastructure as Cloud templates. Now, when I say they can be exactly the same, in some cases, we probably want to scale down the hardware a little bit.
00:03:18
Speaker
Unless we're doing performance testing, most of the time these environments can have smaller instances than they do in production. The key factor here is to have the topography of the environment the same. For example, you might be running a C5 4XL instance in production where you're going to get thousands of hits per hour, but you might be fine using a T3 large in development where you're only going to have one or two users.
00:03:42
Speaker
When we start looking into AWS and the topic of infrastructures, code comes up, we're presented with a couple of different options. To start with, we have the oldest and most prolific option, CloudFormation. CloudFormation launched back in about 2010 and provides users with the ability to build out complete sets of infrastructure, known as a stack.
00:04:03
Speaker
Along the way, CloudFormation has added functionality included nested stacks, stack sets, and change sets to help us to manage more and more complicated infrastructure requirements. We also got drift detection to help us identify when the infrastructure has changed from what was deployed. If you have a CloudFormation template for your environment, you can easily deploy it across multiple AWS accounts or multiple times in the same account to build out your differing environments.
00:04:31
Speaker
A top tip here, it's best practice to separate environments into different AWS accounts. This really creates a hard boundary between the environments and limits the potential for accidental cross pollination of data. We really don't want our dev code to accidentally connect to our production database. CloudFormation is a great tool and you can build some very complex deployments.
00:04:53
Speaker
How complex? Well, check out the AWS Quickstarts, or the AWS Launch Wizard for some examples. Launch Wizard will help you spin up full sets of infrastructure to support SAP or Microsoft Exchange. If you're fluent in cloud formation, pretty much the entirety of AWS is open to you.
00:05:12
Speaker
Now, I'm gonna let you in a little secret here. Under the covers, all these tools I'm talking about today, baked down to cloud formation deployments. You can build an entire data center environment in a cloud development kit project using C sharp. What actually gets deployed into your account is a cloud formation template.
00:05:30
Speaker
Using CloudFormation natively will give you the widest set of services and functionality. Because everything else generates CloudFormation in the end, none of the other tools can create something that isn't already supported by CloudFormation. So if there's a new service or a new feature, you'll be able to leverage it in CloudFormation before any of the other options will have a feature available.
00:05:53
Speaker
Also, CloudFormation is designed to work across the entire landscape of AWS services, where some of the other options we're going to talk about have a more limited scope. However, CloudFormation expresses its constructs in either YAML or JSON. Its syntax is unique to CloudFormation as well, which can result in a learning curve for those who are unfamiliar with the syntax. It does take some effort to really learn CloudFormation well.
Tools for Python Developers: Chalice & SAM
00:06:22
Speaker
A little later, there was an open source product called Chalice. Chalice was designed to simplify the creation of infrastructure specifically for Lambda functions and related functionality. Chalice was focused on Python developers, and so Python developers were able to build out Lambda functions very easily and, for the most part, could completely ignore cloud formation if they wanted to.
00:06:45
Speaker
Chalice would also help out build the infrastructure for your CI-CD pipeline, it would create and link up API Gateway, and create IAM roles for your Lambda function to use. If you're a Python developer and you're focused on creation of Lambda functions, then Chalice makes the process very easy for you, to get started and to get some quick whims. In fact, you may find that Chalice becomes your toolset of choice long-term.
00:07:10
Speaker
The biggest advantage of using Chalice is that instead of focusing your attention on building infrastructure, you can focus on the high value code of your business logic.
00:07:20
Speaker
Chalice does, however, have some restrictions that will limit who can use it. First and foremost, as I said, Chalice works with Python code. If, like me, your primary programming language is C sharp, then you aren't able to use Chalice. The other limitation is the number of services supported. Once you start moving outside of Lambda and API Gateway, you're gonna need to find another toolset. The next item on the list is the AWS Serverless Application Model, or SAM.
00:07:50
Speaker
Now I have to admit here, I'm a bit of a Sam fanboy. I love Sam because like Chalice, it allows you to focus on the high value business logic and really gets the infrastructure out of the way. Sam is a broader scope and encompasses more services than Chalice. For example, Sam can not only create APIs and Lambda functions, but it can also create Lambda layers, DynamoDB tables, and state machines.
00:08:16
Speaker
A SAM template is really a cut down and drastically simplified version of CloudFormation. Because of this, the SAM specification allows you to include other CloudFormation snippets. So if something isn't a SAM resource, it can still be included in the SAM template.
00:08:33
Speaker
I was once working on a project where I had an SQS queue that my lambda function was receiving messages from. I was able to create the queue in the SAM template by providing the cloud formation for the queue along with my SAM definition. Another benefit of SAM over Chalice is that you have your choice of programming languages available. SAM can use any programming language that is supported by a native lambda runtime.
00:09:00
Speaker
Like Chalice, Sam abstracts away most of the cloud information work so you can focus on what's important, your business logic. However, because Sam does have a broader scope, it does set some complexity over Chalice.
00:09:15
Speaker
So when would you use SAM and why would you use SAM over Chalice? First off, you would use SAM if the majority of your work you're performing falls into the idea of a serverless application. So you really focus here on things like Lambda and DynamoDB. You'll also use SAM over Chalice if you want a programming language other than Python. Now, if you're a Python developer and you don't need any of the extra services that SAM provides, you can really choose either option depending on which one you like better.
Exploring the Cloud Development Kit (CDK)
00:09:45
Speaker
Now we come to the cloud development kit, or CDK. To be honest here, it took me a while to warm up to CDK, and mainly because, as I said before, I'm a bit of a Sam fanboy. I thought why would I use CDK when I can just use Sam?
00:10:00
Speaker
However, CDK does fill a different need, and once I understood that, I dove in and I have to say I really like CDK. CDK is much closer to cloud formation in that it opens up a large set of services across the AWS ecosystem. Unlike SAM, you can use CDK constructs to create S3 buckets, create EC2 instances, and configure CloudFront.
00:10:23
Speaker
CDK's big benefit to me is the fact I can write what is centrally cloud formation code using a programming language I'm proficient with. In my case, C sharp. However, you can also use Java, TypeScript or JavaScript. CDK also provides you with some higher level constructs that make things simpler by automatically having safe defaults for common properties, or in some cases, providing multiple related entities into a single higher level construct.
00:10:53
Speaker
One example, CDK has an instance class and a CFN instance class. The instance class has a higher level construct that simplifies the creation of an EC2 instance, while the CFN instance is a one-to-one map to the underlying CloudFormation EC2 instance.
00:11:13
Speaker
General bit of advice here, use the higher level constructs where possible and fall back to the classes that are prefixed with CFN if you find you can't use the higher level classes because you need some more control. So how do I choose whether I want to use CDK or SAM? If I'm building out a full set of infrastructure, I'm going to use CDK every time.
00:11:35
Speaker
If, however, I'm building out something that is really focused on serverless apps, and I might have one or two things outside of the SAM spec, I'm going to opt for SAM. Obviously, your choice for which option to use is going to depend a lot on which technology you're most comfortable with. Everything you can do in SAM, you can do in CDK, if that's what you're most comfortable with.
Choosing the Right Productivity Tool
00:11:57
Speaker
And everything you can do in any of these options can be done in CloudFormation.
00:12:03
Speaker
Now when it comes to infrastructure as code, there's also a lot of third party products as well that offer similar functionality to many of these tools. I won't cover any of the third party tools here because if I did, the episode would be extremely long. How do these products are out there and they are in use?
00:12:21
Speaker
One of the most important things to consider when selecting your tool from the list is going to be what's going to give you the most productivity. And keep in mind, one size does not necessarily fit all use cases. Personally, I mix and match between CDK and SAM depending on the project I'm working on.
Final Advice on Pipelines & Contact Information
00:12:38
Speaker
One thing that is for certain though, once you settle on a tool for the project, do yourself a favor and get it into a pipeline. This will help you in six months when you can't remember what you did, or how you did it, or even why.
00:12:51
Speaker
Also, as I alluded to from the start, once you have a working set of infrastructure as code, use the same template to deploy all your environments. This way, the developers don't get hit with surprises when they're suddenly operating across multiple web servers, instead of a single instance with local access to the database.
00:13:09
Speaker
So there you have it, my discussion on the different options of infrastructure as code and where they all fit. Hopefully you enjoyed the episode. Feel free to provide me any feedback. The address is Tom, that's T-O-M, at basementprogrammer.com, and the link can also be found on the Basement Programmer webpage. Thanks, and see you next time.