Become a Creator today!Start creating today - Share your story with the world!
Start for free
00:00:00
00:00:01
What's new in Lambda
7 Plays2 years ago
AWS Lambda has been one of my favorite services since my early days working with AWS. Some recent releases are making Lambda even more accessible, and easier to program for more and more people. In this episode I am diving into some of the recent development that will make your life easier.
This includes:
The Lambda Powertools for .NET
https://github.com/awslabs/aws-lambda-powertools-dotnet
and the .NET Lamnda Annotations Framework
https://aws.amazon.com/blogs/developer/introducing-net-annotations-lambda-framework-preview/
Recommended
Transcript
Introduction and Disclaimer
00:00:00
Speaker
Hello, basement programmers and welcome to the basementprogrammer.com podcast. My name is Tom Moore and I'm a developer advocate working for Amazon Web Services. The opinions expressed in this podcast are my own and should not be assumed to be the opinions of my employer or any other organizations I may be associated with. Also, basementprogrammer.com is not associated with Amazon in any way.
AWS Lambda Evolution and Versatility
00:00:24
Speaker
In this month's episode, we're going to talk about the AWS Lambda service and some of the new developments that have happened.
00:00:31
Speaker
Back when I started at Amazon Web Services, there was only one serverless option, and that was AWS Lambda. Even back then, Lambda was extremely useful and versatile. Throughout the years, some of Lambda's original constraints have been relaxed, new runtime options have been added and updated, and as a result, the amount of use cases that can be covered by Lambda has really exploded.
00:00:55
Speaker
These days, there are more serverless options available to give us additional serverless compute options. However, Lambda remains one of my favorite services because it's highly versatile in nature.
Demystifying Serverless Computing
00:01:07
Speaker
Now if you've ever attended any of my presentations, especially my introductory presentations, you may have heard me say something like, there's no magic to the cloud. And this applies to serverless just as much as it does everywhere. Serverless really isn't magic. It still does actually run on servers. What serverless does mean, however, is there's no servers for you to manage.
00:01:30
Speaker
All the hard work is taken care of by somebody else. Because of this, you can focus on the real value, the business logic that goes into your function, rather than focusing on the undifferentiated heavy lifting of standing up infrastructure and plumbing code.
Lambda's Role and Vendor Lock-In Concerns
00:01:44
Speaker
Now fundamentally, Lambda is an execution environment. The main parts of the service are dedicated to taking your code, putting that code into the execution environment, connecting the execution environment to a trigger, and then executing it for you.
00:01:58
Speaker
Obviously, there's a lot of work that has gone into Lambda over the years, and I'm really oversimplifying things when I suggest it's just an execution environment. However, from the end user perspective, that's what we see.
00:02:12
Speaker
Now, the idea that Lambda is just an execution environment leads me to the first point I want to make. Over the years, I've seen and heard a lot of customers suggest that they might not want to use Lambda because they're afraid of vendor lock-in. These people are concerned because there is no other Lambda out there, and they could not use Lambda on-premises if they ever decided to move away from AWS, for example. Once again, though, Lambda is just an execution environment. The code you're writing is the real value.
00:02:41
Speaker
and that code can have as little knowledge of AWS as you like. The code you are writing really is just JavaScript or Python, or in my case, C sharp and .NET.
00:02:53
Speaker
If your lambda function is doing some form of business value function, you can easily write that code in a way that knows absolutely nothing about the execution environment. Now, once you've done that, you simply write the lambda event handler and call your business logic from that handler. If you decide to change where you're running the code, it's a simple manner of writing a new entry point for your new execution environment and calling the same business logic code from that endpoint.
Lambda Use Case: S3 Data Processing
00:03:20
Speaker
Let's look at a very common use case these days.
00:03:23
Speaker
Let's imagine that I have some data that's streaming in from a whole bunch of endpoints, and that data is all landing on S3 and data files. Maybe these data files are JSON, maybe the CSV files, whatever the case may be. Now in this case, let's assume that there's some data in those records that I need to process. Maybe I need to translate it, or even delete some fields as the records come in.
00:03:46
Speaker
The specifics don't really matter so much as the idea that I'm using an event-driven Lambda to do that processing. Lambda's a perfect solution in this use case as there are built-in integrations for Lambda to be triggered from S3 as files arrive. However, I'm also concerned with vendor lock-in. I can write all my record and file processing logic and functions that know absolutely nothing about AWS Lambda or S3.
00:04:14
Speaker
In this case, once I've got the file data coming into the function, I process the data and send it back. Then, all I have to do is write the lambda event handler to retrieve the lambda event, pull the file from S3, and pass that file data to my processing function. This separates my high value business logic from the AWS specific pieces that are really just plumbing code.
00:04:38
Speaker
Now let's say I need or want to run that code on premises. I absolutely can. All I need to do is write some code that handles whatever the alternative event processing is and wire up that code to the same business logic. Maybe it's an app that periodically checks a server directory for files. Maybe it's an integration with some third party scheduler.
00:05:00
Speaker
The point is there's no vendor lock-in. The value of my code is the business logic, and you can completely abstract that knowledge away from the execution environment.
00:05:10
Speaker
Now I hear some people saying, but I still have to rewrite some code. The reality is, you're probably going to have to rewrite some code any time you move from one execution environment from another. Maybe you shift from running your code as a console app to a Windows service. Maybe you change from a service to a container. Each of these changes require some form of change to your code, and Linda really is no different. By keeping the business logic separate from the plumbing, you're in pretty good shape.
00:05:40
Speaker
and you can have any number of different execution environments handled in the same code base.
00:05:46
Speaker
The point I'm trying to make here is that it's even more valuable when you think about the lambda engine room. All that stuff you don't see that gives you that just-in-execution environment. Behind the scenes are years of development effort putting into building services out that is highly resilient and scalable for you. It's a massive amount of learning and experience that you don't have to go through.
00:06:10
Speaker
And, if we consider purely from a cost perspective, there are fleets of infrastructure that you aren't paying for standing by so you can run your function. My point here is, though, don't let the idea of vendor lock-in block you from taking advantage of something like Lambda, especially where you can easily build in that escape hatch into your code that, quite frankly, you probably won't use anyway.
AWS Power Tools for .NET Overview
00:06:34
Speaker
Now diving into that idea of building out business logic as Lambda, this is where we can dive into some new functionality that's really exciting. Let's start with the AWS Power Tools for .NET. It's an exciting release because it's fairly new. Now what are the Power Tools? Simply put, the Power Tools are a set of common functionality that you can add into your Lambda code that AWS has already written for you.
00:06:58
Speaker
You can add them in, and because this is .NET we're talking about here, it's a simple NuGet package. So the PowerTools includes utilities for logging, publishing metrics, and adding tracing into your Lambda function. Now the PowerTools is logger output structured JSON. It takes care of things like serializing complex objects to make your logging easier.
00:07:22
Speaker
The Metrics tool is designed to help you publish custom metrics out of your Lambda function and land them into CloudWatch. This is really helpful when you need to understand what your application as a whole is doing across multiple Lambdas and multiple executions.
00:07:39
Speaker
And finally, the tracing power tool helps you connect your custom code into AWS X-Ray so you can get valuable insights into how well your Lambda is performing and where all your execution time is going. Does it seem like your Lambda is taking a long time to execute? Well, tracing will tell you where all that execution time is going and help you find out where you really need to make improvements to help with bottlenecks.
00:08:04
Speaker
Lambda Power Tools is available in .NET, which is obviously why I get excited about it, but it's also available in Python, Java, and TypeScript. Given that the Power Tools address some very common requirements, I suggest giving the Power Tools a look as you start writing new Lambda functions, or maybe even as you start updating some of your older functions.
00:08:25
Speaker
Not only are these power tools free, they're open source. So you can find them on GitHub under AWS labs. That's github.com forward slash AWS labs. So discussing the idea of power tools leads us to the idea of Lambda development and how we're going to go about writing Lambda functions.
Developing with AWS Toolkit and SAM
00:08:46
Speaker
Now, like most people, I wrote my first Lambda function by opening the AWS console and typing a JavaScript function into the console.
00:08:54
Speaker
This really gave the impression that Lambda was magic. Now, of course, this is fine when you're using interpreted languages like Python or Node, and when your function isn't too complex, assuming you don't really care about things like source control. When AWS brought out Cloud9, the development experience inside of the console got a major upgrade.
00:09:16
Speaker
But then again, coding Lambda and the console has some pretty limited use cases. It's great for playing around, but seriously, don't do this for anything real. There are much better ways. Now if like me, you're cutting code in .NET, you have a couple of really great options available for you to develop your Lambda code. First off, there's the AWS Toolkit for Visual Studio. This comes with pre-baked templates for running Lambda in Visual Studio.
00:09:42
Speaker
If you use Visual Studio and you're working in AWS for anything, you really owe it to yourself to check out the AWS Toolkit for Visual Studio if you haven't already. It's incredibly helpful, and it connects your Visual Studio environment into your AWS account. There's a wealth of functionality there, including the AWS Explorer, templates for common projects, and the deployment tools.
00:10:04
Speaker
In fact, another new development that's come out fairly recently is the new deployment experience that can help you deploy your project into AWS, even if you don't know the best way or the best service to deploy your project to. The streamlined deployment experience will figure it out for you and guide you through that process.
00:10:26
Speaker
Another way to develop your Lambda functions for .NET is to have a look at the AWS serverless application model, or SAM. If you listen to my previous podcast on infrastructure as code, I'm a self-admitted SAM fanboy. SAM is a really great toolkit that helps you build serverless applications in a shorthand format. It focuses on the serverless ecosystem and includes tools that help you make your life easier.
00:10:53
Speaker
Now once again with SAM, you're focusing your efforts on high value business logic and not plumbing code. Much like the power tools, SAM is open source as well. So you can have a look at it and even contribute to the code base.
00:11:06
Speaker
SAM supports any programming language that is supported by a Lambda runtime, and has tools to allow you to test your Lambda functions locally using a Docker container to host your Lambda code and runtime. SAM also has functionality that will help you generate copies of the JSON that gets fed into your Lambda as a mock-up for event data, which really helps testing locally as well as in the console.
00:11:29
Speaker
The lambda code you develop with SAM can also employ the AWS lambda power tools for .NET that I mentioned earlier. So the two work nicely together. Simply add the packages into your project and you get and you're ready to go. Personally, I will often code functions for SAM using Visual Studio code. However, you can just as easily use Visual Studio or any other editor that you are comfortable with. This gives SAMs a huge amount of flexibility in how you develop functionality.
CI/CD Pipelines with SAM
00:12:00
Speaker
Another really great addition to SAM, and it's been out for a while, but I really do like to mention this, is the ability for SAM to automatically create your CICD pipeline. Now, SAM has a command line interface, so it's always been easy to incorporate this into any CICD build process. However, for customers starting out completely from scratch, this meant a learning curve, as they got familiar with the tools.
00:12:24
Speaker
Now with this functionality in place, users who don't currently have a CI-CD pipeline for the lambda functions can have one created for them in a simple command line. Use it as it is, or use it as a foundation and customize as needed. It's completely up to you.
00:12:40
Speaker
One challenge that many customers had when they were starting out with developing lambda functions was the idea of adjusting to the lambda programming model. Many of us were used to the programming model where we presented a function that was called directly. It had arguments that we specified in the function signature and we just went down and got down to programming.
00:13:00
Speaker
Lambda created a whole different programming model where we had to respond to events. The event structure means we didn't simply create the parameters that we needed, rather we had to contend with the event data that was being handed to us and had to unwrap that data into something we could work with. Well, that's all about to change with some new exciting functionality called the .NET annotations Lambda framework.
Simplifying Development with .NET Annotations
00:13:25
Speaker
It's going to completely change how you're able to develop Lambda functions.
00:13:29
Speaker
Now don't worry if you're currently using older methods for writing lambda. These extensions won't change anything for you. And your code will continue to work just fine. However, the annotations framework will allow you to create function signatures as you normally would. So with your typical parameters getting passed into the function, as well as typical return values that you would expect. So maybe just a string or an integer or a named parameter and some typical data as your return value.
00:14:00
Speaker
The lambda annotations framework then allows you to add attributes to your function to take care of all the hard work for you. These attributes act at compile time to create the code necessary for your named parameter, to pull your name parameters from the request object that's been passed into your function, and also to wrap the return function into an appropriate lambda return code for you.
00:14:26
Speaker
Not only does this allow you to simplify your function design by eliminating a lot of the plumbing code, it also goes directly to the earlier issue that we were talking about about the idea of not worrying about vendor lock-in when constructing your lambdas. In some cases, the only changes you'll have to make to your existing code now are adding a few attributes. It doesn't get much easier than that.
00:14:50
Speaker
Now the annotation framework is still in preview, so you can start testing it out easily enough, but you probably don't want to jump directly into using it in production code. But you can get started today. There's even a project template available in the Visual Studio Toolkit that I mentioned earlier. See, you really should install that toolkit as soon as you finish with this episode. You should also do a quick search in the AWS Developer Tools blog for the post announcing the preview of the annotations framework.
00:15:21
Speaker
It was published on April 13th by Norm Johnson.
Conclusion and Listener Engagement
00:15:26
Speaker
While Lambda is definitely a superstar in the serverless space, there are other serverless compute options around. I don't have time to dive into them in this episode, but we certainly can feature it in a future podcast. If you have a service or a topic you want me to cover, drop me a line and let me know.
00:15:44
Speaker
So with that, I'll wrap up this episode of the Basement Programmer podcast. I hope you enjoyed it. If you did, please subscribe. As always, if you have comments about the podcast, please feel free to drop me an email. The address is tom, that's T-O-M, at basementprogrammer.com. Always happy and eager to receive feedback. Thanks for listening, and I'll catch you next month.