What is AWS Lambda (Best Tutorial 2019)

What is AWS Lambda

What is AWS Lambda Tutorial 2019

AWS Lambda provides is a code execution environment. You use AWS Lambda to run the code.This tutorial explains the what is AWS lambda and how AWS lambda works with best examples 2019.

 

The AWS Lambda environment can work with any application or back-end process, and Amazon automatically provides the scaling needed to ensure that the code you upload runs as expected (keeping in mind that the cloud environment can’t run code as quickly as a dedicated host server can).

 

Using and configuring AWS Lambda is free, but you do pay for the computing time and other resources that the code uses, so keep costs in mind when you use this service. Typically, you use AWS Lambda in one of two ways

As a response to an event triggered by a service or application

 

As part of a direct call from a mobile application or web page, AWS Lambda doesn’t cost you anything. However, Amazon does charge you for each request that your code makes, the time that your code runs, and any nonfree ­services that your code depends on to perform useful work. In some cases, you may find that a given service doesn’t actually cost anything.

 

For example, you could use S3 with AWS Lambda at the free-tier level to perform experimentation and pay only for the code requests and running time. The examples in this blog don’t require that you actually run any code — you simply set up the application to run should you desire to do so, but the setup itself doesn’t incur any cost.

 

Considering the AWS Lambda Features

AWS Lambda Features

Before you can actually work with AWS Lambda, you need to know more about it. Saying that AWS Lambda is a code-execution environment is a bit simplistic;

 

Lambda provides more functionality because it helps you do things like respond to events. However, starting with the concept of a serverless code-execution environment, one that you don’t have to manage, is a good beginning.

 

The following sections fill in the details of the AWS Lambda feature set. Even though this information appears as an overview, you really need to know it when working through the examples that follow in this blog.

 

Working with an AWS server

What is AWS Lambda

Most applications today rely on a specific server environmentAn administrator creates a server environment, either physical or virtual, configures it, and then provides any required resources a developer may need.

 

The developer then places an application created and tested on a test server of (you hope) precisely the same characteristics on the server. After some testing, the administrator comes back and performs additional configuration, such as setting up accounts for users.

 

Other people may get involved as well. For example, a DBA may set up a database environment for the application, and a web designer may create a mobile interface for it. The point is that a lot of people get involved in the process of getting this application ready for use, and they remain involved as the application evolves.

 

The time and money spent to maintain the application are relatively large. However, the application environment you create provides a number of important features you must consider before moving to a serverless environment:

  • The server is completely under the organization’s control, so the organization chooses every feature about the server.
  • The application environment tends to run faster than even the best cloud server can provide (much less a serverless environment, in which you have no control over the server configuration).
  • Any data managed by the application remains with the organization, so the organization can reduce the potential for data breaches and can better adhere to any legal requirements for the data.

 

Adding more features to the server tends to cost less after the organization pays for the initial outlay.

  • A third party can’t limit the organization’s choice of support and other software to use with the application, nor can it suddenly choose to stop supporting certain software functionality (thereby forcing an unexpected application upgrade).
  • Security tends to be less of a concern when using a localized server as long as the organization adheres to best practices.

 

Working in a serverless environment

Lanbda_server

Using a localized server does have some significant benefits, but the building, developing, and maintaining servers is incredibly expensive because of the staffing requirements and the need to buy licenses for the various pieces of software.

 

(You can mitigate software costs by using lower-cost open source products, but the open source products may not do everything you need or may provide the same services in a less efficient environment.)

 

However, organizations have more than just cost concerns to consider when it comes to servers. Users want applications that are flexible and work anywhere today. With this in mind, here are some reasons  that  you  may  want  to  consider  a  serverless  environment  for  your application:

 

Lower hardware and administration cost: You don’t have hardware costs because Amazon provides the hardware, and the administration costs are theoretically zero as well. However, you do pay for the service and need to consider the trade-off between the cost of the hardware, administration, and services.

 

Automatic scaling: You can bring on additional hardware immediately without any startup time or costs.

Automatic scaling

Low learning curve: Working with AWS Lambda doesn’t require that you learn any new programming languages. In fact, you can continue to use the third-party libraries that you like, even if those libraries rely on native code. Lambda provides an execution environment, not an actual coding environment.

 

You use a Lambda function (explained in the “Creating a Basic AWS Lambda Application” section, later in this blog) to define the specifics of how your application runs. AWS Lambda does provide a number of prebuilt function templates for common tasks, and you may find that you can use one of these templates instead of building your own.

 

It pays to become familiar with the prebuilt templates because using them can save you considerable time and effort. You just need to tell Lambda to use a particular template with your server resources.

 

Increased reliability: In general, because Amazon can provide additional systems immediately, a failure at Amazon doesn’t spell a failure for your application. What you get is akin to having multiple sets of redundant failover systems.

 

Many of Amazon’s services come with hidden assumptions that could cause problems. For example, with Lambda, Amazon fully expects that you use other Amazon services as well. A Lambda app can react to an event such as a file being dropped into an S3 bucket by a user, but it can’t react to an event on your own system.

 

The user may drop a file onto a folder on your server, but that event doesn’t create an event that Lambda can see. What you really get with Lambda is an incredible level of flexibility with significantly reduced costs as long as you want to use the services that Amazon provides with it.

 

In the long run, you may actually find that AWS Lambda locks you into using Amazon services that don’t really meet your needs, so be sure to think about the ramifications of the choices you make during the experimentation stage.

 

Starting the Lambda Console

Lambda Console

The Lambda console is where you interact with AWS Lambda and gives you a method for telling AWS Lambda what to do with the code you upload. Using the Lambda Console takes what could be a complex task and makes it considerably easier so that you can focus on what you need to do, rather than on the code-execution details.

 

Lambda automatically addresses many of the mundane server setup and configuration tasks for you. With this time savings in mind, use these steps to open a copy of the Lambda console:

 

1. Sign into AWS using your administrator account.

2. Navigate to the Lambda Console at https://console.aws.amazon.com/ lambda. You see a Welcome page that contains interesting information about Lambda and what it can do for you. However, you don’t see the actual console at this point.

 

3. Click Get Started Now.

You see the Select Blueprint page. This blog assumes that you use blueprints to perform most tasks as an administrator (in contrast to a developer, who would commonly rely on the associated Application Programming Interface, or API).

 

If you’d like to use the API instead, you want to start by reviewing the developer-oriented documentation at https://docs.aws.amazon.com/lambda/latest/dg/lambda-introduction.html and then proceed to the API documentation at https://docs.aws.amazon.com/lambda/latest/dg/API_Reference.html.

 

However, using the approach in this blog works as a good starting point for everyone, even developers. You can access the AWS documentation pages at https://aws.amazon.com/documentation/ at any time, even if you aren’t logged into an account, so you can review this material at any time.

Creating a Basic AWS Lambda Application

lambda-app

The previous section discusses the Lambda console and shows how to start it. Of course, just starting the console doesn’t accomplish much. In order to make Lambda useful, you need to upload code and then tell Lambda how to interact with it. To make things easier, Lambda relies on the concept of a blueprint, which works much as the name implies.

 

It provides a basic structure for creating the function that houses the code you want to execute. The following sections describe how to create an AWS Lambda application and interact with the application in various ways (including deleting the function when you finish with it). Creating, configuring, and deleting a function won’t cost you anything.

 

However, if you actually test your function and view metrics that it produces, you may end up with a charge on your credit card. Be sure to keep the requirement to pay for code-execution resources in mind when going through the following sections. 

 

Selecting an AWS Lambda blueprint

AWS Lambda blueprint

Lambda supports events from a number of Amazon-specific sources such as S3, DynamoDB, Kinesis, SNS, and CloudWatch. This blog relies on S3 as an event source, but the techniques it demonstrates work with any Amazon service that produces events that Lambda can monitor.

 

When working with blueprints, you need to know in advance the requirements for using that blueprint. The blueprint name tells you a little about the blueprint, but the description adds to it.

 

However, the important information appears at the bottom of the box. In this case, you see that the blueprint uses Python 2.7 and S3. Every blueprint includes these features, so you know what resources the blueprint requires before you use it.

 

Determine the requirements for using a blueprint at the outset.

Amazon provides a number of blueprints and finding the one you need can be time-consuming. Adding a condition to the Filter field or choosing a programming language from the Language field reduces the search time.

 

For example, to locate all the S3-specific blueprints, type S3 in the Filter field. Likewise, to locate all the Python 2.7 blueprints, choose Python 2.7 in the Languages field.

 

A WORD ABOUT PRODUCT VERSIONS

PRODUCT VERSIONS

An interesting detail about the use of Python 2.7 is that it isn’t the most current version of Python available. Many people have moved to Python 3.4.4 (see the downloads page at https://www.python.org/downloads/ for details).

 

In fact, you can find versions as high as 3.5.1 used for applications now, so you may question the wisdom of using an older version of Python for your lambda code.

 

Python is unique in that some groups use the 2.7.x version and other groups use the 3.4.x and above version. Because developers, data scientists, and others who perform data-analysis tasks mainly use the 2.7.x version of Python, Amazon has wisely chosen to concentrate on that version.

 

(Eventually, all development tasks will move to the 3.x version of the product.) Using the 2.7.x version means that you’re better able to work with other people who perform data-analysis tasks.

 

In addition, if Amazon used the 3.x version instead, you might find locating real-world application examples difficult. The Python 2.7.x code does have compatibility issues with Python 3.x, so if you choose to use Python 3.x anyway, you also need to update the Amazon code. 

 

You may find that Amazon uses odd versions of other languages and products as well. In some cases, the choice of language or product version has to do with updating the Amazon code, but in other cases, you may find that the older version has advantages, such as library support (as is the case with Python).

 

Be sure to look at the versions of products when supplied because you need to use the right version to get good results when working with Lambda.

 

Amazon licenses most of the blueprints under the Creative Commons Zero (CC0) rules. This means that Amazon has given up all copyright to the work, and you don’t need to worry about getting permission to use the blueprint as part of anything you do.

 

However, the operative word in the Amazon wording on the blueprint page is “most,” which means that you need to verify the copyright for every blueprint you use to ensure that no hidden requirements exist that oblige you to get a license.

 

Configuring a function

Lambda_function

 

Using the Lambda console and a blueprint means that the function-creation process is less about coding and more about configuration. You need to tell Lambda which blueprints to use, but the blueprint contains the code needed to perform the task. In addition, you tell Lambda which resources to use, but again, it’s a matter of configuration and not actual coding.

 

The only time that you might need to perform any significant coding is when a blueprint comes close to doing what you want to do but doesn’t quite meet expectations.

 

However, you can use any bucket desired. The bucket simply holds objects that you want to process, so it’s a matter of choosing the right bucket to perform the required work. The blueprint used in this section, s3-get-object-python, simply reports the metadata from the objects dropped into the bucket.

 

1. Click s3-get-object-python. You see the Configure Event Sources page.

Even though the blueprint automatically chooses event-source information for you, you can still control the event source in detail. For example, you can change the Event Source Type field to choose a service other than S3, such as Kinesis, S3, CloudWatch, or DynamoDB.

 

2. Select an object source in the Bucket field.

The example assumes that you want to use the bucket tells you how to create. However, any bucket you can access that receives objects regularly will work fine for this example. AWS simply chooses the first S3 bucket, so configuring this field is essential.

 Bucket field

3. Choose the Object Created (All) option in the Event Type field.

S3 supports three event types: Object Created (All); Object Removed (All), and Reduced Redundancy Lost Object. Even though Lambda receives all the events, you use the entries in the Prefix and Suffix fields to filter the events so that you react only to the important events. 

 

For example, you can choose to include a folder path or part of a filename as a prefix to control events based on location or name. Adding a file extension as the suffix means that Lambda will process only files of a specific type.

 

The example provides simple processing in that it reacts to any item created in the bucket, so it doesn’t use either the Prefix or Suffix fields.

 

4. Click Next.

You see the Configure Function page. As with the Configure Event Sources page, it pays to check out the Runtime field. In this case, you can choose between Python 2.7 and Java 8. Even when the blueprint description tells you that it supports a specific language, you often have a choice of other languages you can use as well.

 

5. Type MyFunction in the Name field.

Normally, you provide a more descriptive function name, but this name will do for the example and make it easier to locate the function later when you want to remove it

 

6. Scroll down to the next section of the page.

You see the function code. At this point, you can use the online editor to make small coding changes as needed. However, you can also upload a .zip file containing the source code you want to use, or you can upload existing source code from S3.

 

This example relies on the example code that Amazon provides, so you don’t need to make any changes.

 

Notice that the Python code contains a function (specified by the def keyword) named lambda_handler. This function handles (processes) the information that S3 passes to it.

 

Every language you use has a particular method for defining functions; Python uses this method. As part of configuring the Lambda function, you need to know the name of the handler function.

 

7. Scroll down to the next section of the page.

You see two sections: the Lambda Function Handler and Role section and the Advanced Settings section. The blueprint automatically defines the Handler field for you.

 

Note that it contains the name lambda_ handler as the handler name. When you use custom code, you must provide the name of the function that handles the code in this section.

 

The first part of the entry, lambda_function, is the name of the file that contains the handler function. As with the function name, the blueprint automatically provides the appropriate name for you.

 

However, if you upload a file containing the code, you must provide the filename (without extension) as the first entry. Consequently, lambda_function.lambda_handler provides the name of the file and the associated handler function. The filename is separated from the handler function name by a period.

 

8. Choose S3 Execution Role in the Role field.

S3 Execution Role

You must tell AWS what rights to use when executing the lambda code. The environment provides several default roles, or you can create a custom role to use instead.

 

AWS opens a new page containing the role definition. AWS fills in the details for you. However, you can click View Policy Document to see precisely what rights you’re granting to the Lambda function.

 

9. Click Allow. AWS returns you to the Lambda Function Handler and Role section.

The Advanced Settings section contains the settings you need to use for this example. The Memory field contains the amount of memory you want to set aside for the Lambda function (with 128MB being the smallest amount you can provide). The Timeout field determines how long the Lambda function can run before AWS stops it.

 

Setting the value too high can make your application pause when it encounters an error (resulting in frustrated users); setting it too low may unnecessarily terminate a function that could complete given more time.

 

In most cases, you must experiment to find the best setting to use for your particular function, but the default setting provided by the blueprint gives you a good starting point. You use the VPC field to define which VPC to use to access specific resources. This particular example doesn’t require the use of a VPC.

 

10. Click Next.

Pay particular attention to the Enable Event Source checkbox. If you select this option and create the function, the function becomes active immediately. Amazon recommends that you leave this check box blank so that you can test the function before you enable it.

 

11. Click Create Function.

If you chose not to enable the function, the function exists, but it doesn’t do anything. You aren’t incurring any costs at this point. AWS displays. Note the Test button in the upper-left corner.

 

Using ensembles of functions

functions

Sometimes you can accomplish some incredibly interesting tasks without performing any coding at all by creating ensembles of functions available as blueprints.

 

For example, you can use the s3-get-object blueprint to retrieve objects from an S3 bucket of specific types and then pass the object onto DynamoDB, where another Lambda function, such as microservice-http-endpoint, passes it onto a microservice that your company owns for further processing.

 

You can even double up on blueprints. The same DynamoDB content can trigger another Lambda function, such as simple-mobile-backend, to send alerts to mobile users about new content.

 

You can achieve all these tasks without any significant coding. All you really need to do is think a little outside the box as to how you can employ the blueprints that Amazon provides and combine them in interesting ways.

 

For an example of a combined-service use, check out the article at https://micropyramid.com/blog/using-aws-lambda-with-s3-and-dynamodb/, which is about using S3 with DynamoDB.

 

These blueprints get real-world use by third-party companies that use the blueprint as a starting point to do something a lot more detailed and interesting.

 

[Note: You can free download the complete Office 365 and Office 2019 com setup Guide for here]

 

AWS Lambda Background

AWS Lambda Background

From its conception, Lambda was designed with a sort of “run and forget” model in mind. In other words, the developer provides the code and describes when it should be run (whether on-demand or in response to some event) and AWS takes care of the rest.

 

They provision the compute power, deal with code storage and updates, deploy the code to the required locations, scale up to match the appropriate demand, manage the disk space, memory, and CPU requirements of the underlying resources, and manage the networking, OS updates, and all the other requirements in between.

 

Compared to “traditional” models, or even workloads running on EC2, this is a vast amount of work that is being offloaded from developers to AWS. This is not without its downsides, so be sure to continue reading to make sure a Lambda replacement is appropriate for your project.

 

The Internals

AWS resources

Under the hood, Lambda really just uses all of the same AWS resources that you’re likely already familiar with. The code is stored on S3, metadata for the function is stored in DynamoDB, execution occurs from Amazon Linux EC2 instances, and the function assumes an IAM role.

 

While AWS likely added many new features to these existing services to create Lambda, they really didn’t reinvent the wheel for the service.

 

Despite the fact that many well-known services are in use, Lambda does require a different thought process and development pattern. We’ll be sure to cover how developing for Lambda differs from traditional computing in the coming blogs. For now, let’s look at the basics of a Lambda function.

 

Working with Events

Working with Events

In the previous blog, our “Hello World” function did not require any specific input; it was entirely self-contained and would return the same text regardless of how it was called.

 

In almost all cases, programmers want their programs to react to outside input and behave differently depending on what that input is. In this blog, we will look at the event object and show how it can be used to process user, or system, input.

 

To start, let’s modify our previous “Hello World” function to say “hello” to a specific user:

exports.handler = function(event, context) {

context.succeed(“Hello, ” + event.username);

};

As you can probably tell, the event object will need to contain a “username” property. Ideally, we would check for the existence of this property before attempting to access it (as well as pass it through a sanity check to ensure it’s a valid string), but let’s keep it simple for now.

 

Modify your function, resave it as a ZIP, and then return to the Lambda console for your “Hello World” function. Upload your new ZIP and then click “Save.”

Now, let’s test it again, but this time, change the sample event object to:

{

“username” : “Sally”

}

Your test output should now read “Hello, Sally.”

 

AWS Events

AWS Events

You are probably wondering how an event is created outside of the test dialog. In most cases, the event object is created by AWS when an event occurs. For example, S3 creates events when objects are created and removed.

 

If you’ve configured your Lambda function to act in response to such an event, the event object will contain information from S3 such as the bucket, key, request ID, time information, and much more. To properly create a Lambda function, you will need to look at the format of these events and process them accordingly.

 

At the time of this blog’s publication, AWS supports triggering Lambda functions from S3 create and remove events, DynamoDB row updates, CloudWatch Logs “put” events, SNS notifications, Kinesis events, and Alexa voice commands.

 

Exploring each of these services is beyond the scope of this blog, but I will be using S3 events frequently in my examples. As of now, they are relatively easy to test and don’t require setting up additional, costly resources.

 

The following sample event comes from an S3 “put object” request:

{

“Records”: [
{
“eventVersion”: “2.0”,
“eventTime”: “1970-01-01T00:00:00.000Z”,
“requestParameters”: {
“sourceIPAddress”: “127.0.0.1”
},
“s3”: {
“configurationId”: “testConfigRule”,
“object”: {
“eTag”: “0123456789abcdef0123456789abcdef”,
“sequencer”: “0A1B2C3D4E5F678901”,
“key”: “HappyFace.jpg”,
“size”: 1024
},
“bucket”: {
“arn”: “arn:aws:s3:::mybucket”,
“name”: “sourcebucket”,
“ownerIdentity”: {
“principalId”: “EXAMPLE”
}
},
“s3SchemaVersion”: “1.0”
},
“responseElements”: {
“x-amz-id-2”: “EXAMPLE123/e/yzABCDEFGH”,
“x-amz-request-id”: “EXAMPLE123456789”
},
“awsRegion”: “us-east-1”,
“eventName”: “ObjectCreated:Put”,
“userIdentity”: {
“principalId”: “EXAMPLE”
},
“eventSource”: “aws:s3”
}
]
}

As you can see, the event object contains a lot of information about the trigger of this event. Most useful to your function will likely be “event.Records[0].http://s3.bucket.name” and “event.Records[0].object.key”. Together, these can be used to download the object from S3 for future processing.

 

Custom Events

Custom Events

Earlier, when we tested our “Hello, {user}” function, we defined a custom event. In the cases of AWS-triggered events (like S3 objects being added), the event object is predefined for us.

 

However, when directly invoking a Lambda function, whether through the command line, one of the AWS SDKs, or through the “test function” feature used above, the makeup of the event object is left to the user.

 

This allows us to create functions that can consume pretty much any input (as long as it’s valid JSON). Take a look at the following example of triggering a Lambda function from the Node.js SDK:

var params = {

FunctionName: ‘test-function’,
InvocationType: ‘Event’,
Payload: JSON.stringify({username:“Sally”})
};
lambda.invoke(params, function(err, data){
// Handle err and data
});

 

This code is defining a custom event object, delivered in the “Payload,” which will be ingested and processed by our function. In fact, the Lambda test console is doing a very similar call behind the scenes.

 

At this point, we’ve covered enough information about events to move into more detailed concepts of Lambda. When working with Lambda, always keep in mind that it is event-driven; every time it executes, it’s because it was invoked by some event. These events are usually AWS-specific, but can also be user-generated.

 

The Context Object

Context Object

The second argument to the handler of a Lambda function is the context object. Whereas the event object provided information about the event that triggered the Lambda function, the context object provides details about the function itself and ways to terminate its execution, either successfully or with an error.

 

Properties

To demonstrate the properties of the context object, let’s update our “Hello, {user}” function to log some more information:

exports.handler = function(event, context) { console.log(“Request ID: ” + context.awsRequestId); console.log(“Log Group Name: ” + context.logGroupName); console.log(“Log Stream Name: ” + context.logStreamName); console.log(“Identity: ” + context.identity); console.log(“Function Name: ” + context.functionName); context.succeed(“Hello, ” + event.username);

};

 

Now, when you test the event in the console, you’ll still see the same “Hello, Sally” output, but your logs should contain many more details. The context properties are useful for debugging and can even be included within the app to make control flow decisions.

 

Methods

Methods

Besides its properties that provide information about the function and its current execution, you can also call methods on the context object. These are helpful for managing the closure of the function’s execution. The following methods deal with exiting a Lambda function:

 

●context.succeed()

Indicates that the function has completed successfully. You may optionally pass an object or string as a parameter which can be used by the calling event (for example, the API Gateway service can use the object in the HTTP response to the user).

 

●context.fail()

Indicates that the function has failed during execution. In some cases, this is used to requeue the function. For example, if a function fails in response to an S3 event, AWS will attempt to rerun the Lambda function two more times before giving up. Again, you can pass an optional parameter to context.fail to contain the reason for the failure.

 

●context.done()

This method simulates both the “succeed” and “fail” methods in traditional Node.js callback style. You can call context.done(err, data) where err is an optional error message (or null) and data is an optional success object.

 

●context.getRemainingTimeInMillis()

Returns the number of milliseconds remaining before Lambda will terminate the function. This is helpful to check if a function will have enough time to complete before taking on an additional workload.

 

The context is a very useful part of the Lambda function, and we will utilize its “succeed” and “fail” methods extensively, especially when working with the API Gateway.

 

Roles and Permissions

Roles and Permissions

AWS has encouraged the move from hard-coded AWS access keys and secrets to IAM roles in EC2 for quite some time.

 

IAM roles are a much more secure feature that allows a specific resource, such as an EC2 instance, to assume the privileges required to interact with other AWS services. It does this through AWS’s temporary tokens feature, which means that access for a specific resource can easily be modified or revoked at any time.

 

Policies

IAM roles are required for all Lambda functions. These roles define a specific policy which grant “allow” or “deny” permissions to other AWS resources. Below is a sample IAM role assigned to the execution of our previous “Hello World” function.

{

“Version”: “2012-10-17”,
“Statement”: [
{
“Effect”: “Allow”,
“Action”: [
“logs:*”
],
“Resource”: “arn:aws:logs:*:*:*”
}
]
}

 

As you can see, this permissions document allows the AWS Lambda function to perform CloudWatch Logs actions such as creating a log group, creating a log stream, and saving log entries into that stream. 

 

The above set of permissions is the minimum required for all functions. While it is possible to launch a function without these permissions, it will not be able to create any logs, thus leading to a very difficult debugging process.

 

If you’re concerned about log storage costs, AWS CloudWatch is pretty cheap, and also includes a free tier of several gigabytes. You can always reduce the log retention time to a few days or a week which will further reduce your costs. 

 

Unlike our “Hello World” function, most AWS Lambda functions are going to need to interact with other AWS resources. For example, a Lambda function that processes images uploaded to S3 will need permissions to read those objects from S3.

 

Fortunately, AWS has adopted its same IAM permissions model for Lambda, so creating policy documents should be both familiar and simple.

 

The following example gives the AWS Lambda function permissions to download, upload, and delete objects from a specific S3 bucket called “example-my-org”. Policies can be extended to any other resources as well. You can also attach AWS managed policies for more flexibility.

{

“Version”: “2012-10-17”,
“Statement”: [
{
“Effect”: “Allow”,
“Action”: [
“s3:DeleteObject”,
“s3:GetObject”,
“s3:PutObject”,
“s3:PutObjectAcl”
],
“Resource”: [
“arn:aws:s3:::example-my-org/*”
]
}
]
}
This should be very familiar if you’ve worked with IAM in the past.

 

Trust Relationships

AWS IAM

AWS IAM provides the ability for roles to be restricted to specific resources. For example, the above policy enabling access to S3 objects can be tied to the Lambda service so that new Lambda instances can assume the role.

 

This concept is called a “trust relationship” because it defines which components of the infrastructure you trust to assume the policy model you’ve developed. To apply this relationship to AWS Lambda, you must add the following policy document to the “Trust Relationships” section of the IAM role.

{

“Version”: “2018-10-7”,
“Statement”: [
{
“Sid”: ””,
“Effect”: “Allow”,
“Principal”: {
“Service”: “http://lambda.amazonaws.com”
},
“Action”: “sts:AssumeRole”
}
]
}

 

If you are using the AWS Lambda console creation wizard to create your functions, it will automatically create the policies and trust relationships required. However, I prefer to create the Lambda roles required directly within IAM (usually through CloudFormation) and then attach them when I create the function.

 

Console Popups

AWS is attempting to connect an event trigger (an inbound SES email in this example) with the execution of a specific Lambda function (“hello world” above). Clicking “Add permissions” will allow AWS to make this connection and give the source service the necessary IAM permissions to invoke the function.

 

Cross-Account Access

Cross-Account Access

In some cases, you may need to give permission to one AWS account that you trust to launch a Lambda function in your account. I’ve found that this is common in larger organizations where one team may operate the front-end of the environment (API servers, load balancers, etc.) and then pass traffic to a backend managed by another team.

 

In this situation, the front-end team could be given permission to invoke a function in the backend team’s account. This can be done via the command line with the following command using the AWS CLI.

aws lambda add-permission \
—function-name your-function-name \
—region us-east-1 \
—action “lambda:InvokeFunction” \
—principal 123456789012

To see all of the permissions assigned to the function, you can then run:

aws lambda get-policy —function-name your-function-name

If you need to authorize additional functionality, such as triggering a Lambda function in one account from an S3 upload event in another, AWS has written a blog post which dives into much greater detail than I can here.

 

Dependencies and Resources Node Modules

At their core, Lambda functions are simply Node.js (or Java or Python) code.

For this reason, dependencies function similarly to traditional Node.js programs in that they are installed via “npm install” to the “node_modules” directory. 

When uploading your code, the packaged ZIP must contain all dependencies required.

 

There are, however, a few exceptions: the AWS SDK module (“aws-sdk”), the AWS Lambda module (“aws lambda”), a DynamoDB interaction module (“dynamodb-doc”), and an image processing tool (“ImageMagick”). 

If your code relies on modules not listed, it will need to include them in the “node_modules” directory. You can then reference them:

var mymodule = require(“mymodule”);

 

OS Dependencies

OS Dependencies

Under the hood, AWS is running its Lambda functions in a secure, segregated environment on top of a traditional Amazon Linux EC2 instance. According to the Lambda docs, this instance is built from the publicly available Amazon Linux AMI, so that should be the machine of choice for developing and testing your functions with OS-level dependencies. 

 

Keep in mind that any additional libraries that are not included as part of the operating system must be included in your packaged function contents.

 

An example of a pre-included library is ImageMagick, an image processing library for easy image manipulation. The ImageMagick node module typically requires the underlying ImageMagick library to be installed.

 

But on the AMI used for Lambda, AWS has already installed these dependencies. We will explore image manipulation with Lambda in more depth in the coming blogs.

 

OS Resources

In addition to the pre-installed libraries and modules, AWS also provisions a small amount of space to use for temporary storage with your Lambda functions. As of publication time, this amounts to 500 MB of disk space located in the “/tmp” directory.

 

CPU and memory are allocated as part of the function creation process created above and network calls can be made, but you do not have access to additional directories on the underlying instance.

 

OS Commands

OS Commands

Occasionally, Node.js programs make use of OS-level commands via “exec” and “spawn”. In Lambda, you can utilize these OS functions to spawn native system calls.

 

In fact, up until AWS introduced Python support, a common workaround was to use Node.js to “exec” Python code in the background. Below is a sample program that calls “spawn” to list the files in a directory.

exports.handler = function(event, context) {
var spawn = require(‘child_process’).spawn;
var ls = spawn(‘ls’, [‘-lah’]);
ls.stdout.on(‘data’, function (data) {
console.log(‘stdout: ‘ + data);
});
ls.stderr.on(‘data’, function (data) {
console.log(‘stderr: ‘ + data);
});
ls.on(‘close’, function (code) {
console.log(‘child process exited with code ‘ + code);
context.succeed();
});
};
The logs for this function will produce something similar to:
START RequestId: 9754a20f-ad5b-4471-aaae-8639e00b034e Version: $LATEST
2015-01-01T21:12:02.263Z9754a20f-ad5b-4471-aaae-8639e00b034e
stdout: total 12K
drwxr-xr-x 2 slicer
497 4.0K Jan 1 21:12 .
drwxr-xr-x 20 root
root 4.0K Jan 01 20:06 ..
-rw-rw-r— 1 slicer
497 454 Jan 01 21:12 index.js
2015-01-01T21:12:02.303Z
9754a20f-ad5b-4471-aaae-8639e00b034e
child process
exited with code 0
END RequestId: 9754a20f-ad5b-4471-aaae-8639e00b034e
REPORT RequestId: 9754a20f-ad5b-4471-aaae-8639e00b034e
Duration: 178.50 ms
Billed Duration: 200 ms
Memory Size: 128 MB
Max Memory Used: 28 MB

 

If you wanted to do any kind of bash programming, this would be a great way to create your function in bash and then execute it via Lambda with Node.

 

Logging

Lambda functions

Lambda functions are actually relatively difficult to debug. Despite the fact that all “console.log” statements will be saved to CloudWatch, there is a huge amount of information included with each execution that makes locating relevant log entries difficult.

 

Each execution results in a log entry with the start time, a large, UUID string, the relevant event data, and another entry containing the end of the request, as well as a final line for the reported memory and time usage.

 

Additionally, multiple executions of the same function can be interspersed in the logs, resulting in a dizzying array of content that isn’t easily readable. For this reason, I recommend using a log format that enables you to easily filter out your application logs from the AWS-provided logs.

 

Personally, I’ve created a small NPM module that I can install in each project that can handle logging. Below is the code I’ve used to do this. It’s quite simple, but it does the trick (you could also use existing tools like “Winston” or “Bunyan” but they may be overkill).

module.exports = function(level) {
var levelValue = 100;
switch (level) {
case ‘TRACE’:
levelValue = 0;
break;
case ‘DEBUG’:
levelValue = 1;
break;
case ‘INFO’:
levelValue = 2;
break;
case ‘WARN’:
levelValue = 3;
break;
case ‘ERROR’:
levelValue = 4;
break;
case ‘FATAL’:
levelValue = 5;
break;
}
// Override all logs if testing with mocha
if (process.argv.join(”).indexOf(‘mocha’) > -1) {
levelValue = 100;
}
return {
trace: function(message) {
if (levelValue <= 0) { console.log(‘TRACE: ‘ + message); }
},
debug: function(message) {
if (levelValue <= 1) { console.log(‘DEBUG: ‘ + message); }
},
info: function(message) {
if (levelValue <= 2) { console.log(‘INFO: ‘ + message); }
},
warn: function(message) {
if (levelValue <= 3) { console.log(‘WARN: ‘ + message); }
},
error: function(message) {
if (levelValue <= 4) { console.log(‘ERROR: ‘ + message); }
},
fatal: function(message) {
if (levelValue <= 5) { console.log(‘FATAL: ‘ + message); }
}
};
};
Then, in my main code, I can do the following:
var logs = require(__dirname + ‘logger.js’);
var logger = new logs(‘INFO’);
// Application code…
logger.info(‘Some informational statement’);
logger.debug(‘Some debug statement’);

 

Because each log message is appended to its level, I can easily search CloudWatch for “INFO” or “DEBUG” to find the messages needed. Also, I can adjust the log level depending on the environment; staging may require a debug level, while production may only need warnings.

 

Searching Logs

Searching Logs

After your function executes, its logs will appear in CloudWatch within a few minutes. Each Lambda function creates a separate log group. Within the group, each execution instance creates a new stream. The actual logs are then added to the streams.

 

Do not confuse a stream with an individual execution of a Lambda function; oftentimes the same function will execute multiple times on the same underlying instance (especially if the function is invoked several times within a short time period).

 

Each underlying instance writes to its own stream, which may result in several executions being written to the same stream.

 

Finding the logs you’re looking for is as simple as entering a search term in the box. However, CloudWatch is notoriously limited in functionality, so if detailed log searching is important to you, I recommend a proper third party platform.

 

One additional feature of CloudWatch that I’ve found useful is the ability to configure metrics and alerts based on log patterns.

 

For example, if you’d like to receive an email every time your Lambda function logs the words “error: invalid”, you can. I will not delve into this in this blog since it is not Lambda-specific, but AWS has already created many useful tutorials for setting up CloudWatch metric alerts.

 

Testing Your Function

Testing Your Function

As with any programming cycle, it is very likely that you will need the ability to code, test, iterate the code, test again, and repeat. Because Lambda executes in a custom environment managed by AWS, it can be difficult to work through this cycle.

 

Fortunately, a number of open sources third-party libraries have sprung up to fill the need for testing.

 

Ultimately, these tools mimic the “event” and “context” objects, along with their properties and method to simulate the environment in which Lambda functions launch once uploaded.

 

In this blog, I’ll first work through the process of using the built-in testing functionality, then move to recommend a few third-party tools which I believe are much easier to work with.

 

Lambda Console Tests

Lambda Console Tests

When you upload your Lambda function to the AWS Lambda console, the ability to test it is built directly into the page. You simply need to provide an event (configurable via the interface) and you can see the logs and results immediately.

 

Within the console, click on your Lambda function and then select “Configure test event” from the “Actions” drop-down menu. In this window, you will be able to either work from a template provided by AWS or configure your own JSON body to send as an event. Remember, this event is parsed as the incoming event by your function.

 

Once you select an applicable event and submit, the function will run and output its logs and result to the page. As you can probably tell, re-uploading your function and navigating through these screens is not feasible. When developing, I’m constantly saving, re-running, editing, saving, and running again.

 

Third-Party Testing Libraries

Testing Libraries

Because of the difficulties described above, several third-party tools have been developed to simulate Lambda testing locally. Personally, I prefer “node-lambda” (node-lambda) because of its slim package size and rapid prototyping. Simulating events is as simple as describing them in a JSON file.

 

The node-lambda package has a lot more functionality that can help you create, edit, test, and deploy your functions. However, the inner workings of third-party modules are beyond the scope of this blog, so I suggest you simply install the module and test it out according to its README.

npm install -g node-lambda

Some other third-party Lambda testing modules include:

  • ●local-node-lambda: local-node-lambda
  • ●lambda-local: ashiina/lambda-local

 

Simulating Context

Simulating Context

If you’d rather create a method of testing Lambda locally on your own, you will need to simulate the event and context objects. however, to put this into the code, you can use the following template:

//

Create a sample event to test var event = {
key1: ‘value1’,
key2: ‘value2’
};
var context = {
succeed: function(event) {
console.log(‘Success’);
//
Do something with the event console.log(JSON.stringify(event,null,2));
},
fail: function(event) {
console.log(‘Fail’);
console.log(JSON.stringify(event,null,2));
},
done: function(event) {
console.log(JSON.stringify(event,null,2));
}
};
//
Call your handler handler(event, context);

 

As you can see, this is pretty simple; all you need is the ability to “succeed” or “fail” from the context.

 

Hello S3 Object

At this point, I believe we have covered enough basics to create a more complex Lambda function. The next function we’ll create will extend our existing “Hello World” function, but will instead download an object from S3 containing a user’s name, and respond with “Hello” to that user.

 

Additionally, we will setup Lambda to trigger this event when new objects are uploaded to an S3 bucket.

 

The Bucket

S3 bucket

First, let’s set up an S3 bucket that will be used to hold the uploaded objects which will trigger the Lambda function. Make sure the bucket is in the same region as the Lambda function you will create.

 

The Role

Since the Lambda function will need to download objects from the S3 bucket you just created, you will need to give it permissions to do this. Just like IAM roles associated with EC2 instances, Lambda uses IAM roles to assume the privileges provided to it.

 

Log into the IAM console and create a new role, giving it the correct trust relationship described in blog 8. Make sure that the necessary logging permissions are declared in addition to the S3 “GetObject” permission required for your code to execute. When finished, your IAM policy should look similar to:

 

{

“Statement” : [
{
“Effect” : “Allow”,
“Action” : [
“logs:CreateLogGroup”,
“logs:CreateLogStream”,
“logs:PutLogEvents”
],
“Resource” : “arn:aws:logs:*:*:*”
},
{
“Effect” : “Allow”,
“Action” : [
“s3:GetObject”
],
“Resource” : “arn:aws:s3:::your-bucket-name”
}
]
}
The Code
Note: this code is slightly adapted from the sample code available in the “Blueprints” section of the Lambda console.
var aws = require(‘aws-sdk’);
var s3 = new aws.S3();
exports.handler = function(event, context) {
//
Get the object from the event and show its content type var bucket = event.Records[0].http://s3.bucket.name;
var key = decodeURIComponent( event.Records[0].s3.object.key.replace(/\+/g, ‘ ‘)); var params = {
Bucket: bucket,
Key: key
};
s3.getObject(params, function(err, data) {
if (err) {
console.log(err);
context.fail(‘Error getting object ‘ +
key + ‘ from bucket ‘ + bucket +
‘. Make sure they exist and your ‘ + ‘bucket is in the same region as ‘ + ‘this function.’);
} else {
context.succeed(‘Hello, ‘ + data.Body);
}
});
};

 

This code will download the object that triggered the event from S3 and read its contents (“data.Body”), appending it to “Hello, “ to create the success response.

 

Save the above code in a file called “index.js” and ZIP it up. Then log into the Lambda console and create a new Lambda function using the ZIP. Make sure your handler matches “index.handler” and you select the correct role that you created above. You can select the lowest memory allotment and give the function a couple of seconds for processing time.

 

The Event

To fully understand the code above, This event would be used by AWS when triggering this function and contains data about the S3 event. In our code above, the bucket name and key of the object are used to download that same object.

 

While there is a lot of other information exposed in the S3 “Put” event (region, uploader’s IP, size of the object, etc.) we won’t be using them for now.

 

The Trigger

Trigger

After completing the steps above, you will need to create the trigger that connects an upload event on S3 to the execution of the function you’ve created. This can easily be done within the Lambda console by clicking on the “Event Sources” tab and adding a new source.

 

Select “S3” from the source list, and then choose your bucket. You can optionally provide suffix and prefix requirements so that only objects uploaded with the starting or ending string you define trigger the event. To determine which S3 events trigger an execution, select an event type from the drop-down list.

 

It is important to understand the differences between “Put,” (HTTP PUT by a client) “Post,” (HTTP POST by a client) “Copy,” (through the S3 console or API) and “Complete Multipart Upload” (direct multipart upload from the client) since they all are considered “Object Created” events by S3.

 

If you want each of these event types to trigger your function, you can simply select the parent element in the dropdown list.

 

Testing

Testing

To make sure everything works, create a simple text file containing your name and upload it to S3. You can then navigate to the “Monitoring” tab within the Lambda console and see if your function executed.

You can then click the link to be taken to CloudWatch Logs where you can see the full logs of your function’s execution.

 

When AWS Lambda Isn’t the Answer

As promised at the beginning of this blog, I will take a pause in the technical documentation for a bit to discuss the theory and hypotheticals involved when considering a switch to Lambda.

 

As I have mentioned, the event-driven computing model is one designed to solve a slightly different set of problems than traditional computing, but still shares some overlap. Working with Lambda requires a fundamental shift in the way developers treat the environment in which their code runs.

 

Host Access

Host Access

The first, and most obvious difference, between Lambda and traditional AWS setups is that access to the host instance is much more restricted. This affects everything from logging and monitoring to troubleshooting and development.

 

Previously, developers launching EC2 servers were essentially given free rein over the environment; they could SSH into the instance, download specific packages, utilize package managers, apply specific sets of updates, create periodic tasks, script workloads, and essentially have root or administrative access to a Linux or Windows server. With Lambda, none of this is true.

 

The concept of a “host” is abstracted away in favor of a code-focused development environment. While AWS has given us some clues into the actual host environment (an Amazon Linux AMI), everything else is quite opaque and access to this instance is severely restricted.

 

If you’re considering using Lambda, first ask the following questions:

  • Does your application rely on a heavily configured AMI?
  • Do you install multiple dependencies before running your application?
  • Do you frequently SSH into your host instances?
  • Does your application rely on native OS interaction (such as user accounts)?
  • Does your application require extensive testing or reconfiguration each time the host OS updates?
  • Do you tend to apply the latest OS patches and security updates minutes after they’re released?
  • Do you have multiple user accounts accessing your host instance via SSH/Telnet/some other protocol?
  • Do you require the ability to directly access system, access, and security logs of the host?
  • Does your application utilize utilities that change underlying OS settings or preferences (i.e. max open network connections, max file handlers, etc.)?

 

Do your auditing, compliance, or regulatory needs require you to save snapshots of the host OS during a security incident?

If you answered “yes” to some or most of these questions, Lambda may not be the best fit for your application. If, however, your application is not designed to be OS-dependent, and you can work without the traditional access to SSH, system logs, and other OS-level runtimes, then you should be able to begin converting your application to Lambda.

 

Fine-Tuned Configuration

Fine-Tuned Configuration

One of the most appealing features of Lambda is that AWS manages the entire host environment; the developer does not need to administer memory, CPU, disk, and networking performance.

 

However, this appeal can also be a drawback to some power users or simply to developers requiring more fine-tuned control over the environment in which their applications run.

 

One major caveat of using Lambda is that you must adjust the expected memory requirements of a function in order to increase CPU and networking performance.

 

While this is to be expected (almost all of the AWS EC2 instance classes scale other system attributes in tandem), it doesn’t allow for the fine-tuned configurations provided by EC2 instance classes that prioritize CPU, GPU, networking, or disk space and speed.

 

For example, the G2 class of GPU-optimized instances can provide massive improvements in GPU processing while the I2 class focuses on storage. With Lambda, these considerations are removed in favor of a single memory selection.

 

The developer has no insight into disk speed, GPU, networking, or CPU beyond the fact that “increasing memory will increase the other attributes similarly.”

 

Adding to the confusion, AWS does not make known exactly what instance class is being used on the host-level, so there is no way to tell if Lambda functions will have improved performance over a traditional EC2 service.

 

Purely from experimenting with various applications and performing stress tests across thousands of Lambda executions, I’ve determined that the most likely instance class is general purpose (M).

 

I have no way of independently verifying this, but as a rule of thumb, I would be sure to heavily test any applications you are considering moving from a different instance class before migrating it to Lambda.

 

For example, if your application is extremely GPU-intensive, I fear it would not function well on Lambda, and you may be better off leaving it on a GPU-enhanced EC2 instance.

 

Here are a number of questions you can use to determine if a move to Lambda would benefit your application:

  • Does your application currently run on any optimized EC2 instance class?
  • Do you perform custom, OS-level tuning before running your application?
  • Would you be uncomfortable not knowing the allotted networking speeds, CPU class, or other environment details of the instance running your application?
  • Is your application extremely dependent on GPU processing?

 

Again, if your answers are mostly answered in the affirmative, you may want to reconsider a move to Lambda. Unfortunately (or fortunately, depending on your viewpoint), Lambda removes the granular options available to users of its EC2 platform. If you can live with a single, memory-based, sliding scale of performance, then Lambda is an excellent choice.

 

Security

Security

One of AWS’s most-touted security features is the ability to create a completely private (software-based) network in which EC2 instances and infrastructure can be launched called a virtual private cloud (VPC). With the announcement of Lambda, VPC support was noticeably absent.

 

Additionally, AWS invests a lot of time, effort, and money in ensuring that its services are compliant across a wide range of privacy and security requirement programs. A majority of AWS services are PCI, ISO 9001, ISO 27001, SOC, and HIPAA compliant. Lambda has not yet obtained these same certifications.

 

While AWS has announced that VPC support is on the way and that bringing Lambda under the umbrella of many of their existing compliance certifications is a priority, the fact that they are not currently included may immediately disqualify Lambda from being used in a number of sensitive industries.

 

I have no doubt that Lambda was built with the utmost concern for security, but until the official paperwork is complete, most government, financial, and medical companies cannot use it for their workloads.

 

Beyond compliance requirements, Lambda functions quite similarly to launching new EC2 instances. Each function is given an IAM role, access can be shared across accounts using the cross-account role feature, and policy documents help determine which infrastructure resources (such as S3) can trigger a Lambda function with an event.

 

Because of the fundamental shift in architecture from a traditional host-based application to one where the application is running in an arbitrary container on an arbitrary instance determined by AWS, some host-level security insight may be lost.

 

For example, host-based intrusion detection systems cannot be installed, system-level access logs are not made available, and the developer has no way of knowing which steps AWS has taken to harden the host instance.

 

The understanding is that “AWS is taking care of security,” but that doesn’t quite provide a concrete answer to these questions, especially for security-paranoid industries.

 

The following questions can help you decide if relinquishing this tight management of security to AWS is beneficial to your organization:

  • Does your environment have specific compliance requirements or require certifications such as PCI, HIPAA, or ISO 9001?
  • Does your application consume, store, or manage sensitive user, financial, or company data (SSNs, addresses, credit card numbers, trade secrets, etc.)?
  • Do your security and compliance teams restrict yours from running applications in a non-VPC environment?
  • Do you install specific host-based security software (antivirus, firewall, intrusion detection, etc.)?
  • Can you launch applications without access to host security and access logs?
  • Do you trust AWS to manage your OS-level security requirements (patching, updates, etc.)?*

 

* Note: Despite the framing of this question, AWS is quite responsive to updating their Amazon Linux AMIs and responding to security incidents. In fact, AWS often receives embargoed security disclosures before the general public, so a particular security vulnerability may actually be patched before you’re even aware of it.

 

My personal suggestion is to wait for AWS to announce VPC support for Lambda before using it for any security-sensitive projects, but to consider it heavily for any other projects in your environment. Of course, if your industry requires a certain certification, you’re at the mercy of AWS obtaining it before you can begin working with Lambda.

 

Long-Running Tasks

Long-Running Tasks

Lambda excels at providing immediate, event-driven responses to triggers within an environment. Currently, it is not designed as a catch-all replacement for a traditional server.

 

The current timeout of 300 seconds makes it quite clear that AWS is designing this as a reactive, rather than long-running, service. Lambda cannot currently be considered for projects where any kind of extended access is required.

 

Even though Lambda provides access to a /tmp storage directory and you could technically have one function trigger itself again at the end of its execution time, there is no guarantee that successive functions will execute on the same underlying host.

 

Additionally, Lambda’s pricing model is not very favorable to this setup, as hypothetically having a single Lambda function with 1GB of memory running for an entire month would cost over three times as much as a comparable EC2 instance ($37 vs $9 at current costs).

 

If, however, you need to have a specific task execute at predefined times, and that task takes under 300 seconds to run, then it would be advantageous to create a Lambda function using the scheduled execution model.

 

Some applications, such as build systems like Jenkins or messaging servers like IRC, simply have no way of running as Lambda functions. Any application that requires the service to listen for incoming connections or that must wait for work to be complete elsewhere will not run properly on the Lambda platform.

 

Creating a New API, Resource, and Method

Creating a new API Gateway API is quite simple; just open the console and click “New API.” Give it a name and description, then click create. Click into the newly created API and create a new resource.

 

We’ll call it “hello.” Note that the resource path can be either a hard-coded string or a parameter denoted with brackets (such as {user}. While we’ll use a query string for the user’s name in this example, you could alternatively create a {user} child resource under “hello” (creating /hello/{user}).

 

Once the resource is created, add a method to it and select “GET.” This will then prompt you for the integration settings.

From here, you can connect your Lambda function by selecting the “Lambda Function” option and then locating the function you wish to use.

 

You’ll see a pop up asking you to approve the assignment of IAM permissions to your function which will allow the API Gateway to invoke the function. When finished, you’ll be presented with the settings page for the resource’s method.

 

While I won’t dive into each of these sections (they can become quite complicated very quickly), here is a brief overview that will provide a base level of understanding in order to configure the Lambda function properly.

 

●Method Request

Defines the format of the incoming request from the client. Possible URL query strings are added here, along with potential authorization options.

 

●Integration Request

Determines how the incoming request is mapped to a backend resource. It could be Lambda, a mock response, or proxied to another URL. Additionally, a mapping template is defined to convert the HTTP request into useable data on the backend.

 

●Integration Response

Maps the backend response to an expected format for the client. HTTP status codes and response headers can be set here (must be defined in Method Response first).

 

●Method Response

Defines the format of the outgoing response from the backend. Possible response headers are added here, along with potential HTTP status codes.

 

Initial Configuration

If we tested the API function now, the response will be the value of what is passed to “context.succeed()” in the Lambda function from earlier.

 

However, the CloudWatch logs for the Lambda function will not print anything for the value of the event. The reason is that we have not defined an input mapping to convert the incoming HTTP request information into an event object that is usable by Lambda.

 

Mapping Templates

To properly pass information from the API Gateway to Lambda, you must create a mapping template. To do this, go to the “Integration Request” section and add a new mapping template.

 

Enter “application/json” for the Content-Type and then click the check. In the box that appears to the right, select the pencil icon and change the selection from “Input passthrough” to “Mapping template.”

 

In the template area, enter the following code:

{
“httpMethod”: “$context.httpMethod”,
“resourcePath”: “$context.resourcePath”,
“body” : “$input.json(‘$’)”,
“headers” : “$input.params().header”,
“query” : “$input.params().querystring”,
“params” : “$input.params().path”
}

 

Testing the function again and then checking the Lambda CloudWatch logs reveals the following:

2015-01-01T10:30:00.630Z 8273ecfe-a373-11e5-384e-32a8288f87ba { httpMethod: ‘GET’,

resourcePath: ‘/hello’, body: {}, headers: {}, query: {}, params: {} }

As you can see, we’re now receiving information in the Lambda event that can be useful for creating an appropriate response.

 

Adding A Query String

Next, let’s add a query string for the user’s name to the API Gateway. This is done in the “Method Request” section by adding a new entry under “URL Query String Parameters.” Create one for “name” (be sure to click the check to save).

 

When testing the function again, you should now be prompted to enter a name. Provide some input and click “test” to observe how it is mapped within the event object.

2015-01-01T10:30:00.630Z 1353ecfe-f373-31e5-384e-32a8288f87ac { httpMethod: ‘GET’, resourcePath: ‘/hello’, body: ‘{}’, headers: ‘{}’, query: ‘{name=bob}’, params: ‘{}’ }

 

The query string value is now passed to the Lambda function as part of the event object. While it is possible to parse “name=bob” within the code, I personally prefer to offload as much work as possible from Lambda to the API Gateway.

 

The cost structure favors more work being done by the API Gateway since work performed there is not counted towards the execution time of the Lambda function. Instead of parsing the query string within Lambda, let’s update the API Gateway’s mapping template like so:

{
“httpMethod”: “$context.httpMethod”,
“resourcePath”: “$context.resourcePath”,
“body” : $input.json(‘$’),
“headers” : $input.params().header,
“query”: {
#foreach($query in $input.params().querystring.keySet())
“$query”: “$util.escapeJavaScript($input.params().querystring.get($query))” #if($foreach.hasNext),#end
#end
},
“params” : $input.params().path
}

 

After saving and testing again, the Lambda logs should now show the query string as an object instead of a string.

2015-01-01T10:30:00.630Z 5553ecfe-f373-31e5-384e-32a8288f87ae { httpMethod: ‘GET’, resourcePath: ‘/hello’, body: {}, headers: {}, query: { name: ‘bob’ }, params: {} }

 

This input mapping makes use of a language called “Velocity Template Language,” which you can read more about in the Apache documentation. You can update the “headers” and “params” to use the same process of converting the string to a useable object.

 

Using HTTP Request Information Within Lambda

Now that we have easy access to the “name” query string, let’s update the Lambda function.

exports.handler = function(event, context) {
console.log(event);
context.succeed({code:0, data: “Hello, ” + http://event.query.name});
};

 

Running the API Gateway test again should now display “Hello,” with the value of “name” that you entered.

 

Deploying the API

Once you have a working API, you need to deploy it. The API Gateway uses stages to allow you to update your code in one place without affecting what is running. You can then make updates and test on one stage while keeping a working production stage in use.

 

To deploy your API, simply click the “Deploy API” button and follow the steps to create the first stage. You can then create additional stages from the “stages” page. Once you deploy the API to a stage, you will be given a URL that looks like https://abc123def.execute-api.us-east-1.amazonaws.com/stage-name

 

While you can certainly use that URL directly, you can also create a custom domain name mapping in the API Gateway settings page. You’ll need to upload an SSL certificate, as the API Gateway enforces HTTPS-only usage.

 

Additional Use Cases

The demo above is only scratching the surface of what is possible when the API Gateway is connected to Lambda. Additional information from the HTTP request can be used to craft an entire REST API.

 

The URL parameter values can be used to determine which resource is created, modified, or deleted based on the HTTP method value. AWS has a number of examples of further integrating the REST API concept with DynamoDB for management of data.

 

Numerous projects have been developed that allow for easy creation of API Gateway and Lambda resources for a complete API. The most popular of these projects is called “serverless” (previously “JAWS”). It provides easy methods for developing, testing, and deploying a complete API to multiple regions.

 

As you can see, AWS expects the combination of API Gateway and Lambda to be able to replace a traditional EC2 API server. While the API Gateway is still a bit cumbersome to use, AWS is heavily invested in the future of Lambda and serverless computing and is continually providing updates.

 

Lambda Competitors

To call Lambda a completely new idea would be unfair to the countless companies developed around “serverless” approaches to computing. While I have not extensively used the following services, they all aim to solve the similar problem of wanting to execute code in an isolated environment without managing the underlying infrastructure.

 

If your organization is not entirely invested with AWS, you may find that these solutions work better for you. However, keep in mind that if you’re looking to develop Lambda functions that respond to other AWS events, it may be best to stick with other AWS products for compatibility and simplicity purposes.

 

DevOps Solutions from Startups to Enterprise

DevOps Solutions from Startups to Enterprise’s WebWorker service is perhaps the most similar service to Lambda. While both platforms support the same setup (uploading a Node.js file), DevOps Solutions from Startups to Enterprise has a longer timeout period (one hour vs Lambda’s five minutes).

 

A larger maximum memory options (2 GB vs 1.5 GB), and support for more languages (Go, PHP, Python, Node, Java, Scala, Ruby, and .NET vs Node, Python, and Java).

 

Predictably, DevOps Solutions from Startups to Enterprise also aims to be less tied to AWS and has the ability to process events from all major cloud providers. The pricing model is difficult to compare since DevOps Solutions from Startups to Enterprise works with monthly plans instead of per-request pricing.

 

StackHut

At its core, Lambda is really just a container running on top of an EC2 server. StackHut has a similar approach but uses a more open and familiar technology: Docker.

 

The service allows you to run Docker container microservices on-demand, similar to invoking a Lambda function. However, the service is more oriented towards long-running processes rather than many concurrent executions of the same process.

Read more about StackHut on their website or GitHub

 

Web task

Similar to StackHut, Web task is solving the server management problem by providing access to long-running containers on managed infrastructure. After your code is uploaded, Web task provides an API for invoking it. Pricing is based on the number of containers as well as the number of requests per second that each container can support.

 

One helpful difference between Web task and Lambda is the ability to respond to webhooks. While Lambda can be configured with the API Gateway to do this, WebTask tasks expose the functionality natively. Read more at their website.

 

Existing Cloud Providers

Google has a Lambda competitor called “Cloud Dataflow,” while Microsoft Azure has “Stream Analytics.” Even though these services are all designed to abstract away the management of the underlying infrastructure, they vary widely in their approach, pricing, and requirements.

 

The Future of Lambda

Lambda will continue to grow and evolve as a service that is tightly integrated with other AWS services. Even though S3, DynamoDB, SNS, Kinesis, and the API Gateway are currently set up to work with Lambda, I predict that, in the future, almost every AWS service will be tightly coupled to Lambda in some way.

 

One of AWS’s objectives is to tie users to the AWS ecosystem while still supporting a flexible development environment. By integrating Lambda in as many places as possible, AWS is essentially creating this vendor lock-in.

 

Lambda has many potential use cases beyond short-lived, moderate-memory processing. I anticipate that AWS will drastically raise not only the timeout but also the memory limits.

 

In the future, expect to see Lambda timeouts in the range of hours instead of just minutes. I also hope that AWS adds the ability to optionally declare CPU, disk, and network requirements in addition to memory.

 

This will make Lambda a much better drop-in replacement for EC2. Imagine having a function that can be designed for GPU processing, bandwidth intensive tasks, or tasks requiring extremely fast disk access. Regardless of how Amazon chooses to develop Lambda going forward, it has certainly shifted the realm of what is possible within AWS today.

 

More Resources

There are more possibilities with Lambda than I could ever fit into a single blog. Because of its flexibility, developers have been using Lambda for more scenarios than AWS likely imagined possible.

 

Throughout the web, there are numerous blog posts, third-party tutorials, walkthroughs, and resources that can aid you in your use of Lambda. I’ll provide a few of my favorites here, but the list is growing all the time.

 

Alestic.com

Eric Hammond is an AWS Power User who has been sharing his thoughts on AWS for some time. He has published a number of blog posts that dive into the technical specifics of Lambda as well as numerous use cases.

 

Before AWS announced scheduled Lambda tasks, he was responsible for the “Unreliable Town Clock (UTC)” Project which would run Lambda jobs at an interval.

 

●GitHub

AWS services tend to have large numbers of related open-source applications created by third parties and Lambda is no exception. I’ve found many new and helpful repositories just by periodically searching “Lambda.”

 

●AWS Blogs

Many AWS employees and guest authors post on the AWS Blog about Lambda. Most of their posts dive into concepts that are much more complex than what is explored in the traditional AWS documentation.

 

●Company Blogs

Numerous organizations, from startups to huge enterprises are either running Lambda in production or are considering it for numerous workloads. A good number of them also like to share their experiences with Lambda on their own company blogs.

 

It’s always great to read about Lambda from the perspective of an operations team that has actually implemented it in a real-world project. I’m a fan of posts by Airpair, AdRoll, and Enterprise Web Data Extraction and Analysis - Import.io - Data Extraction, Web Extraction, Web Data, Web Harvestingamong many others.

 

●Cloud Academy

Cloud Academy frequently creates course material designed to train users on various aspects of AWS. They have a number of blog posts as well as a complete lesson on Lambda.

 

●Udemy Courses

If you prefer hands-on exercises and content delivered in easily digestible chunks, then you’ll likely enjoy Udemy’s course software. I can’t personally vouch for the only Lambda course available currently but it does have a number of positive ratings from users who have completed it.

 

●Yourself

The best way to learn Lambda is to actually just try using it! Since Lambda costs are so small, you can upload multiple test functions, test them repeatedly, and learn how the service can work in your environment, all without spending a dime.

 

Conclusion

Computing has come a long way since the days of provisioning physical hardware for each server required. While AWS users are obviously quite familiar with the fairly standard practice of virtualization of server environments, Lambda represents one step beyond what has been the industry standard for some time.

 

In this rapidly advancing field, AWS has put forward what it believes is a solution to the problems of server management and system administration. By using proprietary technology, AWS can provide not only an easy-to-use system for its customers but also one that will tie them more closely to the overall AWS ecosystem.

 

As with any technology, Lambda is not the answer to every problem. As we’ve discussed in this blog, there are numerous use cases where Lambda just isn’t suited for the job. However, the number of possibilities for Lambda will only continue to grow as AWS continues to integrate it into additional services and build upon the platform.

 

My recommendation for your next steps is to take a high-level look at your environment. Ask yourself if you provide any services, run any scripts, or execute specific functionality at periodic intervals that could be replaced with Lambda.

At best, you’ll find that a majority of your current EC2-based platform can be replaced with Lambda. At worst, you’ll understand a new service and can consider its potential for future projects.

Recommend