How to Create API Gateway Using Terraform & AWS Lambda
Enterprise companies use Terraform to deploy their API implementations efficiently and reliably because it allows them to manage their infrastructure as code. This means they can automate deployments, ensure consistency across environments, and easily scale their operations.
- Automate Deployments: Instead of manually creating resources via the UI, developers can define and manage them as code with Terraform scripts, saving time and reducing errors.
- Ensure Consistency: Terraform ensures that all environments (development, testing, production) are set up the same way every time, preventing unexpected issues.
- Easily Scale Operations: When more resources are needed to handle increased traffic, Terraform makes it easy to add them quickly without manual setup.
In this blog, we'll guide you through the process of deploying a REST API on AWS using Terraform, a powerful Infrastructure as Code (IaC) tool, aditionaly you can find the source code here.
We'll start by explaining the basics of IaC and how Terraform can simplify and streamline your infrastructure management. Then, we'll dive into the practical steps, from setting up API Gateway and Lambda functions to configuring authentication with Cognito and securing your API with TLS certificates from ACM.
Introduction to AWS Services and Prerequisites
To follow along with this example, you'll need a basic understanding of the following AWS services:
What is a Serverless Architecture? Serverless architecture allows developers to focus solely on their code without managing the underlying infrastructure. AWS Lambda, coupled with API Gateway, dynamically scales resources in response to demand, eliminating the need for server management.
What is AWS Lambda? AWS Lambda is a serverless computing service that runs your code in response to events. It frees developers from provisioning and managing servers, ensuring optimal resource utilization and automatic scaling.
What is API Gateway? API Gateway is a managed service that makes it easy to create, publish, and manage APIs at any scale. It handles tasks such as accepting and processing API calls, including traffic management, authorization, and access control.
What is Amazon Cognito? Amazon Cognito simplifies user identity management, enabling user sign-up, sign-in, and access control. It supports various identity sources, making it easy to add authentication and authorization to your applications.
What is AWS Certificate Manager (ACM)? AWS Certificate Manager manages SSL/TLS certificates for your AWS-based websites and applications, ensuring secure communication. It automates certificate renewal and provisioning, maintaining the confidentiality and integrity of data.
What is Route 53? Route 53 is a scalable domain name system (DNS) service that translates domain names into IP addresses. It offers domain registration, routing, and health-checking capabilities, ensuring high availability and performance.
Step-by-Step Guide: Preparing the environment
1. Install Terraform:
- Download and install Terraform from http://terraform.io .
- Verify the installation by running terraform --version in your terminal.
2. Configure AWS CLI:
- Install AWS CLI from aws.amazon.com/cli.
- Configure your credentials by running aws configure and providing your Access Key, Secret Key, region, and output format.
- Or on the file /username/.aws/credentials paste the credentials following this steps
3. Folder Structure:
- We use modules to organize and reuse infrastructure configurations efficiently. Modules allow encapsulating a set of related resources and providing a simplified interface for their use. This promotes modularity, reuse, and easier maintenance of the infrastructure, and here is the folder structure:
/rest-api-aws-terraform
├── /src
│ ├── /lambdas
│ │ └── users.ts # Lambda function file for users
│ └── index.ts # Main entry point for src
└── /terraform
├── /modules
│ ├── /vpc
│ │ ├── main.tf
│ │ ├── variables.tf
│ │ └── outputs.tf
│ ├── /api-gateway
│ │ ├── main.tf
│ │ ├── variables.tf
│ │ └── outputs.tf
│ └── /lambda
│ ├── main.tf
│ ├── variables.tf
│ └── outputs.tf
├── /templates
│ └── swagger.yaml # Swagger template file
├── main.tf # Main configuration file for Terraform
├── variables.tf # Input variables for Terraform
├── outputs.tf # Output values for Terraform
└── terraform.tfvars # Variable values for Terraform4. Create a Project Directory:
- Create a new directory for your project, e.g., rest-api-aws-terraform.
- Navigate to this directory and create a file named /terraform/main.tf.
5. Define the AWS Provider:
- In /terraform/provider.tf, add the AWS provider configuration:
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
}
}
required_version = ">= 0.13"
}
provider "aws" {
region = var.aws_region
}6. Define the Necessary Resources: In /terraform/main.tf Define the necessary modules
- VPC MODULE: The VPC module creates a secure and isolated network environment in AWS.
module "vpc" {
source = "./modules/vpc" # Path to the VPC module source
}- API Gateway Module: The API Gateway module sets up an API Gateway that allows us to expose our Lambda functions as HTTP endpoints.
module "api-gateway" {
source = "./modules/api-gateway" # Path to the API Gateway module source
users_lambda_invoke_arn = module.lambda.user_lambda_arn # ARN for user Lambda invocation
aws_region = var.aws_region # AWS region for deployment
}- Lambda Module: The Lambda module creates and configures AWS Lambda functions.
module "lambda" {
source = "./modules/lambda" # Path to the Lambda module source
subnets_ids = module.vpc.subnets_ids # IDs of subnets from the VPC module
lambda_vpc_id = module.vpc.lambda_vpc_id # VPC ID for Lambda functions
}Configuring the VPC Module with Terraform
Creating and configuring a Virtual Private Cloud (VPC) in AWS using Terraform involves several steps. We'll walk through the process of setting up a VPC, creating private and public subnets, configuring NAT and Internet Gateways, and defining route tables and security groups.
1. Create the VPC
The VPC is the foundation of your AWS network infrastructure. It allows you to create an isolated network within the AWS cloud.
In your terraform/modules/vpc/main.tf file, add the following resource:
resource "aws_vpc" "lambda_vpc" {
cidr_block = "10.0.0.0/16" // Define the CIDR block for the VPC
enable_dns_support = true // Enable DNS support
enable_dns_hostnames = true // Enable DNS hostnames
tags = {
Name = "lambda_vpc_example" // Tag for identification
}
}2. Create Private Subnets
Private subnets are used for resources that do not require direct access to the internet.
Add the following resources to your terraform/modules/vpc/main.tf file:
// Create the first private subnet
resource "aws_subnet" "lambda_subnet_a" {
vpc_id = aws_vpc.lambda_vpc.id // Attach the subnet to the VPC
cidr_block = "10.0.1.0/24" // Define the CIDR block for the first private subnet
availability_zone = "us-west-2a" // Specify the availability zone
tags = {
Name = "lambda_subnet_a_example" // Tag for identification
}
}
// Create the second private subnet
resource "aws_subnet" "lambda_subnet_b" {
vpc_id = aws_vpc.lambda_vpc.id // Attach the subnet to the VPC
cidr_block = "10.0.2.0/24" // Define the CIDR block for the second private subnet
availability_zone = "us-west-2b" // Specify a different availability zone
tags = {
Name = "lambda_subnet_b_example" // Tag for identification
}
}3. Create a Public Subnet
Public subnets are used for resources that need direct access to the internet.
Add the following resource to your terraform/modules/vpc/main.tf file:
// Create a public subnet
resource "aws_subnet" "public_subnet" {
vpc_id = aws_vpc.lambda_vpc.id // Attach the subnet to the VPC
cidr_block = "10.0.3.0/24" // Define the CIDR block for the public subnet
availability_zone = "us-west-2c" // Specify the availability zone
map_public_ip_on_launch = true // Assign public IPs to instances launched in this subnet
tags = {
Name = "public_subnet_example" // Tag for identification
}
}4. Configure NAT and Internet Gateways
To allow instances in private subnets to access the internet, we need to set up NAT and Internet Gateways.
Add the following resources to your terraform/modules/vpc/main.tf file:
// Allocate an Elastic IP for the NAT gateway
resource "aws_eip" "nat" {
domain = "vpc" // This EIP will be used with a NAT gateway in a VPC
}
// Create a NAT gateway in the public subnet
resource "aws_nat_gateway" "nat_gw" {
allocation_id = aws_eip.nat.id // Associate the EIP with the NAT gateway
subnet_id = aws_subnet.public_subnet.id // Specify the public subnet
}
// Create an Internet Gateway for the VPC
resource "aws_internet_gateway" "igw" {
vpc_id = aws_vpc.lambda_vpc.id // Attach the Internet Gateway to the VPC
}5. Create Route Tables
Route tables direct traffic within the VPC.
Add the following resources to your terraform/modules/vpc/main.tf file:
// Define a route table for the public subnet
resource "aws_route_table" "public_rt" {
vpc_id = aws_vpc.lambda_vpc.id // Attach the route table to the VPC
// Route to the Internet Gateway for public subnet access
route {
cidr_block = "0.0.0.0/0" // Route all outbound traffic
gateway_id = aws_internet_gateway.igw.id // Use the Internet Gateway
}
}
// Define a route table for the private subnets
resource "aws_route_table" "private_rt" {
vpc_id = aws_vpc.lambda_vpc.id // Attach the route table to the VPC
// Route outbound traffic to the NAT Gateway
route {
cidr_block = "0.0.0.0/0" // Route all outbound traffic
nat_gateway_id = aws_nat_gateway.nat_gw.id // Use the NAT Gateway
}
}
// Associate the public route table with the public subnet
resource "aws_route_table_association" "public" {
subnet_id = aws_subnet.public_subnet.id // Public subnet ID
route_table_id = aws_route_table.public_rt.id // Public route table ID
}
// Associate the private route table with the first private subnet
resource "aws_route_table_association" "private_a" {
subnet_id = aws_subnet.lambda_subnet_a.id // Private subnet A ID
route_table_id = aws_route_table.private_rt.id // Private route table ID
}
// Associate the private route table with the second private subnet
resource "aws_route_table_association" "private_b" {
subnet_id = aws_subnet.lambda_subnet_b.id // Private subnet B ID
route_table_id = aws_route_table.private_rt.id // Private route table ID
}6. Define Security Groups
Security groups act as virtual firewalls for your instances to control inbound and outbound traffic.
Add the following resource to your terraform/modules/vpc/main.tf file:
resource "aws_security_group" "primary_default" {
name_prefix = "default-example-" // Prefix for security group name
description = "Default security group for all instances in ${aws_vpc.lambda_vpc.id}" // Description
vpc_id = aws_vpc.lambda_vpc.id // Attach the security group to the VPC
// Allow all inbound traffic
ingress {
from_port = 0
to_port = 0
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"] // Allow traffic from any IP
}
// Allow all outbound traffic
egress {
from_port = 0
to_port = 0
protocol = "-1" // All protocols
cidr_blocks = ["0.0.0.0/0"] // Allow traffic to any IP
}
}Configuring the API Gateway REST API Module with Terraform and using Swagger
1. Define api gateway
Defines the API Gateway and uses Swagger to configure the API’s routes and methods.
resource "aws_api_gateway_rest_api" "api" {
name = "example-api"
binary_media_types = ["multipart/form-data"]
description = "Example API Gateway"
body = templatefile("./templates/swagger.yaml", {
userLambdaArn = var.users_lambda_invoke_arn
cognito_user_pool_arn = aws_cognito_user_pool.main.arn
})
}2. IAM Role for API Gateway to Push Logs to CloudWatch
Creates an IAM role that API Gateway uses to push logs to CloudWatch.
resource "aws_iam_role" "api_gateway_cloudwatch_role" {
name = "api-gateway-example-cloudwatch-role"
assume_role_policy = jsonencode({
Version = "2012-10-17",
Statement = [
{
Action = "sts:AssumeRole",
Effect = "Allow",
Principal = {
Service = "apigateway.amazonaws.com"
}
}
]
})
}3. Attach Policy to IAM Role
Attaches a policy to the IAM role to allow API Gateway to push logs.
resource "aws_iam_role_policy_attachment" "api_gateway_cloudwatch_role_policy" {
role = aws_iam_role.api_gateway_cloudwatch_role.name
policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonAPIGatewayPushToCloudWatchLogs"
}4. Configure API Gateway Account to Use IAM Role
Associates the IAM role with the API Gateway account for CloudWatch logging.
resource "aws_api_gateway_account" "api_gw_account" {
cloudwatch_role_arn = aws_iam_role.api_gateway_cloudwatch_role.arn
}5. CloudWatch Log Group
Creates a CloudWatch log group for API Gateway logs.
resource "aws_cloudwatch_log_group" "api_logs" {
name = "/aws/api_gateway/example-api"
}7. API Gateway Deployment
Deploy the API Gateway REST API
resource "aws_api_gateway_deployment" "api" {
rest_api_id = aws_api_gateway_rest_api.api.id # The ID of the REST API to deploy
triggers = {
redeployment = sha256(file("./templates/swagger.yaml")) # Redeploy if the Swagger definition changes
}
lifecycle {
create_before_destroy = true # Ensure a new deployment is created before the old one is destroyed
}
}7. API Gateway Stage
Sets up a stage for the API Gateway, enabling access logging and X-Ray tracing.
resource "aws_api_gateway_stage" "api_gateway_stage" {
deployment_id = aws_api_gateway_deployment.api.id
rest_api_id = aws_api_gateway_rest_api.api.id
stage_name = "dev-example"
access_log_settings {
destination_arn = aws_cloudwatch_log_group.api_logs.arn
format = "{ \"requestId\":\"$context.requestId\", \"ip\": \"$context.identity.sourceIp\", \"caller\":\"$context.identity.caller\", \"user\":\"$context.identity.user\", \"requestTime\":\"$context.requestTime\", \"httpMethod\":\"$context.httpMethod\", \"resourcePath\":\"$context.resourcePath\", \"status\":\"$context.status\", \"protocol\":\"$context.protocol\", \"responseLength\":\"$context.responseLength\" }"
}
xray_tracing_enabled = true
}8. Method Settings for API Gateway Stage
Configures method settings for the API Gateway stage, including logging and metrics.
resource "aws_api_gateway_method_settings" "method_settings" {
rest_api_id = aws_api_gateway_rest_api.api.id
stage_name = aws_api_gateway_stage.api_gateway_stage.stage_name
method_path = "*/*"
settings {
metrics_enabled = true
logging_level = "INFO"
}
}9. Cognito User Pool
Creates a Cognito User Pool for managing user authentication.
resource "aws_cognito_user_pool" "main" {
name = "example_user_pool"
auto_verified_attributes = ["email"]
email_configuration {
email_sending_account = "COGNITO_DEFAULT"
}
}10. Cognito User Pool Domain
Sets up a domain for the Cognito User Pool.
resource "aws_cognito_user_pool_domain" "main" {
domain = "example-pool-domain"
user_pool_id = aws_cognito_user_pool.main.id
}11. Cognito User Pool Client
Defines a client application within the Cognito User Pool with token validity settings.
resource "aws_cognito_user_pool_client" "main" {
name = "example_pool_client"
user_pool_id = aws_cognito_user_pool.main.id
access_token_validity = 1
id_token_validity = 1
refresh_token_validity = 1
}12. Cognito User Pool Authorizer for API Gateway
Configures a Cognito User Pool authorizer for the API Gateway.
resource "aws_api_gateway_authorizer" "cognito_authorizer" {
name = "cognito_authorizer_example"
rest_api_id = aws_api_gateway_rest_api.api.id
type = "COGNITO_USER_POOLS"
provider_arns = [aws_cognito_user_pool.main.arn]
identity_source = "method.request.header.Authorization"
}13. Swagger (OpenAPI) File
Defines the API’s structure, paths, and security settings using Swagger.
swagger: "2.0"
info:
version: "1.0"
title: REST API AWS TERRAFORM
securityDefinitions:
CognitoUserPool:
type: "apiKey"
name: "Authorization"
in: "header"
x-amazon-apigateway-authtype: "cognito_user_pools"
x-amazon-apigateway-authorizer:
type: "cognito_user_pools"
providerARNs:
- "${cognito_user_pool_arn}"
paths:
/users:
options:
summary: CORS support
description: Send a preflight request to check for CORS
consumes:
- application/json
produces:
- application/json
responses:
'200':
description: CORS response
headers:
Access-Control-Allow-Origin:
type: string
default: "'*'"
Access-Control-Allow-Methods:
type: string
default: "'GET'"
Access-Control-Allow-Headers:
type: string
default: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'"
x-amazon-apigateway-integration:
type: mock
requestTemplates:
application/json: '{"statusCode": 200}'
responses:
default:
statusCode: '200'
responseParameters:
method.response.header.Access-Control-Allow-Origin: "'*'"
method.response.header.Access-Control-Allow-Methods: "'GET'"
method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'"
responseTemplates:
application/json: ''
get:
summary: Fetch the users
security:
- CognitoUserPool: ["aws.cognito.signin.user.admin"]
x-amazon-apigateway-integration:
uri: "arn:aws:apigateway:us-west-2:lambda:path/2015-03-31/functions/${userLambdaArn}/invocations"
responses:
default:
statusCode: "200"
responseParameters:
method.response.header.Access-Control-Allow-Origin: "'*'"
httpMethod: "POST"
type: "aws_proxy"
responses:
200:
description: "Successful response"
headers:
Access-Control-Allow-Origin:
type: string
default: "'*'"
Access-Control-Allow-Methods:
type: string
default: "'GET'"
Access-Control-Allow-Headers:
type: string
default: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'"Creating Lambda source and Lambda Module with Terraform
Lambda Function Code /src/lambdas/user.ts
Defines a Lambda function that processes HTTP requests. Handles
requests to fetch users and responds with JSON.
import { APIGatewayProxyEvent, APIGatewayProxyResult } from "aws-lambda";
export const handler = async (
event: APIGatewayProxyEvent
): Promise<APIGatewayProxyResult> => {
console.log("Received event: ", event);
try {
switch (event.httpMethod) {
case "GET":
if (event.resource === "/users") {
return await getAllUsers();
}
break;
default:
return createResponse(400, "Invalid request method");
}
} catch (error) {
console.error("Error processing event", error);
return createResponse(500, "Internal Server Error " + error.msg);
}
};
async function getAllUsers(): Promise<APIGatewayProxyResult> {
const fakeUsers = [
{ username: "user1", email: "user1@user1.com" },
{ username: "user2", email: "user2@user2.com" },
];
return createResponse(200, { users: fakeUsers });
}
function createResponse(statusCode: number, body: any): APIGatewayProxyResult {
return {
statusCode,
body: JSON.stringify(body),
};
}Export Handler /src/index.ts
Exports the Lambda function handler for deployment.
export { handler as userHandler } from './lambdas/user';Create Deployment Package
Compile TypeScript, copy node_modules, and create a ZIP file.
@echo "test node pkg"
rm -rf dist && npx tsc && \\
cp -r node_modules dist/ && \\
cd dist && zip -r test-lambda.zip .Alternatively, use a Makefile target make node_pkg.
Create module Lambda in terraform/modules/lambda/main.tf
Deploys the Lambda function using the ZIP file and specifies configuration details.
resource "aws_lambda_function" "users" {
function_name = "usersExampleLambda"
filename = "../dist/test-lambda.zip"
source_code_hash = filebase64sha256("../dist/test-lambda.zip")
handler = "index.userHandler"
runtime = "nodejs18.x"
role = aws_iam_role.lambda_exec.arn
vpc_config {
subnet_ids = var.subnets_ids
security_group_ids = [aws_security_group.lambda_sg.id]
}
environment {
variables = {}
}
timeout = 900
}Create Lambda Permissions
Grants API Gateway permission to invoke the Lambda function.
resource "aws_lambda_permission" "apigw_users" {
statement_id = "AllowAPIGatewayInvoke"
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.users.function_name
principal = "apigateway.amazonaws.com"
}Create IAM Role for Lambda Execution
resource "aws_lambda_permission" "apigw_users" {
statement_id = "AllowAPIGatewayInvoke"
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.users.function_name
principal = "apigateway.amazonaws.com"
}IAM Policy for CloudWatch Logging
Allows Lambda to log to CloudWatch Logs.
resource "aws_iam_policy" "lambda_logging" {
name = "LambdaLoggingExample"
description = "Allow Lambda to log to CloudWatch Logs"
policy = jsonencode({
Version = "2012-10-17",
Statement = [
{
Action = [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
Effect = "Allow",
Resource = "*"
}
]
})
}Attach Logging Policy to IAM Role
Attaches the logging policy to the IAM role.
resource "aws_iam_role_policy_attachment" "lambda_logging" {
role = aws_iam_role.lambda_exec.name
policy_arn = aws_iam_policy.lambda_logging.arn
}IAM Policy for VPC Access
Allows Lambda to manage ENIs for VPC access.
resource "aws_iam_policy" "lambda_vpc_access" {
name = "LambdaVPCAccessExample"
description = "Allow Lambda to manage ENIs for VPC access"
policy = jsonencode({
Version = "2012-10-17",
Statement = [
{
Action = [
"ec2:CreateNetworkInterface",
"ec2:DescribeNetworkInterfaces",
"ec2:DeleteNetworkInterface"
],
Effect = "Allow",
Resource = "*"
}
]
})
}Attach VPC Access Policy to IAM Role
Attaches the VPC access policy to the IAM role.
resource "aws_iam_role_policy_attachment" "lambda_vpc_access_attachment" {
role = aws_iam_role.lambda_exec.name
policy_arn = aws_iam_policy.lambda_vpc_access.arn
}Create Security Group for Lambda
Defines a security group for the Lambda function.
resource "aws_security_group" "lambda_sg" {
vpc_id = var.lambda_vpc_id
name = "lambda_sg_example"
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
ingress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
}Plan and Apply with Terraform
Validate the terraform folder
$ make validate
Executes: cd terraform && terraform validatePlan the terrraform logic
$ make plan
executes: cd terraform && terraform planApply the changes
$ make apply
executes: cd terraform && terraform apply
...
Apply complete! Resources: 29 added, 0 changed, 0 destroyed.
Outputs:
api_gateway_invoke_url = "arn:aws:execute-api:AWS_REGION:ACCOUNT_ID:API_ID/STAGE/"See the source code on Github!
Each resource defined in Terraform plays a crucial role in building a robust and scalable infrastructure for your REST API on AWS. From creating an isolated network environment with VPC and subnets, to setting up a secure entry point with API Gateway, and running code with AWS Lambda, each component contributes to a well-organized and efficient architecture. By using Terraform, you can manage and scale this infrastructure with ease, ensuring that your REST API is always ready to handle the demands of your application.
Happy coding!
