Creating a basic YAML CloudFormation template from scratch

Being a web developer nowadays is not an easy task. Thousands of tools and technologies have been created to help us develop software faster and more efficiently, achieving results in an increasingly shorter time.

But, in the web world, having a clean and working code won’t get its deserved recognition if you can’t get the application up running nicely on a web server for people to try and use it. That’s where developers often have a headache.

Different cloud solutions exist to make this stage less painful, such as AWS and Azure. They require some basic understanding in infrastructure and their services though, which adds another learning curve to the whole process.

Due to being among the first to deliver IaaS and PaaS, AWS is widely used and is the most common choice by companies. Provisioning on demand computer resources makes it cheaper and scalable against data centers.

Recently my team was charged to bring all of our applications to AWS. We wanted to automate the entire process of deployment as much as possible and, for that, we needed first to find a way to automate provisioning of application resources.

And then AWS CloudFormation comes into play.

AWS CloudFormation is a service that allows you to use programming languages or a simple text file to model and provision, in an automated and secure manner, all the resources needed for your applications across all regions and accounts.

Exactly what we need.

In an attempt to help other developers, this article gives a glimpse about CloudFormation template creation with a working template example.

Getting our hands dirty

So, let’s get directly to our template. It describes a stack, that is, a collection of AWS resources which can be managed as a single unit. In our stack, we define our resources based on following services:

- AWS VPC: Lets you define a network infrastructure in an isolated section of the AWS Cloud.
- AWS EC2: Lets you define the computer capacity in the cloud in a scalable manner.
- AWS Elastic Beanstalk: Lets you deploy an application with automatic load-balancing and auto-scaling.
- AWS CloudWatch: Lets you monitor applications by collecting operational data as logs, metrics and events.

For each resource, we define settings that can be pre-existing (already created on AWS console or cli) or not. For settings that are not defined on template, AWS creates a default one with default values such as, for example, a Security Group (resource that defines inbound and outbound traffic rules for an EC2 instance).

The template is divided in two basic sections: Parameters and Resources. Parameters defines values to be used by the stack itself and the application environment. Resources defines the building blocks of the stack. This division was made to make it easier to reuse the template for multiple applications.

Code please!

Here’s the code for Parameters section.

Parameters:
  AppEnvironment:
    Type: String
    Default: production
  EC2InstanceType:
    Type: String
    Default: a1.medium  
  AccountVpc:
    Type: AWS::EC2::VPC::Id
    Description: Account VPC
    Default: vpc-id
  EC2Subnet:
    Type: AWS::EC2::Subnet::Id
    Description: Subnet subnet-1
    Default: subnet-1-id
  LoadBalancerSubnet:
    Type: AWS::EC2::Subnet::Id
    Description: Subnet subnet-1
    Default: subnet-1-id
  LoadBalancerVisibility:
    Type: String
    Default: public

AppEnvironment is an environment variable for our application. In this case, we define only one environment with one environment variable for our application, but it is possible to define n environments with n environment variables.

EC2InstanceType sets which instance type our EC2 instance will be. A list of available instance types can be found at https://aws.amazon.com/ec2/instance-types/.

In our case, network infrastructure is already defined so we only need to set which VPC and Subnets we want to use (to have a public or private app for example). For this, we only need to tell IDs for AccountVpc, EC2Subnet and LoadBalancerSubnet. These IDs are obtained under “Services” > “VPC” menu on AWS Management Console.

LoadBalancerVisibility can be defined as public or internal according to LoadBalancerSubnet subnet (if it is public or private), defining if the application will be reacheable from internet (public) or not (internal).

All these Parameters values are reflected on Resources section.

Resources:
  CloudWatchLogsRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow 
          Principal:
            Service: ["ec2.amazonaws.com"]
          Action: ["sts:AssumeRole"]
      Policies:
        - PolicyName: AppLogsPolicy
        PolicyDocument:
          Version: "2012-10-17"
          Statement:
            - Effect: Allow
            Action:
              [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:GetLogEvents",
                "logs:PutLogEvents",
                "logs:DescribeLogGroups",
                "logs:DescribeLogStreams",
                "logs:FilterLogEvents",
                "logs:PutRetentionPolicy"
              ]
            Resource: "*"  InstanceProfile:
    Type: "AWS::IAM::InstanceProfile"
    Properties:
      InstanceProfileName: !Sub ${App}-${AppEnvironment}-CloudWatch
      Roles:
        - !Ref CloudWatchLogsRole  LogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: !Sub /aws/elasticbeanstalk/${App}-${AppEnvironment}/var/log/eb-docker/containers/eb-current-app/stdouterr.log
      RetentionInDays: 7  App:
    Type: "AWS::ElasticBeanstalk::Application"
    Properties:
      ApplicationName: app-name
      Description: Application description  AppVersion:
    Type: "AWS::ElasticBeanstalk::ApplicationVersion"
    Properties:
      ApplicationName: !Ref App
      Description: String
      SourceBundle:
        S3Bucket: s3-bucket-name
        S3Key: s3-key-name.zip  AppEnvironment:
    Type: "AWS::ElasticBeanstalk::Environment"
    Properties:
      EnvironmentName: !Sub ${App}-${AppEnvironment}
      ApplicationName: !Ref App
      SolutionStackName: "64bit Amazon Linux 2018.03 v2.15.1 running Docker 19.03.6-ce"
      Tier:
        Name: WebServer
        Type: Standard
        Version: "1.0"
      VersionLabel: !Ref AppVersion
      OptionSettings:
        - Namespace: aws:elasticbeanstalk:application:environment
          OptionName: APP_ENV
          Value: !Ref AppEnvironment
        - Namespace: aws:elasticbeanstalk:command
          OptionName: DeploymentPolicy
          Value: RollingWithAdditionalBatch
        - Namespace: aws:elasticbeanstalk:command
          OptionName: BatchSizeType
          Value: Fixed
        - Namespace: aws:elasticbeanstalk:command
          OptionName: BatchSize
          Value: 1        
        - Namespace: aws:ec2:vpc
          OptionName: VPCId
          Value: !Ref AccountVpc
        - Namespace: aws:ec2:vpc
          OptionName: Subnets
          Value: !Ref EC2Subnet
        - Namespace: "aws:autoscaling:launchconfiguration"
          OptionName: InstanceType
          Value: !Ref EC2InstanceType
        - Namespace: aws:elasticbeanstalk:environment
          OptionName: EnvironmentType
          Value: LoadBalanced
        - Namespace: "aws:elasticbeanstalk:environment"
          OptionName: LoadBalancerType
          Value: network
        - Namespace: "aws:ec2:vpc"
          OptionName: ELBSubnets
          Value: !Ref LoadBalancerSubnet
        - Namespace: "aws:ec2:vpc"
          OptionName: ELBScheme
          Value: !Sub ${LoadBalancerVisibility}
        - Namespace: "aws:autoscaling:launchconfiguration"
          OptionName: IamInstanceProfile
          Value: !Ref InstanceProfile
        - Namespace: aws:elasticbeanstalk:cloudwatch:logs
          OptionName: StreamLogs
          Value: true

Whoa! Now that’s a little bit big, so we will go over it step-by-step.

CloudWatchLogsRole defines an IAM Role (Profile) with Policies (Permissions) to enable integration with CloudWatch Logs. These Policies determines the usage of Log Groups, Log Streams and Log Retention. A Log Stream is a sequence of logs events. A Log Group is a group of Log Streams with same retention. Log Retention represents logs expiration time. Examples of IAM Policies can be found here: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/iam-identity-based-access-control-cwl.html.

InstanceProfile attachs the CloudWatchLogsRole information to our EC2 instance for it to integrate with CloudWatch Logs. A EC2 instance can have multiple instance profiles, thus it is a good ideia to give a well defined and clear name for each one as it is the only way to identify them after their creation.

LogGroup defines from what kind of log we want to make a Log Group. In our case we will deploy an Elastic Beanstalk app with a Docker image, meaning that application logs are streamed to the log group with the following naming scheme: /aws/elasticbeanstalk/environment_name/var/log/eb-docker/containers/eb-current-app/stdouterr.log. For a list of log groups by platform, see https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/AWSHowTo.cloudwatchlogs.html.

App defines the application name in the Elastic Beanstalk service.

AppVersion defines the current version of App as a SourceBundle. A source bundle is a ZIP or WAR file containing the source code of the application and configuration files for the Beanstalk environment that makes the application deployment possible. Some source bundle creation examples: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/applications-sourcebundle.html. Note: A S3 Bucket with s3-bucket-name name containing a zip file with s3-key-name.zip name must be created before using this template on AWS CloudFormation console.

AppEnvironment defines all environment properties of our application. First, set EnvironmentName since an application can have multiple environments. Then, our platform of choice specified in SolutionStackName (Docker container). For supported platforms, see https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/concepts.platforms.html.

Last but not least, OptionSettings that designates all defined parameters from Parameters section. Detailing these options:

1. Setting an environment variable

- Namespace: aws:elasticbeanstalk:application:environment
  OptionName: APP_ENV
  Value: !Ref AppEnvironment

2. Setting deployment policy

- Namespace: aws:elasticbeanstalk:command
  OptionName: DeploymentPolicy
  Value: RollingWithAdditionalBatch
- Namespace: aws:elasticbeanstalk:command
  OptionName: BatchSizeType
  Value: Fixed
- Namespace: aws:elasticbeanstalk:command
  OptionName: BatchSize
  Value: 1

3. Setting VPC to be used

- Namespace: aws:ec2:vpc
  OptionName: VPCId
  Value: !Ref AccountVpc

4. Setting EC2 instance subnet

- Namespace: aws:ec2:vpc
  OptionName: Subnets
  Value: !Ref EC2Subnet

5. Setting EC2 instance type

- Namespace: "aws:autoscaling:launchconfiguration"
  OptionName: InstanceType
  Value: !Ref EC2InstanceType

6. Setting Load Balancer type

- Namespace: aws:elasticbeanstalk:environment
  OptionName: EnvironmentType
  Value: LoadBalanced
- Namespace: "aws:elasticbeanstalk:environment"
  OptionName: LoadBalancerType
  Value: network

7. Setting Load Balancer subnet and visibility

- Namespace: "aws:ec2:vpc"
  OptionName: ELBSubnets
  Value: !Ref LoadBalancerSubnet
- Namespace: "aws:ec2:vpc"
  OptionName: ELBScheme
  Value: !Sub ${LoadBalancerVisibility}

8. Setting instance profile

- Namespace: "aws:autoscaling:launchconfiguration"
  OptionName: IamInstanceProfile
  Value: !Ref InstanceProfile

9. Setting stream logs to true (enabling log streaming to CloudWatch)

- Namespace: aws:elasticbeanstalk:cloudwatch:logs
  OptionName: StreamLogs
  Value: true

Conclusion

By combining both Parameters and Resources sections, a simple CloudFormation template is created and can be used to deploy applications from different languages. It makes the management of application and cloud resources secure, easy and fast. For a team with a lot of applications, this solution represents a standardization of deployment process at the same time it allows personalization according to each project/application demands.

Here follows the complete template.