Seeds of Inception - Part 1

Part 1 discusses Seeding your Account with an Inception Pipeline - A practical example for DevOps on AWS

What’s The Problem

As we all know, we are supposed to automate everything today. Everything must be ‘as code’ and there are no manual steps to doing anything. However when pressed with project deadlines, production fires and the rare moments when we get to work on something new, we just don’t get around to it.

While working on a recent project, building out a continuous deployment pipeline, I got to thinking about the number of upcoming future projects. Each project would need at least one pipeline per application, and a dedicated one for the AWS Account. Rolling a unique snowflake pipeline everytime just seemed evil and wasteful and wrong and totally against everything we stand for at Mechanical Rock!

So welcome to the Inception Pipeline; a CloudFormation template that plants itself inside an AWS Account and then self manages and self updates itself using nothing more than off-the-shelf AWS services.

What Technologies Are We Going To Use

• CloudFormation
• CodeCommit
• CodePipeline
• IAM Roles
• S3

What Are The Prerequisites?

1. An AWS Account to create the pipeline in.
2. The AWS CLI installed and configured with access credentials.
3. CodeCommit git credentials configured.

While not strictly required, a passing familiarity of Bash Shell scripts, Git, JSON & YAML, and CloudFormation templates will make understanding everything easier.

How It All Works

At a high-level, the Inception Pipeline works by executing a CloudFormation template which then creates a CodeCommit repository, a CodePipeline pipeline and a few other supporting resources. The first non-source action in the pipeline is a CloudFormation Deployment Action.

This blog post won’t dive deeply into the CloudFormation template (I’ll leave that as an exercise for you dear reader), instead I’ll just discuss the really juicy bits.

The secret-sauce to the Inception Pipeline is the using the same CloudFormation stackname in the CLI/console invocation and the CodePipeline Action (CloudFormation parameter StageAdministerPipelineStackName). Once the initial CloudFormation stack has been created, the CodePipeline action below maintains the pipeline.

- Name: 'AdministerPipeline'
    Actions:
    - Name: 'AdministerPipeline'
        ActionTypeId:
            Category: 'Deploy'
            Owner: 'AWS'
            Provider: 'CloudFormation'
            Version: '1'
        Configuration:
            ActionMode: 'REPLACE_ON_FAILURE'
            Capabilities: 'CAPABILITY_NAMED_IAM'
            RoleArn: !GetAtt [CloudFormationDeployActionRole, Arn]
            StackName: !Ref StageAdministerPipelineStackName
            TemplateConfiguration: !Join ['', [!Ref RepositoryName, 'Source', '::aws_seed.json']]
            TemplatePath: !Join ['', [!Ref RepositoryName, 'Source', '::aws_seed.yml']]
        InputArtifacts:
            - Name: !Join ['', [!Ref RepositoryName, 'Source']]
        RunOrder: '1'

The CodePipeline is also configured to restart execution if it updates itself (RestartExecutionOnUpdate: true). This means when you add new stages, actions, etc, the pipeline will run the above action, realise it has changed and then restart itself.

While not strictly necessary, I would recommend that:

1. The AdministerPipeline stage is only used for the AdministerPipeline action, i.e. don’t add any other actions in.
2. The AdministerPipeline stage is the first stage after the Source stage. This allows the pipeline to update itself before anything else runs.

Where Do I Get The Seed Files

Go grab a copy from the GitHub repository.

I’ll wait while you do.

What Are The Files

Taking It For A Spin

1. Download the zip file from the GitHub repository. Do not clone the repo, as the shell script will perform a git init. Copy the unziped files into a folder which will become your project folder.
2. Open all the files into your editor-of-choice. An editor that allows global search-and-replace will be super helpful.

3. Open aws_seed-cli-parameters.json. Perform a global replace for all values between the @@:

   Parameter	Description
   RepositoryName	The name of the CodeCommit repository to create
   CodePipelineS3BucketName	The name of the S3 bucket to create that will be used to store artifacts from CodePipeline execution runs.
   CloudFormationDeployActionRoleName	The name of the IAM role that will be created to run the CloudFormation deployment action as.
   CodePipelineRoleName	The name of the IAM role that will be created for the CodePipeline to use.
   CodePipelineName	The name of the CodePipeline to create.
   StageAdministerPipelineStackName	The name of this stack that is creating the pipeline. If the names do not match then the pipeline will not be able to update itself.

4. Run the init.sh|init.bat and watch the output.
5. If all goes well, within a few minutes you will have a running Inception Pipeline in your account!

Wrapping Up

If you do end up running an Inception Pipeline please let me know! I’m especially keen to hear any and all improvements, suggestions and critiques.