Discord Bot Pipeline - GitHub Actions Setup

This post is part 5 of a 10 part series:

• Part 1 - Discord Bot Pipeline - Intro
• Part 2 - Discord Bot Pipeline - GCP Setup
• Part 3 - Discord Bot Pipeline - GitHub Setup
• Part 4 - Discord Bot Pipeline - Terraform Setup
• Part 5 - Discord Bot Pipeline - GitHub Actions Setup (This Post)
• Part 6 - Discord Bot Pipeline - Making Terraform Changes
• Part 7 - Discord Bot Pipeline - Discord Specific Terraform Setup
• Part 8 - Discord Bot Pipeline - More Setup
• Part 9 - Discord Bot Pipeline - Creating The Bot
• Part 10 - Discord Bot Pipeline - Conclusion

Previously in this series, we created a GitHub project and populated it with Terraform configurations. In this post, we’ll go over how to leverage GitHub Workflows as our CI/CD infrastructure for deploying Terraform updates.

GitHub Workflows

We can use Terraform in GitHub by creating workflows to support the following:

1. A dispatch (manually-triggered) workflow that can be used to create new GCP projects.
2. A workflow that runs when Pull Requests are opened that triggers terraform plan. This workflow should output the result of terraform plan so that you can visually inspect the proposed changes. This workflow will be read-only; no changes will be made to your infrastructure when it runs.
3. A workflow that runs when Pull Requests are merged that triggers terraform apply. This workflow will perform actual changes to your infrastructure when it runs.

Creating new projects

Create a file .github/workflows/new_project.yaml with the following contents:

name: New Project
on:
  workflow_dispatch:
    inputs:
      project_name:
        description: "GCP project name"
        required: true
      region:
        description: "GCP region for terraform configs"
        required: true
        default: "us-east1-a"

env:
  GOOGLE_CREDENTIALS: ${{ secrets.GCP_SA_KEY }}
  TF_WORKING_DIR: terraform

permissions:
  contents: write

jobs:
  new_project:
    runs-on: ubuntu-latest
    defaults:
      run:
        working-directory: ${{ env.TF_WORKING_DIR }}
    steps:
      - uses: actions/checkout@v3
      - uses: hashicorp/setup-terraform@v1
        with:
          terraform_wrapper: false
      - name: Create GitHub branch
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          git config --global user.name "Terraform Admin"
          git config --global user.email "you@example.com"
          git checkout -b ${{ inputs.project_name }}_deploy
          zone="$(echo ${{ inputs.region }} | cut -d'-' -f-2)"
          sed -i -e "s/{{PROJECT_NAME}}/${{ inputs.project_name }}/g" *
          sed -i -e "s/{{BILLING_ACCOUNT}}/${{ secrets.GCP_BILLING_ACCOUNT_ID }}/g" *
          sed -i -e "s/{{ORGANIZATION_ID}}/${{ secrets.GCP_ORGANIZATION_ID }}/g" *
          sed -i -e "s/{{REGION}}/${{ inputs.region }}/g" *
          sed -i -e "s/{{ZONE}}/${zone}/g" *
          rm ../.github/workflows/new_project.yaml ../admin_setup.sh
          git commit -am "Initial setup for ${{ inputs.project_name }}"
          git push origin ${{ inputs.project_name }}_deploy
      - name: terraform init
        run: terraform init
      - name: terraform plan
        run: terraform plan -no-color -lock-timeout=5m
      - name: terraform apply
        run: terraform apply -no-color -lock-timeout=5m -auto-approve

Note that there is a global git user.email configuration that can be set to your custom domain.

Terraform plan

Add the following file to .github/workflows/terraform_plan.yaml:

name: Terraform Plan
on:
  pull_request:
    paths:
      - "terraform/**"

env:
  GOOGLE_CREDENTIALS: ${{ secrets.GCP_SA_KEY }}
  TF_WORKING_DIR: terraform

permissions:
  pull-requests: write

jobs:
  terraform_plan:
    runs-on: ubuntu-latest
    defaults:
      run:
        working-directory: ${{ env.TF_WORKING_DIR }}
    steps:
      - uses: actions/checkout@v3
      - uses: hashicorp/setup-terraform@v2
      - name: terraform fmt
        id: fmt
        run: terraform fmt -check
        continue-on-error: true
      - name: terraform init
        id: init
        run: terraform init
      - name: terraform validate
        id: validate
        run: terraform validate -no-color
        continue-on-error: true
      - name: terraform plan
        id: plan
        run: terraform plan -no-color -lock-timeout=5m
      - uses: actions/github-script@v6
        if: github.event_name == 'pull_request'
        env:
          PLAN: "terraform\n${{ steps.plan.outputs.stdout }}"
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          script: |
            const output = `
            #### Terraform Format and Style 🖌\`${{ steps.fmt.outcome }}\`
            #### Terraform Initialization ⚙️\`${{ steps.init.outcome }}\`
            #### Terraform Validation 🤖\`${{ steps.validate.outcome }}\`
            #### Terraform Plan 📖\`${{ steps.plan.outcome }}\`

            <details><summary>Show Plan</summary>

            \`\`\`\n
            ${process.env.PLAN}
            \`\`\`

            </details>`

            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: output
            })

Terraform apply

Add the following file to .github/workflows/terraform_apply.yaml

name: Terraform Apply
on:
  push:
    branches:
      - "**_deploy"
  workflow_dispatch:

env:
  GOOGLE_CREDENTIALS: ${{ secrets.GCP_SA_KEY }}
  TF_WORKING_DIR: terraform

jobs:
  terraform:
    runs-on: ubuntu-latest
    defaults:
      run:
        working-directory: ${{ env.TF_WORKING_DIR }}
    steps:
      - uses: actions/checkout@v3
      - uses: hashicorp/setup-terraform@v1
        with:
          terraform_wrapper: false
      - name: terraform init
        run: terraform init
      - name: terraform plan
        run: terraform plan -no-color -lock-timeout=5m
      - name: terraform apply
        run: terraform apply -no-color -lock-timeout=5m -auto-approve

You should end up with something very similar to https://github.com/thompsonja/terraform-controller.

Creating your first project

From your Terraform controller repository, click on Actions and select New Project from the left-hand panel. Click the Run Workflow button, which will bring up a menu with various arguments. Add your domain name (without the top level domain like .com) and set the project name. This project name will be used as a prefix for the GCP project. The GCP region defaults to us-east1-a, leave it as is or override it if you prefer your bot to be hosted somewhere else.

• 1 - Running the New Project Workflow
• 2 - Setting Workflow Arguments

Clicking on Run workflow will cause the workflow to begin. Once it starts, an entry will appear in the workflow runs page. Clicking on it will reveal a status page. You can expand the dropdowns to see logs for each step of the workflow. For instance, expanding terraform plan will show you Terraform logs for the plan stage, which generally includes which resources will be created, modified, or destroyed, and similarly for terraform apply, which shows which resources actually were created, modified, or destroyed.

• 1 - Checking Status
• 2 - Expanding Plan Logs
• 3 - Expanding Apply Logs

In the next section, we will go over how to view this newly created project and how to use our new pipeline to make changes to our project with Terraform.