Adding Continuous Deployment to your CI

CI what now?! Continuous Integration is the practice of automating your testing so you can frequently merge your changes to your main branch. The smaller and more frequent the changes the better. But this is only half of the story. Having your GitHub repo up to date is great but what if you could also have those changes deployed automatically? Welcome to Continuous Deployment!

GitHub Actions

We have our example GitHub Action which checks out our code and runs some basic testing

name: Example Pipeline

on:
  push:
    branches: [ main ]
  pull_request:
  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

jobs:
  Testing:
    runs-on: ubuntu
    steps:
      - name: Checkout Code
        uses: actions/checkout@v3
      
      - name: Set up Python 3.10
        uses: actions/setup-python@v4
        with:
          python-version: '3.10'

      - name: Install Pipenv and run PyLint
        run: |
          pipenv install
          pipenv run pylint *.py

      - name: PyTesting
        run: pipenv run pytest -v tests/

This would live in your .github/workflow/example_pipeline.yml file. If you are unfamiliar spend some time reading the docs: https://docs.github.com/en/actions

Creating our Tornado Script

On our running server we are going to create a simple tornado script which will listen for a webhook request. Upon receiving the webhook request it will update our repo.

#!/usr/bin/env python

from pathlib import Path
import json
from tornado.web import Application, RequestHandler
from tornado.ioloop import IOLoop

class Main(RequestHandler):
  def post(self) -> None:
    body = json.loads(self.request.body.decode('utf-8'))
    if 'secret' not in body.keys():
      # Fail anything that doesn't provide our secret
      logging.warning('%s failed to provide an API key', remote_ip)
      self.set_status(404)
      self.write({'message':'API Key not specified'})
      return
    if body['secret'] == 'YoUrReAlLyHorrIbLeSecrEtKey!':
      # Only if our secret matches
      Path(f"{envs['base']}/{body['repo']}/.update").touch()
      self.set_status(200)
      self.write({'message': 'Thank you for request'})

def app():
  urls = [
    ("/api/", Main)
  ]
  return Application(urls, debug=True)

app = make_app()
app.listen(3000)
IOLoop.instance().start()

Our script takes a message body of {'secret': 'Your Key', 'repo': 'Your repo name'}. This allows us to authenticate the request and to control the scope of what our tornado script can do should a malicous person guess our super difficult secret. Its important whenever we are exposing things on the internet that we think about attack vectors.

The cron job and the update

Our cron is going to be super simple for this example as will the bash script. I would encourage you to write something more defensive that will look for possible errors and alert to problems.

# Every 5 minutes call our update script
*/5 * * * * ubuntu /bin/bash ${HOME}/MyRepo/update.sh

Our Update script

#!/bin/bash

if [[ -f "${HOME}/MyRepo/.update" ]]; then
  cd "${HOME}/MyRepo/" || exit 1
  rm .update || exit 1
  git checkout main && git pull
  echo "$(date)" > .last_updated
fi

Putting it all together

We should now have our tornado script running on our server listening for the webhook. Upon receiving the webhook a .update file will be recreated. That in turn will be picked up by our cron’d script running every 5 minutes resulting in the repo being updated and the .last_updated file recording the datetime.

The last step is to add the following to the bottom of our GitHub Action

      - name: PyTesting
        run: pipenv run pytest -v tests/
      - name: Deployment
        # Only deploy if on branch == main
        if: github.ref == 'refs/heads/master' 
        run: curl -X POST http://<your home IP>:3000 -d '{'secret': 'YoUrReAlLyHorrIbLeSecrEtKey!', 'repo': 'MyRepo'}

If you have a dynamic IP address and want to use a static domain name have a look at services like https://www.noip.com/.

And that’s it! You now have Continuous Deployment to your Continuous Integration!