Simplifying CI/CD with GitHub actions

In the world of software development, making sure your code works well and gets to users quickly is very important. This is where CI/CD comes in. CI stands for Continuous Integration, and CD stands for Continuous Deployment or Continuous Delivery. These practices help developers work together better and release new features faster.

GitHub Actions is a tool that makes CI/CD easier. It lets you set up workflows that automatically test and deploy your code whenever you make changes. This article will explain how to use GitHub Actions to simplify your CI/CD process, using examples from a real project.

Understanding CI/CD

Before we explore GitHub Actions, let’s understand what CI/CD means:

Continuous Integration (CI)

CI is about combining code changes from different team members into one main codebase often.

When someone adds new code, it’s automatically tested to make sure it doesn’t break anything. This helps catch problems early.

Continuous Deployment/Delivery (CD)

CD takes CI a step further. After the code passes all tests, it’s automatically prepared for release.

In Continuous Deployment, the code is automatically put live for users. In Continuous Delivery, the code is ready to go live, but a person decides when to actually release it.

Introduction to GitHub actions

GitHub Actions is a powerful tool built right into GitHub. It lets you create custom workflows to automatically build, test, and deploy your code.

Here’s why it’s great:

1. It’s already in GitHub, so you don’t need a separate tool.
2. You can use pre-made actions or create your own.
3. It works with many programming languages and tools.
4. You can test your code on different operating systems easily.

Setting up a GitHub actions workflow

Let’s look at how to set up a GitHub Actions workflow using the ci.yml file from an example project.

Structure of GitHub related files:

.github
├── dependabot.yml
└── workflows
    └── ci.yml

And the ci.yml file itself:

name: CI

env:
  DOCKER_BUILDKIT: 1
  COMPOSE_DOCKER_CLI_BUILD: 1

on:
  pull_request:
    branches: ['master', 'main']
    paths-ignore: ['docs/**']

  push:
    branches: ['master', 'main']
    paths-ignore: ['docs/**']

concurrency:
  group: ${{ github.head_ref || github.run_id }}
  cancel-in-progress: true

jobs:
  linter:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Code Repository
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - name: Run pre-commit
        uses: pre-commit/[email protected]

  pytest:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout Code Repository
        uses: actions/checkout@v4

      - name: Build the Stack
        run: docker compose -f local.yml build django postgres redis

      - name: Run DB Migrations
        run: docker compose -f local.yml run --rm django python manage.py migrate

      - name: Run Django Tests
        run: docker compose -f local.yml run django pytest

      - name: Tear down the Stack
        run: docker compose -f local.yml down

Now, let’s break down this workflow and explain each part:

Workflow Trigger

on:
  pull_request:
    branches: ['master', 'main']
    paths-ignore: ['docs/**']

  push:
    branches: ['master', 'main']
    paths-ignore: ['docs/**']

This part tells GitHub when to run the workflow. It runs when:

• Someone creates a pull request to the ‘master’ or ‘main’ branch
• Someone pushes code to the ‘master’ or ‘main’ branch
• It ignores changes in the ‘docs’ folder

Concurrency control

concurrency:
  group: ${{ github.head_ref || github.run_id }}
  cancel-in-progress: true

This prevents multiple workflows from running at the same time for the same branch or pull request. If a new workflow starts, it cancels any old ones still running.

Jobs

The workflow has two main jobs: ‘linter’ and ‘pytest’.

Linter Job

linter:
  runs-on: ubuntu-latest
  steps:
    - name: Checkout Code Repository
      uses: actions/checkout@v4

    - name: Set up Python
      uses: actions/setup-python@v5
      with:
        python-version: '3.11'

    - name: Run pre-commit
      uses: pre-commit/[email protected]

This job:

1. Runs on the latest version of Ubuntu
2. Checks out the code from the repository
3. Sets up Python 3.11
4. Runs pre-commit hooks to check code style and quality

Pytest Job

pytest:
  runs-on: ubuntu-latest

  steps:
    - name: Checkout Code Repository
      uses: actions/checkout@v4

    - name: Build the Stack
      run: docker compose -f local.yml build django postgres redis

    - name: Run DB Migrations
      run: docker compose -f local.yml run --rm django python manage.py migrate

    - name: Run Django Tests
      run: docker compose -f local.yml run django pytest

    - name: Tear down the Stack
      run: docker compose -f local.yml down

This job:

1. Runs on the latest version of Ubuntu
2. Checks out the code
3. Builds Docker containers for Django, Postgres, and Redis
4. Runs database migrations
5. Runs Django tests using pytest
6. Tears down the Docker stack when done

Benefits of this workflow

1. Automated testing: Every time someone pushes code or creates a pull request, tests run automatically.
2. Code quality: The linter job checks code style and quality before merging.
3. Realistic environment: Tests run in a Docker environment similar to production.
4. Fast feedback: Developers quickly know if their changes broke something.

Customizing your workflow

You can customize this workflow in many ways:

1. Add more jobs: You could add jobs for security scanning, performance testing, or building documentation.
2. Use different actions: GitHub has a marketplace with many pre-made actions you can use.
3. Deploy automatically: Add a job to deploy your code if all tests pass.
4. Notify team members: Set up notifications for when workflows succeed or fail.

Best practices for GitHub actions

1. Keep secrets safe: Use GitHub’s secret storage for sensitive information like API keys.
2. Cache dependencies: Speed up workflows by caching things that don’t change often.
3. Use specific versions: For actions and Docker images, use specific versions to avoid surprises.
4. Limit workflow permissions: Only give workflows the permissions they need.
5. Monitor usage: Keep an eye on how much you’re using GitHub Actions, especially for private repositories.

Dependabot: Keeping dependencies up-to-date

An important part of maintaining a healthy codebase is keeping dependencies up-to-date.

GitHub’s Dependabot tool automates this process. Here’s how it’s configured in an example project:

version: 2
updates:
  - package-ecosystem: 'github-actions'
    directory: '/'
    schedule:
      interval: 'daily'

  # Multiple Docker ecosystems...

  - package-ecosystem: 'pip'
    directory: '/'
    schedule:
      interval: 'daily'

  - package-ecosystem: 'npm'
    directory: '/'
    schedule:
      interval: 'daily'

This configuration tells Dependabot to check for updates daily in several areas:

1. GitHub Actions workflows
2. Docker images in various directories
3. Python packages (pip)
4. JavaScript packages (npm)

Dependabot creates pull requests when it finds updates, allowing you to review and test changes before merging. This helps keep your project secure and up-to-date without manual effort.

For Docker images, the configuration ignores major and minor version updates, focusing only on patch updates. This reduces the risk of breaking changes while still getting important security updates.

Integrating Dependabot with your CI/CD workflow

You can make Dependabot even more powerful by integrating it with your CI/CD workflow:

1. Automatic testing: When Dependabot creates a pull request, your CI workflow will automatically run, testing the updated dependencies.
2. Review process: Set up required reviews for Dependabot pull requests to ensure a team member checks the changes.
3. Gradual updates: If you have many outdated dependencies, consider updating them gradually to avoid overwhelming your CI system.
4. Security patches: Configure your workflow to automatically merge security patches that pass all tests, ensuring quick fixes for vulnerabilities.

Challenges and solutions in CI/CD

While CI/CD greatly improves the development process, it can come with challenges:

Challenge 1: slow builds

As your project grows, CI/CD pipelines can become slow.

Solutions:

• Use caching to speed up dependency installation
• Run tests in parallel
• Only run necessary tests for each change

Challenge 2: flaky tests

Sometimes tests fail randomly, not because of actual code problems.

Solutions:

• Identify and fix flaky tests
• Set up automatic retries for flaky tests
• Quarantine unreliable tests until fixed

Challenge 3: environment differences

Tests passing in CI doesn’t always mean they’ll work in production.

Solutions:

• Use Docker to ensure consistent environments
• Have a staging environment that mirrors production
• Implement feature flags for safer releases

Future of CI/CD and GitHub actions

The field of CI/CD is always evolving. Here are some trends to watch:

1. AI-assisted testing: Tools that use AI to generate and improve tests automatically.
2. Serverless CI/CD: More cloud-based, serverless options for running CI/CD pipelines.
3. Security integration: Tighter integration of security tools in the CI/CD process.
4. Cross-platform testing: Easier ways to test applications across different operating systems and devices.
5. GitOps: Using Git as the single source of truth for both application code and infrastructure configuration.

Conclusion

GitHub Actions provides a powerful, flexible way to implement CI/CD directly within your GitHub repository. By automating testing, linting, and even deployment, you can catch errors early, maintain code quality, and release features faster.

The example workflow we’ve explored demonstrates how to set up a basic CI pipeline for a Python project using Docker. By understanding and customizing this workflow, you can create a CI/CD process that fits your project’s specific needs.

Remember, the key to successful CI/CD is continuous improvement. Regularly review and refine your workflows, keep your dependencies up-to-date with tools like Dependabot, and stay informed about new GitHub Actions features and best practices.