Implementing Remote Caching in Docker Builds for 5x Faster CI

Waiting 15 minutes for a Docker image to build during a critical hotfix is unacceptable. By implementing Docker Buildx and remote layer caching in GitHub Actions, we consistently drop client build times down to under 3 minutes.

The Ephemeral CI Problem

The fundamental problem with CI/CD runners (like GitHub Actions or GitLab CI) is that they are ephemeral. Every time a pipeline triggers, it spins up a completely fresh virtual machine. This means the local Docker cache is completely empty.

If your Dockerfile spends 8 minutes running npm install or pip install -r requirements.txt, it will do that every single time you push a commit, wasting hundreds of hours of engineering time per month.

The Solution: Docker Buildx with Inline Caching

Instead of relying on the local machine's Docker cache, we can push the cache layers to a remote registry (like AWS ECR or GitHub Container Registry) directly alongside the actual image.

When the next CI runner boots up, it pulls those specific cached layers from the remote registry instead of rebuilding them from scratch.

The Implementation (GitHub Actions)

To achieve this, we use the docker/build-push-action and specify the cache-from and cache-to parameters using the gha (GitHub Actions) cache backend.

name: Build and Push Docker Image

on:
  push:
    branches:
      - main

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Login to Amazon ECR
        uses: docker/login-action@v3
        with:
          registry: ${{ secrets.AWS_ACCOUNT_ID }}.dkr.ecr.us-east-1.amazonaws.com

      - name: Build and push with remote caching
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: my-api:latest
          # The magic happens here:
          cache-from: type=gha
          cache-to: type=gha,mode=max

Understanding "mode=max"

By default, Docker only caches the layers that were explicitly built in the final image. However, in multi-stage Dockerfiles, this means builder stages (where heavy dependency compilation happens) aren't cached.

Setting mode=max forces Docker to cache every single layer across all stages of your multi-stage build, resulting in maximum cache hits and dramatically faster pipelines.