How to Set Up a CI/CD Pipeline for ML Models Using GitHub Actions and Docker
Most ML teams build great models that never make it to production. The model gets trained, the notebook looks impressive, and then it sits in a folder while the team argues about how to deploy it. Meanwhile, the business is not getting any value from the work.
A CI/CD pipeline for ML solves this. Every time you push code, your model gets tested, containerized, and deployed automatically. Here is how to build one using GitHub Actions, Docker, and a dedicated GPU server. No Kubernetes cluster required.
What You Will Build
By the end of this guide, you will have a pipeline that:
- Runs your model tests on every push
- Builds a Docker image with your model and dependencies
- Pushes the image to a container registry
- Deploys it to a GPU server as a running inference API
You can run this on any infrastructure. We tested it on a dedicated L40S GPU server, but it works on any box with Docker and an NVIDIA GPU.
Step 1: Structure Your Project
Your ML project should look like this:
ml-model/
model.py # your model code
requirements.txt # Python dependencies
Dockerfile # container definition
tests/
test_model.py # model tests
.github/
workflows/
deploy.yml # GitHub Actions pipeline
Keep it simple. The model is just a Python file. The tests verify it loads and produces output. No fancy frameworks. This is production, not a research notebook.
Step 2: Write the Dockerfile
FROM nvidia/cuda:12.4-runtime-ubuntu22.04
RUN apt-get update && apt-get install -y python3 python3-pip
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY model.py .
EXPOSE 8000
CMD ["python3", "-m", "uvicorn", "model:app", "--host", "0.0.0.0", "--port", "8000"]
This uses the NVIDIA CUDA base image so your model can access the GPU. The requirements file should pin your exact versions:
torch==2.5.0
transformers==4.45.0
fastapi==0.115.0
uvicorn==0.30.0
Step 3: Create the Model API
Your model.py exposes a FastAPI endpoint:
from fastapi import FastAPI
from pydantic import BaseModel
import torch
app = FastAPI()
# Load model once at startup (not on every request)
model = torch.load("model.pt", map_location="cuda")
model.eval()
class PredictRequest(BaseModel):
text: str
class PredictResponse(BaseModel):
label: str
confidence: float
@app.post("/predict", response_model=PredictResponse)
def predict(req: PredictRequest):
with torch.no_grad():
output = model(req.text)
pred = output.argmax().item()
confidence = output.softmax(dim=-1)[pred].item()
return PredictResponse(label=str(pred), confidence=confidence)
@app.get("/health")
def health():
return {"status": "ok"}
Two endpoints. One for predictions. One for health checks. That is all you need.
Step 4: Write Tests
Your tests should verify the model loads and produces output of the right shape:
from model import app
from fastapi.testclient import TestClient
client = TestClient(app)
def test_health():
response = client.get("/health")
assert response.status_code == 200
def test_predict():
response = client.post("/predict", json={"text": "sample input"})
assert response.status_code == 200
data = response.json()
assert "label" in data
assert "confidence" in data
assert 0 <= data["confidence"] <= 1
No GPU needed for unit tests. Tests run on the GitHub Actions runner, which has no GPU. This is by design - you test the API logic, not the GPU hardware. GPU validation happens in staging.
Step 5: The GitHub Actions Pipeline
Create .github/workflows/deploy.yml:
name: ML Model CI/CD
on:
push:
branches: [main]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- run: pip install -r requirements.txt
- run: pip install pytest httpx
- run: pytest tests/ -v
build-and-deploy:
needs: test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Build Docker image
run: docker build -t my-model:latest .
- name: Deploy to GPU server
uses: appleboy/ssh-action@v1
with:
host: ${{ secrets.GPU_HOST }}
username: ${{ secrets.GPU_USER }}
key: ${{ secrets.GPU_SSH_KEY }}
script: |
docker pull my-model:latest
docker stop model-api 2>/dev/null || true
docker run -d --name model-api \
--gpus all -p 8000:8000 \
--restart unless-stopped \
my-model:latest
sleep 5
curl -f http://localhost:8000/health
The pipeline runs tests first. If they pass, it builds the Docker image and deploys it to your GPU server via SSH. The health check after deployment ensures the model is actually serving before the pipeline reports success.
Step 6: Deploy and Verify
Push your code to GitHub. Watch the Actions tab. In about three minutes, your model will be running on your GPU server.
Test it:
curl -X POST http://your-gpu-server:8000/predict \
-H "Content-Type: application/json" \
-d '{"text": "this is great"}'
You should get back a label and confidence score. Your model is live.
When You Need More Than One GPU
This pipeline works for a single model on a single GPU server. If you have multiple models or need auto-scaling, you will eventually want Kubernetes. But start simple. Most teams never outgrow a single dedicated GPU server for inference. The ones that do usually know they need more before they start.
If you are serving multiple models, you can run separate Docker containers on different ports. Use a reverse proxy like nginx to route requests by model name. Add Prometheus metrics to track latency and throughput. The pipeline stays the same. You just add more deploy jobs.
What About AWS SageMaker?
SageMaker is a great product for teams already deep in the AWS ecosystem. It handles model registry, endpoint management, and auto-scaling out of the box. But SageMaker endpoints cost money even when idle, and the per-hour GPU pricing is higher than a dedicated server.
For teams just starting with ML deployments, a single GPU server with this CI/CD pipeline is simpler, cheaper, and easier to understand. You can always add SageMaker later when you need its orchestration features. The Docker image you built in Step 2 deploys to SageMaker with zero changes - just point it at an ECR repository instead of your GPU server.
How ServerGurus Helps
Our GPU cloud gives you dedicated NVIDIA GPUs with root access. You own the server, you run your Docker containers, you set up your CI/CD pipeline however you want. No per-hour billing anxiety. No cold starts. No noisy neighbors.
Every GPU plan includes a dedicated IP, NVMe storage, and private networking. If you need help setting up the pipeline in this guide, our DevOps team can have you running in under an hour. That is the difference between buying infrastructure and having a partner who helps you use it.