Want your ML model to go from “works on my laptop” to “serving real users behind a load balancer”? This tutorial walks you end-to-end: project layout, training a model, packaging an API, writing tests, containerizing, and setting up a production-grade CI/CD pipeline that builds, scans, pushes, and deploys to AWS.You’ll finish with:<ul><li>A working /predict API backed by your model</li><li>Automated tests + linting</li><li>A secure GitHub Actions pipeline using AWS OIDC, Amazon ECR and ECS Fargate</li><li>Zero server management (fully managed containers)</li></ul><hr><h2>What we’re building</h2>Flow: Commit → GitHub Actions: test → build → scan → push image to ECR → deploy ECS task → behind ALB → <code>/predict</code> live.Stack: Python, scikit-learn, FastAPI, Docker, pytest, GitHub Actions, AWS ECR, ECS Fargate, ALB.<hr><h2>1) Repo structure</h2><pre><code>ml-cicd-aws/
├─ app/
│ ├─ main.py
│ ├─ model.py
│ ├─ schemas.py
│ ├─ __init__.py
├─ data/
│ └─ iris.csv # (optional; we’ll use sklearn’s dataset)
├─ models/
│ └─ model.pkl # produced by training job
├─ tests/
│ ├─ test_api.py
│ └─ test_model.py
├─ training/
│ └─ train.py
├─ Dockerfile
├─ requirements.txt
├─ runtime.txt # optional pin for build tooling
├─ .dockerignore
├─ .gitignore
├─ Makefile
└─ .github/
 └─ workflows/
 └─ ci-cd.yml</code></pre><hr><h2>2) Minimal ML model (training script)</h2>We’ll train a simple classifier (Iris) and persist it to <code>models/model.pkl</code>.<code>training/train.py</code><pre><code>import os
import joblib
from pathlib import Path
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier

OUT_DIR = Path(__file__).resolve().parents[1] / "models"
OUT_DIR.mkdir(parents=True, exist_ok=True)
MODEL_PATH = OUT_DIR / "model.pkl"

def train():
 iris = load_iris()
 X, y = iris.data, iris.target
 X_train, X_test, y_train, y_test = train_test_split(
 X, y, test_size=0.2, stratify=y, random_state=42
 )
 clf = RandomForestClassifier(n_estimators=120, random_state=42)
 clf.fit(X_train, y_train)
 joblib.dump({"model": clf, "target_names": iris.target_names}, MODEL_PATH)
 print(f"Saved: {MODEL_PATH}")

if __name__ == "__main__":
 train()</code></pre>Run it locally:<pre><code>python -m venv .venv &amp;&amp; source .venv/bin/activate
pip install -r requirements.txt
python training/train.py</code></pre><hr><h2>3) FastAPI app to serve predictions</h2><code>app/schemas.py</code><pre><code>from pydantic import BaseModel, Field
from typing import List

class PredictRequest(BaseModel):
 # For iris: sepal_length, sepal_width, petal_length, petal_width
 instances: List[List[float]] = Field(..., example=[[5.1, 3.5, 1.4, 0.2]])</code></pre><code>app/model.py</code><pre><code>import joblib
from pathlib import Path

class ModelService:
 def __init__(self, path: str = "models/model.pkl"):
 p = Path(path)
 if not p.exists():
 raise FileNotFoundError(f"Model not found at {p.resolve()}")
 blob = joblib.load(p)
 self.model = blob["model"]
 self.target_names = blob["target_names"]

 def predict(self, X):
 labels = self.model.predict(X)
 return [self.target_names[i] for i in labels]</code></pre><code>app/main.py</code><pre><code>from fastapi import FastAPI
from app.schemas import PredictRequest
from app.model import ModelService

app = FastAPI(title="ML Iris Predictor")
svc = ModelService() # loads on startup

@app.get("/health")
def health():
 return {"status": "ok"}

@app.post("/predict")
def predict(payload: PredictRequest):
 preds = svc.predict(payload.instances)
 return {"predictions": preds}</code></pre><hr><h2>4) Tests</h2><code>tests/test_model.py</code><pre><code>import os
from app.model import ModelService

def test_model_loads():
 assert os.path.exists("models/model.pkl"), "Run training/train.py first"
 svc = ModelService()
 assert svc.model is not None</code></pre><code>tests/test_api.py</code><pre><code>from fastapi.testclient import TestClient
from app.main import app

client = TestClient(app)

def test_health():
 r = client.get("/health")
 assert r.status_code == 200
 assert r.json()["status"] == "ok"

def test_predict():
 payload = {"instances": [[5.1, 3.5, 1.4, 0.2]]}
 r = client.post("/predict", json=payload)
 assert r.status_code == 200
 assert "predictions" in r.json()
 assert isinstance(r.json()["predictions"], list)</code></pre><hr><h2>5) Requirements</h2><code>requirements.txt</code><pre><code>fastapi==0.115.2
uvicorn[standard]==0.30.6
scikit-learn==1.5.2
joblib==1.4.2
pydantic==2.9.2
pytest==8.3.3</code></pre><hr><h2>6) Containerization</h2><code>.dockerignore</code><pre><code>.venv
__pycache__
*.pyc
*.pyo
.git
.gitignore
*.ipynb
data/</code></pre><code>Dockerfile</code><pre><code>FROM python:3.11-slim

# System deps
RUN apt-get update &amp;&amp; apt-get install -y --no-install-recommends \
 gcc build-essential &amp;&amp; \
 rm -rf /var/lib/apt/lists/*

WORKDIR /app

# Pre-copy requirements for better layer caching
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy the rest
COPY app/ app/
COPY models/ models/

# FastAPI runs on 8000
EXPOSE 8000

# Healthcheck (ECS respects container exit, ALB health checks the /health)
HEALTHCHECK --interval=30s --timeout=3s \
 CMD curl -f http://localhost:8000/health || exit 1

# Start the API
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]</code></pre>Build &amp; run locally:<pre><code>docker build -t ml-iris:local .
docker run -p 8000:8000 ml-iris:local
# test
curl -s http://localhost:8000/health
curl -s -X POST http://localhost:8000/predict -H "Content-Type: application/json" \
 -d '{"instances": [[5.1, 3.5, 1.4, 0.2],[6.0, 2.7, 5.1, 1.6]]}'</code></pre><hr><h2>7) Makefile (handy shortcuts)</h2><code>Makefile</code><pre><code>venv:
	python -m venv .venv &amp;&amp; . .venv/bin/activate &amp;&amp; pip install -r requirements.txt

train:
	. .venv/bin/activate || true; python training/train.py

test:
	pytest -q

build:
	docker build -t $(IMAGE):$(TAG) .

run:
	docker run -p 8000:8000 $(IMAGE):$(TAG)

.PHONY: venv train test build run</code></pre><hr><h2>8) AWS setup (one-time)</h2><h3>8.1 Create ECR repository</h3><pre><code>aws ecr create-repository --repository-name ml-iris --image-scanning-configuration scanOnPush=true</code></pre>Note the ECR URI: <code>ACCOUNT_ID.dkr.ecr.ap-south-1.amazonaws.com/ml-iris</code><h3>8.2 Create ECS Cluster + Fargate Service (with ALB)</h3>You can click-through in the console (ECS → Create cluster → Fargate), or use IaC later. For the first pass, console is fine:<ul><li>Cluster: <code>ml-cluster</code></li><li>Task Definition: Fargate, CPU 0.25 vCPU, memory 0.5–1 GB, container port <code>8000</code></li><li>Load Balancer: Application Load Balancer, health check path <code>/health</code>, target type IP</li><li>Desired tasks: 1 (scale later)</li></ul>Keep AWSVPC networking; pick public subnets (or private with NAT).<hr><h2>9) GitHub Actions with OIDC (no long-lived AWS keys)</h2><h3>9.1 AWS IAM Role for GitHub OIDC</h3><ol><li>Create OIDC provider (usually already exists if you’ve done it before):</li></ol><ul><li>Provider URL: <code>https://token.actions.githubusercontent.com</code></li><li>Audience: <code>sts.amazonaws.com</code></li></ul><ol start="2"><li>Create IAM role (trusted by that provider), attach policy to push to ECR and deploy to ECS.</li></ol>Trust policy (example):<pre><code>{
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Principal": { "Federated": "arn:aws:iam::&lt;ACCOUNT_ID&gt;:oidc-provider/token.actions.githubusercontent.com" },
 "Action": "sts:AssumeRoleWithWebIdentity",
 "Condition": {
 "StringEquals": {
 "token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
 },
 "StringLike": {
 "token.actions.githubusercontent.com:sub": "repo:&lt;GITHUB_OWNER&gt;/&lt;REPO_NAME&gt;:*"
 }
 }
 }]
}</code></pre>Permissions policy (minimal to ECR/ECS):<pre><code>{
 "Version": "2012-10-17",
 "Statement": [
 {"Effect": "Allow","Action": ["ecr:GetAuthorizationToken"],"Resource": "*"},
 {"Effect": "Allow","Action": ["ecr:BatchCheckLayerAvailability","ecr:CompleteLayerUpload","ecr:UploadLayerPart","ecr:InitiateLayerUpload","ecr:PutImage","ecr:DescribeRepositories"],"Resource": "arn:aws:ecr:ap-south-1:&lt;ACCOUNT_ID&gt;:repository/ml-iris"},
 {"Effect": "Allow","Action": ["ecs:DescribeServices","ecs:DescribeTaskDefinition","ecs:RegisterTaskDefinition","ecs:UpdateService"],"Resource": "*"},
 {"Effect": "Allow","Action": ["iam:PassRole"],"Resource": "*","Condition": {"StringEquals":{"iam:PassedToService":"ecs-tasks.amazonaws.com"}}}
 ]
}</code></pre>Save the role ARN as a GitHub Actions secret: <code>AWS_ROLE_TO_ASSUME</code>.Also add these Actions secrets:<ul><li><code>AWS_REGION</code> = <code>ap-south-1</code> (or your region)</li><li><code>ECR_REPO</code> = <code>ml-iris</code></li><li><code>ECS_CLUSTER</code>= <code>ml-cluster</code></li><li><code>ECS_SERVICE</code>= <code>ml-iris-service</code> (the service you’ll create)</li><li>(Optional) <code>IMAGE_TAG</code> defaults to <code>github.sha</code> in workflow</li></ul><hr><h2>10) CI/CD pipeline</h2><code>.github/workflows/ci-cd.yml</code><pre><code>name: CI/CD - ML to AWS Fargate

on:
 push:
 branches: [ "main" ]
 workflow_dispatch:

permissions:
 id-token: write # for OIDC
 contents: read

env:
 AWS_REGION: ${{ secrets.AWS_REGION }}
 ECR_REPO: ${{ secrets.ECR_REPO }}
 ECS_CLUSTER: ${{ secrets.ECS_CLUSTER }}
 ECS_SERVICE: ${{ secrets.ECS_SERVICE }}
 IMAGE_TAG: ${{ github.sha }}

jobs:
 build-test:
 runs-on: ubuntu-latest
 steps:
 - name: Checkout
 uses: actions/checkout@v4

 - name: Set up Python
 uses: actions/setup-python@v5
 with:
 python-version: "3.11"

 - name: Install deps
 run: pip install -r requirements.txt

 - name: Train model (fresh artifact)
 run: python training/train.py

 - name: Run tests
 run: pytest -q

 - name: Build Docker image
 run: |
 IMAGE_URI=${{ secrets.AWS_ACCOUNT_ID }}.dkr.ecr.${{ env.AWS_REGION }}.amazonaws.com/${{ env.ECR_REPO }}:${{ env.IMAGE_TAG }}
 docker build -t $IMAGE_URI .

 - name: Trivy scan (optional but recommended)
 uses: aquasecurity/trivy-action@0.28.0
 with:
 image-ref: ${{ secrets.AWS_ACCOUNT_ID }}.dkr.ecr.${{ env.AWS_REGION }}.amazonaws.com/${{ env.ECR_REPO }}:${{ env.IMAGE_TAG }}
 format: 'table'
 vuln-type: 'os,library'
 exit-code: '0' # don't fail build initially; tighten later

 - name: Configure AWS credentials (OIDC)
 uses: aws-actions/configure-aws-credentials@v4
 with:
 role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }}
 aws-region: ${{ env.AWS_REGION }}

 - name: Login to ECR
 id: ecr
 uses: aws-actions/amazon-ecr-login@v2

 - name: Push to ECR
 run: |
 IMAGE_URI=${{ secrets.AWS_ACCOUNT_ID }}.dkr.ecr.${{ env.AWS_REGION }}.amazonaws.com/${{ env.ECR_REPO }}:${{ env.IMAGE_TAG }}
 docker push $IMAGE_URI

 - name: Render task definition
 id: taskdef
 run: |
 IMAGE_URI=${{ secrets.AWS_ACCOUNT_ID }}.dkr.ecr.${{ env.AWS_REGION }}.amazonaws.com/${{ env.ECR_REPO }}:${{ env.IMAGE_TAG }}
 cat &gt; taskdef.json &lt;&lt; 'JSON'
 {
 "family": "ml-iris-td",
 "networkMode": "awsvpc",
 "requiresCompatibilities": ["FARGATE"],
 "cpu": "256",
 "memory": "512",
 "executionRoleArn": "arn:aws:iam::${ACCOUNT_ID}:role/ecsTaskExecutionRole",
 "taskRoleArn": "arn:aws:iam::${ACCOUNT_ID}:role/ecsTaskRole",
 "containerDefinitions": [{
 "name": "ml-iris",
 "image": "${IMAGE_URI}",
 "essential": true,
 "portMappings": [{"containerPort": 8000, "protocol": "tcp"}],
 "logConfiguration": {
 "logDriver": "awslogs",
 "options": {
 "awslogs-group": "/ecs/ml-iris",
 "awslogs-region": "${REGION}",
 "awslogs-stream-prefix": "ecs"
 }
 },
 "healthCheck": {
 "command": ["CMD-SHELL", "curl -f http://localhost:8000/health || exit 1"],
 "interval": 30,
 "timeout": 5,
 "retries": 3,
 "startPeriod": 10
 }
 }]
 }
 JSON
 sed -i "s|\${IMAGE_URI}|$IMAGE_URI|g" taskdef.json
 sed -i "s|\${ACCOUNT_ID}|${{ secrets.AWS_ACCOUNT_ID }}|g" taskdef.json
 sed -i "s|\${REGION}|${{ env.AWS_REGION }}|g" taskdef.json
 cat taskdef.json

 - name: Register new task definition
 id: register
 run: |
 ARN=$(aws ecs register-task-definition --cli-input-json file://taskdef.json --query 'taskDefinition.taskDefinitionArn' --output text)
 echo "TASK_DEF_ARN=$ARN" &gt;&gt; $GITHUB_OUTPUT

 - name: Deploy service (rolling update)
 run: |
 aws ecs update-service \
 --cluster "${{ env.ECS_CLUSTER }}" \
 --service "${{ env.ECS_SERVICE }}" \
 --task-definition "${{ steps.register.outputs.TASK_DEF_ARN }}" \
 --force-new-deployment

 - name: Wait for stability
 run: |
 aws ecs wait services-stable \
 --cluster "${{ env.ECS_CLUSTER }}" \
 --services "${{ env.ECS_SERVICE }}"</code></pre><blockquote>Replace <code>secrets.AWS_ACCOUNT_ID</code>, roles, cluster &amp; service names to match your account.</blockquote><hr><h2>11) Create the ECS service (once)</h2>If you didn’t do the console wizard, you can create the service after the first task definition registers:<pre><code># Example (adjust subnets/securitygroups/ALB target group ARNs accordingly)
aws ecs create-service \
 --cluster ml-cluster \
 --service-name ml-iris-service \
 --task-definition ml-iris-td \
 --desired-count 1 \
 --launch-type FARGATE \
 --network-configuration "awsvpcConfiguration={subnets=[subnet-xxx,subnet-yyy],securityGroups=[sg-zzz],assignPublicIp=ENABLED}" \
 --load-balancers "targetGroupArn=arn:aws:elasticloadbalancing:ap-south-1:&lt;ACCOUNT_ID&gt;:targetgroup/ml-tg/abc123,containerName=ml-iris,containerPort=8000"</code></pre>Point your ALB listener (port 80/443) to that target group. Health check path: <code>/health</code>.<hr><h2>12) Try it live</h2>Once the workflow finishes, hit the ALB DNS:<pre><code>curl http://&lt;your-alb-dns&gt;/health
curl -X POST http://&lt;your-alb-dns}/predict -H "Content-Type: application/json" \
 -d '{"instances": [[6.2, 2.8, 4.8, 1.8]]}'</code></pre>You should see a species prediction (e.g., <code>virginica</code>).<hr><h2>13) Production tips</h2><ul><li>Reproducible training: Move training into a separate job/artifact, version artifacts (<code>model-YYYYMMDD.pkl</code>) and pin them in releases.</li><li>Model registry: Store artifacts in S3 with versioning; pass S3 URL via ECS task env var.</li><li>Secrets: Use SSM Parameter Store or Secrets Manager; mount via task definition.</li><li>Observability: Enable AWS Logs Insights for <code>/ecs/ml-iris</code>, add request metrics via a sidecar (e.g., Prometheus exporter) or API middleware.</li><li>Autoscaling: Configure Target Tracking on ECS service (CPU/Memory or ALB RequestCount).</li><li>Blue/Green: Use CodeDeploy for zero-downtime if you need canaries/linear rollouts.</li><li>Security: Restrict IAM to least privilege, use private subnets + NAT in production.</li></ul><hr><h2>14) Local developer experience (optional)</h2><code>docker-compose.yml</code> for quick spins:<pre><code>version: "3.9"
services:
 api:
 build: .
 ports:
 - "8000:8000"</code></pre><hr><h2>15) Common pitfalls &amp; fixes</h2><ul><li>Model file missing in container Make sure <code>models/model.pkl</code> exists before building the image (run training in CI first).</li><li>ALB health check failing Confirm containerPort 8000, target group health check path <code>/health</code>, security groups allow ALB → ECS.</li><li>Permission errors in CI Recheck IAM role trust policy <code>sub</code> matches <code>repo:OWNER/REPO:*</code>, and <code>id-token: write</code> is enabled.</li></ul>In this tutorial, you trained a scikit-learn model, exposed it via a FastAPI endpoint, containerized it with Docker, and built a secure CI/CD pipeline using GitHub Actions’ OIDC to deploy on AWS ECS Fargate behind an Application Load Balancer. Every push to <code>main</code> tests your code, scans your image, pushes to ECR, and rolls out a new task—giving you a reproducible, auditable path from experimentation to real-world traffic.

Machine Learning

Ship Your ML Model Like a Pro End-to-End CI/CD to AWS (ECR and ECS Fargate) with FastAPI & GitHub Actions