Machine Learning Model Deployment: From Jupyter to Production

TL;DR

• ML model deployment has 5 stages: serialise → wrap in API → containerise → deploy → monitor
• FastAPI + Docker is the standard production stack for Python ML models
• Feature preprocessing must be serialised alongside the model: not just the model weights
• Monitoring for model drift is as important as uptime monitoring: models degrade silently
• Batch inference (overnight jobs) is simpler than real-time inference: start there if latency doesn't matter

The Deployment Pipeline Overview

Stage 1: Model Serialisation

The most common mistake in ML deployment is serialising only the model weights. This forgets that the preprocessing pipeline (scalers, encoders, imputers) must also be saved and loaded identically. If your training data was scaled with StandardScaler, production data must be scaled with the same fitted scaler, not a new one.

Save model + preprocessor together

import joblib
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import GradientBoostingClassifier

# Build pipeline, preprocessing + model in one object
pipeline = Pipeline([
 ('scaler', StandardScaler()),
 ('classifier', GradientBoostingClassifier(n_estimators=200))
])

pipeline.fit(X_train, y_train)

# Save the entire pipeline (scaler params are saved with it)
joblib.dump(pipeline, 'model_pipeline.joblib')

# Load in production, scaler params are restored
pipeline = joblib.load('model_pipeline.joblib')
prediction = pipeline.predict(new_data)

Stage 2: REST API with FastAPI

FastAPI is the standard choice for Python ML model APIs: async-native, auto-validated inputs via Pydantic, and auto-generated OpenAPI documentation.

FastAPI prediction endpoint

from fastapi import FastAPI
from pydantic import BaseModel
import joblib
import numpy as np

app = FastAPI()
pipeline = joblib.load('model_pipeline.joblib')

class ChurnInput(BaseModel):
 tenure_months: int
 monthly_spend: float
 support_tickets_90d: int
 last_login_days_ago: int

class ChurnPrediction(BaseModel):
 churn_probability: float
 churn_predicted: bool

@app.post("/predict/churn", response_model=ChurnPrediction)
async def predict_churn(data: ChurnInput):
 features = np.array([[
 data.tenure_months,
 data.monthly_spend,
 data.support_tickets_90d,
 data.last_login_days_ago
 ]])
 prob = pipeline.predict_proba(features)[0][1]
 return ChurnPrediction(
 churn_probability=round(prob, 4),
 churn_predicted=prob > 0.5
 )

Stage 3: Docker Containerisation

Docker ensures your model runs identically on any machine: dev, staging, and production. The Dockerfile captures Python version, library versions, and the model file.

Dockerfile for ML API

FROM python:3.11-slim

WORKDIR /app

COPY requirements.txt.
RUN pip install --no-cache-dir -r requirements.txt

COPY model_pipeline.joblib.
COPY main.py.

EXPOSE 8000
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Stage 4: Cloud Deployment Options

Stage 5: Monitoring and Model Drift

ML models degrade silently. Unlike a crashed API (immediately visible), a model that starts returning wrong predictions can go unnoticed for weeks. You need two layers of monitoring:

Infrastructure Monitoring

• API latency (P50, P95, P99)
• Error rates (5xx responses)
• Request throughput
• Memory / CPU utilisation

Model Quality Monitoring

• Prediction distribution drift
• Feature distribution drift
• Accuracy (when labels available)
• Confidence score distribution

Log predictions for drift monitoring

import datetime
import json

@app.post("/predict/churn", response_model=ChurnPrediction)
async def predict_churn(data: ChurnInput):
 features = np.array([[...]])
 prob = pipeline.predict_proba(features)[0][1]

 # Log every prediction for drift monitoring
 log_entry = {
 "timestamp": datetime.utcnow().isoformat(),
 "input_features": data.dict(),
 "prediction": float(prob),
 "model_version": "v1.2.0"
 }
 # Write to your monitoring store (S3, BigQuery, etc.)
 prediction_logger.log(log_entry)

 return ChurnPrediction(churn_probability=round(prob, 4), churn_predicted=prob > 0.5)

Batch vs Real-Time Inference