From notebook to service: packaging, a FastAPI endpoint, a smoke test

A model in a notebook helps one person. A model behind an HTTP endpoint helps any program that can make a request. This lesson walks the shortest honest path between the two: fit a scikit-learn pipeline, save it to a file, load it in a small FastAPI application, validate every request, and test the whole thing in-process before any server is started. It is for anyone who has a model that works in a notebook and has been asked to “put it somewhere the app can call it”. Everything here runs on a laptop CPU in seconds and the only packages beyond scikit-learn are FastAPI and uvicorn.

The model is deliberately small, a logistic regression on the iris flowers dataset, so that none of the lesson is about the model. The same layout serves the gradient boosting model from lesson 05 or the network from lesson 07 with a different predict body.

Save the pipeline, not the estimator

The first rule of serving is that preprocessing is part of the model. In lesson 02 you learned to fit the scaler on the training data only. If the scaler is a separate object from the classifier, the service has to remember to apply it, in the right order, with the right fitted statistics, and that is a bug waiting to happen. A scikit-learn Pipeline bundles the steps into one object with one predict, so what is saved and what is served is the same thing that was evaluated.

The block below trains the pipeline, checks it, and saves it with joblib alongside the feature names, the class names and a version string. The metadata matters: six months from now the file will be loaded by someone who does not remember which column was which.

import joblib
import numpy as np
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler

iris = load_iris()
X_train, X_test, y_train, y_test = train_test_split(
    iris.data, iris.target, test_size=0.25, stratify=iris.target, random_state=0
)
pipeline = Pipeline([
    ("scale", StandardScaler()),
    ("clf", LogisticRegression(max_iter=1000)),
])
pipeline.fit(X_train, y_train)
print(f"test accuracy: {pipeline.score(X_test, y_test):.3f}")

bundle = {
    "pipeline": pipeline,
    "feature_names": list(iris.feature_names),
    "class_names": [str(c) for c in iris.target_names],
    "version": "2026-11-13",
}
joblib.dump(bundle, "iris_model.joblib")
loaded = joblib.load("iris_model.joblib")
print("reloaded classes:", loaded["class_names"])
print("prediction for one flower:", loaded["pipeline"].predict([[5.1, 3.5, 1.4, 0.2]]))

test accuracy: 0.974
reloaded classes: ['setosa', 'versicolor', 'virginica']
prediction for one flower: [0]

joblib is the serialiser scikit-learn recommends. It is pickle underneath, with better handling of large numpy arrays. Two warnings come with that. A pickle file executes code when it is loaded, so never load one from a source you do not trust. And a pickle is tied to the library versions that wrote it: scikit-learn warns when a file was written by a different version and makes no promise that it loads at all. Pin the version in requirements.txt and record it in the bundle. For models that must outlive library versions, ONNX is the portable format, and the MLOps and deployment hub covers when it is worth the conversion.

The service

The diagram shows the three parts and what crosses each boundary. The client sends JSON over HTTP. The API validates it, converts it to the array the model expects, and converts the model’s output back to JSON. The model never sees HTTP and the client never sees numpy.

Here is the application file. It is marked # snippet because it defines an app to be run by uvicorn, and the guard that runs every Python block on this site cannot start a server that waits for connections. The runnable version follows in the next section.

# snippet
# app.py
import joblib
from fastapi import FastAPI
from pydantic import BaseModel, Field

bundle = joblib.load("iris_model.joblib")       # load once, at import, not per request
app = FastAPI(title="iris-service")


class Measurements(BaseModel):
    sepal_length: float = Field(gt=0, lt=20)    # centimetres
    sepal_width: float = Field(gt=0, lt=20)
    petal_length: float = Field(gt=0, lt=20)
    petal_width: float = Field(gt=0, lt=20)


@app.get("/health")
def health() -> dict:
    return {"status": "ok", "model_version": bundle["version"]}


@app.post("/predict")
def predict(m: Measurements) -> dict:
    row = [[m.sepal_length, m.sepal_width, m.petal_length, m.petal_width]]
    probs = bundle["pipeline"].predict_proba(row)[0]
    best = int(probs.argmax())
    return {"species": bundle["class_names"][best], "probability": round(float(probs[best]), 3)}

Read it top to bottom. The model is loaded once when the module is imported, which is the single most important performance decision in a small service: loading a joblib file per request would turn a millisecond prediction into a hundred-millisecond one. Measurements is a pydantic model. FastAPI uses it to parse the JSON body, to reject any body that does not match with a 422 response listing every problem, and to generate interactive documentation at /docs. The bounds on each field are the first line of defence against garbage in: a negative petal length is not a flower, and the model would happily return a confident answer for it.

The predict function converts the validated fields into the 1 by 4 list the pipeline expects, in the same column order as training, calls predict_proba, and returns the best class with its probability. The explicit int(...) and float(...) matter, because numpy scalars are not JSON serialisable and the error you get without them is unhelpful.

The smoke test, without a server

FastAPI ships TestClient, which calls the app in-process: no port, no network, no subprocess. The block below is complete and runnable. It trains and saves the model (restating the first block, so that the block stands alone), defines the same app, and exercises both endpoints, including one deliberately bad request.

import joblib
from fastapi import FastAPI
from fastapi.testclient import TestClient
from pydantic import BaseModel, Field
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler

# Train and save, as in the first block, so this block stands alone.
iris = load_iris()
pipeline = Pipeline([("scale", StandardScaler()), ("clf", LogisticRegression(max_iter=1000))])
pipeline.fit(iris.data, iris.target)
joblib.dump({"pipeline": pipeline, "feature_names": list(iris.feature_names),
             "class_names": [str(c) for c in iris.target_names], "version": "2026-11-13"}, "iris_model.joblib")

# The same app as app.py, loaded from the saved file.
bundle = joblib.load("iris_model.joblib")
app = FastAPI(title="iris-service")


class Measurements(BaseModel):
    sepal_length: float = Field(gt=0, lt=20)
    sepal_width: float = Field(gt=0, lt=20)
    petal_length: float = Field(gt=0, lt=20)
    petal_width: float = Field(gt=0, lt=20)


@app.get("/health")
def health() -> dict:
    return {"status": "ok", "model_version": bundle["version"]}


@app.post("/predict")
def predict(m: Measurements) -> dict:
    row = [[m.sepal_length, m.sepal_width, m.petal_length, m.petal_width]]
    probs = bundle["pipeline"].predict_proba(row)[0]
    best = int(probs.argmax())
    return {"species": bundle["class_names"][best], "probability": round(float(probs[best]), 3)}


# The smoke test: no server, no network, the client calls the app in-process.
client = TestClient(app)

r = client.get("/health")
print(r.status_code, r.json())

r = client.post("/predict", json={"sepal_length": 5.1, "sepal_width": 3.5, "petal_length": 1.4, "petal_width": 0.2})
print(r.status_code, r.json())

r = client.post("/predict", json={"sepal_length": 6.7, "sepal_width": 3.0, "petal_length": 5.2, "petal_width": 2.3})
print(r.status_code, r.json())

r = client.post("/predict", json={"sepal_length": -1, "sepal_width": 3.0, "petal_length": 5.2})
print(r.status_code, r.json()["detail"][0]["msg"], "|", r.json()["detail"][1]["msg"])

200 {'status': 'ok', 'model_version': '2026-11-13'}
200 {'species': 'setosa', 'probability': 0.985}
200 {'species': 'virginica', 'probability': 0.961}
422 Input should be greater than 0 | Field required

The last request has a negative length and a missing field, and the response lists both problems in one 422 rather than failing on the first. Nothing reached the model. That is the behaviour to check in a test, because it is the behaviour that protects the model from inputs it was never trained on.

In a real project the smoke test lives in tests/test_app.py as functions that assert on the status codes and the species, and runs with pytest before every deployment. A test that takes a second and catches a missing model file, a renamed field or a broken dependency is worth more than any amount of monitoring after the fact. The tabular classifier project builds that out with a larger model and a Dockerfile.

Project layout

A service someone else can run needs more than the two files. The minimum layout is this.

iris-service/
  app.py              the FastAPI application above
  train.py            the first block: fits the pipeline and writes iris_model.joblib
  iris_model.joblib   the artefact (commit it for a tiny model, or build it in CI for a large one)
  requirements.txt    pinned dependencies
  tests/
    test_app.py       the smoke test, as pytest functions
  README.md           how to train, run and call it

requirements.txt pins exact versions, because a model file and the library that reads it have to agree. Replace the versions with the ones from pip freeze in the environment where the model was trained.

scikit-learn==1.9.1
numpy==2.5.4
joblib==1.6.0
fastapi==0.162.7
uvicorn==0.54.0
pydantic==2.14.3
httpx==0.28.1
pytest

httpx is there because TestClient is built on it. The test runner is left unpinned because it does not touch the model file. The exact versions above are the ones in the Field Guide’s environment at the time of writing, and will be out of date by the time you read this: run pip freeze and use your own.

Run it

Train once, start the server, and call it from another terminal. --reload restarts the server when app.py changes, which is convenient in development and must not be used in production.

python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
python train.py
uvicorn app:app --host 127.0.0.1 --port 8000 --reload

curl -s http://127.0.0.1:8000/health

curl -s -X POST http://127.0.0.1:8000/predict \
  -H "Content-Type: application/json" \
  -d '{"sepal_length": 6.7, "sepal_width": 3.0, "petal_length": 5.2, "petal_width": 2.3}'

The first call returns {"status":"ok","model_version":"2026-11-13"} and the second {"species":"virginica","probability":0.961}. Open http://127.0.0.1:8000/docs in a browser for the generated documentation, which includes a form for each endpoint built from the pydantic model.

The server is bound to 127.0.0.1, so only programs on the same machine can reach it. Binding to 0.0.0.0 exposes it to the network, and at that point the service needs authentication, rate limiting and HTTPS in front of it, none of which this lesson provides. Those belong to a reverse proxy or an API gateway, not to the application.

What was left out, on purpose

This is a minimum, and it is useful to know what the next steps are so that you add them when they are needed and not before. Batch prediction, where the client sends many rows at once, is a second endpoint taking a list of Measurements. Logging every request and its prediction, with the model version, is the basis of all later monitoring and takes five lines with the standard logging module. A Dockerfile makes the environment reproducible on another machine. Model loading via FastAPI’s lifespan handler, rather than at import, lets the service report that it is starting rather than failing a health check while the file loads. And for a PyTorch model, torch.set_num_threads and a single worker process avoid the server and the model fighting over CPU cores. The MLOps and deployment hub orders these by how soon each one usually becomes necessary.

Exercise

1. Add a POST /predict_batch endpoint that accepts a JSON list of measurement objects and returns a list of results, with a limit of 100 rows per request enforced by pydantic. Extend the smoke test to send 3 rows, then 101.
2. Add a created_at timestamp and the scikit-learn version to the saved bundle, and return both from /health. Then load the bundle in a fresh environment with a different scikit-learn version and record exactly what warning or error you see.
3. Replace the iris model with the gradient boosting classifier from lesson 05 on load_breast_cancer (30 features). Decide how the 30 fields should be named in the pydantic model, and whether a single features: list[float] of fixed length would be a better interface. Write down the trade-off in three sentences.

Summary

• Save the whole pipeline, preprocessing included, with joblib, together with feature names, class names and a version string. Pickle files are tied to library versions and must only be loaded from trusted sources.
• Load the model once at start-up, never per request. The API layer converts between JSON and arrays so the model code stays the same as in the notebook.
• A pydantic model validates every request and returns a 422 with all problems before the model is called. Bounds on fields are the first defence against inputs the model never saw.
• TestClient exercises the app in-process with no server, which makes a one-second smoke test possible in CI.
• Pin dependencies, bind to localhost until authentication and a proxy are in place, and keep the layout small enough that the README fits on one screen.

Further reading

• FastAPI documentation: request bodies, path operations and the generated documentation.
• FastAPI documentation, Testing: TestClient and the pytest layout used above.
• scikit-learn user guide, Model persistence: joblib, its security and version caveats, and the ONNX and skops alternatives.
• Pydantic documentation, Fields: the constraints available on request fields.
• Uvicorn documentation, Deployment: workers, process managers and running behind a proxy.