Training and serving have opposite shapes. Training reads large historical data sets in batches, can take hours and runs on schedule. Serving answers one request at a time within a latency budget and must stay up. A model registry connects them: every trained model is versioned with its data, code and metrics, and deployment promotes a specific version.
Not every prediction needs to be computed on request. Batch inference precomputes scores for known entities overnight; online inference computes them when a request arrives with fresh inputs. New models should prove themselves before taking all traffic: shadow deployments score requests without affecting users, and canary rollouts send a small share of traffic to the new model while metrics are compared.
Separate training from serving, choose batch or online inference and roll models out safely.