Truss

Truss: Model Packaging & Deployment

Overview
Truss is an open-source framework (by Baseten) for packaging AI/ML models. It solves the "it works on my machine" problem for ML models by creating a standardized structure that runs locally and deploys anywhere (Docker).

The Problem
Deploying a model requires:
- Correct Python version.
- System packages (apt-get install libGL).
- Python requirements (pip install torch).
- Serialization checks.

Truss handles this automatically.

How to use

pip install truss

import truss
from transformers import pipeline

# Load model
pipe = pipeline("text-classification")

# Create truss
truss.create(pipe, target_directory="./my-model")

This creates a folder with:
- `model/model.py`: Inference logic.
- `config.yaml`: Dependencies and settings.
- `data/`: Model weights.

Live Reload
Truss supports "live reload" during development. You can tweak the `model.py` code and verify the API response instantly in Docker without rebuilding the image from scratch.

Deployment
- Baseten: Native deployment (one click).
- Docker: `truss build-image ./my-model` → deploy to AWS/GCP.

Truss is a modern alternative to BentoML, focusing on developer experience and rapid iteration.