Torchscript
TorchScript is PyTorch's intermediate representation (IR) system for converting dynamic Python models into serializable, optimizable, static graphs that can run in C++ production environments without the Python runtime — using either tracing (recording operations on example inputs) or scripting (analyzing Python source code) to capture model logic into a portable format that eliminates the Python Global Interpreter Lock (GIL) bottleneck and enables deployment on servers, mobile devices, and embedded systems.
What Is TorchScript?
- Definition: A statically-typed subset of Python that PyTorch can compile into an intermediate representation — enabling models to be saved as `.pt` files and loaded in C++, Java, or other runtimes without requiring a Python interpreter.
- Two Capture Modes: Tracing (`torch.jit.trace`) records the exact sequence of operations executed on example inputs — fast and simple but fails on data-dependent control flow (if statements, variable-length loops). Scripting (`torch.jit.script`) analyzes the Python source code and compiles it — supports control flow but requires TorchScript-compatible Python syntax.
- Production Deployment: The primary use case — export a model from Python research code and deploy it in a C++ inference server, mobile app (iOS/Android via PyTorch Mobile), or embedded system without shipping a Python environment.
- Optimization: The TorchScript IR enables graph-level optimizations — constant folding, dead code elimination, operator fusion, and memory planning that are impossible with Python's dynamic execution model.
Tracing vs Scripting
| Mode | How It Works | Control Flow | Ease of Use | Best For |
|---|---|---|---|---|
| Tracing | Records ops on example input | No (flattened) | Easy | Simple feed-forward models |
| Scripting | Analyzes Python source | Yes (if/for) | Harder | Models with dynamic logic |
| Hybrid | Trace outer, script inner | Partial | Medium | Complex models |
TorchScript vs Alternatives
- torch.compile (PyTorch 2.0): The modern replacement — uses TorchDynamo to capture computation graphs with full Python support, largely superseding TorchScript for optimization.
- ONNX Export: Alternative serialization path — export to ONNX format for cross-framework deployment (ONNX Runtime, TensorRT).
- torch.export (PyTorch 2.1+): The newest export API — captures a clean graph representation for AOT compilation, designed to replace both TorchScript and the old ONNX exporter.
TorchScript is PyTorch's original model serialization and optimization system — converting dynamic Python models into static, portable representations that run in C++ without the Python runtime, now being gradually superseded by torch.compile and torch.export but still widely used in production deployments.