Per-Scene Optimization

Per-scene optimization is the training paradigm where a separate neural representation is optimized for each individual scene - it emphasizes scene-specific quality rather than cross-scene generalization.

What Is Per-scene optimization?

• Definition: Model parameters are fit from scratch or fine-tuned for one target scene.
• Typical Use: Classic NeRF pipelines optimize independently per capture sequence.
• Benefit: Can produce very high fidelity for the trained scene.
• Cost: Requires substantial compute and time per new scene.

Why Per-scene optimization Matters

• Quality Ceiling: Scene-specific fitting can outperform generic feed-forward models.
• Research Baseline: Provides strong reference quality for evaluating faster methods.
• Control: Allows tailored hyperparameters for unique scene characteristics.
• Scalability Limit: Not ideal for large-scale deployment across many scenes.
• Motivation: Drove development of accelerated methods such as Instant NGP and Gaussian splatting.

How It Is Used in Practice

• Initialization: Use good pose estimates and normalized scene scale before optimization.
• Budget Planning: Set convergence criteria to avoid unnecessary long-tail training.
• Use Case Fit: Reserve per-scene optimization for high-value or offline rendering tasks.

Per-scene optimization is a high-quality but compute-intensive reconstruction strategy - per-scene optimization is best when maximum fidelity is required and throughput is secondary.