One-Shot Nas

One-Shot NAS is a weight-sharing NAS approach where a single "supernet" is trained that contains all candidate architectures as sub-networks — enabling architecture evaluation without training each candidate from scratch, reducing search cost from thousands of GPU-hours to hours.

How Does One-Shot NAS Work?

• Supernet: A single overparameterized network containing all possible operations and connections.
• Training: Train the supernet with random path sampling (at each iteration, activate a random sub-network).
• Evaluation: To evaluate a candidate architecture, simply activate its corresponding paths in the trained supernet. No separate training needed.
• Search: Use evolutionary search or RL to find the best sub-network within the trained supernet.

Why It Matters

• Massive Speedup: Train once, evaluate thousands of architectures by inheritance.
• Practical: Makes NAS accessible on a single GPU (SPOS, OFA, FairNAS).
• Challenge: Weight entanglement — shared weights may not accurately represent independently trained networks.

One-Shot NAS is all architectures in one network — a clever weight-sharing trick that trades absolute accuracy for enormous search efficiency.