Proxylessnas

ProxylessNAS is a NAS method that directly searches on the target hardware and target dataset — eliminating the need for proxy tasks (smaller datasets, shorter training) that introduce a gap between the searched and deployed architecture.

How Does ProxylessNAS Work?

• Direct Search: Searches directly on ImageNet (not CIFAR-10 proxy) and on the target hardware (GPU, mobile, etc.).
• Path-Level Binarization: At each step, only one path (operation) is active -> memory-efficient (don't need to run all operations simultaneously like DARTS).
• Latency Loss: Includes a differentiable latency predictor in the search objective: $\mathcal{L} = \mathcal{L}_{CE} + \lambda \cdot Latency$.

Why It Matters

• No Proxy Gap: Architectures searched directly on the target task & hardware generalize better.
• Hardware-Aware: Different architectures for GPU, mobile CPU, and edge TPU — each optimized for its platform.
• Memory Efficient: Binary path sampling uses ~50% less memory than DARTS.

ProxylessNAS is searching where you deploy — finding the best architecture directly on the target hardware and dataset without approximation.