neural architecture search (NAS)

Neural architecture search automates the design of the network itself — how many layers, which operations, what connections — instead of hand-crafting it. You specify a search space of possible architectures, a search strategy to explore it, and a way to estimate each candidate's quality, then let the procedure discover designs a human might not have tried.

Early NAS trained thousands of architectures under reinforcement learning or evolution and cost thousands of GPU-days. Modern NAS slashes this with weight sharing: a single over-parameterized supernet contains every candidate as a sub-path, so a child architecture's accuracy can be read off without training it from scratch. Differentiable approaches like DARTS relax the discrete operator choice into a continuous mixture trained by gradient descent, while predictor-based and one-shot methods estimate performance cheaply.

NAS produced influential model families such as EfficientNet, MobileNetV3, and the MnasNet line. It is most valuable when the objective includes deployment constraints rather than accuracy alone, which is the focus of hardware-aware NAS.