RL for chip design
Designing a chip means placing many components on a two-dimensional canvas so that wires stay short, heat is manageable, and timing constraints are met — a puzzle so vast that human engineers can spend weeks on it. RL can treat placement as a sequence of decisions: where does this block go, then the next, then the next.
The agent places macro blocks one at a time; the state is the partial layout, the action is the next position, and the reward combines wirelength, congestion, and timing, usually estimated quickly so the agent gets fast feedback. A learned policy can generalize across designs, proposing floorplans in hours that rival or beat hand-tuned ones.
Google reported using RL for parts of its TPU floorplanning; the work drew both excitement and debate over how fairly its results were compared with strong classical optimizers — a healthy reminder that any "RL beat the experts" claim deserves careful, like-for-like baselines.