accelerator amortization
Buying an industrial bread oven makes sense for a bakery that sells thousands of loaves a day — spread across all those loaves, the oven's cost per loaf is tiny. Buying the same oven to bake one loaf a week would be ridiculous; the cost would never be repaid. Accelerator amortization is this exact reasoning applied to specialized hardware: the large, fixed cost of building (and using) an accelerator only pays off when it is spread across enough work. It is the honest test of whether an accelerator is actually worth it.
Concretely, an accelerator carries two kinds of cost that must be amortized. First, the upfront design cost: an ASIC can take millions of dollars and many months to design, verify, and fabricate, plus the matching compiler and software stack. That cost is repaid only if the chip then runs an enormous volume of work — which requires a workload that is large (lots of it) and stable (it will not change before the chip is obsolete). Second, the per-use overhead: every time you offload a task, you pay to move data to the accelerator and back and to start it. That is repaid only if each task is big enough that the actual computation dwarfs the setup. The classic rule is that an accelerator is worth it when the workload is large, stable, and parallel enough to amortize both costs.
Why this is the field's most important honest check: it is the reason most computing is not done on custom accelerators. If a workload is small, changing fast, or not parallel, a general-purpose CPU is the right tool — its flexibility and zero design cost win. Skip this analysis and you get expensive silicon that sits idle, or an offload that is slower than just using the CPU because the data-movement overhead swamped a tiny task. The whole specialization trend rests on workloads (like deep learning) that happen to be huge, durable, and parallel — exactly the conditions under which amortization works out. When those conditions fail, 'right tool for the job' means the humble general processor.
Google built the TPU because billions of inference requests per day are a huge, stable, parallel workload — the design cost amortizes across an astronomical number of uses. A small startup with a workload that changes every few months and runs occasionally would be foolish to tape out an ASIC; for them an FPGA, a GPU, or just a CPU amortizes far better.
An accelerator is worth it only when the workload is large, stable, and parallel enough to repay its cost.
An accelerator is not a free upgrade. If the workload is small, fast-changing, or serial, amortization fails and a general CPU is genuinely the better choice — the per-use data-movement overhead alone can make a small offload slower than not offloading at all.