response time

You press a key, and you want the letter to appear now. You tap a search button, and you wait for the page. The gap between asking and getting an answer — the time you actually sit there waiting — is response time, also called latency. It is the most human measure of speed, because it is the one you feel. When a website feels sluggish or a game stutters, it is response time, not throughput, that you are noticing.

Precisely, response time is the total elapsed wall-clock time from the start of a task to its finish, including everything: the CPU doing work, waiting for memory, waiting for disk, even waiting for other programs to get off the processor. Architects often separate this into CPU time (the part the processor actually spent on your program) and the rest (input/output and time stolen by other tasks). The iron law applies to that CPU-time part: CPU time = instruction count times cycles per instruction times clock-cycle time. Lowering any of those three lowers response time.

The key honesty here is that response time is not the same as throughput, and optimizing one can worsen the other. A pipeline raises throughput (jobs per second) but does not shorten the latency of any single instruction. A batch system that crams many jobs together to keep the machine busy gets great throughput but may make each individual user wait longer. If you care how soon one answer is ready — a keystroke, a database query a person is staring at — response time is the metric that matters, and a fast average throughput is cold comfort to the one request stuck waiting.