computational graph
/ kom-pyoo-TAY-shun-ul graf /
A computational graph is a map of a calculation, drawn as a network of boxes and arrows. Each box is a single small operation — an addition, a multiplication, applying an activation function — and each arrow shows where its result flows next. Instead of treating a big formula as one tangled lump, you break it into these tiny steps and lay out how they connect. It's a bit like a recipe written as a flowchart: this step takes the output of that step, which took the output of two earlier steps.
Why bother? Because once a calculation is laid out as a graph, a computer can reason about it mechanically. A neural network's entire forward pass — millions of multiplications and additions — is one enormous computational graph, from the input at one end to the loss (the error) at the other. Modern deep-learning tools build this graph automatically as you write ordinary code, then use it to run the computation efficiently, often spreading it across many processors.
The graph's real superpower is making learning possible. To improve, the network needs to know how a tiny change in each weight would affect the final error. Walking backward through the graph, step by step, and applying the chain rule of calculus at each box, lets a computer compute all those sensitivities automatically and exactly. This backward walk is automatic differentiation, and it is the engine underneath backpropagation. Without the graph to follow, training networks with millions of parameters would be hopeless.
Take the small expression (a + b) × c. Its graph has three input boxes a, b, c; a "+" box that takes a and b; and a "×" box that takes the sum and c. To find how much the final answer changes when a wiggles, you just follow the arrows backward from the output, multiplying the local effect at each box. Scale this idea to millions of boxes and you have how a neural network learns.
A computational graph breaks any formula into tiny linked steps you can walk forward or backward.
The same graph is traversed in two directions: forward to compute the answer, backward to compute how to improve it. That reusable structure is precisely what makes automatic differentiation, and hence training, practical at scale.