the finite-difference method
A derivative is a limit of a ratio, the slope you approach as two nearby points slide together. A computer cannot take that limit, but it can compute the ratio for points a small but finite distance apart. The finite-difference method rests on that single trade: lay a grid over space (and time), replace every derivative by a difference between neighbouring grid values, and a differential equation becomes ordinary arithmetic you can solve by brute force.
The standard building blocks come from Taylor expansion. A first derivative can be approximated by the centred difference df/dx ~= (f(x + h) - f(x - h)) / (2 h), whose error is of order h^2, or the one-sided forward difference (f(x + h) - f(x)) / h of order h. The second derivative becomes d^2f/dx^2 ~= (f(x + h) - 2 f(x) + f(x - h)) / h^2. Substituting these stencils into a partial differential equation, say Poisson's equation or the heat equation, turns it into a large system of algebraic equations linking the values at the grid nodes, which you solve with linear algebra or by stepping forward in time.
You meet it everywhere numerical physics is done: solving the Schrodinger equation on a lattice, computing electrostatic potentials, modelling heat flow and fluid dynamics. The essential caveat is stability. An explicit time-stepping scheme is only stable if the time step is small enough relative to the spatial step, a bound expressed by the Courant-Friedrichs-Lewy (CFL) condition; violate it and the numerical solution explodes into meaningless oscillations. There is also a tension: a finer grid lowers the discretization error but forces smaller time steps and more work, and it never touches round-off error.
To find the electrostatic potential in a charge-free region you discretize Laplace's equation on a grid; the second-difference stencil makes each interior node's potential the average of its four neighbours, so iterating that averaging (relaxation) until it stops changing yields the solution.
Laplace's equation on a grid: each node relaxes toward the average of its neighbours.
Refining the grid lowers discretization error but not round-off, and in an explicit time-stepping scheme a smaller spatial step forces a smaller time step through the CFL stability condition, so cost can grow faster than resolution.