well-conditioned vs ill-conditioned problems
Think of two roads to a viewpoint. On the gentle road, drifting a step off course leaves you essentially where you meant to be. On the knife-edge ridge, one step sideways drops you off a cliff. A well-conditioned problem is the gentle road: small input errors give small output errors. An ill-conditioned problem is the ridge: tiny input errors can throw the answer wildly off. The dividing line is the size of the condition number.
Quantitatively, 'well' versus 'ill' is a matter of degree, read off the condition number kappa. kappa near 1 is excellent; kappa up to a few hundred or thousand is comfortably well-conditioned; kappa around 10^8 means you have lost about half of your 16 double-precision digits and should be uneasy; kappa near 10^16 means the problem is, for double precision, effectively unsolvable — there is no accuracy left to lose. The rule of thumb is that you forfeit about log10(kappa) significant digits purely because of conditioning.
The honest, often-surprising point: ill-conditioning is not a bug you can debug away. If the underlying problem is ill-conditioned, switching algorithms, adding checks, or polishing your code will not rescue accuracy — only reformulating the problem (e.g. avoiding a near-cancellation, using a better-posed quantity) or carrying more precision can. Many real ill-conditioned problems hide behind innocent appearances: differencing nearly equal numbers, fitting high-degree polynomials, or inverting matrices whose columns nearly line up.
The roots of (x-1)(x-2)...(x-20) are well-separated and seem harmless, but as Wilkinson's polynomial famously shows, perturbing a single coefficient by one part in 10^10 shifts some roots by 2 or 3 — finding those roots from the coefficients is severely ill-conditioned.
Looks tame, behaves wildly — the classic shock that launched the field.
Well- and ill-conditioned describe the problem, not your method. A common error is blaming the algorithm for a bad answer when the true culprit is an ill-conditioned problem.