The Phase Problem and Solving a Structure

What the detector throws away

Recall from the diffraction rung that every reflection carries a structure factor F_hkl, and that F is a complex number — it has an amplitude |F_hkl| (how tall the scattered wave is) and a phase phi_hkl (where its crest falls, the timing that says how this wave lines up with all the others). But a detector is just a photon counter. It cannot sense timing; it can only record how many X-rays land in a spot, which is the intensity I_hkl proportional to |F_hkl|^2. Take the square root and you recover the amplitude |F_hkl|. The phase phi_hkl is simply gone — never measured, nowhere in the data.

Why should losing a phase matter so much? Because the atoms only appear when you add all the scattered waves back together in the right register, and it is the phases that set the register. Picture recording a piano chord but writing down only how LOUD each note is, never when each note is struck. You can name the notes, yet you can never rebuild the melody, because the tune lives in the timing you discarded. Losing the phase of every reflection is exactly this, in three dimensions and thousands of notes at once. This missing-timing catastrophe has a name — the phase problem — and it is the single deepest obstacle between a diffraction pattern and a crystal structure.

If you had the phases: the electron-density map

It helps to first see what you could do if the phases were handed to you, because that names exactly what is missing. The diffraction pattern and the crystal's electron density are a Fourier transform pair: one lives in reciprocal space, the other in real space, and each is the exact transform of the other. Concretely, every reflection (hkl) contributes one cosine ripple of electron density — a density wave whose repeat spacing is d_hkl, whose height is set by the amplitude |F_hkl|, and whose crests are positioned by the phase phi_hkl. Sum all those ripples over every measured reflection and the electron density rho(x,y,z) rebuilds itself. This summation is called a Fourier synthesis.

FROM A DIFFRACTION EXPERIMENT TO A STRUCTURE

  crystal --> diffraction --> detector counts photons
                                    |
                                    v
                        I(hkl) = |F(hkl)|^2      <-- measured
                                    |
                  sqrt()            |   the PHASE phi(hkl) is LOST here
                                    v
     you HAVE:  |F(hkl)|                you LACK:  phi(hkl)

  electron density (the answer you want):
     rho(x,y,z) = (1/V) * sum over hkl of
                  |F(hkl)| * cos( 2pi(hx+ky+lz) - phi(hkl) )
                  ^^^^^^^^                        ^^^^^^^^^
                  have this                       need this

  atoms = the spots where thousands of density waves crest together

The atoms in the finished map are nothing more than the peaks where thousands of these density waves happen to crest at the same place; the empty space between atoms is where they cancel. How sharp those peaks come out depends on how many reflections you fed in, and how far out in reciprocal space they reach. This is the inverse relationship between the two spaces again: the finest real-space detail is carried by the highest-angle reflections, so to separate two atoms 1 angstrom apart you must have measured data out to about d = 1 angstrom. Cut the data off too soon and the atoms blur together; the map's resolution is only as good as the outermost reflection you collected.

So the entire task of solving a structure collapses to one sentence: get the phases. You already own the amplitudes for free. If someone whispered you the phase of every reflection, you would type them into the Fourier synthesis, press go, and read the atoms straight off the map. Everything that follows — Patterson, direct methods, the whole toolbox — is just different ways of manufacturing a starting set of phases you were never allowed to measure.

Patterson: the map you can always draw

Here is a beautiful trick. What if you ran a Fourier synthesis but used the quantities you actually possess — the intensities |F_hkl|^2 — as the coefficients, and set every phase to zero? You get a real, computable map with no phase information needed at all, because |F|^2 is exactly what you measured. This is the Patterson function P(u,v,w), and its great virtue is that you can always draw it, for any crystal, straight from the raw data.

The catch is that a Patterson map does not show you atoms — it shows you the vectors BETWEEN atoms. A peak at position (u,v,w) in the Patterson map means: somewhere in the real structure there is a pair of atoms separated by exactly that vector. If atoms sit at r_1 and r_2, the Patterson has a peak at r_2 - r_1 (and one at r_1 - r_2). Crucially, the height of each Patterson peak is proportional to the product of the two atoms' atomic numbers, Z_i times Z_j — because heavier atoms scatter X-rays more strongly, so the vector between two heavy atoms shouts loudest.

That Z times Z weighting is what makes the Patterson practically useful, through the heavy-atom method. Suppose an organic molecule made mostly of carbon (Z = 6) contains a single bromine atom (Z = 35). A carbon-carbon vector has weight about 6 times 6 = 36, but the bromine-bromine vector has weight 35 times 35 = 1225 — roughly 34 times taller, towering over the forest of light-atom vectors. It is like a pitch-black party where you cannot make out anyone, except the one guest in a bright white suit: find that dominant Br-Br peak, and you have pinned down the heavy atom's position. From that single known atom you can compute approximate phases, run the Fourier synthesis, and start pulling the lighter atoms out of the resulting map one by one.

Direct methods: phases hidden in the amplitudes

The second great route refuses to give up on the phases so easily. Its founding insight is that the phases are not actually free to be anything — they are quietly constrained by physics you already know. The electron density must be non-negative everywhere (you cannot have less than no electrons at a point), and it is not a smooth blur but a scatter of sharp, discrete atomic peaks sitting on flat empty space. These two facts — positivity and atomicity — link the phases to the amplitudes statistically. The amplitudes alone do not fix the phases uniquely, but they make some phase combinations overwhelmingly more probable than others.

The workhorse of direct methods is the triplet relation. Among the strong reflections, phases obey, with high probability, phi_h is approximately phi_k plus phi_(h-k) — a three-way relationship linking reflections whose indices add up. No single triplet is certain, but each is a probabilistic vote, and the probability grows the larger the three amplitudes involved. So you fix a few starting phases, let thousands of triplet relations propagate their votes across the whole reflection list, and search for the self-consistent phase set that satisfies the most of them at once. Feed that phase set into the Fourier synthesis and, if it worked, atoms snap into focus.

It sounds like magic that amplitudes could confess their own phases, so hold onto the constraint that makes it work. It is like a jigsaw puzzle where you also happen to know the finished picture must be a real photograph of separated dots on a blank field — that single demand slashes the astronomically many phase combinations down to a tiny handful that could possibly be physical. Direct methods now solve most small-molecule structures (up to hundreds of atoms) almost automatically, and the idea earned Herbert Hauptman and Jerome Karle the 1985 Nobel Prize in Chemistry. Be clear on its limits, though: it needs data to near-atomic resolution (many strong reflections for the triplets to vote), and it works best on structures of roughly equal atoms — so very large proteins usually still lean on heavy-atom and anomalous-scattering methods instead.

From a first map to a finished structure

Neither Patterson nor direct methods hands you a perfect structure — each hands you a rough starting model: a few atoms in roughly the right places. From there the model is polished by refinement. You calculate what the intensities WOULD be for your trial atoms, compare against the measured ones, and nudge the atomic positions and thermal factors to shrink the disagreement. For single-crystal diffraction, where every reflection is measured separately, this is a least-squares refinement; for powder data, where the whole profile is fitted at once, it is Rietveld refinement — the subject of the next guide. Either way the loop is the same: model gives phases, phases give a better map, the map reveals atoms you had missed, and you refine again, the phases sharpening at every turn.

Two honest cautions close the loop. First, a solved structure is a model that fits the data, not a literal photograph of atoms; how well it fits is reported by an R-factor, and a low R means the model and the measurements agree, not that the answer is guaranteed unique. Second, powder data makes all of this harder than single-crystal data, because reflections at nearly the same 2-theta overlap and their individual |F| values cannot be cleanly separated — which is exactly why a full three-dimensional single-crystal dataset, giving every (hkl) on its own, is the gold standard for solving a new structure from scratch.