the domain of the generator

Because the generator A of a strongly continuous semigroup is typically unbounded, it cannot be applied to every function in the Banach space — only to those on which the defining limit (P_t f - f)/t converges. That set is the domain D(A). Specifying D(A) is not a technicality to be waved away: for a differential generator it pins down boundary behaviour, conservativeness and which Markov process you actually have, so two operators with the same formula but different domains are different operators generating different processes.

Formally, D(A) = { f in X : lim as t -> 0+ of (P_t f - f)/t exists in X }, and A : D(A) -> X is the resulting linear map. Several structural facts hold for any C_0 semigroup: D(A) is a DENSE linear subspace of X (so A is densely defined); A is a CLOSED operator (its graph is closed, which is the right substitute for continuity of an unbounded map); the semigroup leaves the domain invariant, P_t : D(A) -> D(A), and on D(A) the semigroup is differentiable with d/dt P_t f = A P_t f = P_t A f. In practice the abstract D(A) is awkward, so one works with a CORE: a smaller, more concrete subspace D_0 contained in D(A) that is dense in D(A) in the graph norm and on which A is essentially determined (the closure of A restricted to a core equals A). For a diffusion, the smooth compactly supported functions C_c^infinity form a natural candidate core.

Why it matters: the martingale problem, Dynkin's formula and Hille-Yosida are all stated 'for f in D(A)' (or on a core), and getting the domain right is what makes uniqueness theorems true. The classic warning is the heat semigroup on a half-line: the formula A f = (1/2) f'' is the same whether you impose absorbing (Dirichlet) or reflecting (Neumann) boundary conditions, but those are different domains, hence different processes (killed versus reflected Brownian motion). Identifying a core is usually the hard analytic step; without it, claims that two processes 'have the same generator' can be quietly false.