Experiment Design & A/B Testing

Randomized Controlled Experiment

A study where you split people (or whatever you are testing) into groups purely by chance, change one thing for some groups and not others, then compare what happens. Because the split is random, the groups start out alike on average, so any later difference can be credited to the change you made rather than to who happened to land in each group.

This is the gold standard for answering “did my change actually cause this?” because randomization (隨機分派) balances both the things you measured and the things you never thought to measure. Without it you can only see correlation; with it you can claim causation. The catch is you need enough people and a clean, uncontaminated split for the magic to work.

A hospital randomly gives half its incoming patients a new drug and half the old one, then compares recovery rates.

Also called
RCTcontrolled experiment