The device trial ladder
Device trials do not use the drug world's Phase I/II/III labels. They climb a different ladder: an Early Feasibility Study (a first-in-human cohort of a few participants to assess basic safety and gather design feedback), a traditional feasibility study (a somewhat larger group to refine the protocol and endpoints), and finally a pivotal trial sized to confirm safety and effectiveness for market authorization. The philosophy differs too: a drug's mechanism is fixed once dosed, but a device — especially an adaptive BCI — is co-developed with its trial, iterating hardware, decoder and workflow between cohorts.
The N-of-few problem
The classic power calculation assumes you can enrol enough people to detect an effect. For BCI the target conditions are severe and rare, the surgery is major, and each participant is followed intensively for years — so cohorts are often single digits. The sample-size relation below shows why brute-force enrolment is hopeless: to shrink the detectable effect \delta you need n to grow with 1/\delta^2, and to lower the noise's grip you fight \sigma^2 in the numerator.
n \;\approx\; \frac{\bigl(z_{1-\alpha/2}+z_{1-\beta}\bigr)^{2}\,\sigma^{2}}{\delta^{2}}Per-group sample size to detect an effect δ at significance α and power 1−β; halving the detectable effect quadruples n — untenable for rare severe conditions.
How many participants per group you need to reliably detect a real effect. The killer feature is the \delta^2 in the denominator: the effect you're hunting enters squared, so a treatment whose benefit is half as large needs four times as many people — brutal for a trial in rare, severe conditions with few candidates.
- n
- The number of participants needed per group.
- \delta
- The effect size you want to be able to detect.
- \sigma
- The variability (spread) of the outcome.
- z_{1-\alpha/2},\,z_{1-\beta}
- Constants set by your significance level \alpha and power 1-\beta.
Halving the target effect \delta divides the denominator by four, so n quadruples — turning a feasible 40-patient arm into an unrealistic 160.
The response is to change the design, not chase impossible n. Within-subject designs make each participant their own control (decoder-on vs decoder-off blocks). Single-case experimental designs and N-of-1 protocols use dense, repeated measurement in one person with formal statistics. Bayesian adaptive designs update a posterior as data arrive and can stop early for success or futility. Each buys statistical credibility from repetition and structure rather than from population size.
Choosing endpoints that mean something
A clinical outcome measure must capture benefit a regulator and a patient both recognize. Engineers reach for performance metrics (bits per minute, cursor accuracy); clinicians want clinical benefit (can she communicate her needs?) and quality of life or activities-of-daily-living gains. Performance is a surrogate — useful, objective, but not automatically meaningful. The mature stance is to report performance and a patient-relevant outcome, and to be explicit about which is which.
The most honest common currency is throughput. The Wolpaw information transfer rate turns accuracy P and a choice among N options into bits, then bits per minute given selection time T. It rewards accuracy and speed together and penalizes wide-but-error-prone menus.
\text{ITR} \;=\; \underbrace{\left[\log_2 N + P\log_2 P + (1-P)\log_2\!\frac{1-P}{N-1}\right]}_{\text{bits per selection } B}\times\frac{60}{T}Wolpaw ITR (bits/min): assumes N equiprobable targets and errors spread evenly over the wrong ones — a model, so report the assumptions with the number.
The same bits-per-selection idea, now turned into a speed: multiply the information each choice carries by how many choices you make per minute. It's the standard way to put "how fast and accurate is this BCI?" into one number — but only under its assumptions (equal-probability targets, evenly spread errors), so quote those alongside it.
- \text{ITR}
- Information transfer rate, in bits per minute.
- B
- Bits per selection — the bracketed Wolpaw term.
- N,\,P
- The number of targets and the per-selection accuracy.
- 60/T
- Selections per minute, where T is seconds per selection.
Two bits per selection at one selection every 3 seconds — that is twenty selections a minute — gives about 40 bits per minute.
For continuous control (moving a cursor or arm) the natural yardstick is Fitts's law throughput: harder targets — far away (D) and small (W) — carry more bits, and dividing by movement time gives an achieved bit-rate directly comparable across studies and to able-bodied hands.
\text{ID} = \log_2\!\left(\frac{D}{W}+1\right),\qquad \text{Throughput} = \frac{\text{ID}}{\text{MT}}\;\;(\text{bits/s})Fitts's index of difficulty (ID) and throughput; a target-agnostic way to compare cursor and arm control across labs and against natural movement.
A fair yardstick for pointing tasks: a target that's far away (D) and small (W) is harder, and "difficulty" grows only with the logarithm of that ratio. Divide difficulty by the time it took and you get throughput in bits per second — a way to compare a BCI cursor against a mouse or a natural arm on equal terms.
- \text{ID}
- The index of difficulty of a movement.
- D
- The distance to the target.
- W
- The width of the target — bigger is easier.
- \text{MT}
- Movement time — how long the move took.
A target twice as far but also twice as wide has the same D/W, hence the same difficulty — Fitts's law says only the ratio matters, not the absolute size.
The control problem you cannot sham
Rigorous trials crave blinding and a placebo arm, and here BCI collides with reality. You cannot ethically perform a sham craniotomy to blind an implant. So sham control and blinding must be engineered where it is possible: blind stimulation ON vs OFF (as adaptive-DBS trials do), compare decoder-on vs decoder-off epochs within a session, randomize the order of conditions, and lean on objective performance so expectation effects cannot masquerade as benefit. Staggered enrolment (a multiple-baseline design) provides a further internal control without a sham surgery.
Who gets enrolled
Participant selection shapes the science and the ethics at once. Inclusion turns on the clinical indication (paralysis, ALS, stroke), disease stage and trajectory, intact cortical anatomy where you plan to record, ability to give (or have a surrogate give) consent, and crucially a psychosocial support structure able to sustain months of demanding study visits and home use. Narrow criteria make a cleaner trial but a less generalizable claim — a tension every protocol must state openly rather than hide.