Agency, Authorship and the Responsibility Gap

The sense of agency and the comparator model

The sense of agency is the ordinary feeling that I am causing an action and its effects. Cognitive science models it, in part, with a comparator: the brain predicts the sensory consequences of its own motor commands, then compares the prediction with what actually happens. When prediction and outcome match, agency is high; when they diverge, the action feels externally caused, delayed, or not mine.

\text{PE}_t = \big\lVert\, s_t - \hat{s}_t \,\big\rVert, \qquad \hat{s}_t = g(u_t)

A comparator prediction error: the mismatch between the actual sensory feedback s and the brain's forward-model prediction ŝ of its own command u. Agency degrades as PE grows.

Your brain predicts what its own actions will feel like, then compares that prediction to what actually happens. The mismatch is the prediction error. A small error feels like "I did that" (agency); when a BCI adds lag or mistakes, the mismatch grows and the sense of ownership over the action fades.

\text{PE}_t: The prediction error at time t — the felt mismatch.
s_t: The actual sensory feedback you receive.
\hat{s}_t: The forward model's prediction of that feedback.
u_t: Your own motor command, from which the prediction is made.

Move a cursor and see it respond instantly and correctly, and \text{PE} stays low, so it feels like yours; add half a second of lag and the rising error erodes that feeling.

A BCI sits directly in this loop and can break it in ways ordinary tools cannot. Latency inserts a delay between intention and effect; an adaptive decoder silently changes the mapping so yesterday's command produces today's different result; a smoothing filter makes the effector move when you did not command it. Each enlarges PE, and users report the felt consequence: the cursor 'has a mind of its own,' the arm 'is not doing what I want.' Agency is not a vague comfort; it is a measurable property of a control loop you are engineering.

When the decoder acts with you: shared control

High-performance neuroprosthetics rarely execute raw neural commands. They blend the user's decoded intent with autonomous assistance — obstacle avoidance, grasp shaping, target snapping. Formally, the command sent to the effector is a convex combination of the user term and an autonomous term, weighted by an arbitration coefficient \alpha_t:

u_t = (1-\alpha_t)\,u^{\text{user}}_t + \alpha_t\,u^{\text{auto}}_t, \qquad \alpha_t \in [0,1]

Shared / assistive control. α=0 is pure user command; α=1 is full autonomy. The ethically loaded question is who sets α, whether the user knows its value, and whether it can rise without consent.

The command that actually gets executed is a blend of what the user intends and what the autonomous system suggests, mixed by a knob \alpha. At \alpha=0 the user is fully in charge; at \alpha=1 the machine drives. The ethical weight is not in the math but in who sets \alpha, whether the user knows its value, and whether it can quietly rise. See shared/assistive control.

u_t: The command actually carried out at time t.
u^{\text{user}}_t: The user's own intended command.
u^{\text{auto}}_t: The autonomous system's suggested command.
\alpha_t: The mixing weight in [0,1] — the share given to automation.

At \alpha=0.2 the machine only nudges the movement gently; slide \alpha toward one and the user's intent is increasingly overridden, perhaps without them realizing.

Assistance is genuinely good: it makes clinical BCIs usable and less exhausting. But it trades control for performance, and the trade is often invisible to the user. If \alpha is high and undisclosed, the person may feel authorship over an action the machine largely produced — a comfortable illusion — or, worse, be blamed for an outcome the autonomy chose. The design duty is not to minimise \alpha but to make it legible and governed: the user should be able to know, and ideally shape, how much of each action is theirs.

Algorithmic mediation of intention

A decoder never observes intention directly; it infers it. Bayesian decoders make the inference explicit: the estimate combines the neural likelihood with a prior over intentions, often shaped by context (a language model in a speech BCI, a task model in a cursor BCI).

p(\text{intent}\mid \text{neural},\,\text{context}) \;\propto\; p(\text{neural}\mid \text{intent})\; p(\text{intent}\mid \text{context})

Bayesian intention estimation. The prior p(intent|context) is chosen by designers. When it is strong, the output reflects the model's expectations as much as the user's neural signal — the intent has been algorithmically mediated.

The system's guess at what you intend combines two things: the evidence in your neural signal, and a prior expectation, baked in by designers, about what you're likely to want given the context. When that prior is strong, the output leans on the designers' assumptions as much as on your brain — your intention has been algorithmically mediated.

p(\text{intent}\mid \text{neural},\,\text{context}): The system's updated belief about your intent.
p(\text{neural}\mid \text{intent}): How well the neural signal fits each candidate intent (the evidence).
p(\text{intent}\mid \text{context}): The prior — what the designers assume is likely in this context.
\propto: "Proportional to" — the pieces multiply, then get normalized.

If the context prior strongly favors "yes" in a yes/no speller, a weak or ambiguous neural signal can still be read as "yes" — the answer partly reflects the prior, not just the user.

This is algorithmic mediation of intention, and it is a double-edged sword. A strong language-model prior is what makes a speech neuroprosthesis fluent; it also means the words that appear are partly the model's guess, not solely your neural output. In the limit — a very confident prior and a weak signal — the system can 'complete' an intention you did not fully form, or autocorrect toward a word you did not mean. Who authored the sentence: you, or the prior?

The responsibility gap

Now the hard question. A BCI-controlled wheelchair injures a bystander; a speech BCI emits a defamatory sentence; an assisted grasp drops a fragile object. Who is responsible? The user did not fully cause it (assistance and priors intervened). The decoder cannot be a moral agent. The manufacturer built a tool that behaves partly stochastically and adapts after deployment. This is the responsibility gap: a harm for which no party cleanly satisfies the conditions for responsibility — causal contribution plus control plus foreseeability.

The gap is not unique to BCI (it appears with any autonomous system), but neurotech sharpens it because the human is inside the loop, contributing a real but unmeasurable share. Two moves help without dissolving the problem. First, instrument the loop: log \alpha_t, the prior's influence, and decoder confidence, so post-hoc one can estimate how much of an action was user versus machine. Second, shift from asking 'who is to blame?' to allocating duties in advance — design standards, disclosure requirements, and liability rules set before deployment, so no one is retroactively saddled with a share they could not see.

Design principles for preserving agency

Make authorship legible: expose how much of each action is user versus autonomy (a visible or logged α), never let assistance masquerade as pure user control.
Keep priors inspectable and adjustable, and disclose them when they materially shape output.
Bound and signal latency and adaptation: users should be able to anticipate the mapping, and be told when it changes under them.
Provide a clean stop: an unambiguous, low-latency way to halt or veto the effector preserves agency even when moment-to-moment control is shared.
Instrument for accountability: log the quantities (α, confidence, prior weight) that a later inquiry would need to allocate responsibility fairly.