False-Positive Calls of Reimplemented Published Analyses for Covert Command-Following on Electroencephalograms Recorded Without Study Commands

A. Gorenshtein; Y. Adiniaev; Y. Barash; E. Klang; O. Daniel

doi:10.64898/2026.11.03.26367200

AI summary · DOI: 10.64898/2026.11.03.26367200 · medRxiv: Neurology

False-Positive Calls of Reimplemented Published Analyses for Covert Command-Following on Electroencephalograms Recorded Without Study Commands

A. Gorenshtein, Y. Adiniaev, Y. Barash, E. Klang, O. Daniel

Original paper Published on 04 Nov. 2026 medRxiv preprint License: CC BY 4.0 Also in: fr

How to read this summary? Illustrated guide

The essentials

Three reimplemented EEG analyses frequently called patients positive on recordings without study commands; false-call rates depended on the complete analysis and patient-level decision rule.

Question

How often do published EEG analyses for covert command-following call patients positive when no study command was given, and what contributes to these false calls? The study also examines whether randomizing trial order, testing genuine-task controls, or calibrating thresholds changes false calls and detection.

Method

The authors reimplemented 11 published analyses and ran them on command-free recordings, including a development set of 974 clinical EEG patients, a fixed replication set of 730, 38 patients with prolonged disorders of consciousness, and 544 post-cardiac-arrest patients. They also tested repeated runs and randomized trial order, used genuine-task and sham-label controls, and calibrated thresholds on null recordings to target a 5% patient-level false-call probability over 1, 2, 5, or 10 assessments.

Findings

Result Three of the 10 command-following analyses called 13.8% to 23.3% of clinical EEG patients positive without study commands, and these elevated rates were confirmed in the replication set.
See the passage in the paper
Goldfine 2011 called 164 of 703 patients (23.3%; 95% CI, 20.2%-26.6%), du Bois 2026 157 of 707 (22.2%; 95% CI, 19.2%-25.5%) and Guger 2018 117 of 850 (13.8%; 95% CI, 11.5%-16.3%)
Result The same three rules called 11.5% to 27.3% of comatose patients positive on EEG recorded about 24 hours after cardiac arrest.
See the passage in the paper
Applied unchanged from clinical EEG to the 544 patients with a usable window, du Bois 2026 called 140 of 513 (27.3%; 95% CI, 23.5%-31.4%), including 29 of 83 with a suppressed background, Guger 2018 75 of 534 (14.0%; 95% CI, 11.2%-17.3%) and Goldfine 2011 57 of 494 (11.5%; 95% CI, 8.9%-14.7%)
Result Permitting more Guger 2018 runs increased the proportion of patients called positive, reaching 62.9% among patients with seven runs.
See the passage in the paper
In a fixed cohort of 639 patients with 7 runs, the share rose from 14.1% with 1 run to 62.9% with 7
Result For Goldfine 2011, changing the jackknife unit from individual tapered estimates to whole commands reduced false calls while retaining high detection in stroke imagery sessions.
See the passage in the paper
deleting whole commands rather than single tapered estimates in its jackknife gave a variance a median 3.9 times that expected for independent estimates and lowered false calls from 79 (23.1%) to 2 (0.6%; 95% CI, 0.1%-2.1%)
Authors’ interpretation Independent implementations of two analyses disagreed, underscoring that a publication may not specify a unique executable analysis.
See the passage in the paper
Implementations written independently from the same texts disagreed with ours, so a publication does not fix a single executable analysis.

Limitations

The analyses were reimplemented from publications and public code rather than tested with vendor software or reviewed by the original authors or an independent EEG expert, and implementations could disagree.
See the passage in the paper
First, the analyses were reimplemented from publications and public code, without vendor software, the original authors’ review or review by an independent EEG expert; independent implementations of 2 analyses disagreed with ours
The genuine-task controls were not conducted in patients with disorders of consciousness, and no recordings from the original studies' patients were available.
See the passage in the paper
Second, the real-task controls used healthy volunteers and patients after stroke, not patients with disorders of consciousness, and a tactile Braille task rather than the vendor’s vibrotactile paradigm; no recordings of the original studies’ patients were available
The replication was internal to one corpus, and the exploratory disorders-of-consciousness cohort was small.
See the passage in the paper
Third, the choice rule and 2 corrections were set after development rates had been seen; the replication was internal to one corpus and had no background labels

Why it matters

The authors argue that a positive EEG command-following result should be interpreted using the complete patient-level call rule, including its false-call rate on relevant command-free recordings and detection of genuine responses at that rate. They recommend reporting these measures rather than relying on an algorithm name or nominal threshold alone.

Written by AI from the paper. Passages are checked word for word against the text we analysed; the rest may contain errors.

AI-generated keywords: EEG covert command-following false-positive calls disorders of consciousness statistical validation

Read further

Comprehensive Summary
Layman's Summary
Blog article
Original abstract

The paper examines how often published EEG analyses for covert command-following produce positive results when patients have not received a study command, and which analytical features may contribute to those results. This question matters because a positive finding can affect prognostic counseling and treatment decisions for unresponsive patients. The authors stress that a positive call on command-free EEG is not equivalent to a clinical false positive: it does not show what happens when a patient hears a command but fails to follow it. The researchers reimplemented 11 published analyses and applied their command or stimulus schedules to EEG recordings made without study commands. The clinical EEG development set comprised 974 patients, followed by a fixed-rule replication set of 730 patients. They also examined recordings from 38 patients with prolonged disorders of consciousness and 544 comatose patients recorded about 24 hours after cardiac arrest. To assess detection as well as false calls, they tested selected analyses on recordings with genuine tasks, including motor imagery in stroke patients and a tactile-attention task in volunteers. Additional tests examined randomized trial order, repeated runs, and thresholds calibrated using command-free or sham-label data. Because some publications leave analytical choices unspecified, the team tested multiple plausible options. For the primary results, they selected the option with the fewest positive calls among those considered consistent with each publication. Three of the ten command-following analyses produced high false-call rates on clinical EEG. Goldfine 2011 called 23.3% of patients positive, du Bois 2026 called 22.2%, and Guger 2018 called 13.8%. The results replicated in the 730-patient set, with corresponding rates of 22.9%, 24.8%, and 12.5%; each was significantly above 5%. The other seven analyses called between 0% and 6.9% of patients positive in the development set. The same three analyses called 11.5% to 27.3% of comatose patients positive after cardiac arrest. Repeated testing increased the chance of a positive call. For Guger 2018, the proportion of a fixed group called positive rose from 14.1% after one run to 62.9% after seven runs. The excess calls had different explanations. In Goldfine 2011, the published jackknife method treated tapered estimates within a command as independent. Recalculating the variance with whole commands as the unit increased the estimated variance by a median factor of 3.9 and reduced calls from 23.1% to 0.6% in locally held development recordings. With the correction fixed in advance, the call rates were 0.8% in the replication set and 2.4% after cardiac arrest. The analysis still detected 93.4% of stroke motor-imagery sessions against rest. The authors caution that this finding does not show that the test specifically identifies motor imagery. For du Bois 2026, results depended strongly on choices such as whether cross-validation respected task blocks and how its configuration search was handled. Its call rate fell substantially when the publication’s permutation evaluation was included. Guger 2018 used a threshold close to its chance rate of about 12%. Although it detected genuine tactile-attention responses well, it also frequently called sham labels positive. Randomizing trial order reduced false calls to roughly 3.6%–6.0% in clinical EEG, but reduced detection for some methods. The paper concludes that an algorithm’s name or nominal threshold is insufficient to interpret a positive result. Reports should specify the complete patient-level decision rule, including the permitted number of runs, the false-call rate on relevant command-free recordings, and the method’s ability to detect genuine responses under comparable conditions.

When EEG Suggests a Response to a Command That Was Never Given

For patients who cannot communicate, an EEG analysis that appears to detect a response to a command can have profound consequences. It may influence how clinicians discuss prognosis and make treatment decisions. But how often might an analysis report command-following even when no study command was presented?

The paper “False-Positive Calls of Reimplemented Published Analyses for Covert Command-Following on Electroencephalograms Recorded Without Study Commands,” by A. Gorenshtein, Y. Adiniaev, Y. Barash, E. Klang, and O. Daniel, investigates that question. The researchers reimplemented published EEG analyses and tested them on recordings made without study commands, examining both how often they produced positive calls and which analytical choices might explain those results.

What a “false call” does—and does not—mean

The authors make an important distinction: a positive result on EEG recorded without a study command is not automatically a clinical false positive. It cannot tell us what would happen when a patient actually hears a command but does not follow it. Instead, these tests measure how often an analysis produces a positive call in the absence of the study’s command or task.

That distinction matters when interpreting the findings. The study evaluates a potential source of misleading results, not whether any particular patient was aware or capable of following commands.

Testing published methods on command-free recordings

The team reimplemented 11 published analyses and applied their command or stimulus schedules to EEG recordings collected without study commands. Their main clinical EEG development set included 974 patients. They then tested fixed versions of the analyses on a separate replication set of 730 patients. The study also included recordings from 38 patients with prolonged disorders of consciousness and 544 comatose patients recorded about 24 hours after cardiac arrest.

To check whether methods could detect genuine responses as well as generate command-free calls, the researchers tested selected analyses on recordings with real tasks. These included motor imagery in stroke patients and a tactile-attention task in volunteers. They also explored the effects of randomized trial order, repeated runs, and thresholds calibrated using command-free or sham-label data.

Published descriptions do not always specify every analytical choice. Where details were ambiguous, the team tested multiple plausible options. For its primary results, it chose the option that yielded the fewest positive calls among those considered consistent with each publication.

Three analyses produced notably high call rates

Three of the ten command-following analyses showed high rates of positive calls on clinical EEG recorded without study commands:

Goldfine 2011 called 23.3% of patients positive in the development set.
du Bois 2026 called 22.2% positive.
Guger 2018 called 13.8% positive.

The pattern persisted in the separate replication set, where the corresponding rates were 22.9%, 24.8%, and 12.5%. Each was significantly higher than 5%. The other seven analyses produced rates between 0% and 6.9% in the development set. The three high-call methods also produced positive results in 11.5% to 27.3% of the post-cardiac-arrest recordings.

Why the results differed

Goldfine 2011: how the analysis estimated uncertainty

For Goldfine 2011, the researchers identified a problem with how the published jackknife method treated tapered estimates within a command as independent. When they instead treated whole commands as the unit for calculating variance, the estimated variance rose by a median factor of 3.9. In locally held development recordings, positive calls fell from 23.1% to 0.6%.

With that correction fixed in advance, the call rate was 0.8% in the replication set and 2.4% after cardiac arrest. The corrected analysis detected 93.4% of stroke motor-imagery sessions against rest. The authors caution, however, that this does not prove the test specifically identifies motor imagery.

du Bois 2026: choices in validation and configuration

Results for du Bois 2026 depended strongly on implementation choices, including whether cross-validation kept task blocks together and how the search for an analysis configuration was handled. Its positive-call rate fell substantially when the publication’s permutation evaluation was included. This illustrates how apparently small procedural details can change the behavior of an analysis.

Guger 2018: a threshold close to chance

Guger 2018 detected genuine tactile-attention responses well, but it also frequently called sham labels positive. Its threshold was close to its chance rate of about 12%, making the interpretation of positive results especially dependent on how chance performance was assessed.

Repeated testing can change the odds

The number of runs also mattered. For Guger 2018, the share of a fixed group called positive increased from 14.1% after one run to 62.9% after seven runs. In other tests, randomizing trial order reduced command-free positive calls to roughly 3.6%–6.0%, though this also reduced detection for some methods.

These findings show why a result cannot be interpreted apart from the testing procedure. Repeating a method, changing the trial order, or adjusting validation and thresholds may alter both its chance of detecting a genuine response and its chance of producing a positive call without a study command.

What should EEG reports include?

The central message is that an algorithm’s name or nominal threshold is not enough to judge a positive result. Reports should describe the complete patient-level decision rule: how many runs are permitted, how results across runs are combined, and how the threshold is set.

They should also report performance on relevant command-free recordings and show whether the method can detect genuine responses under comparable conditions. Clear reporting would help clinicians and researchers understand what a positive EEG finding supports—and what it does not—before that finding informs consequential decisions.

Created on 09 Nov. 2026

Was this synthesis useful?

Score: 0

Ask questions about this paper to our AI assistant

How to chat with this paper? Illustrated guide

You can also chat with multiple papers at once here.

AI assistant instructions?

Tools for this paper

Explain it differently 1 free rewrite left today

Save to a collection Group papers in collections to compare them, chat with them and export them. Continue with Google Create a free account Log in

Export BibTeX · RIS (Zotero, Mendeley, EndNote) · Obsidian (Markdown) · Notion (Markdown)

Advanced tools Premium

For researchers who follow a field closely: citation alerts, journal-club slides, version comparison and Zotero export. Discover Premium

Zotero

Citation alerts Citation alerts guide

Who works on it? The people and institutions who publish on its topic

Weekly letter Presented in "Brain and behaviour this week" of 9 November 2026. Brain and behaviour this week: What persists when the brain changes

Journal club Premium+ Slides to present it Propose for the journal club

Talk about it in public Premium+ Press kit: release, questions, hype check Press kit guide

Papers around this one

See them on a map

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

Close Broader context
How large an effect would we have seen? Positive controls, detection floors and dataset pitfalls in scalp-EEG speech decoding
B. Adeika, O. Osuagwu · 2026 · cited 1 time · Not summarized yet

This audit tests imagined-speech EEG decoding with injected effects and positive controls, addressing detection limits and validation pitfalls rather than command-free false calls.

Abstract
Objective. Studies that decode imagined speech from scalp electroencephalography (EEG) commonly validate their pipelines with a positive control, usually overt speech, and report null results without stating what effect size the pipeline could have detected. Approach. Using ds003626 ("Thinking out loud"), KaraOne and Track 3 of the 2020 International Brain-Computer Interface Competition, we audited decoding of overt speech, articulator preparation and imagined speech with artifact-channel subsets, band-limited decoding and trial selection, under a protocol that enforces split integrity by assertion. We injected class-dependent effects of known amplitude into the real recordings to measure th…
Close Broader context
Beyond Classification Accuracy: Building Trustworthy Artificial Intelligence for Clinical Electroencephalography.
Sha-Sha Hu, Yun-Lei Huang, Jie Zhang · 2026 · Brain Research Bulletin · Not summarized yet

This article discusses why high EEG classification accuracy does not establish clinical readiness, while the paper measures false calls of command-following analyses.

Abstract
Electroencephalography (EEG) remains important in clinical neuroscience because it records brain activity as a time-resolved physiological process. It offers millisecond temporal resolution, can be used at the bedside and supports prolonged monitoring. Artificial intelligence (AI) has expanded EEG analysis through automated pattern recognition, representation learning, large-scale screening and decision support in epilepsy, sleep medicine, cognitive disorders, psychiatric conditions, intensive care, disorders of consciousness and anaesthesia monitoring. High classification accuracy in a research dataset, however, is not clinical readiness. EEG signals are shaped by physiological state, artef…
Close Broader context
Decoding sequence auditory expectations for clinical diagnostic and prognostic assessment of prolonged disorders of consciousness.
Rui-Jie Luo, Yue-Hong Huang, Chen-Xia Xue et al. · 2026 · Journal of Neural Engineering · Not summarized yet

This study decodes auditory expectations in prolonged disorders of consciousness, a patient population also included in the paper's command-free false-call tests.

Abstract
OBJECTIVE Electroencephalogram (EEG) features are important objective markers for diagnostic and prognostic evaluation of prolonged disorders of consciousness (DOC). However, task-based EEG paradigms are limited by mismatches between task demands and patients' residual awareness, as well as the limited individual assessment capacity of machine-learning-based detection frameworks. APPROACH In this study, we proposed a lowdemand DOC detection strategy by decoding expectations of global regularities in sequence auditory stimulation using a modified local-global paradigm. Three single-trial classification frameworks using canonical correlation analysis, principal component analysis, and xDAWN fo…
Close Broader context
Outcome-Blind EEG Microstate Topography After Cardiac Arrest: Spatial Expression of a Lateralized Microstate and Association with Neurological Outcome
Calixto Machado · 2026 · Brain Topography · Not summarized yet

This study analyzes EEG microstates after cardiac arrest and relates them to neurological outcome, whereas the paper tests command-following calls in post-arrest recordings.

Abstract
Electroencephalography (EEG) is central to neurological assessment after cardiac arrest, yet conventional approaches predominantly characterize amplitude, frequency content, continuity, suppression, reactivity, or predefined pathological patterns. EEG microstates provide a complementary framework by describing quasi-stable configurations of the global scalp electric field. We investigated whether an outcome-blind microstate decomposition of EEG from comatose survivors of cardiac arrest identifies reproducible spatial states associated with neurological outcome. We performed a secondary analysis of the multicenter I-CARE/PhysioNet database. Following prespecified EEG screening requiring conti…
Related Another approach to the problem
Same pipelines, different answers: how evaluation design shapes EEG seizure prediction
z. song, M. Pedersen, D. Henare · 2026 · medRxiv preprint · AI summary available

This study holds seizure-prediction pipelines fixed while changing evaluation splits, examining how evaluation design affects EEG results rather than false command-following calls.

In short
Fixed EEG seizure-prediction pipelines performed near the 0.5 AUC reference on unseen patients, while window allocation and threshold transfer substantially changed their apparent performance.
Related Review
Variability in Preprocessing and Analysis of the Error‐Related Negativity in Adult Response‐Conflict Studies: A Systematic Review
Katharina Paul, Johanna Paping, Jan Wacker · 2026 · Psychophysiology · Not summarized yet

This systematic review catalogs preprocessing and analysis choices in EEG error-related negativity studies, rather than testing command-following analyses on command-free recordings.

Abstract
ABSTRACT The analysis of EEG (electroencephalography) data affords many defensible choices, creating substantial researcher degrees of freedom. While stimulus‐locked components such as the N400 show wide methodological variability and incomplete reporting, it has been unclear whether this generalizes to the response‐locked error‐related negativity (ERN). We conducted a PRISMA‐guided systematic review of ERN studies published 1995–2025. From a chronologically stratified sample of 202 studies of adults using conventional response‐conflict tasks and conventional ERP analyses, we extracted preprocessing and quantification choices, assessed reporting completeness, examined temporal trends, and te…
Related Broader context
Assessing consciousness by near-nerve electrostimulation in intensive rehabilitation
Piergiuseppe Liuzzi, Sara Ballanti, A. Mannini et al. · 2026 · Journal of Neural Engineering · Not summarized yet

This study uses stimulation-evoked EEG measures to assess consciousness in prolonged disorders of consciousness, a neighboring clinical application to covert command-following.

Abstract
Objective. Consciousness diagnosis in prolonged disorders of consciousness (pDoC) relies on the Coma Recovery Scale–Revised, yet consciousness or its potential for recovery may escape behavioural detection. We tested whether near-nerve stimulation (NNS) of the median nerve, compared to standard surface cutaneous stimulation (SCS), could provide an EEG-based marker of consciousness and its recovery, by specifically eliciting sensory responses. Approach. Nineteen pDoC patients (median age 69 years, time post-onset 74 d) underwent NNS and SCS with high-density EEG. We quantified evoked potentials, global field power and perturbational complexity, and defined recovery as improvement in clinical …
Related Broader context
Measuring Evaluation Leakage in EEG Motor Imagery Decoding, with an Application to Stroke Patients
Aria Chadha · 2026 · Proceedings of the OJSS Virtual Research Conference 2026 · Not summarized yet

This study quantifies evaluation leakage in EEG motor-imagery decoding, while the paper examines false-positive calls and detection in covert command-following analyses.

Abstract
Brain-computer interfaces (BCIs) that decode imagined movement from EEG are often evaluated using Common Spatial Patterns (CSP), a supervised filtering method. If CSP is fitted to all data before splitting into training and test sets, information from the test labels leaks into the model and inflates reported accuracy. This study measures the size of that effect. In a null experiment where true accuracy should be at chance (50%), fitting CSP before the split produced 91% within-subject accuracy, a 41-percentage-point inflation, compared with about 5 points in cross-subject evaluation. On real data from 87 healthy participants and 46 post-stroke patients, leakage added 14.3 and 10.6 points re…

Go further

How are these papers found?

We compare the title and abstract of this paper with those of every paper summarized on this site, and with the papers Semantic Scholar recommends, those it cites and those citing it (for a much-cited paper, its most recent citations and, from OpenAlex, the most cited ones). Among the prior work and the follow-ups, the papers that use each other in depth, a method it uses and the much-cited follow-ups come a little higher. The levels (very close, close, related) come from the similarity of the texts, not from a judgment of quality; * marks a paper without an abstract, compared by its title. The sentence under a paper is written by an AI from the two abstracts: check it in the papers. A paper marked not relevant by three readers leaves the list.

Updated on 9 November 2026.

Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.

False-Positive Calls of Reimplemented Published Analyses for Covert Command-Following on Electroencephalograms Recorded Without Study Commands

The essentials

Question

Method

Findings

Limitations

Why it matters

Read further

Definitions

When EEG Suggests a Response to a Command That Was Never Given

What a “false call” does—and does not—mean

Testing published methods on command-free recordings

Three analyses produced notably high call rates

Why the results differed

Goldfine 2011: how the analysis estimated uncertainty

du Bois 2026: choices in validation and configuration

Guger 2018: a threshold close to chance

Repeated testing can change the odds

What should EEG reports include?

Ask questions about this paper to our AI assistant

Tools for this paper

Advanced tools Premium

Papers around this one

Closest papers

Go further

Cite this paper (BibTeX)