Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -41,3 +41,5 @@ data/
bids/
!bcipy/simulator/data

# Ignore paper build files
paper/jats
43 changes: 20 additions & 23 deletions paper/paper.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,67 +116,64 @@ Evidence of use is reflected in 76,000 estimated downloads (from PyPI) and 39 ex

BciPy is positioned for near-term impact within the BCI community due to its emphasis on reproducibility, clear documentation, and compatibility with common hardware, software, and analysis tools. By providing a simple and extensible interface for neural data acquisition and control, our work helps accelerate rapid prototyping and experimental iteration in BCI research.

# BciPy 2.0 Overview
# BciPy 2.0 Change Overview

BciPy 2.0 is a major update to the original BciPy 1.0, with significant improvements in architecture, functionality, and usability. The following sections describe the key additions in detail: task and experiment orchestration, multimodal data acquisition and evidence fusion, and a new simulation module for offline evaluation of system parameters.
BciPy 2.0 is a major update to the original BciPy 1.0, with significant improvements in architecture, functionality, and usability. The following subsections describe the key additions: task and experiment orchestration, multimodal data acquisition and evidence fusion, and a new simulation module for offline evaluation of system parameters.

# Task & Experiment Support
## Task & Experiment Support

BciPy manages the execution of experimental `Task`s using the `SessionOrchestrator` class (see \autoref{fig:orchestrator}), which ensures `Task`s are run in the correct order and that all data are properly persisted. Researchers can run individual `Task`s, such as `Calibration`, directly via the client, or define complete experimental protocols for reproducible studies.
BciPy manages the execution of experimental `Task`s using the `SessionOrchestrator` class, which ensures `Task`s are run in the correct order and that all data are properly persisted. Researchers can run individual `Task`s, such as `Calibration`, directly via the client, or define complete experimental protocols for reproducible studies.

Experiment protocols are specified in the `experiments.json` file as ordered sequences of `Task`s. The orchestrator reads this protocol, executes each `Task` sequentially, and logs all relevant information. Each `Task` writes its own logs to a dedicated subdirectory for easy tracking and analysis.

In addition to standard `Task`s, protocols can include `Action`s, which are lightweight subclasses of `Task`. `Action`s do not require display or data acquisition and are useful for simple steps such as prompting the researcher with a dialog or indicating experiment progress. For example, an `IntertaskAction` can be inserted to request input from the researcher before continuing to the next `Task` in the sequence.

<!-- Add Figure3 from static/ -->
![**Session Orchestration.** The `SessionOrchestrator` executes a sequence of `Task`s defined in an experiment protocol. Each `Task` is initialized with the current parameters and any data needed from previous `Task`s. The orchestrator manages the flow of data between `Task`s and ensures that each `Task` is executed in the correct order. The `SessionOrchestrator`, once initialized with parameters and optional metadata, is ready for `Task`s to be added using the `add_tasks()` or `add_task()` methods. These can be defined and loaded using the `experiment.json` and defined protocol or manually added to the `SessionOrchestrator`. The experiment can then be run using `execute()`. This method loops over `Task`s, providing all parameters and a log needed for operation. The `Task`s are then responsible for initializing any objects required for operation, such as the `DataAcquisitionClient`, `Display`, or `LanguageModel`. After each `Task` and the entire execution loop, the data persists on disk.\label{fig:orchestrator}](static/Figure3.png){width=50%}
<!-- ![**Session Orchestration.** The `SessionOrchestrator` executes a sequence of `Task`s defined in an experiment protocol. Each `Task` is initialized with the current parameters and any data needed from previous `Task`s. The orchestrator manages the flow of data between `Task`s and ensures that each `Task` is executed in the correct order. The `SessionOrchestrator`, once initialized with parameters and optional metadata, is ready for `Task`s to be added using the `add_tasks()` or `add_task()` methods. These can be defined and loaded using the `experiment.json` and defined protocol or manually added to the `SessionOrchestrator`. The experiment can then be run using `execute()`. This method loops over `Task`s, providing all parameters and a log needed for operation. The `Task`s are then responsible for initializing any objects required for operation, such as the `DataAcquisitionClient`, `Display`, or `LanguageModel`. After each `Task` and the entire execution loop, the data persists on disk.\label{fig:orchestrator}](static/Figure3.png){width=50%} -->

# Multimodal Data Acquisition and Fusion
## Multimodal Data Acquisition and Fusion

BciPy 2.0 introduces support for multimodal data acquisition and evidence fusion. The system can consider information from multiple devices when making typing decisions. The BciPy 1.0 data acquisition module supported TCP-based connections as well as connections through LabStreamingLayer (LSL) [@Kothe:2025]. After extensive testing, we leaned more heavily on LSL to support multimodal acquisition in BciPy 2.0. LSL is well-supported across the industry, with many devices providing compatible drivers. This decision allowed us to drastically simplify the acquisition module while increasing functionality.
BciPy 2.0 introduces support for multimodal data acquisition and evidence fusion, allowing the system to consider information from multiple devices when making typing decisions. After extensive testing, we leaned more heavily on LabStreamingLayer (LSL) [@Kothe:2025] to support multimodal acquisition in BciPy 2.0. LSL is well-supported across the industry, with many devices providing compatible drivers. This decision allowed us to drastically simplify the acquisition module compared to BciPy 1.0 while increasing functionality.

The data acquisition module in BciPy has two primary responsibilities: passively recording streaming device data to disk for later processing, and querying data in real time for use in a typing task. BciPy 2.0 streamlines the query interface from BciPy 1.0, replacing the previous unconstrained session-wide data access with a more efficient, event-driven approach suited to multimodal workflows.
<!-- The data acquisition module in BciPy has two primary responsibilities: passively recording streaming device data to disk for later processing, and querying data in real time for use in a typing task. BciPy 2.0 streamlines the query interface from BciPy 1.0, replacing the previous unconstrained session-wide data access with a more efficient, event-driven approach suited to multimodal workflows. -->

The approach for combining multimodal data sources is described briefly. Under the conditional independence assumption, a Bayesian probabilistic algorithm that fuses information from multiple sources of evidence is employed to achieve multimodal classification. The posterior probabilities for user intent ($\theta$) given biosignals data ($x_{1:s}$) are computed using Bayes' rule as follows:
<!-- The approach for combining multimodal data sources is described briefly. Under the conditional independence assumption, a Bayesian probabilistic algorithm that fuses information from multiple sources of evidence is employed to achieve multimodal classification. The posterior probabilities for user intent ($\theta$) given biosignals data ($x_{1:s}$) are computed using Bayes' rule as follows:

$$
p(\theta \mid x_{1:s}) \propto p(\theta) \prod_{j=1}^{s} p(x_j \mid \theta) \tag{1},
$$

where $s$ denotes the number of sources ($s > 1$ for multimodal) and $p(\theta)$ is the class prior.
where $s$ denotes the number of sources ($s > 1$ for multimodal) and $p(\theta)$ is the class prior. -->

In addition to the EEG signal model developed in BciPy 1.0, BciPy 2.0 introduces a gaze model for classification of eye gaze trajectory data that can be acquired in parallel with the EEG data stream through an eye tracker. Both the EEG and gaze models inherit the same `SignalModel` structure and are compatible with the scikit-learn estimators API [@Pedregosa:2011].

The provided gaze model assumes positional and temporal dependence in gaze data. It is also assumed that the gaze trajectory ($x_g$) obeys a Gaussian Process distribution, that is:
<!-- The provided gaze model assumes positional and temporal dependence in gaze data. It is also assumed that the gaze trajectory ($x_g$) obeys a Gaussian Process distribution, that is:

$$
p(x_g \mid \theta) \sim \mathcal{GP}(\mu_{g,\theta}, \Sigma_g) \tag{2},
$$

where $\mu_{g,\theta}$ and $\Sigma_g$ are the multidimensional means and shared covariances corresponding to the class labels, respectively. Details of the multimodal fusion method for EEG and gaze data are further described in the repository.
where $\mu_{g,\theta}$ and $\Sigma_g$ are the multidimensional means and shared covariances corresponding to the class labels, respectively. Details of the multimodal fusion method for EEG and gaze data are further described in the repository. -->

BciPy 2.0 expands upon the language modeling capabilities of BciPy 1.0 and removes the language model (LM) module from the previous Docker image, opting instead for direct function calls in Python. The goal of the language model remains the same as the prior version—to accelerate the text input task by providing additional evidence to the system. The LM module takes the context, or the text that the user has written so far, and produces an initial likelihood distribution over the system's symbol set. This distribution can be used to present more likely characters to the user sooner, in a paradigm like RSVP Keyboard [@Orhan:2012], or simply to fuse with the evidence gathered from the user (e.g., EEG, gaze, etc.).
BciPy 2.0 expands upon the language modeling capabilities of BciPy 1.0 and removes the language model (LM) module from the previous Docker image, opting instead for direct function calls in Python. The LM module takes the text that the user has written so far and produces a likelihood distribution over the system's symbol set. This distribution can be used to present more likely characters to the user sooner, in a paradigm like RSVP Keyboard [@Orhan:2012], or simply to fuse with the evidence gathered from the user (e.g., EEG, gaze, etc.). BciPy 2.0 introduces inference by large language models (LLMs) via the TextSlinger API, which allows the use of causal transformer models from Hugging Face using the search algorithm from @Gaines:2025. BciPy 2.0 also supports TextSlinger's `NGramLanguageModel` class, which leverages the KenLM package [@Heafield:2011].

The underlying models that drive the predictions are modular, and custom LM classes can be created or modified to suit users' needs. BciPy 2.0 introduces inference by large language models (LLMs) via the TextSlinger API, which allows the use of causal transformer models from Hugging Face using the search algorithm from @Gaines:2025. Wrapper classes such as `CausalLanguageModelAdapter` handle the initialization of TextSlinger models for seamless integration with BciPy. In addition to the causal LLM, BciPy 2.0 supports TextSlinger's `NGramLanguageModel` class, which leverages the KenLM package [@Heafield:2011].
<!-- The underlying models that drive the predictions are modular, and custom LM classes can be created or modified to suit users' needs. BciPy 2.0 introduces inference by large language models (LLMs) via the TextSlinger API, which allows the use of causal transformer models from Hugging Face using the search algorithm from @Gaines:2025. Wrapper classes such as `CausalLanguageModelAdapter` handle the initialization of TextSlinger models for seamless integration with BciPy. In addition to the causal LLM, BciPy 2.0 supports TextSlinger's `NGramLanguageModel` class, which leverages the KenLM package [@Heafield:2011].

BciPy 2.0 also provides a `UniformLanguageModel` class, which returns an equal probability distribution among all characters in the symbol set. This is useful when researchers want to remove language model influence as a control condition or independent variable, and it serves as an example of the methods required to create custom `LanguageModel` subclasses.
BciPy 2.0 also provides a `UniformLanguageModel` class, which returns an equal probability distribution among all characters in the symbol set. This is useful when researchers want to remove language model influence as a control condition or independent variable, and it serves as an example of the methods required to create custom `LanguageModel` subclasses. -->

# BciPy Simulator
## BciPy Simulator

A major advancement in BciPy 2.0 is the introduction of the simulator module. This module allows researchers to use previously recorded typing data to systematically evaluate how changes in parameters, signal models, and language models impact typing performance. The simulator supports tasks such as customizing experiment parameters, conducting large-scale comparisons of language model implementations, and testing multimodal evidence fusion strategies. Additionally, the simulator framework can be extended to train new EEG signal models.

As illustrated in \autoref{fig:simulator}, the simulator architecture consists of several key components:
<!-- As illustrated in \autoref{fig:simulator}, the simulator architecture consists of several key components:

- The `SimulatorTask` (e.g., `CopyPhrase`),
- A `TaskRunner` for managing multiple iterations,
- A `DataEngine` for loading and querying data samples,
- A `DataProcessor` for formatting data for classification, and
- A `Sampler` that selects samples from the `DataEngine` using user-defined strategies.
- A `Sampler` that selects samples from the `DataEngine` using user-defined strategies. -->

The simulator collects metrics for each run and summarizes across all runs to assess performance. The simulator provides both a graphical user interface for designing simulation parameters and input sources, as well as a command line interface for scripting and automation.
The simulator collects metrics for each run and summarizes across all runs to assess performance. It provides both a graphical user interface for designing simulation parameters and input sources as well as a command line interface for scripting and automation.

<!-- Add Figure4 from static/ -->
![**BciPy Simulator Architecture.** The BciPy Simulator consists of several components that work together to simulate a typing task using previously recorded data. The `SimulatorTask` defines the task to be performed, such as `CopyPhrase`. The `TaskRunner` manages the execution of multiple iterations of the task, collecting metrics for each run. The `DataEngine` loads and queries data samples from the provided dataset, while the `DataProcessor` prepares the data to match the input format required by the classification model. The `Sampler` draws samples from the `DataEngine` based on a user-selected sampling strategy, such as random sampling or sequential sampling. The `SignalModel` and `LanguageModel` are used to classify the sampled data and provide predictions, respectively. Finally, the collected metrics are summarized across all runs to evaluate performance.\label{fig:simulator}](static/Figure4.png)
<!-- ![**BciPy Simulator Architecture.** The BciPy Simulator consists of several components that work together to simulate a typing task using previously recorded data. The `SimulatorTask` defines the task to be performed, such as `CopyPhrase`. The `TaskRunner` manages the execution of multiple iterations of the task, collecting metrics for each run. The `DataEngine` loads and queries data samples from the provided dataset, while the `DataProcessor` prepares the data to match the input format required by the classification model. The `Sampler` draws samples from the `DataEngine` based on a user-selected sampling strategy, such as random sampling or sequential sampling. The `SignalModel` and `LanguageModel` are used to classify the sampled data and provide predictions, respectively. Finally, the collected metrics are summarized across all runs to evaluate performance.\label{fig:simulator}](static/Figure4.png) -->

# AI usage disclosure

Expand Down
Loading