Bessy Unity Documentation

This is looking to capture some of the key documentation pieces needed in understanding and getting ready to use BCI Essentials Unity. This will try to provide some documentation and tutorial support from a “top-down” approach, leaving the actual documentation generation for the “bottom-up” approach to something like https://uvasgd.github.io/sgd-docs/unity/documentation.html .

Drawing goes here:

Concepts & Definitions

BCI Paradigm - The specific pattern or way a user controls the BCI. These are generally “named” after the primary way in which the user interacts with the system.

Visual Paradigms - Use visual stimuli for the primary control of the BCI, which is synchronized to the data recorded from the brain. These are also sometimes called exogenous paradigms, as it relies on an outside stimulus for control.

P300 Paradigm - Relies on the positive 300ms (P300) event-related potential (ERP), which is an evoked neural response which occurs when an unexpected (oddball) stimulus shows up (e.g. the object flashes at a pseudorandomly). The P300 Paradigm can also be detected using other methods, including audio or physical stimulation. Fundamentally, any number of objects* on screen can be selected using this paradigm, as it is trying to categorize a “target” object (i.e. the one focused on) vs. all the other “non-target” objects (i.e. everything else). This paradigm scales well with increasing the number of objects on screen because it is binary classification scheme (target vs. non-target), but has a trade off in having an ever increasing time required for selection (flashing each of the objects on screen). Here is a simple video explaining the P300 in high-level terms.

*There must be at least 3 objects on the screen for this to work in typical visual systems. This is due to the need for a pseudorandom flashing of the objects.

Steady-State Visual Evoked Potential (SSVEP) Paradigm - Relies on the brain synchronizing to a light or object flashing at a specific consistent frequency (hence the “steady-state” name). Multiple objects can be differentiated for selection on screen by providing discrete steady-state frequencies for each object. This paradigm does not scale well with more targets on screen as each new target adds another classification required (i.e. it is a multi-class problem at nature). However it requires only a little time for selection as it can flash all objects at once, meaning it can be faster for selecting.

Transient Visual Evoked Potential (tVEP) Paradigm - Closely linked to the SSVEP, except that it flashes the objects at a very low (< 3 Hz) frequency. As such it behaves somewhere between the SSVEP and the P300 paradigms, where peaks are looked at from the slower cortical responses, more similar to the event-related potential of the P300. The use of tVEPs is under construction for BCI-Essentials Unity.

Auditory Paradigms - Uses auditory stimuli for primary control of the BCI. These are analogous to their visual counter-parts, with auditory outside stimuli being used for control. There are currently no Auditory BCI paradigms supported in BCI Essentials Unity.

Motor Imagery (MI) Paradigms - Uses imagined movements of the body to control the BCI system. This is part of a wider category of “endogenous” paradigms, because this doesn’t rely on any outside stimulus and instead require “internal” stimulus to run. In this case, it is the imagination of moving a body part which can be picked up. There are only a finite number of internal imagined actions that we can distinguish, and with each one there is an exponentially increasing burden both on the user and the classification scheme. This does not scale well with selection options on screen, as typically only 1-4 (maximum) actions can be detected.

Stimulus Presenting Object (SPO) - This is the core “building block” of what you interact with in BCI-Essentials Unity. The SPO is fundamentally the “object” you may want to select. You can have multiple SPOs on screen at once, as you may want the user to select from any number of them. The pool of SPO objects consists of all objects that are interactable by the BCI system, which may be limited based on your chosen paradigm.

Example: Imagine you are making an action game where there multiple enemies on screen. Each enemy could be its own SPO for selecting to do an attack of some sort. You may want to turn on/off active SPOs through the “Selectable” property and update the pool of SPOs when changing game states/actions.

Examples

Designing with Visual Paradigms

TODO

Designing with Motor Imagery Paradigm

TODO

Scripts - High Level Purpose

BCI Control Scripts

BCI Controller

BCI Behavior Type

BCI Training Type

SPO Population Method

IBCI Behavior

BCIControllerBehavior

MIControllerBehavior

P300ControllerBehavior

SSVEPControllerBehavior

Deprecated/Unused

Switch Controller

Controller

MIController

P300Controller

SSVEPController

Networking

LSL Marker Receiver Settings

LSL Marker Response

ILSL Service

ILSL Marker Subscriber

ILSL Marker Receviever

Stimulus/Selection Related Scripts

SPO

Color Flash Effect

Setup/Misc.

Matrix Setup