User Reference:BCI2000Analysis

Introduction

The BCI2000Analysis program is a versatile tool for

Applying Signal Processing filters to existing data,
Testing and Benchmarking new Signal Processing filters,
Training and testing Linear Classifier weights using various methods.

The program may be viewed as a more versatile version of the BCI2000 Offline Analysis tool, and a complement of the P300Classifier tool. Unlike BCI2000 Offline Analysis, it does not require Matlab to run. Unlike the P300Classifier tool, it allows to use classifier training methods other than SWLDA, and it allows to visualize time courses and topographic plots of P300 responses. For data recorded with the P300 speller online component, it will also translate classifications into spelling results.

The BCI2000Analysis main window

The main window consists of

areas for training and test data,
a pane containing the filter graph,
a log area that holds log information produced by filters (if any).
In the lower left, two condition fields exist to hold the conditions for collecting data into either of two sets that may then be used to train a classifier, or to view data as seen by the application node.
In the lower right, a selection area allows to select the "target" of analysis, i.e. the filter graph node at which to observe and visualize data. By default, this is set to "Linear Classifier", and the Linear Classifier node in the filter graph is correspondingly highlighted by a black frame.
At the bottom, there is an "Analysis Log" area that holds log messages originating from the BCI2000Analysis program itself, such as information about newly loaded data files, modifications to the filter graph, or application of filter parameters from trained classifiers.

Clicking the “Add analysis files” or the “+” button below the analysis files field will open a file dialog that allows you to choose BCI2000 data files for analysis. Similarly, “Add testing files” or the “+” button below the testing files field will allow you to add BCI2000 data files for testing after a classifier has been trained. Analysis and testing files are checked for consistency when added.

By default, the filter graph is automatically populated when one or more training data files are added to the list, and so are the condition fields. If necessary, filter graph and conditions may be edited to fit special needs (see below).

Above the condition fields, a button will show the standard BCI2000 parameter configuration dialog when clicked. By default, parameter configuration is taken from the first data file in the “training data files” list, and from parameter defaults for any parameters that are not present in that data file.

Editing the Filter Graph

By default, a suitable filter graph is automatically chosen when specifying a data file. Still, you may edit such a default graph, choose a different one from the Analysis→FilterGraph menu, or build one from scratch by selecting Analysis→FilterGraph→User defined.

As it suits the BCI2000 model, a filter graph will always begin with a single Source node, and end in a single Application node. In between, filtering may be split into multiple branches but must finally be joined into a single node. By the way filter graph editing works, it is not possible to produce invalid graphs. Whenever you insert a splitter node, a corresponding joint is automatically inserted for you. You may add and remove filters into and from the new branches but when you remove a splitter, all its branches and its joint node are removed.

Internally, a splitter and its corresponding joint node are implemented by a ParallelCombination object. A splitter's two branches are each implemented as a SubchainFilter.

You insert a filter between two nodes by right-clicking the connection between them, and choosing “Filter…” from the popup menu. A dialog window appears that contains a menu with all pre-existing types of filters, so you may select e.g. “SpatialFilter”, “P3TemporalFilter”, and so on. As a special entry, the list contains “Filter from DLL…” which allows you to choose a BCI2000 filter DLL. This is especially useful when developing a filter that is not part of the standard set of BCI2000 filters (see here for a description how to build your own filter as a BCI2000 filter DLL).

A special role plays the LinearClassifier filter, which is displayed in red. At its location, data is collected according to the conditions specified in the “Analysis conditions” field. When performing classifier training (see below), this data is then used to train the classifier, and training results are applied to test the classifier.

Similarly, there is a "CSPSpatialFilter" which is also displayed in red. Normally, it acts like a SpatialFilter, but when selected as an analysis target, it will observe its input data according to the conditions specified in the "Analysis conditions" field. It will then be able to do a "Common Spatial Pattern" analysis on this data, resulting in a SpatialFilter configuration that provides an optimal signal-to-noise ratio between the two conditions.

From the “File” menu, filter graphs may be saved to, and loaded from, .bcigraph files, which are human readable and may be edited using a text editor. Also, filter graphs may written into C++ PipeDefinition.cpp files suitable for use as a signal processing module’s pipe definition file.

Performing Analysis

You perform an analysis by pressing “Run analysis…” in the button bar, or by selecting “Run analysis…” from the “Analysis” menu. During analysis, a progress bar is displayed. Once done, a new window opens up, showing an r^2 plot. Depending on the input of the LinearClassifier filter, the x axis displays either time in milliseconds, or frequency in Hertz, whereas channels are numbered on the y axis. This is the same plot as displayed by the Offline Analysis tool, and shows the squared correlation between features and conditions at the input of the LinearClassifier filter.

In the r^2 plot, you may view an individual value by clicking its location, which will make a tooltip appear. Right-clicking a location in the plot will turn up a context menu offering to view a cross-section across channels (topographic plot), or a cross-section across frequencies/time points (curve plot). You may also manually enter the frequency/time point, or channel, in question by selecting the appropriate entry from the “View” menu.

Using the Profiler

If you choose “Profiler” as the analysis target, the behavior of “Run Analysis” will change. It will still run the specified data through the pipeline, but it will not produce an r^2 graph as a result. Rather, it will open a window that shows how much time each filter has been using, in terms of a sample block size. If any of these numbers approaches 100%, this will be problematic for real-time operation, and the respective lines will be shown in red.

Training a Classifier

The analysis window provides controls that allow you to train a linear classifier using four distinct methods: Linear Discriminant Analysis (LDA), Step-Wise Linear Discriminant Analysis (SWLDA), Support Vector Machine (SVM), and "Manual", i.e. choosing classifier weights by hand.

For all training algorithms, a downsampling factor may be chosen. For P300 data sampled at 250Hz, this would typically be 10; for ERD data, no downsampling would be appropriate (downsampling factor of 1). Note that downsampling will typically improve classifier performance because it reduces the dimensionality of the classifier, and results in a better ratio of sampled data to degrees of freedom in the classifier.

For the SWLDA training algorithm, three additional parameters are available. p_enter is the p-value for which a feature will be included in classification, p_remove is the p-value for which a feature will be removed from classification. max_iterations limits the number of iterations to perform. Typically, default values are fine.

Once a classifier has been trained, a window appears which shows classifier information on the left side, and a classifier plot on the right side. The classifier plot is similar to the r^2 plot but shows absolute values of classifier weights to give an impression which features are weighted strongly by the classifier.

On the left, the classifier’s offset and variance are displayed, plus its performance on training and test data. Note that performance on training data is typically much better than on test data, which is due to overfitting of the classifier. Only the performance on test data should be taken as an indication how well the classifier will perform on unseen data. Performance is shown in % of correct classifications, for both the “normalized” and “raw” case. The difference is that the “normalized” case takes the bias due to unequal number of data samples in the two data sets into account. If the number of data samples in the two data sets agree, “normalized” and “raw” performance will agree as well.

Saving a Classifier as a Parameter File

From the classifier window, choose File→Save Classifier Parameters… in order to save a parameter file with only LinearClassifier related parameters. You can then load this parameter file into BCI2000 using the Operator’s Config dialog, or the Load Parameterfile scripting command. Similarly, you may save all parameters from the current analysis chain, including LinearClassifier, by choosing File→Save All Filter Parameters…

Applying a Classifier to the Filter Chain

If you want to apply the LinearClassifier to the chain without saving it to a file, choose File→Apply Classifier Parameters.

Training a Spatial Filter

If there is a "CSPSpatialFilter" node present in the filter graph (a red "SpatialFilter" node), it may be selected as the analysis target. Then, clicking "Run analysis..." will result in a window like this:

At the top, CSP components are displayed in form of topographic plots. Below the topographic plot, there is a slider that allows to move through components; this works as well using shift-left arrow and shift-right arrow. Using the "Display pattern" and "Display field" radio buttons, a component may be viewed as a "pattern" (weights for component extraction) or "field" (weights for synthesizing the component). The "Retain" checkbox allows to select which components are supposed to be retained or removed from the data after applying the spatial filter. Finally, there is a graph showing the power log ratio of all components. Positive values correspond to greater amplitudes for condition 1, and negative values to greater amplitudes for condition 2. For the current component, power log ratio is also displayed at the top of the window, in both natural log and binary log units (bits). As can be seen from the graph, components are sorted according to power log ratio, such that the first components carry most information about the given conditions.

Saving a Spatial Filter as a Parameter File

From the CSP window, choose File→Save Spatial Filter Parameters… in order to save a parameter file with only CSP spatial filter parameters. You can then load this parameter file into BCI2000 using the Operator’s Config dialog, or the Load Parameterfile scripting command. Similarly, you may save all parameters from the current analysis chain, including CSP spatial filter, by choosing File→Save All Filter Parameters…

Applying a Spatial Filter to the Filter Chain

If you want to apply the CSP Spatial Filter to the chain without saving it to a file, choose File→Apply Spatial Filter Parameters.

Viewing data as seen by the Application

Finally, after training and applying classifier and/or spatial filter, you may be interested in the distribution of data as seen by the Application module. To this end, select the Application node as a target (its actual name depends on the kind of data analyzed).

SMR/ERD Data

For SMR/ERD data, the result may look like this:

For training data, the classifier is able to perform a perfect separation.

For test data, only partial separation is obtained.

By integrating over trials (as the actual SMR application does), good separation is still obtained for both training and test data.

P300 Spelling Data

For data recorded with the P300Speller online component, the result is not only displayed as a distribution but also evaluated in terms of spelling accuracy:

Debugging a filter DLL

To debug a filter contained in a DLL, you have two options:

Setting breakpoints in the IDE

Start up BCI2000Analysis, and attach a debugger to the process. In the debugger IDE, navigate to your filter's cpp file, and set breakpoints as you see fit. Then, activate the DLL by adding a filter DLL node and choosing your filter, and run the filter chain by clicking "Run analysis". Execution should stop when any of the breakpoints is hit.

Using a `SuggestDebugging` Statement

In your filter's cpp file, include Debugging.h, and add a line

SuggestDebugging << "Foo";

anywhere in your filter's functions. When the filter DLL is activated and the SuggestDebugging statement is executed, it will display a message box. You may then attach a debugger to the BCI2000Analysis process, and click "yes" in the message box to drop into the debugger.

NOTE: For these debugging instructions to work, it is necessary that both BCI2000Analysis and the filter DLL have been compiled in Debug mode.