User Tutorial:UnityBCI2000 Barebones
Red text blocks contain detailed instructions for the higher-level instructions preceding them.
This tutorial goes over how to use BCI2000 as a logging service for an existing Unity project. For a more comprehensive tutorial on integrating Unity with BCI2000, see User Tutorial:UnityBCI2000. This tutorial will use a sample Unity project, the same as the full tutorial.
Download Unity
Download Unity Hub from Unity.
Setting Up BCI2000
Follow the instructions starting with [1] to download BCI2000.
Setting Up UnityBCI2000
Download UnityBCI2000 from its GitHub page, [2].
Click on the "Releases" section on the right side of the page. This tutorial is intended to be used with UnityBCI2000 version 2.0.0, so scroll to that release and click on the files BCI2000RemoteNETStandard.dll and UnityBCI2000.cs to download them.
Download Tutorial Project
Download the CursorTask3 repository from GitHub.
Either: -Clone the git repository -Download the respository directly from GitHub -> Click the green "Code" button -> Select "Download ZIP" -> Extract the zip file
The download directory contains two versions of the CursorTask3 project. One, CursorTask3-BCI2K, contains an example implementation of the task with BCI2000 integration. The other, CursorTask3-NoBCI2K, contains the same task without BCI2000 integration. This tutorial is a step by step guide on turning that second project into the first.
Opening the project
Open the project in the Unity Editor, and open the CursorTask scene.
Open Unity Hub. Click the "Add" button in the top right corner. Navigate to the CursorTask3 directory downloaded previously. Select the CursorTask3-NoBCI2K directory to add it to Unity Hub. Click the CursorTask3-NoBCI2K project in the Unity Hub. If prompted to select a version of Unity to use or install, select Unity 2022.3. The specific release number of Unity 2022.3 should not matter, so select the one labeled "LTS" for consistency. The lower panel (the asset browser) should already show the Assets/Scenes directory, which contains a single scene called SampleScene. Double-click SampleScene. This should open a scene with a box containing various lights and objects.
If you wish to view the completed implementation, repeat these steps, but instead open the CursorTask3-BCI2K-Barebones directory. Note that in order to use the completed implementation, you will still need to set the Operator Path value of the UnityBCI2000 component of the BCI2000 game object.
Adding UnityBCI2000 to the project
Add BCI2000RemoteNETStandard.dll and UnityBCI2000.cs to your project's assets.
In the Unity Editor's menu bar, click Assets > Open Containing Folder to open your project directory. Place the BCI2000RemoteNETStandard.dll and UnityBCI2000.cs files in the Assets folder.
Add an empty GameObject called 'BCI2000'. This will hold the scripts for controlling BCI2000.
In the Unity Editor's menu bar, click GameObject > Create Empty to create an empty GameObject, and name it 'BCI2000'. (The object's specific name does not actually matter, it is only needed in order to reference it within other scripts) You will notice that it now appears in the Scene Hierarchy panel, on the left side of the screen.
Add the UnityBCI2000 component to the object.
Click on the object in the Hierarchy panel. This will open it in the Inspector panel (the panel on the right side of the screen) In the inspector, below the new object's Transform component, click the "Add Component" button, which will open up a small window for adding components. Within this window, select Scripts > Unity BCI2000 to add the UnityBCI2000 component.
Configuring UnityBCI2000
The inspector panel now contains the configuration options for UnityBCI2000. Set the options as follows:
Start Local Operator: Checked
This will start the operator on your computer when the Unity scene initializes. If we were instead connecting to an already-running instance of BCI2000, or an instance on another computer, this box would be deselected.
Operator Path: The path to the Operator executable. This will look something like this on Windows (C://path/to/bci2000/prog/Operator.exe)
Operator Address: 127.0.0.1
This is the address of the machine on which BCI2000 is running. Since we are running BCI2000 on the same computer as Unity, we leave it as 127.0.0.1, the loopback address.
Operator Port: 3999
This is the port on which BCI2000 is listening for commands. By default, it is 3999.
Start Modules: Checked
This tells BCI2000 to start the requested Signal Source, Signal Processing, and Application modules when the Unity scene initializes. Similarly to Start Local Operator, we would deselect this box if connecting to an already-running instance.
Start With Scene: Checked
This tells BCI2000 to start a data collection run when the scene starts. Since we will be using BCI2000 itself to set experiment parameters, we will instead wait for BCI2000 to start from Unity, and thus will leave the box unchecked.
Stop With Scene: Checked
This tells BCI2000 to stop collecting data when the scene stops. Since we will be controlling BCI2000 directly, rather than entirely through Unity, we will leave this unchecked.
Shutdown With Scene: Unchecked
This tells BCI2000 to shut down alongside the Unity scene. Whether or not this value is set ultimately doesn't matter much, especially if Start Local Operator is checked. The data will be saved whether or not BCI2000 shuts down.
Module 1: SignalGenerator
The signal source module to start. We will use SignalGenerator, which generates a signal without any connected hardware.
Module 2: DummySignalProcessing
The signal processing module to start. We will use DummySignalProcessing, as there is no processing to do.
Module 3: DummyApplication
The Application module to start. Since we are using Unity, we will use DummyApplication.
Setting References
In order for the game's scripts to communicate with BCI2000, they need to hold a reference to the UnityBCI2000 component.
There are three scripts which will need to communicate with BCI2000. They are GameControl.cs, BallControl.cs, and MCursorControl.cs.
These scripts are each located within the Assets directory of the Unity project.
As before, select Assets > Open Containing Folder to open the project directory, then open the Assets folder.
For each script, open it in a text editor.
Add a data member of type UnityBCI2000 to the class, like so:
UnityBCI2000 bci;
Place the member definition above the Awake() method, for readability.
Within the Awake() method, set this reference to the UnityBCI2000 component.
bci = GameObject.Find("BCI2000").GetComponent<UnityBCI2000>();
Each of the three scripts should contain a section like this:
...
UnityBCI2000 bci;
void Awake() {
bci = GameObject.Find("BCI2000").GetComponent<UnityBCI2000>();
...
Adding Events
Events are the primary way that non-signal experiment data is recorded in BCI2000. They are timestamped integer values which are encoded alongside the signal data in BCI2000 output files. Due to BCI2000's design, events must be added during a very specific part of its startup sequence, which is immediately after the BCI2000 operator starts, and before any of the modules start. As such, we cannot simply call AddEvent() whenever we want. Furthermore, the order in which Unity objects initialize is undefined, so it cannot even be guaranteed that calling AddEvent at a specific time will be consistent across multiple projects. As such, UnityBCI2000 provides a couple of methods for sending commands at well-defined points within the startup sequence. These two methods, OnIdle and OnIdle and OnConnected allow BCI2000 commands to be sent while the operator is in the state Idle (immediately before starting its modules) and when the operator is in the state Connected (after starting and connecting to the modules. Below is an example of using those methods to add and show an event in BCI2000.
class Script : MonoBehaviour {
UnityBCI2000 bci;
void Awake() {
bci = GameObject.Find("BCI2000Object").GetComponent<UnityBCI2000>();
bci.OnIdle(remote => {
remote.AddEvent("AnEvent", 32);
});
}
void Start() {
bci.OnConnected(remote => {
remote.Visualize("AnEvent");
}
}
}
As seen above, OnIdle and OnConnected take a delegate (C#'s term for a callback/closure/functor/etc.) with a single parameter of type BCI2000Remote (in this case called "remote"). The lambda expression given to the call to OnIdle uses the method BCI2000Remote.AddEvent() to add an event called "AnEvent" with a bit width of 32 bits. The call to OnConnected tells BCI2000 to show the event's value in a graphical window. A description of the BCI2000Remote class can be found here, and API documentation can be found here.
We will now add the events relevant to the Cursor Task. Open the GameControl.cs script, and modify its Awake() to add the events PreFeedback, Feedback, PostFeedback, TargetHit, and Timeout with bit width 1. Additionally, add the events TrialNumber, TargetPositionX, and TargetPositionY with width 16. These will encode the task state and number of trials.
We also need to set a parameter value so that the signal will be set to the mouse position. Using BCI2000Remote.SetParameter(), we will set the ModulateAmplitude parameter to 1.
Open the BallControl.cs script, and within the Awake() function, add events CursorPositionX and CursorPositionY, with width 16. Show these events in a visualization window with Visualize
Your two scripts should now look like this:
>>> GameControl.cs
...
void Awake() {
...
bci.OnIdle(remote => {
remote.AddEvent("PreFeedback", 1);
remote.AddEvent("Feedback", 1);
remote.AddEvent("PostFeedback", 1);
remote.AddEvent("TargetHit", 1);
remote.AddEvent("Timeout", 1);
remote.AddEvent("TrialNumber", 16);
remote.AddEvent("TargetPositionX", 16);
remote.AddEvent("TargetPositionY", 16);
remote.SetParameter("ModulateAmplitude", "1");
});
}
>>> BallControl.cs
...
void Awake() {
bci.OnIdle(remote => {
remote.AddEvent("CursorPositionX", 16);
remote.AddEvent("CursorPositionY", 16);
});
bci.OnConnected(remote => {
remote.Visualize("CursorPositionX");
remote.Visualize("CursorPositionY");
}
Reading the control signal
We will be using the control signal to control the cursor. Open the MCursorControl.cs script.
The commented out section of the GetPos() method contains the code required to turn the control signal waveform coming from the Signal Source and Signal Processing modules into screen coordinates.
Uncomment the commented part and delete the line Ray r = camera.ScreenPointToRay(Input.MousePosition);.
Change the double signalX = 0; and double signalY = 0; to.
Your GetPos() method should look like this:
Vector3 GetPos() {
double signalX = bci.Control.GetSignal(1, bci.CurrentSampleOffset());
double signalY = bci.Control.GetSignal(2, bci.CurrentSampleOffset());
signalsX[signalIndex] = signalX;
signalsY[signalIndex] = signalY;
signalIndex = signalIndex + 1 >= rollingMaximumAmount ? 0 : signalIndex + 1;
double max_x = signalsX.Max();
double max_y = signalsY.Max();
Ray r = camera.ScreenPointToRay(new Vector3((float) max_x * Screen.width, (float) max_y * Screen.height, 0));
float p;
if (!plane.Raycast(r, out p)) {
throw new Exception("error casting ray to plane, invalid mouse position?");
}
return r.GetPoint(p);
}
Note the use of the UnityBCI2000.CurrentSampleOffset() method. This is a special method which gets the offset into the current block such that the sample is exactly one block length later than when it was collected by the hardware. This is to normalize the latency between when the hardware collects the signal and the software receives the signal, due to how BCI2000 processes data in blocks.
Sending events back to BCI2000
The primary way to communicate game state back to BCI2000 is via the use of Events, which are integer values encoded alongside the signal data.
We will send the events that we added in a previous section.
First, we will send back the current position of the cursor. Within the BallControl.cs script, within the Update() method, immediately after Move() is called, set the CursorPositionX and CursorPositionY events.
>>> BallControl.cs
void Update() {
...
if (isTrialRunning) {
...
Move();
bci.Control.SetEvent("CursorPositionX", (uint) ((transform.position.x + 7) * 1000));
bci.Control.SetEvent("CursorPositionY", (uint) ((transform.position.y + 4.5) * 1000));
}
...
Notice that we transform the value of the cursor's position. This is because events in BCI2000 are represented as unsigned integers, so, for example, the cursor's range of movement in the x axis, -7 to 7, would not be directly representable within a BCI2000 event, so we add 7 so it is positive, and multiply by 1000 so that we have a more precise measure of the cursor's position. The range of [-7,7] becomes [0,14000].
We will also set the events corresponding to the game state within GameControl.cs. First we will set PreFeedback, Feedback, and PostFeedback to represent when the game is in these states. To do this, at the beginning and end of PreTrial(), Trial(), and PostTrial(), we will set the corresponding event values to 1 and 0, respectively.
>>> GameControl.cs
IEnumerator PreTrial() {
bci.Control.SetEvent("PreFeedback", 1);
...
bci.Control.SetEvent("PreFeedback", 0);
}
IEnumerator Trial() {
bci.Control.SetEvent("Feedback", 1);
...
bci.Control.SetEvent("Feedback", 0);
}
IEnumerator PostTrial() {
bci.Control.SetEvent("PostFeedback", 1);
...
bci.Control.SetEvent("PostFeedback", 0);
}
We will also set the events which happen at the end of each trial. The TargetHit event is activated when the subject hits the target, and the Timeout event is activated when the subject runs out of time.
>>> GameControl.cs
IEnumerator Trial() {
...
bci.Control.SetEvent("Feedback", 0);
if (lastTrialSucceeded) {
bci.Control.PulseEvent("TargetHit", 1);
} else {
bci.Control.PulseEvent("Timeout", 1);
}
}
Notice that we use <code>BCI2000Remote.PulseEvent()</code> rather than <code>BCI2000Remote.SetEvent()</code>. This results in the event being set to the value <code>1</code> for exactly one sample duration, then returning to zero.
We will also record the position of the target. Similarly to the cursor, we will transform the target's coordinates to be a positive integer.
<pre>
>>> GameControl.cs
IEnumerator PreTrial() {
...
target.SetActive(true);
bci.Control.SetEvent("TargetPositionX", (uint) ((target.transform.position.x + 7) * 1000));
bci.Control.SetEvent("TargetPositionY", (uint) ((target.transform.position.y + 4.5) * 1000));
...
}
The last event we need to set is the trial number.
IEnumerator ControlLoop() {
...
while (IsContinue() && trials < n_trials) {
bci.Control.SetEvent("TrialNumber", trials + 1);
...
}
}
Unity Player Settings
Additionally, due to how Unity detects changes in BCI2000 state, it must be allowed to run in the background.
In the Unity menu bar:
Edit > Project Settings > Player > Resolution and Presentation
Check the "Run In Background" box.
Usage
Now, when the Unity application runs, BCI2000 will open and start collecting data.