Contributions:AVMeetingExtension

Synopsis

This extension displays video from a remote location, plus local video in a user application window. It works in conjunction with the SimpleRelayServer application running within a local network, or on the internet. Audio may be transferred in one direction as well. Bidirectional audio is possible but will lead to feedback unless earphones are used.

Authors

Jürgen Mellinger (mellinger@neurotechcenter.org)

SimpleRelayServer

To allow connection between machines which are located at different locations, and possibly in different local networks, it is necessary to use a bridge that is accessible to both machines. AVMeetingExtension is designed to use a relay server for this bridge. In order to bridge different local networks, the relay server must be located somewhere in the open internet, on a Linux server under the experimenter's control.

Follow these steps to build and run the necessary SimpleRelayServer executable on a Linux machine:

• Download SimpleRelayServer.cpp from

https://bci2000.org/svn/trunk/src/contrib/Extensions/AVMeetingExtension/SimpleRelayServer/SimpleRelayServer.cpp

• cd to the download's directory and execute

g++ SimpleRelayServer.cpp -o SimpleRelayServer

• Start the server with

./SimpleRelayServer 2>/dev/null

or just

./SimpleRelayServer 

if you want to see log messages on the console. By default, the server process will listen on all of the machine's interfaces, at port 1915. A different port may be specified by providing a command line argument:

./SimpleRelayServer 1928 2>/dev/null

NOTE: For testing purposes, BCI2000 provides a SimpleRelayServer instance at upload.bci2000.org:1915.

Example Batch File

The following example batch file starts up BCI2000 with the SignalGenerator source module, the DummySignalProcessing module, and the GuiltTask with the AVMeetingExtension enabled.

 #! ../prog/BCI2000Shell
@cls & ..\prog\BCI2000Shell %0 %* #! && exit /b 0 || exit /b 1\n

Change directory $BCI2000LAUNCHDIR
Show window; Set title ${Extract file base $0}
Reset system
Startup system localhost
Start executable SignalGenerator --local --LogKeyboard=1
Start executable DummySignalProcessing --local 
Start executable GuiltTask --local --EnableAVMeeting=1 --Port=1234 --IPAddress=3.216.172.59 --SharedStates=isReadyStart0,8&isReadyStart1,8&ClientNumber,8&CompensationTrans,32&CompensationVictim,32
Wait for Connected

Load parameterfile "../parms/GuiltTask/guiltTaskLocal.prm"

Parameters

EnableAVMeeting

If nonzero, enables the AVMeeting extension.

StreamingServer

The streaming server's address with port, as in upload.bci2000.org:1915. This must match the SimpleRelayServer's listening address. For testing purposes, an instance of the streaming server is running at the BCI2000 server's address given above.

CameraIndex

The local camera's 0-based index into the video backend's list of detected cameras. Typically 0 unless multiple cameras are attached. A value of -1 disables video streaming.

AudioDeviceIndex

The local audio input device's 0-based index into the audio backend's list of detected audio devices. Specify -1 to disable audio streaming from the local machine.

VideoEncoder

The name of the desired video encoder to use, e.g. h264_mf or libx264. Specify auto to use h264_mf on Windows, and libx264 on non-Windows systems.

RemoteVideoRect

The display rectangle for remote video, in relative coordinates. Values are left, top, bottom, right. Defaults to 0 0 1 1, i.e. the entire window.

RemoteVideoZ

The remote video display's Z coordinate. Objects with a smaller Z coordinate are displayed on top objects with larger Z coordinates.

RemoteVideoVisibility

A BCI2000 expression which may entail BCI2000 state names. Whenever this expression evaluates to a nonzero value, remote video is displayed. Otherwise, remote video is hidden.

LocalVideoRect

The display rectangle for local video, in relative coordinates. Values are left, top, bottom, right. Defaults to 0.75 0 1 0.25, i.e. the upper right corner of the display window.

LocalVideoZ

The local video display's Z coordinate. Objects with a smaller Z coordinate are displayed on top objects with larger Z coordinates.

Note that by default the local video rectangle overlaps with the remote video rectangle, and local video is on top of the remote.

LocalVideoVisibility

A BCI2000 expression which may entail BCI2000 state names. Whenever this expression evaluates to a nonzero value, local video is displayed. Otherwise, local video is hidden.

States

VideoStreamingLocalTotalFrames

A 32-bit state holding the current local frame number since the beginning of the current recording (BCI2000 run).

VideoStreamingRemoteTotalFrames

A 32-bit state holding the current remote frame number since the beginning of the current recording (BCI2000 run).

VideoStreamingLocalFrame

A 32-bit state holding the current local frame number since the beginning of the current file.

VideoStreamingRemoteFrame

A 32-bit state holding the current remote frame number since the beginning of the current file.

Files

Both local and remote video is being recorded into files in mp4 format. These files bear the current BCI2000 file's basename, followed with _VideoStreamingLocal and _VideoStreamingRemote, respectively.

NOTE: Seeking to individual frames is not possible for mp4 files unless frame time stamps exactly match the frame rate specified at the beginning of the file. To work around this, video files are created with the camera's nominal frame rate, and artificial time stamps proportional to frame number are used when writing frames to a file. In analysis, it will be possible to reconstruct actual time stamps using one of the frame count states.

However, replaying mp4 files written by the AVMeetingExtension with a standard video player may not result in proper playback speed.

See also

Contributions:Extensions