Contributions:ScreenCaptureLogger: Difference between revisions

Latest revision as of 16:39, 20 May 2026

Synopsis

An extension that records video from one or more screen areas to mp4 files, and stores frame numbers as (event-type) state variables.

Location

http://www.bci2000.org/svn/trunk/src/contrib/Extensions/ScreenCaptureLogger
http://www.bci2000.org/svn/trunk/src/contrib/Extensions/ScreenCaptureLogger_Wayland

Versioning

Authors

Juergen Mellinger (mellinger@neurotechcenter.org)

Version History

May 6, 2026: initial version

Source Code Revisions

Initial development: 9343
Tested under: 9357
Known to compile under: 9357

Known Issues

On Linux, the Wayland version has never been working. It connects to the XDG screen capturing portal and displays a screen selector dialog but subsequently fails to establish a connection to the PipeWire daemon. Due to lack of error information and documentation on the PipeWire side, it is currently not possible to fix the issue with reasonable effort.

Functional Description

For many experiments, a documentation of screen content is desired. While the content of the user's task window is available in the ApplicationWindow visualization, this has its limitations, especially if the participant's screen is not displayed by a BCI2000 application module but a third party stimulus presentation kit such as PsychoPy or PsychToolBox.

This extension allows capturing of any number of desktop areas, and saving each into its own MP4 video file. It also records the current frame number as a state variable, so that the video can be synchronized with brain data at sample resolution (max. 1ms).

The video file bears the name of the BCI2000 data file, with the extension removed and _ScreenCaptureM.mp4 appended where M is the one-based index of the screen area in question. For the format of the video file, see the notes below.

Integration into BCI2000

Compile the extension into your source module by enabling contributed extensions in your CMake configuration. In your root build folder, run Configure.sh.cmd in order to bring up the CMake GUI; there, enable EXTENSIONS_SCREENCAPTURELOGGER for the Windows/macOS/Linux-X11 version of the module. In order to use the Linux-Wayland version disable EXTENSIONS_SCREENCAPTURELOGGER, and enable EXTENSIONS_SCREENCAPTURELOGGER_WAYLAND. You cannot enable both the default and the Wayland version.

Further requirements are:

Windows

No further requirements must be met.

macOS

On macOS, the extension requires at least macOS 15.2 (Sequoia) to compile and run.

Linux-X11

The X11 version of the extension requires the xcb (X11 C bindings) and the xcb-shm (XCB shared memory extension) libraries and headers. On Debian-based distributions, this resolved by installing the libxcb-dev and libxcd-shm0-dev packages. On other distributions, installing libxcb may be sufficient.

Linux-Wayland

To build the (currently broken) Wayland version of the extension, it its necessary to install the sdbus-c++ library (libsdbus-c++-dev on Debian-based systems, sdbus-cpp on Arch) as well as the PipeWire library (libpipewire-dev on Debian-based distributions, libpipewire on Arch).

Once the extension is built into the source module, enable it by starting the source module with the --LogScreenCapture=1 command line argument.

Parameters

The ScreenCapture logger is configured in the Source tab within the ScreenCaptureLogger section. The configurable parameters are:

LogScreenCapture

Enables/Disables the logger extension.

DisplayCaptureStream

Display the video stream from each captured screen area. Display stream is identical to what is being recorded in the .mp4 file.

ScreenCaptureAreas

A matrix parameter with each row representing an independently capture area (capture areas may overlap). In each row, you may specify a video width, height, spatial decimation, fps, encoder profile, and a compression factor. The encoding profile column allows you to choose an H.264 encoding profile for video output, which may be a number from 1 to 3. 1 corresponds to the "baseline" profile, 2 corresponds to the "main" profile, and 3 corresponds to the "high" profile as defined in the ITU-T H.264 standard. If the encoder uses ffmpeg’s x264 codec (on non-Windows systems), the additional option tune=animation is applied to optimize encoding for screen content. On Windows, the mf_h264 codec is used which does not provide such an option.

The compression factor determines the amount of data compression applied when encoding the stream of images into an .mp4 file. The smallest possible compression setting is 1 (no compression). Use a setting of 100 to compress the data stream to 1/100 of its original size.

Event Variables

ScreenCaptureFrame<1-n>

The frame number of the screen capture area with row i. The default value of n is 4, allowing for capturing of up to 4 areas. To record more areas, change the NUM_OF_SCREEN_CAPTURE_EVENTS macro in ScreenCaptureLogger.cpp to the number of areas you want to record, and recompile. Event values can be used to synchronize the video with the recorded neural data. It is a 32-bit value, allowing 4294967295 frames before overflowing. If screen capturing is done at 30 frames/sec, this allows more than four years of video before it will go back to frame 0. In other words, this should not be an issue for most users.

Format of the Video File

Each capture area writes its own video file. Files in mp4 format (and related formats) do not allow to seek to locations that represent original frames when writing the file. Rather, the file's nominal frame rate and length are used in conjunction with interpolation to construct a frame at an ideal location. Thus, it is not possible to write individual frames using the screen capture’s original timestamp, at the original frame rate.

Instead, the file is written with the nominal framerate specified in the fps column of the ScreenCaptureAreas parameter, with differences between frame timestamps matching ideal frame duration. This way, loading the video into e.g. Matlab, you will obtain a sequence of frames where each frame index corresponds to the frame index in the ScreenCaptureFrameN state (minus 1).

For a strategy how to synchronize video file frames with brain signal data, and possibly audio data, see this page.

@@ Line 55: / Line 55: @@
 ===ScreenCaptureAreas===
 A matrix parameter with each row representing an independently capture area (capture areas may overlap).
-In each row, you may specify a video width, height, spatial decimation, fps, and encoder profile.
+In each row, you may specify a video width, height, spatial decimation, fps, encoder profile, and a compression factor.
 The ''encoding profile'' column allows you to choose an H.264 encoding profile for video output, which may be a number from 1 to 3. '''1''' corresponds to the "baseline" profile, '''2''' corresponds to the "main" profile, and '''3''' corresponds to the "high" profile as defined in the ''ITU-T H.264'' standard.
 If the encoder uses ffmpeg’s ''x264'' codec (on non-Windows systems), the additional option ''tune=animation'' is applied to optimize encoding for screen content. On Windows, the ''mf_h264'' codec is used which does not provide such an option.
+The ''compression factor'' determines the amount of data compression applied when encoding the stream of images into an .mp4 file. The smallest possible compression setting is 1 (no compression). Use a setting of 100 to compress the data stream to 1/100 of its original size.
 ==Event Variables==