Programming Reference:StimulusTask Class - Revision history

Mellinger: /* Synopsis */

2013-03-25T19:56:40Z

Synopsis

← Older revision		Revision as of 19:56, 25 March 2013
Line 12:		Line 12:

	In the BCI2000 core distribution, both the [[User Reference:P3SpellerTask\|P3SpellerTask]] and the [[User Reference:StimulusPresentationTask\|StimulusPresentationTask]] are based on the <tt>StimulusTask</tt> base class, and implement their own specialized behavior on top of it.		In the BCI2000 core distribution, both the [[User Reference:P3SpellerTask\|P3SpellerTask]] and the [[User Reference:StimulusPresentationTask\|StimulusPresentationTask]] are based on the <tt>StimulusTask</tt> base class, and implement their own specialized behavior on top of it.

			==Introduction==
			The <tt>StimulusTask</tt> class is a base class for applications that present stimuli to the user, and optionally perform selection of a selection target based on a real-time evaluation of the user's brain signals generated in response to stimulus presentations. Straightforward as this seems, complication arises from the fact that there is not always a 1-to-1 relation between stimuli to present, and targets to choose from (e.g., the <tt>P3SpellerTask</tt> will present rows and columns of the selection matrix, but the final choice will be a single matrix entry rather than a row or column). Also, due to the low signal-to-noise ratio in individual evoked response signals, it is generally necessary to perform multiple stimulus presentations before a target may be selected with reasonable accuracy.

			Through its interface to descendant classes, the <tt>StimulusTask</tt> base class allows for efficient implementation of various kinds of such applications. In detail, the following basic concepts are involved in the interaction between <tt>StimulusTask</tt>, and a descendant class.

			===Presentation of stimuli===
			Individual stimuli are instances of stimulus classes, which derive from a common <tt>Stimulus</tt> base class. There are classes for visual and auditory stimuli available.
			A <tt>StimulusTask</tt> descendant class may instantiate any number of stimuli during <tt>OnInitialize()</tt>. Each stimulus must be made known to the <tt>StimulusTask</tt> class by adding it to a so-called "Association" (see below). The <tt>StimulusTask</tt> class will then take care of presenting those stimuli at appropriate points in time, according to user configuration.

			===Grouping of stimuli===
			Stimuli are presented as groups called "Associations". Each Association corresponds to a single ''StimulusCode'' value. No Associations exist when a <tt>StimulusCode</tt> descendant's <tt>OnInitialize()</tt> function is called. As a coding example, the descendant may add a stimulus to an Association with ''StimulusCode'' 23 by calling
			SomeStimulusClass* pStimulus = new SomeStimulusClass;
			... // set stimulus properties
			Associations()[23].Add( pStimulus );
			There, a new Association object for ''StimulusCode'' 23 will be created if it does not exist. Stimulus codes are positive integers. They need not be continuous, may be specified in any order, and chosen as deemed convenient. A zero ''StimulusCode'' represents absence of a stimulus, so 0 cannot be chosen as a ''StimulusCode'' index for an association. Still, an Association object with index 0 may be created, but it will be ignored during stimulus presentation.

			Note that an empty Association object is created on any access to a non-existing index, both read and write accesses.
			Thus, you will need to take care to avoid unwanted empty Associations, i.e., don't use an Association index unless you want that Association to exist. If an empty Association exists, its ''StimulusCode'' may appear in the presentation sequence, taking up a presentation time slot without any stimulus being presented. In case such behavior is actually desired, an empty Association may be created by calling
			Associations()[index];

			Whenever the ''StimulusCode'' state becomes nonzero, all stimuli contained in the corresponding Association are presented simultaneously. An Association may contain a single stimulus only (e.g., when presenting images in the StimulusPresentation module), or it may contain multiple stimuli (e.g., matrix speller rows and columns are made up of multiple stimuli, where each stimulus is a single matrix entry). Also, Associations may contain different kinds of stimuli (e.g., when presenting both images and audio files with StimulusPresentation, or when using auditory stimuli to announce rows or columns in the P3Speller).

			===Sequencing===
			Presentation sequences consist of Associations, represented by their ''StimulusCodes''.

			The <tt>StimulusTask</tt> base class does not do any sequencing by itself. Rather, each time before a presentation happens, the <tt>OnNextStimulusCode()</tt> function is called, which must be implemented by a <tt>StimulusTask</tt> descendant.
			The descendant class indicates the end of a sequence by returning zero as a ''StimulusCode''. Further, it indicates the end of a run by returning a zero ''StimulusCode'' twice in a row (i.e., an empty sequence). The information about existing ''StimulusCode'' values will be known by a descendant because it is responsible for populating the <tt>Associations</tt> object, but to avoid duplication of information, it may be appropriate to either
			* choose stimulus codes from a continous range 1..N, and obtain N by calling <tt>Associations().size()</tt>, or
			* iterate over the <tt>Associations()</tt> object which is a <tt>std::map<int, Association></tt>.

			In the StimulusPresentation module, sequences may be user-defined, and there exists a 1-to-1 correspondence between StimulusCodes, and entries in the Stimuli matrix. In the P3Speller module, sequences are random permutations of an enumeration of all existing rows and columns, and are chosen randomly at runtime.

			===Classification vs Target Selection===
			Brain signals are generated in response to individual stimulus presentations. When used in conjunction with a StimulusTask application, a signal processing module is supposed to provide a classification score for individual brain responses, or an average score for a predefined number of brain responses elicited by presentation of a certain Association/StimulusCode.

			====Specifying Stimulus-Target Relations====
			In general, there is no direct correspondence between stimulus presentations, and possible selection targets. E.g., in the P3Speller, stimulus presentation is done by highlighting rows and columns, but an individual matrix entry will be selected rather than a row or column. Thus, evidence collected in response to presentation or rows and columns needs to be combined into evidence per selection target before selection can be performed. This translation of per-presentation evidence into per-target evidence is done automatically by the <tt>StimulusTask</tt> class, provided <tt>StimulusTask</tt> has been given the necessary information about how stimuli and targets are related to each other.

			A <tt>Target</tt> base class exists for this purpose, which defines a <tt>Select</tt> method to be implemented by <tt>Target</tt> descendants overriding the <tt>Target::OnSelect()</tt> member function. As an example, the P3Speller defines a <tt>SpellerTarget</tt> descendant class which performs an appropriate action on the Speller object if selected as a target.

			The <tt>StimulusTask</tt> class is given information about stimulus-target relations by the Association mechanism described above. In addition to <tt>Stimulus</tt> objects, the <tt>Association::Add()</tt> function may be called with <tt>Target</tt> objects as well, creating a link between presentation of stimuli contained in the Association, and selection of targets contained in the Association. In many cases, there will be a 1-to-1 relation between stimuli, and targets, e.g. a P3Speller matrix entry will create both a <tt>Stimulus</tt> object (so it can be highlighted), and a <tt>Target</tt> object (so it can be selected, and perform some action). However, this is not mandatory -- multiple stimuli may refer to the same target, e.g., neighboring P3Speller matrix elements might select the same target in order to make selection easier for that target; or multiple targets might refer to the same stimulus in a hierarchic selection scheme. Though it may seem unintuitive at first, no 1-to-N or N-to-1 relations between stimuli and targets need to be specified explicitly in such cases; simply adding ''any'' target related to ''any'' stimulus contained in an Association will result in correct target selection. Individual stimulus-target relations are implicitly contained in the resulting Associations, and could be recovered by using set operations on the Associations' stimulus sets, and target sets, but they are not necessary for computing per-target evidence.

			====Accumulation and Combination of Evidence====
			Combination of evidence across individual stimulus presentations is also required if one wants to dynamically choose the number of stimulus presentations that occur before a selection is made. This may be advantageous if the "quality" of brain responses varies, which may be due a number of factors, such as position of a target with respect to the user's eye focus, or the user's attentiveness to the spelling task. In such cases, stimulation sequences may be repeated indefinitely, and evidence from stimulus presentations may be collected into per-target evidence, until a certain threshold is reached. Compared to a system with a fixed number of stimulus repetitions, such a system will improve information flow (bitrate) by avoiding redundant stimulus presentations which would be wasted time in case of good signal-to-noise ratio, and by avoiding selection errors in case of bad signal-to-noise ratio.

			In the case of evoked brain potentials (ERPs, e.g., P300 wave), a linear classifier may be used to classify between the two cases "ERP occurred" vs "no ERP occurred". Quite favorably, it may be shown that in this case the linear classifier's output is a linear function of the log-likelihood ratio representing evidence in favor of the "ERP occurred" case, and against the "no ERP occurred" case. As a result, combined resp. accumulated evidence in favor of single selection targets can be accurately computed from the classifier's output for individual brain responses, by using Association information about how stimuli were grouped in presentation. This allows classification to be performed entirely by the StimulusTask class, omitting the need for classification code within a descendant class. Still, a descendant class may implement its own classification by overriding the OnClassResult() function, or it may modify the default classification result by overriding OnClassResult(), and calling StimulusTask::OnClassResult() within its own implementation of that function.

			===Using a Different Classifier with StimulusTask===
			In general, linear classification may be considered optimal for detection of evoked potential responses, and the ''P3SignalProcessing'' module performs such linear classification.

			Still, you might want to use a different classification algorithm in order to classify the brain's response to a stimulus, e.g. you might want to use oscillatory signal features in addition to or in place of evoked potentials. In such cases, combination and accumulation of evidence may not function properly, unless you transform classification output into the kind of data which is expected by <tt>StimulusTask</tt>-based applications such as the P3Speller application module.

			====Converting a Classifier's Output into a Classification Score====
			As described above, the output of P3SignalProcessing represents a log-likelihood ratio in favor of the existence of a brain response. Thus, when using StimulusTask with a different classifier, that classifier's output should be converted into a log-likelihood ratio before sending it to the application module.

			In terms of the probability <math>p</math> for the case that '''no''' brain response has been observed, the log-likelihood ratio <math>\lambda</math> is defined as

			<math>\lambda = \ln\frac{1-p}{p}</math>,

			where ''ln'' stands for the natural logarithm, which is often termed ''log'' when provided as a function in programming environments.

			Using the above formula, the output of a binary classifier may be converted into a log-likelihood ratio by first determining the probability '''against''' the existence of a response. Depending on the nature of your classification algorithm, you may need to do some analysis by your own in order to determine that probability. In many cases, however, <math>p</math> may be obtained from classifier output by one of the following methods:

			=====Classifier Error Rates=====
			For a classifier with binary output <math>C</math>, if the false positive rate is estimated to be <math>p_{\textrm{fp}}</math>, and the false negative rate is estimated to be <math>p_{\textrm{fn}}</math>, you will have

			*<math>\textrm{for}\, C = 0:\; p = (1 - p_{\textrm{fn}})</math>
			*<math>\textrm{for}\, C = 1:\; p = p_{\textrm{fp}}</math>

			=====Measures of Significance=====
			A <math>p</math>-value from a statistical test is the probability for the null hypothesis to be true. The null hypothesis is "there was no ERP", and thus a <math>p</math>-value represents the desired probability against an ERP.

			Although a <math>p</math>-value may not be directly available, other measures of significance, such as a <math>t</math>-value, may be converted into a <math>p</math>-value. Formulae relating the <math>p</math>-value to other measures of significance may be found in statistics textbooks, or web resources.

			====Transmitting Classification Scores====
			The StimulusTask class expects to receive a control signal with a single channel, and a single element. Whenever the StimulusCodeRes state is nonzero, it will treat the single value of its control signal as an update to the classification score associated with the StimulusCode sent in the StimulusCodeRes state.

			Classification score updates may be sent any time during sequence or post-sequence (PhaseInSequence state is 2 or 3).
			Though the order in which classification scores are sent does not matter, StimulusTask expects to receive exactly one classification result for each StimulusCode that has been presented since the last classification score update was sent.

			Only a single classification score can be sent for each signal data block. If your classifier operates by comparison of multiple brain responses, rather than each response individually, you should store classification scores in a stack-like data structure, and send only the top of the stack at each data block.

			Detailed processing of classification score updates depends on the [[#MinimumEvidence\|MinimumEvidence]] and [[#AccumulateEvidence\|AccumulateEvidence]] parameters.

	==Events==		==Events==

Mellinger: /* Events Summary */

2011-08-03T11:31:08Z

Events Summary

← Older revision		Revision as of 11:31, 3 August 2011
Line 65:		Line 65:
	\|-		\|-
	\|     OnSequenceEnd		\|     OnSequenceEnd
			\|-
			\|     DoPostSequence* \|\|   \|\| PostSequence
	\|-		\|-
	\|     [OnClassResult] \|\|   \|\|   \|\|   \|\| determine selected target (copy/free modes only)		\|     [OnClassResult] \|\|   \|\|   \|\|   \|\| determine selected target (copy/free modes only)
	\|-
	~~\|     DoPostSequence* \|\|   \|\| PostSequence~~
	\|-		\|-
	\| }		\| }

Mellinger: /* Events Summary */

2011-08-03T11:22:42Z

Events Summary

← Older revision		Revision as of 11:22, 3 August 2011
Line 63:		Line 63:
	\|-		\|-
	\|     }		\|     }
			\|-
			\|     OnSequenceEnd
	\|-		\|-
	\|     [OnClassResult] \|\|   \|\|   \|\|   \|\| determine selected target (copy/free modes only)		\|     [OnClassResult] \|\|   \|\|   \|\|   \|\| determine selected target (copy/free modes only)
	\|-
	~~\|     OnSequenceEnd~~
	\|-		\|-
	\|     DoPostSequence* \|\|   \|\| PostSequence		\|     DoPostSequence* \|\|   \|\| PostSequence

Mellinger: /* Events Summary */

2011-08-03T11:19:35Z

Events Summary

← Older revision		Revision as of 11:19, 3 August 2011
Line 42:		Line 42:
	\|     OnNextStimulusCode \|\|   \|\|   \|\|   \|\| provide a stimulus code, or 0 to finish run		\|     OnNextStimulusCode \|\|   \|\|   \|\|   \|\| provide a stimulus code, or 0 to finish run
	\|-		\|-
	\|     OnPreSequence		\|     OnPreSequence \|\|   \|\|   \|\|   \|\| determine attended target
	\|-		\|-
	\|     DoPreSequence* \|\|   \|\| PreSequence		\|     DoPreSequence* \|\|   \|\| PreSequence
	\|-		\|-
	\|     OnSequenceBegin ~~\|\|   \|\|   \|\|   \|\| determine attended target~~		\|     OnSequenceBegin
	\|-		\|-
	\|     Loop {		\|     Loop {
Line 63:		Line 63:
	\|-		\|-
	\|     }		\|     }
			\|-
			\|     [OnClassResult] \|\|   \|\|   \|\|   \|\| determine selected target (copy/free modes only)
	\|-		\|-
	\|     OnSequenceEnd		\|     OnSequenceEnd
	\|-		\|-
	\|     DoPostSequence* \|\|   \|\| PostSequence		\|     DoPostSequence* \|\|   \|\| PostSequence
	\|-
	~~\|     [OnClassResult] \|\|   \|\|   \|\|   \|\| determine selected target (copy/free modes only)~~
	\|-		\|-
	\| }		\| }

Mellinger: /* int OnNextStimulusCode */

2008-11-22T12:15:00Z

int OnNextStimulusCode

← Older revision		Revision as of 12:15, 22 November 2008
Line 83:		Line 83:
	===Stimulus Code Event===		===Stimulus Code Event===
	====int OnNextStimulusCode====		====int OnNextStimulusCode====
	~~By returning a sequence of~~ stimulus ~~codes~~, ~~this event handler~~ determines the sequence in which stimuli (or associations thereof) are presented.		This event handler is called immediately before stimulus presentation, and determines the sequence in which stimuli (or associations thereof) are presented.
	This handler should return the next element of the current sequence of stimulus codes, or zero to indicate the end of the sequence.		This handler should return the next element of the current sequence of stimulus codes, or zero to indicate the end of the sequence.
	A null sequence indicates the end of a [[BCI2000 Glossary#Run\|run]], i.e. the current run will end when ''OnNextStimulusCode'' returns two zeros in a row.		A null sequence indicates the end of a [[BCI2000 Glossary#Run\|run]], i.e. the current run will end when ''OnNextStimulusCode'' returns two zeros in a row.

Mellinger: /* DisplayMessage(string) */

2008-09-11T15:54:48Z

DisplayMessage(string)

← Older revision		Revision as of 15:54, 11 September 2008
Line 167:		Line 167:
	Methods are declared protected, i.e. for use by descendants only.		Methods are declared protected, i.e. for use by descendants only.
	===DisplayMessage(string)===		===DisplayMessage(string)===
	Displays a text message in the application window. This function is provided for convenience and consistency of appearance; for detailed control over appearance of text messages, add your own [[Programming Reference:TextField Class\|TextField]] object to the [[Programming Reference~~:GUI:~~:GraphDisplay Class\|GUI::GraphDisplay]] represented by the ''Display'' property.		Displays a text message in the application window. This function is provided for convenience and consistency of appearance; for detailed control over appearance of text messages, add your own [[Programming Reference:TextField Class\|TextField]] object to the [[Programming Reference:GraphDisplay Class\|GUI::GraphDisplay]] represented by the ''Display'' property.

	==Parameters==		==Parameters==

Mellinger: /* GUI::GraphDisplay Display (r) */

2008-09-11T15:54:34Z

GUI::GraphDisplay Display (r)

← Older revision		Revision as of 15:54, 11 September 2008
Line 161:		Line 161:

	===GUI::GraphDisplay Display (r)===		===GUI::GraphDisplay Display (r)===
	The ''Display'' property provides access to the [[Programming Reference~~:GUI:~~:GraphDisplay Class\|GUI::GraphDisplay]] object representing the application module's output window.		The ''Display'' property provides access to the [[Programming Reference:GraphDisplay Class\|GUI::GraphDisplay]] object representing the application module's output window.
	{{ApplicationBaseProperties\|classname=StimulusTask\|hlevel====}}		{{ApplicationBaseProperties\|classname=StimulusTask\|hlevel====}}

Mellinger: /* Target* AttendedTarget (rw) */

2008-09-10T17:44:57Z

Target* AttendedTarget (rw)

← Older revision		Revision as of 17:44, 10 September 2008
Line 156:		Line 156:

	===Target* AttendedTarget (rw)===		===Target* AttendedTarget (rw)===
	A pointer that refers to a [[Programming Reference:Target Class\|selection target object]], or a null pointer. When non-null, the [[#StimulusType\|StimulusType]] state variable is set based on the ''AttendedTarget'' property such that ''StimulusType'' is set to 1 whenever the current ''StimulusCode'' state variable refers to an [[Programming Reference:Association Class\|Association]] that contains the specified target.		A pointer that refers to a [[Programming Reference:Target Class\|selection target object]], or a null pointer. When non-null, the [[#StimulusType\|StimulusType]] state variable is set based on the ''AttendedTarget'' property such that ''StimulusType'' is set to 1 whenever the current ''StimulusCode'' state variable refers to an [[Programming Reference:AssociationMap Class#Association Class\|Association]] that contains the specified target.

	For a discussion of how stimuli, stimulus codes, and selection targets are related to each other via [[Programming Reference:Association Class\|Association]] objects, refer to [[Programming Reference:AssociationMap Class]].		For a discussion of how stimuli, stimulus codes, and selection targets are related to each other via [[Programming Reference:AssociationMap Class#Association Class\|Association]] objects, refer to [[Programming Reference:AssociationMap Class#AssociationMap Class]].

	===GUI::GraphDisplay Display (r)===		===GUI::GraphDisplay Display (r)===

Mellinger: /* See also */

2008-09-10T17:44:20Z

Mellinger: /* OnClassInput(stimulusCode (r), GenericSignal (r)) */

2008-09-10T17:43:48Z

OnClassInput(stimulusCode (r), GenericSignal (r))

← Older revision		Revision as of 17:43, 10 September 2008
Line 135:		Line 135:
	====OnClassInput(stimulusCode (r), GenericSignal (r))====		====OnClassInput(stimulusCode (r), GenericSignal (r))====
	The ''OnClassInput'' event handler is called each time a classification value has been received from the [[Technical Reference:Core Modules#Signal Processing Module\|signal processing module]].		The ''OnClassInput'' event handler is called each time a classification value has been received from the [[Technical Reference:Core Modules#Signal Processing Module\|signal processing module]].
	Classification values are stored in a [[Programming Reference:ClassResult Class\|<tt>ClassResult</tt>]] object.		Classification values are stored in a [[Programming Reference:AssociationMap Class#ClassResult Class\|<tt>ClassResult</tt>]] object.
	A <tt>StimulusPresentation</tt> descendant class may take additional action by implementing its own <tt>OnClassInput</tt> event handler.		A <tt>StimulusPresentation</tt> descendant class may take additional action by implementing its own <tt>OnClassInput</tt> event handler.

← Older revision		Revision as of 17:44, 10 September 2008
Line 181:		Line 181:
	[[User Reference:P3TemporalFilter]], [[User Reference:P3SpellerTask]], [[User Reference:StimulusPresentationTask]];		[[User Reference:P3TemporalFilter]], [[User Reference:P3SpellerTask]], [[User Reference:StimulusPresentationTask]];

	~~[[Programming Reference:Association Class]],~~ [[Programming Reference:AssociationMap Class]], [[Programming Reference:Stimulus Class]], [[Programming Reference:Target ~~Class]], [[Programming Reference:ClassResult~~ Class]]		[[Programming Reference:AssociationMap Class]], [[Programming Reference:Stimulus Class]], [[Programming Reference:Target Class]]

	[[Programming Reference:GenericFilter Class]]		[[Programming Reference:GenericFilter Class]]

	[[Category:Framework API]][[Category:Development]][[Category:Filters]][[Category:User Application]]		[[Category:Framework API]][[Category:Development]][[Category:Filters]][[Category:User Application]]