Multi-variate Data Analysis using Support Vector Machines (SVMs)

Multi-variate pattern classification (MVPC) is gaining increasing interest in the neuroimaging community for both offline and real-time fMRI data analysis, because it allows detecting differences between conditions with higher sensitivity than conventional univariate analysis (see above), by focusing on the analysis and comparison of distributed patterns of activity. In such a multi-variate approach, data from many sources (e.g., voxels in fMRI, channels in EEG and fNIRS) are jointly analyzed. The high sensitivity of MVPC allows “brain reading” applications that aim to decode (“predict”) specific mental states or representational content from activity patterns. After performing a “training” or “learning” phase, the decoding/prediction phase requires little computational load and it is, thus, suitable for real-time BCI applications, including the decoding of mental states. The current version of Turbo-Satori introduces multi-variate pattern classification based on the widely used support vector machine (SVM) learning algorithm (LIBSVM implementation, Chang & Lin, 2001). Training and testing (prediction) is controlled in two separate steps. In the “Analysis” menu, the “SVM Training” item can be used to open the “Multi-Channel Pattern Classification” dialog, allowing to train an SVM on the data from one or more completed runs of a real-time session (currently only 1 training run is supported); the dialog can also be used to perform offline testing, e.g., on the data of a subsequent run. For trial-by-trial online classification, the “Real-Time SVM Classification” dialog can be invoked by clicking the “Real-Time SVM Classification” item in the “Analysis” menu. After training, this dialog allows to analyze trials of a new run Incrementally, producing prediction values that indicate to which class a distributed activity pattern belongs according to the information extracted by the classifier.

Trial Estimation

The “Multi-Channel Pattern Classification” dialog can be used to train support vector machines (SVMs) to associate distributed activity patterns with class labels that correspond to two or more conditions (“classes”) of a paradigm. The “Trial Estimation” tab (see snapshot below) is used to specify how responses should be estimated for individual trials at each channel. The “Train SVM Classifier” tab is used to create training data suitable for SVMs, and to perform the actual training process. The classifier supports multiclass problems, i.e., distributed patterns can be assigned to two or more different conditions. The trained classifier can then be tested on completed run data using the “Test Classifier” tab. For real-time classification, the “Real-Time SVM Classification” dialog is used.

In order to obtain training exemplars for different classes, response values are estimated for each individual trial, as opposed to estimating a mean response of a condition across the whole time course data in a standard GLM (see above). Estimated single-trial responses across relevant channels (e.g., all non-masked channels of a montage) then form the feature vectors used to train the classifier. An estimated trial response might be as simple as the activity level at a certain time point (e.g., at the time of the expected hemodynamic peak response), or the mean response of a few measurement points around the peak response relative to a pre-stimulus baseline. In Turbo-Satori, however, a GLM is used to fit an expected hemodynamic response to measured single-trial data. The beta value or t value (default), estimating the amplitude of the hemodynamic response, is then used as the trial response value.

To add time course data for training trials from one or more completed runs, the “+” button in the Input data field can be used. The selected FNP (Functional NIRS Project) run data will be listed in the “Time course” data box in the Input data field (see snapshot above). From a selected functional data file, the referenced protocol is indicated in the “Protocol” text field, and the associated experimental conditions are extracted and displayed in the “Protocol condition selection” box for the currently selected file. At present, only the data from the current project can be used. Condition onset and offset times (intervals) can be specified in protocol (PRT) files, either by specifying time points or seconds; in the latter case, the specified data in seconds is transformed into data points using the sampling rate value extracted from the produced NIRScout header file. For the displayed example data, the conditions displayed are “Motor Execution” and “Motor Imagery”. The first two main conditions are selected automatically as targets for single-trial estimation. The first condition is skipped under the assumption that it corresponds to a “baseline” or “rest” condition (if specified during reading of the data). It may be useful to select additional conditions in case multiple pair-wise classifications or multi-class classifications are planned. If more than two conditions are included and the pair-wise training option will be used in the Train “SVM Classifier” tab, any two pairs can be selected for training. If the multi-class option will be chosen later, all conditions included at this stage in the Protocol condition selection box will be used as class labels to train the classifier. The trial estimation step calculates a GLM for each trial in each (non-masked) channel by creating a data window around the trial onset. For estimating beta or t values (see below) with single-trial GLMs, a two-gamma hemodynamic response function (HRF) is used. The data window for a trial is specified in seconds, with respect to the onset of a trial as specified in the protocol(s) of the included FNP data set(s). The “Pre-onset” spin box can be used to specify how many data points (seconds) should be included before the trial onset data point. The offset of the trial data window can be either specified absolutely or with respect to the duration of the trial, as specified in the protocol. If the “Use trial duration” option is turned off, the end of the trial window is specified with the “Post-onset” spin box, specifying the number of seconds included after the stimulus onset point. If, for example, the pre-onset value is “4” and the post-onset value is “22”, the data window used for estimating the trial response would be w = pre-onset + 1 + post-onset = 26 seconds (plus one data point); the provided values in seconds will be converted to the corresponding data points by multiplying with the sampling rate. Clicking the “Auto” button after setting the pre-onset interval will provide a value for the post-onset interval, ensuring that the interval is just long enough to let the expected blood oxygenation level-dependent (BOLD) response turn back (largely) to baseline. The described way of specifying the trial data window is well-suited when all trials have the same duration. If trials of a condition have different durations, the trial duration option should be used to specify the data window. If the “Use trial duration” option is turned on (default), the offset is determined by the trial duration plus the value specified in the “Post-trial” spin box as w = pre-onset + trial-duration + post-trial; in case of a pre-onset value of 4 seconds, a posttrial value of 8 and a trial duration of 10 seconds, this would, for example, result in a data window of w = 4 + 10 + 8 = 22 seconds. Note that for both versions available to specify the trial data window, the duration of a trial, as specified in the protocol, will be used to determine a box-car reference function that is set to 1 at the trial onset data point, and subsequent points reflecting the trial duration. The constructed box-car reference is finally submitted to the two-gamma hemodynamic response function to calculate the shape of the predictor, as used in the single-trial design matrix.

For a more robust fit of the model, a linear trend confound predictor may be added (next to the constant predictor and the main predictor) for estimating (and removing) a linear trend. This can be done by checking the “Linear trend” option. The options in the “Resulting estimates” field allow to specify the resulting type of single-trial estimate, which can be t values (“T-values” option), beta values (“Betas” option), or percent signal change beta values (“Betas [% change]” option). The t values option is selected as default, since it often provides the most robust results. As default, the time course data within each trial window is z-normalized, which can be turned off by un-checking the “z normalization” option. The trial estimation process can be started by clicking the “GO” button. The resulting single-trial estimates per channel are stored in FMP (functional NIRS map) files that are automatically made available in the “Train SVM Classifier” tab. For each included time course FNP file, one corresponding FMP output file will be created.

SVM Classifier Training

The “Train SVM Classifier” tab (see snapshot above) is used to create SVM training data for a specified subset of channels (“region-of-interest” [ROI]) from the data listed in the “Trial data” field (created using the steps described above or loaded from previously stored FNP files). The selected channels-ROI can be as large as all (non-masked) channels, or as small as two channels. At present, all non-masked channels are automatically defined as the ROI for data analysis.

Before the SVM training data can be determined, one needs to specify the classes that should be used for training in the “Selection of classification task” field. For a two-class task, one can select one class in the “First class”, and one class in the “Second class” selection box. If the FNP trial response files contain only estimates for two conditions (as is the case for the depicted example data in the snapshot above), the class boxes are already filled with the two class names, respectively. In case more than two classes are available, the desired class comparisons can be specified using the two selection boxes. For a multi-class task, the “Multi-class” option will be enabled and can be checked; this will prepare creation of a learning task with all conditions available in the FNP trial estimation files. Note that it is not possible (in the present version) to select a subset of three or more classes from four or more available classes: if you want to use more than two classes, but not all available classes, you need to specify the classes in the earlier stage of trial estimation. Note also that the “Multi-class” option will only be enabled if at least three conditions are available. If the option is turned on, the two class selection boxes will be disabled.

When the classes for training have been specified, the actual training data for the SVM classifier can be generated by clicking the “Create” button in the “Create ROI-MVPA data” field. The resulting training data MVP file will appear in the “Training/test data” text box in the “ROI-MVPA” data field. The “For training” and “For testing” options in the “Create ROI-MVPA data” field are only available to aid in distinguishing the purpose of a specific training data file. If the “For training” option is selected (default), the resulting MVP training data file will begin with the “Train_” substring, whereas it will begin with “Test_”, when the “For testing” option is selected.

The snapshot above shows the “ROI-MVPA data” field after clicking the “Create” button in the “Create ROI-MVPA data” field. The name of the resulting training data file reflects the conditions used and how many trials (exemplars) are available for each class. In the displayed example, the names of the two conditions are shown, followed by two values (6-6), indicating that each of the two classes contains 6 trials.

The “Plot” button can be used to create a visualization of the created training data (see snapshot above for the example data). The y-axis represents the training patterns with black lines separating the different classes; the x-axis represents the channels used for training (14 channels in this example). The colored rectangles indicate the value of the estimated responses within a pattern for a given channel (negative values green-to-blue, positive values yellow-to-red). The rows at the bottom of the graph show the mean values averaged over the trials of the available classes. In this example, it is evident that in most channels the trial responses for the first class (“Motor Execution”) are larger than the responses for the second class (“Motor Imagery”).

With the created .MCP training data file (MCP = Multi-Channel Pattern), a support vector machine can now be trained. When the “Train” button is clicked in the “Support Vector Machine (SVM)“ field (see snapshot above), SVM training is performed for the data in the file listed in the “Training/test data” text field; the SVM will use default settings, including a linear kernel and a value of “1” for the penalty (regularization) parameter “C”. One may also let the program find an optimal C value by running a cross-validation procedure (by turning on the “Cross-validation” option and by setting an appropriate “Folds” value) that uses only the training data. While it is possible to select a non-linear kernel using the “Kernel” combo box in the “Support Vector Machine (SVM)” field, it is recommended to stick to the default linear kernel since this allows the interpretation of the weight vector of the trained SVM. The training process results in a .SVM file that is stored to disk, and shown in the “Classifier” text box (see snapshot above). Note that if one wants to run another classifier on the same data with different settings, one needs not to re-create the training data, but can directly select the desired MCP file (if already existing) using the “…” selection button next to the “Training/test” data text box.

Note that there is also a “Log pane” in the “Test Classifier” tab that shows additional information about the trial estimation and SVM learning process, including the values of the classifier weights (one per included channel, see “Weight values” section in the snapshot above). The last information, printed to the log after training, will show the result of applying the trained SVM on the training data itself. This should lead to an accuracy value of 100% (see snapshot above). To test the classifier on new data, the “Test Classifier” can be used for completed run data or, for real-time applications, the “Real-Time SVM Classification” dialog can be used to produce incremental (online) predictions.

SVM Classifier Testing

To test completed run data using the “Test Classifier” tab of the “Multi-Channel Pattern Classification” dialog, one needs to create the .MCP file for the testing data in the same Way, as described for the training data. This includes using the same channels (“ROI”), but selection of different .NMP files. Before clicking the “Create” button in the “Create ROI-MVPA” data field, it is advised to select the “For testing” option in order to name the resulting .MCP file as “Test_…” data. For testing, one does not run the classifier training step, but uses the previously trained classifier on the new data. The previously trained classifier can be loaded using the “…” browse button on the right side of the “Trained classifier” text box. To test the classifier on the data of the loaded run, click the “Test on Run Data” button. The prediction results will be printed in the “Log pane”. Note that the data from the test run(s) should match the previous training run(s), i.e., the data should be collected during the same NIRS session and using the same channel setup; furthermore, the same channels (ROI) as used during training, must be used to create the .MVP file.