Turbo-BrainVoyager v4.4

Semantic Neurofeedback

During semantic neurofeedback semantic information is displayed to participants that is extracted online using real-time representational similarity analysis of evoked activity patterns and visualized in a semantic map using multi-dimensional scaling (MDS). For details see Goebel (2021, Russo et al. (2021), Ciarlo et al. (2022) and Goebel et al. (2024).

TBV 4.4 does not visualize a semantic map (yet), but it calculates and exports useful information incrementally about the current mental state pattern and reference patterns obtained during a localizer run. The export can be controlled using the Semantic Neurofeedback Export dialog that can be invoked by using the Semantic Neurofeedback item in the BCI menu (see screenshot below).

sNF-Export-1

The pattern similarity information exported TR-by-TR to disk can be used by custom software to calculate and visualize a semantic space, usually by employing a form of MDS that 'projects' the pattern similarity structure of the (di)ssimilarity matrix in a two-dimensional space (map) where the Eucleadean distance of the current mental state pattern reflects the (dis)similarity to the reference patterns.

To export pattern information, an ROI condition patterns (RCPS) file must be provided. A .RCPS file is similar as a standard .ROI file but it contains activation effects (usually estimated beta values) for each ROI voxel and condition. To get an RCPS file, select one or more ROIs at the end of a localizer run and then save the selected region(s) to disk as an RCPS file using the Save RCPS File item in the Analysis menu. You can also inspect the current voxel-level effects by using the Show ROI Condition Patterns (RCPS) item in the Analysis menu, which will show the Edit ROI Conditon Patterns dialog.

Before starting a sNF run, open the Semantic Neurofeedback Export dialog and provide the saved RCPS file as input in the Localizer multi-voxel condition patterns field. After starting the sNF run, TBV will then save the currently estimated (beta) values at each voxel into a series of RCPS files that are identified with the current data volume number in the substring before the file extension: [prefix]-[volume-number].rcps.

References

Ciarlo A, Russo AG, Ponticorvo S, Di Salle F, Lührs M, Goebel R, Esposito F (2022). Semantic fMRI neurofeedback: a multi-subject study at 3 tesla. Journal of Neural Engineering, 19, 036020.

Goebel R (2021). Analysis methods for real-time fMRI neurofeedback. In: M. Hampson (Ed.). fMRI Neurofeedback, pp. 23-55. Academic Press.

Goebel R, Lührs M, Ciarlo A, Esposito E, Linden DE (2024). Semantic fMRI neurofeedback: From mechanisms to clinical applications. Submitted for publication.

Russo AG, Lührs M, Di Salle F, Esposito F, Goebel R (2021). Towards semantic fMRI neurofeedback: navigating among mental states using real-time representational similarity analysis. Journal of Neural Engineering, 18(4), 046015.