BrainVoyager v23.0

Deep Neural Network Based Segmentation

Functional MRI studies with sub-millimetre resolution often require highly accurate tissue class segmentations given that small segmentation errors of the inner (white / grey) and outer (pial) boundaries can lead to changes in the localisation and interpretation of cortical activity. However, obtaining such segmentations has been difficult given that automatic segmentation algorithms have often been developed using images collected at lower resolution and conventional field strengths. Furthermore, manual corrections are very costly for sub-millimetre resolution images given the high number of voxels.

With recent advances in deep learning (DL), it is now possible to train fully convolutional neural networks to perform semantic segmentation of 2D and 3D images. Previous works have successfully used such algorithms to tackle segmentation of MRI brain images with conventional resolution or for the inner cortex boundary at high-resolution data. BrainVoyager 22 introduces brain segmentation using a deep neural network (DNN) with an advanced 3D patch-wise Tiramisu architecture (Schneider & Goebel, 2020, see below). The developed DNN addresses segmentation of sub-millimetre resolution images with the goal to solve additional challenges for DL: (i) with its large matrix size, high-resolution volumes are very memory intensive, (ii) labelled training data sets are limited to a few volumes, and (iii) the fine structures revealed by detailed high-res images require fine-grained segmentations.

As tests reveal (Schneider & Goebel, 2020), the trained DNN produces segmentation results that outperform traditional segmentation algorithms used in neuroimging software. The obtained tissue class segmentation enables the reconstruction of both the inner and outer boundary of the cortex with no or minimal editing. The high-quality segmentations are especially suited for cortical thickness measurements and mesoscopic (laminar and columnar) fMRI studies. It is planned to use the developed Tiramisu network architecture in the future also to train more conventional (i.e. 1 mm) datasets.

The tool has been developed in TensorFlow using Python, which must be prepared before using the DNN Segmentator, for details see topic Enabling Python. The DNN Segmentator can then be applied for tissue classification. After the segmentation process has been completed, the raw result can be postprocessed to fine-tune the segmentation, to separate left and right hemispheres and to create standard segmented VMRs and reconstructed cortex meshes to prepare advanced applications such as cortical thickness analysis and laminar and columnar fMRI analysis.

Background: The Tiramisu Architecture

The deep neural network developed for precise grey matter segmentation is based on a state-of-the-art Encoder-Decoder architecture called Tiramisu. [...]

References

Schneider, M. & Goebel, R. (2020). 3D Patchwise Tiramisu Net for Segmentation of Sub-millimetre Resolution 7T Brain Images. Poster, OHBM Virtual Conference.