Multimodal Graph Coarsening for Interpretable, MRI-Based Brain Graph Neural Network

Graph neural networks (GNN s) are a powerful class of model for representation learning on relational data and graph-structured signal, such as brain connectivity graphs derived from neuroimaging. To date, existing work applying graph learning methods to brain connectivity is limited to a single neuroimaging modality such as structural or functional MRI. In practice, the brain is best represented by multiple networks arising from different imaging modalities. We develop a gen-eral framework for jointly pooling multimodal graphs which share the same set of underlying nodes whilst differing in edge connectivity. Building on this approach, we propose a multimodal GNN (MM-GNN) model that incorporates mul-tiple types of neuroimaging-based brain connectivity. When applied to the task of classifying brain images from patients with schizophrenia and healthy control subjects, we observe that incorporating multimodal pooling dramatically improves performance over non-pooled networks and that MM-GNN matches or improves performance over multiple single-modal and non-GNN baselines. Finally, we demonstrate how our approach uses multimodal data to learn a unified, interpretable measure of the salience of individual brain regions of interest. In this way, MM-GNN represents a new method for leveraging diverse brain connectivity data to enhance the detection of mental health disorders and to understand their biological underpinnings.