Encoding the local connectivity patterns of fMRI for cognitive task and state classification

Itir Onal Ertugrul, Mete Ozay, Fatos T. Yarman Vural

Research output: Contribution to journalArticlepeer-review

4 Citations (Scopus)


In this work, we propose a novel framework to encode the local connectivity patterns of brain, using Fisher vectors (FV), vector of locally aggregated descriptors (VLAD) and bag-of-words (BoW) methods. We first obtain local descriptors, called mesh arc descriptors (MADs) from fMRI data, by forming local meshes around anatomical regions, and estimating their relationship within a neighborhood. Then, we extract a dictionary of relationships, called brain connectivity dictionary by fitting a generative Gaussian mixture model (GMM) to a set of MADs, and selecting codewords at the mean of each component of the mixture. Codewords represent connectivity patterns among anatomical regions. We also encode MADs by VLAD and BoW methods using k-Means clustering. We classify cognitive tasks using the Human Connectome Project (HCP) task fMRI dataset and cognitive states using the Emotional Memory Retrieval (EMR). We train support vector machines (SVMs) using the encoded MADs. Results demonstrate that, FV encoding of MADs can be successfully employed for classification of cognitive tasks, and outperform VLAD and BoW representations. Moreover, we identify the significant Gaussians in mixture models by computing energy of their corresponding FV parts, and analyze their effect on classification accuracy. Finally, we suggest a new method to visualize the codewords of the learned brain connectivity dictionary.

Original languageEnglish
Pages (from-to)893-904
Number of pages12
JournalBrain Imaging and Behavior
Issue number4
Publication statusPublished - 2019 Aug 15


  • Brain decoding
  • Fisher vector encoding
  • Mesh arc descriptors
  • fMRI

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging
  • Neurology
  • Cognitive Neuroscience
  • Clinical Neurology
  • Cellular and Molecular Neuroscience
  • Psychiatry and Mental health
  • Behavioral Neuroscience


Dive into the research topics of 'Encoding the local connectivity patterns of fMRI for cognitive task and state classification'. Together they form a unique fingerprint.

Cite this