Indexing the Brain

For the last two decades, Human Brain Mapping (HBM) research has flourished, generating tens of thousands of research papers that describe the anatomical and functional structure of the human brain. This is mainly accredited to the infiltration of fMRI research to every branch related to neuroscience, from cognition, to decision making, neuro-economics, as well as sensory perception and motor actions and planning. There is a huge amount of data out there that is hardly indexed, and the number of fMRI papers is increasing every year.

I believe that current technological tools, mainly web scraping, crawlers and natural language processing (NLP) tools, have reached a point that one can create an automatic tool that "reads" all these papers and aggregate them in a single database according to anatomical as well as functional structure.

In order to understand how to implement this project, as well as how to use its products, a short explanation on fRMI research is in order. A typical research begins with an experimental design, where the most basic one is composed of two conditions, for example presenting the subject with a picture of either a house or a face. Then the experiment begins inside the fMRI research and is repeated many times per subject, as well as on many subjects, in order to get a statistically significant result. Such a result is usually described as: "area A was more active under condition X than condition Y". This result thus suggests that area A is somehow involved in processing condition X. For example, it was shown that an area called the Fusiform Gyrus is more active when seeing faces than when seeing houses and this result (with many others) suggests that the Fusiform Gyrus area is involved in processing visual aspects of faces.

An example Abstract of an fMRI research paper (quite old):

"Using functional magnetic resonance imaging (fMRI), we found an area in the fusiform gyrus in 12 of the 15 subjects tested that was significantly more active when the subjects viewed faces than when they viewed assorted common objects. This face activation was used to define a specific region of interest individually for each subject, within which several new tests of face specificity were run. In each of five subjects tested, the predefined candidate “face area” also responded significantly more strongly to passive viewing of (1) intact than scrambled two-tone faces, (2) full front-view face photos than front-view photos of houses, and (in a different set of five subjects) (3) three-quarter-view face photos (with hair concealed) than photos of human hands; it also responded more strongly during (4) a consecutive matching task performed on three-quarter-view faces versus hands. Our technique of running multiple tests applied to the same region defined functionally within individual subjects provides a solution to two common problems in functional imaging: (1) the requirement to correct for multiple statistical comparisons and (2) the inevitable ambiguity in the interpretation of any study in which only two or three conditions are compared. Our data allow us to reject alternative accounts of the function of the fusiform face area (area “FF”) that appeal to visual attention, subordinate-level classification, or general processing of any animate or human forms, demonstrating that this region is selectively involved in the perception of faces." From: Nancy Kanwisher, Josh McDermott, and Marvin M. Chun, "The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception". The Journal of Neuroscience, 1 June 1997, 17(11):4302-4311

The first sentence has all the relevant information for the project: an area "fusiform gyrus" and condition "subjects viewed faces than when they viewed assorted common objects".

Furthermore, most papers have fMRI images that show the exact area of activation, as well as standard 3D coordinates within the brain call Talaraich coordinates. The project should thus first create an indexed database of papers according to their coordinates and/or function/condition; then using NLP tools to extract the meaning of the area from many such papers. The product is thus a tool for future researchers that can query such a database for a specific area they discover in their own research and get not only proper citations for their papers (which is important), but also a suggestion for the meaning of the areas they discovered. If it works, the tool can be even more powerful; usually each experiment results in several areas that are active, because more recent experiments are much more complex and examine higher cognitive functions (e.g. neuroeconomics). Hence, the tool can actually suggest a reconstruction of the experiment all by itself. How? Given the active areas, the tool knows and connects, via NLP and extensions, what function each area performs and can integrate all of these into a hypothesis of the experiment that gave such activation. This is a huge step in human brain mapping research and neuroscience in general: it is akin to reading the inner as well as outer environment of a person, solely from an fMRI scan (a little different from recent "thought reading" experiments, which focus on much finer details).

How to implement this project?

1.      Crawlers in the internet can search specific sites of neuroscience-related journals (not that many) and automatically scan for specific keywords: fMRI, subjects, research, etc.

2.      Once a candidate paper is found, the relevant areas are searched for (again, not that many); or searching for Talaraich coordinates inside the paper.

3.      Milestone: An indexed 3D map/database of human brain papers.

4.      Scanning all papers in the database for a description of the experiment/condition.

5.      Using state-of-the-art NLP tools to extract and index database of conditions/experiments such that there would be overlap between papers.

6.      Milestone: An indexed 3D map/database of the human brain function.