Indexing and Retrieval of Dynamic Brain Images*: Construction of Time-Space Graphs for Cognitive Processes in Human Brain Ulukbek Ibraev, Ph.D. Candidate, Rutgers University * Sponsored by NSF Grant # EIA P.B. Kantor, Principal Investigator (PI), S.J. Hanson, Co-PI. Functional Magnetic Resonance Imaging (fMRI) detects changes in blood flow to particular areas of the brain. “Firing” neurons consume oxygen. Therefore, to re- supply oxygen blood flows to those areas of the brain that are actively firing. fMRI detects these changes in blood flow and thus effectively provides both anatomical and functional views of the brain over time. Neurons exchange electrical signals to communicate with each other. Our project suggested that traces of these electrical signals over time could be used for indexing and retrieval of dynamic brain images. For example, the figure shows imaginary trace of the signal that two areas of the brain might have exchanged over time. However, the time resolution of current fMRI doesn’t allow us to see individual signals propagating in human brain. What one can see in fMRI images is that particular areas of brain were active during the period of the experiment. We could use thresholding and/or object segmentation to find these meaningful areas of activation and use them to build a graph, where vertices are activation centroids and edges are causational paths. Functional Magnetic Resonance Imaging Electrical Signals in Brain Time-Space Graphs Since the signal propagation speed in human brain is finite, it is more likely that active areas of the brain that are close in space-time exchanged electrical signals. The Euclidean distance function is computed in four dimensional space, where first three coordinates is space and the fourth coordinate is time. Given the time-space graphs for two different fMRI scans, we can use an algorithm developed by Sven Dickinson to compute their similarity. Building and Comparing Graphs We have developed three slightly different types of graph building (centroid matching) algorithms. The first algorithm, called “local”, connects only vertices that have shortest Euclidean distance and are adjacent in time. The second algorithm, called “global”, connects vertices so that the total length of all edges is minimized. This means that not only vertices adjacent in time can be connected. The second algorithm is very expensive computationally since it requires us to consider all possible edges for matching. Therefore, a new heuristic called “sliding- window”, was developed. It defines a window that slides over time and all matching is done inside that window. Centroid Matching Algorithms Testing on simulated data showed that the global and sliding- window algorithms performed better than local algorithm. Although the sliding-window algorithm is significantly less expensive in terms of the computational requirements, it did not perform worse than the global algorithm. Based on the results of testing we have modified our algorithms to allow splitting and merging of the signals. The figure shows results of running the sliding-window algorithm on simulated signals. Conclusions and Results It is important to compare the performance of the centroid matching algorithms with other types of indexing and retrieval algorithms. The vector-space model has proved effective in textual information retrieval (IR). It can be used as the base case for retrieving dynamic brain images. One naïve way is to treat each volume file as a vector in high dimensional space. Thus, each 3D image is a point in this high dimensional space. fMRI scans are represented as a collection of vectors and can be compared using inner product or cosine measure. Some Future Work Another way to use vector-space model is to represent each 3D image as a color histogram. Histogram can be viewed as a multi-dimensional vector. fMRI scans are represented as a collection of color histogram vectors. Two fMRI scans can be compared by computing similarity between their vectors. Color histogram representation is more desirable than simply using activation values since it significantly reduces the dimensionality of the feature space. Futures (cont.) t t G1 G2 t Local Global Sliding-Window D fMRI image … 97 Vector V(5, 15, 12, 26, …, 97) 3D fMRI image … 97