University of Washington Human Brain Project – 2004 Representative Projects Uses for Web-based Visualization of Brain Data Next Generation Visualization and Mapping Pre-surgical visualization of fMRI data on the reconstructed cortex surfaces In an ongoing study, language mapping data are collected from patients undergoing neurosurgery for intractable epilepsy. Structural and functional MRI data are acquired prior to surgery. During the surgery, cortical stimulation mapping is performed and single neuron activity may be recorded from areas that will be resected. Reconstructions of 3D models and fMRI analysis can now be performed prior to surgery and the neurosurgeon can access this integrated data on the web before or during the surgery. In particular, this helps planning experimental recording of single neuron activity in the areas that were found to be active during language tasks, as revealed by fMRI. As reported earlier, we have developed a web interface for our interactive 3D visualization and analysis tools for integrated brain data. We utilize surface- based rendering of the cortex and other 3D models. Functional data from various modalities can be overlaid onto individual subject’s anatomical models. Both scene rendering and data storage are performed on the server, while a simple web client can run on virtually any computer with internet access. James F. Brinkley, David Corina, Dan Suciu, Andrew V. Poliakov, Richard F. Martin, Kevin P. Hinshaw, Eider Moore, Hao Li, Xenia Hertzenberg, Veronica Smith, Erin Gibson, Chris Re, Etorre Lettich, Hansang Cho, Linda Shapiro and George Ojemann Supported by Human Brain Project grant R01 DC02310 Features   Supports neuroimaging data: structural MRI volumes functional volumes (fMRI, ERP etc.) surfaces (cortex, veins, arteries etc.) surgical photographs maps (collections of labeled points)  Written in Java/Java3D: cross-platform hardware accelerated  Supports remote visualization client/server and standalone modes one server can serve multiple clients uses the Java3D off-screen renderingFeatures   Supports neuroimaging data: structural MRI volumes functional volumes (fMRI, ERP etc.) surfaces (cortex, veins, arteries etc.) surgical photographs maps (collections of labeled points)  Written in Java/Java3D: cross-platform hardware accelerated  Supports remote visualization client/server and standalone modes one server can serve multiple clients uses the Java3D off-screen rendering Mapping   Map points using pick operation  Assign any text or number label to point  Display label text and/or a choice of shapes  Nodes can be arbitrarily grouped  Each group has its own appearance.  Can arbitrarily place points in space Visualization Visualization   Slice viewer and 3D surface rendering  Cutaway view utilizes structural volume and brain mask  Overlay of functional volumes onto the structural data  An arbitrary number of volumes can be overlaid  fMRI can be projected to the cortical surface  Multiple models can be displayed at once (Cortex, Veins and Arteries)  Extracts surfaces from masked volumes Workspace   Organizes and manages patient’s data  XML format, easy to customize Current and Planned Work   Incorporate results from other applications  Manage coordinate systems in workspace  Spatial Queries  Basis for further work in visualization and mapping Evaluating Anatomical Normalization Methods P54 P54P117 TMS localization on cortex models Another group of researchers is evaluating TMS (Transcranial Magnetic Stimulation) in severely depressed patients as a potential treatment technique, a less traumatic alternative to ECT ("shock therapy"). Using our tools, they are able to localize the position of the coil (yellow) with respect to the individual patient’s cortex model and show it’s projection on the cortex surface (blue) Integrating functional data of multiple modalities In a study of Autistic patients, researchers are collecting functional data of multiple modalities, including:  Spectroscopy (PEPSI)  Electromagnetic tomography (EEG/ERP source localization)  functional MRI This approach helps facilitate remote collaboration and data sharing among a distributed group of researchers, and our collaborators find it productive and convenient. This system is now being used in several neuroimaging studies by research groups that utilize various techniques. Embedded Data Management for fMRI Analysis Defining new protocol SPM2 with x_batch Goal: Provide transparent data management and batch processing for users of the popular SPM2 fMRI analysis package. Solution: X-Batch, a plug-in utility for SPM2 which links user-defined SPM2 batch scripts to the Data Management ontology developed by the Dartmouth fMRI Data Center. The fMRIDC ontology is implemented in the Protégé knowledge acquisition framework developed at Stanford. SPM2 Metadata automatically stored into the ontology Features Features  Implemented as an SPM toolbox  Accesses the fMRIDC ontology through the Protégé API  Allows user definition and storage of batch processing parameters  Processing sequences may be applied to multiple data sets  Metadata are collected automatically and stored in the ontology  Results files are saved to directories defined by the protocol Features Features  Implemented as an SPM toolbox  Accesses the fMRIDC ontology through the Protégé API  Allows user definition and storage of batch processing parameters  Processing sequences may be applied to multiple data sets  Metadata are collected automatically and stored in the ontology  Results files are saved to directories defined by the protocol SPM toolbox:  Default ontology loads automatically  View/delete/run previously defined processing sequences (protocols)  Create new processing sequences SPM toolbox:  Default ontology loads automatically  View/delete/run previously defined processing sequences (protocols)  Create new processing sequences Batch ‘script’ creation:  No programming necessary  Protocols consist of processing stages, each stage defined by its parameter set  Use previously created parameter sets, define new ones, or use a combination  Specify order of processing  All parameters saved into the ontology Batch ‘script’ creation:  No programming necessary  Protocols consist of processing stages, each stage defined by its parameter set  Use previously created parameter sets, define new ones, or use a combination  Specify order of processing  All parameters saved into the ontology Batch processing:  Process any number of datasets from beginning to end  Relevant result files are moved into a designated directory Batch processing:  Process any number of datasets from beginning to end  Relevant result files are moved into a designated directory Data management: In the background, metadata are collected and stored in the fMRIDC Data Management Tool Data management: In the background, metadata are collected and stored in the fMRIDC Data Management Tool Screenshot of the Brain Visualization client Left Hemisphere w/ CSM sites Flat Map Surface-Based Method: CARET Volume-Based Method: SPM2 ERP source localization BrainJ3D is a portable, Java3D software toolkit for brain visualization and mapping that will permit more widespread distribution of the tools we have developed to date (left hand panel). In addition, it is designed to be general- purpose, flexible and interoperable with popular neuroimaging software packages (SPM, FSL etc.) MRI from database Colin Atlas with normalized CSM sites Motivation Motivation A valuable quantitative approach for relating anatomy and function of multiple human brains employs the deformation or anatomical normalization of individual target brains to a canonical brain chosen as a template. Neuroscientists have a variety of anatomical normalization methods to choose from. The method selection impacts result quality and study validity. Therefore, it is important to establish criteria of success and validity of statistical models for anatomical variation appropriate for the specific research project. We evaluate two anatomical normalization methods, CARET and SPM2, for comparing and analyzing data collected from epileptic patients who have undergone neurosurgery. During surgery, the cerebral cortical surface is electrically stimulated in multiple locations (CSM sites) to identify cortical loci associated with language disruption. The CSM site data for 80 patients have been entered into a relational database. By anatomically normalizing these patients’ brains with the CSM site data, neuroscientists believe that the remaining functional variation of the language sites may exhibit organizational patterns that are not apparent otherwise, providing more insight into language function in the human brain. Methods Methods Normalize ~20 brains to the colin27 brain atlas using both methods. Criteria for success includes preservation of known anatomical locations and reduction of anatomical variation between CSM language sites. Post-normalization, we record 3-D coordinates of CSM sites in MNI152 coordinate system. Metrics Metrics The smaller the average distance between normalized CSM sites, the smaller the observed anatomical variation. A neuroanatomist compares pre- and post- normalization anatomical locations of CSM sites. The greater the number of sites that remain in the same anatomical location after normalization, the better the preservation of anatomical locations. Preliminary Results: Preliminary Results: Motivation Motivation A valuable quantitative approach for relating anatomy and function of multiple human brains employs the deformation or anatomical normalization of individual target brains to a canonical brain chosen as a template. Neuroscientists have a variety of anatomical normalization methods to choose from. The method selection impacts result quality and study validity. Therefore, it is important to establish criteria of success and validity of statistical models for anatomical variation appropriate for the specific research project. We evaluate two anatomical normalization methods, CARET and SPM2, for comparing and analyzing data collected from epileptic patients who have undergone neurosurgery. During surgery, the cerebral cortical surface is electrically stimulated in multiple locations (CSM sites) to identify cortical loci associated with language disruption. The CSM site data for 80 patients have been entered into a relational database. By anatomically normalizing these patients’ brains with the CSM site data, neuroscientists believe that the remaining functional variation of the language sites may exhibit organizational patterns that are not apparent otherwise, providing more insight into language function in the human brain. Methods Methods Normalize ~20 brains to the colin27 brain atlas using both methods. Criteria for success includes preservation of known anatomical locations and reduction of anatomical variation between CSM language sites. Post-normalization, we record 3-D coordinates of CSM sites in MNI152 coordinate system. Metrics Metrics The smaller the average distance between normalized CSM sites, the smaller the observed anatomical variation. A neuroanatomist compares pre- and post- normalization anatomical locations of CSM sites. The greater the number of sites that remain in the same anatomical location after normalization, the better the preservation of anatomical locations. Preliminary Results: Preliminary Results: