INCF Infrastructure for Interoperable Atlases of Rodent Brain Ilya Zaslavsky San Diego Supercomputer Center University of California San Diego

OUTLINE 1.What is INCF-DAI 2.A system of distributed atlas hubs and services Anatomy of a service Spatial transformations 3.INCF Central 4.The current status: introduction to the demo 5.How do you join the INCF atlasing network 6.Summary, current limitations, future work

Purpose of this INCF program: To enhance the interoperability, accessibility, and sharing of spatial data sets in neuroscience: INCF-sponsored standards

The Integration Problem: Whose standard?

What are we trying to do? We have many individual fragmented efforts, each using separate semantic and spatial frameworks Each is a research effort, i.e. constantly changing, imperfect, not necessarily designed for integration Our focus is not on perfecting tools, but integrating them into an interoperable system, by creating an INCF-DAI architecture and standardizing data exchange This is an iterative process: integration experiment -> finding “weakest links” -> prioritizing tool improvement -> new level of integration, with enhanced integration models…

1.What is INCF-DAI 2.A system of distributed atlas hubs and services Anatomy of a service Spatial transformations 3.INCF Central 4.The current status: introduction to the demo 5.How do you join the INCF atlasing network 6.Summary, current limitations, future work

INCF Central, and INCF Services and Hubs INCF Central: Service registry, Registry of spaces, Metadata catalog ABA Service WHS Service UCSD Service EMAGE Service Other…

Atlas Services ABA ServiceWHS ServiceUCSD ServiceEMAGE Service Core Services GetCapabilities, DescribeProcess Known coordinate spaces ABAvoxel, AGEA, ABAreference WHSPaxinosEMAP LookupGetStructureNameBy POI Coordinate translations WHS  AGEA, ABAreference   ABAvoxel, AGEA   ABAvoxel Paxinos   WHSEMAP   WHS Other, based on POI Get2DImageAtPOI, GetCorrelationMap, GetGenesByPOI Get2DImageAtPOIGet2DImageAtPOI, GetGenesByPOI Based on structure GetGenesByStructure Service = some functionality that you can call over the web in a standard manner

Internals of a service – demo implementation (with ABAServiceController as an Example) Service definition resides at the INCF server (as with all current services) Some methods are implemented locally: core methods, all space translations (with mandatory bridge to WHS), also structure lookup by POI Other methods wrap ABA API calls: GetCorrelationMap, Get2DImageatPOI, GetGenesAtPOI, GetGenesByStructure The services return either HTML (currently), or XML/txt GetCapabilities and DescribeProcess comply with OGC’s Web Processing Service (WPS) specification API: Application Programming Interface

Some examples of service signatures Coordinate transformation: – services/service/ABAServiceController?request=SpaceTransformation&FromSRSCod e=WHS&ToSRSCode=AGEA&x=X&y=Y&z=Z&output=txthttp:// :8080/incf- services/service/ABAServiceController?request=SpaceTransformation&FromSRSCod e=WHS&ToSRSCode=AGEA&x=X&y=Y&z=Z&output=txt Structure name lookup: – services/service/ABAServiceController?request=GetStructureNameByPOI&SRScode= ABAVoxel&x=X&y=Y&z=Z&filter=structures:anatomic&output=txthttp:// :8080/incf- services/service/ABAServiceController?request=GetStructureNameByPOI&SRScode= ABAVoxel&x=X&y=Y&z=Z&filter=structures:anatomic&output=txt Find genes: – services/service/EmageServiceController?request=GetGenesByStructure&vocabulary =ABAvoxel&term=term&filter=“structures:anatomic” &output=htmlhttp:// :8080/incf- services/service/EmageServiceController?request=GetGenesByStructure&vocabulary =ABAvoxel&term=term&filter=“structures:anatomic” &output=html Services can be chained (i.e. output of one service becomes input of another) to implement different research workflows. We attempted to make the service signatures as common across different atlas hubs as possible

The Central Role of Waxholm Space

Different types of transformations ABAvoxel to WHS –Large volumes for transforms in both directions, between 3D and 3D of different dimensions –Then simple lookup in the transformation volume ABAvoxel to ABAreference –Collection of conversion formulas for individual slices ABAvoxel to AGEA –Simple scaling WHS to Paxinos Mouse Atlas –Warping appropriate WHS cuts to match with Paxinos slices; translations per slice and brain region

Transformation service issues (looking at WHS to Paxinos as example) WHS is a voxel space, containing continuous information of a particular mouse’s brain, obtained elegantly from an MRI imaging. Paxinos is derived from a mouse brain sample, physically sliced into cross-sections and observed visually: –Different animals, and different shape of the brain –There is no Paxinos information between two adjacent slices –Some Paxinos slides are “out of step” with the adjacent slides –Many Paxinos boundaries are estimated and not consistent with one another, particularly in the cerebellum. Method: –Taking cross-sections of the MRI image, projecting them over Paxinos slides. –The human eye and brain is still a very powerful data processor. By looking at a WHS section over a Paxinos slide, one can see what adjustments are needed: a nudge to the left, a twist clockwise, or a dilation in one direction. Not only can the shape of the brain itself be used, but the light-and-dark patches in the MRI match with Paxinos brain regions. Eventually, each Paxinos slide can match very closely with a WHS cross- section. –Once these cross-sections are found, a clear mathematical correlation in the data gives a formula from one space to another. Additional problems: –It is virtually impossible to make a perfect match, or ascertain accuracy –The anterior commisure region in WHS mouse happened to be located slightly higher than that in the Paxinos mouse. The discrepancy distance is about one diameter. A query in the WHS ac would be likely to miss the Paxinos ac. –The Olfactory Bulb is another difficult region; its cone shape is susceptible to deformation and has different relative dimensions not only in WHS and Paxinos, but in other spaces such as ABA as well. –In some slides, while the internal regions match up very well, the outside extent of the cerebral cortex fails to match.

1.What is INCF-DAI 2.A system of distributed atlas hubs and services Anatomy of a service Spatial transformations 3.INCF Central 4.The current status: introduction to the demo 5.How do you join the INCF atlasing network 6.Summary, current limitations, future work

INCF hubs and central: extended

INCF Central: Registry of Atlas Spaces INCF role may play similar role to EPSG, the authority in coordinate systems –EPSG codes (e.g. EPSG:4326 for WGS84) are ubiquitous in GIS software. EPSG: is the coordinate system used in Google Maps, Yahoo Maps, Virtual Earth. INCF may maintain its own registry of atlas coordinate systems (we have a prototype)

Other INCF Central Registries Service registry for atlas hubs (GetCapability requests) Spatial transformations registry, from all atlas hubs Ontology registry and repository (PONS) Additional metadata: replicas of local metadata catalogs and spatial registries, segmentations, etc.

Managing different data types in INCF-DAI

1.What is INCF-DAI 2.A system of distributed atlas hubs and services Anatomy of a service Spatial transformations 3.INCF Central 4.The current status: introduction to the prototype 5.How do you join the INCF atlasing network 6.Summary, current limitations, ongoing work

Demo Contributors Neuroscience scenarios: Maryann Martone, Stephen Larson (UCSD), Jyl Boline (Informed Minds), Lydia Ng and Mike Hawrylycz (Allen Institute), Al Johnson (Duke University) WHS dataset: Al Johnson and Jeff Brandenburg (Duke University), Jonathan Nissanov and Pablo Burstein (Drexel University) Other datasets and meshes: Maryann Martone and Stephen Larson (UCSD), Lydia Ng and Chris Lau (Allen Institute) INCF Central and atlas service wrappers: Asif Memon (UCSD) INCF hubs and local services: Lydia Ng (Allen Institute), Albert Burger (MRC, UK), Kenneth McLeod (Heriot-Watt Univ, UK), Asif Memon (UCSD) Spatial transformations and anatomic structure lookup: Lydia Ng (Allen Institute), Steven Lamont and Alexander Young (UCSD) Representation of coordinate spaces: Alexander Young (UCSD) Client development: Stephen Larson and UCSD team; Seth Ruffins and UCLA team Hardware and server setup: Larry Lui (UCSD) Architecture: Albert, Fons, Jyl, Mike, Janis, Ilya

Demo overview: with WholeBrainCatalog

Testing transformation chains: from Paxinos Reference Plates to ABA Reference Plates via WHS and AGEA/ABA Initial point: (1.0, 4.3, 1.78) Anterior to Bregma at 1.78 mm, Fig. 16 Structure= AC (anterior commissure)

Testing, Step 1: Paxinos to WHS The original Transform to WHS using transformation service we built: services/service/UCSDServiceController?request=SpaceTransformati on&fromSRSCode=paxinos&toSRSCode=whs&x=1.0&y=4.3&z=1.78 &output=html Result = 308,642,224 WHS coronal cut WHS coronal cut fitted with Paxinos plate

Testing, Step 2: WHS to AGEA The original Transform to AGEA using Steve’s lookup over Lydia’s conversion matrix, wrapped in Asif’s service: services/service/ABAServiceController?request=SpaceTransformation &fromSRSCode=whs&toSRSCode=AGEA&x=308&y=642&z=224&ou tput=html Result: 3825,5650, atlas=whs&view=Y&x=308&y=642&z=224 atlas=whs&view=Z&x=308&y=642&z=224 WHS views Looking at the result in AGEA: map.org/agea/all_coronal ? correlation&seedPoint =3825,5650,4650

Testing, Step 2: WHS to AGEA; Results in AGEA The original map.org/agea/all_coron al?correlation&seed Point=3825,5650,4650 Wrapped in GetCorrelationMap service: mapserver.crbs.ucsd.edu:8080/i ncf- services/service/ABAServiceCon troller?request=GetCorrelationM ap&SRSCode=whs&x=308&y=6 42&z=224&filter=maptype:coron al&output=html

Testing, Steps 3 and 4: AGEA to ABA volume to ABA reference plates The original Transform to ABA (i.e. divide by 25): services/service/ABAServiceController?request=SpaceTransformation&fromSRS Code=agea&toSRSCode=abavoxel&x=3825&y=5650&z=4650&output=html Result: 153,226,186 Transform to ABA reference plate coordinates, using Lydia’s formula implemented in Asif’s service: services/service/ABAServiceController?request=SpaceTransformation&fromSRS Code=abavoxel&toSRSCode=abareference&x=153&y=226&z=186&output=html Result: 1.194, 5.127,1.693 Check the result in ABA reference atlas at map.org/atlas/ARA/Coronal/browser.html map.org/atlas/ARA/Coronal/browser.html (see imageId= ) – we are in Coronal level 38, as predicted! imageId=130973

The results, again The original Paxinos

Placing a probe in the 3D atlas viewer Wholebraincatalog.org

Lateral view, with the probe in hippocampus

Calling ABA services

Calling EMAGE services

Calling WHS services

Calling UCSD (Smart Atlas) services

Response from UCSD Get2DImageAtPOI

Lateral view, rotated

Responses from various ABA requests

Fine Structure Name: DG Anatomic Structure Name: HIP WHC GetStructureNa meByPOI: Hc Responses from various ABA requests - 2

Response from EMAGE GetGenesByStructure

1.What is INCF-DAI 2.A system of distributed atlas hubs and services Anatomy of a service Spatial transformations 3.INCF Central 4.The current status: introduction to the demo 5.How do you join the INCF atlasing network 6.Summary, current limitations, future work

Requirements Flexibility –Individual atlases may use different representations and data types –Atlases may support different functions –Atlases may adhere to known (already registered) or unique (but defined) spatial and semantic frameworks –There may be different amount of funding available to bring the atlas in compliance with INCF-DAI expectations A range of approaches: –Register a GetCapabilities service, then INCF Central will harvest the rest of the metadata (implying that local coordinate system(s) and semantics are formally described and available via services): we need to provide sample software stack. This would be an ideal scenario, though need a monitoring infrastructure –Work with INCF on a hybrid service, where some functions are hosted at the atlas hub, and other are created at INCF (e.g. space descriptions and WHS transformations) –Have INCF host entire atlas

1.What is INCF-DAI 2.A system of distributed atlas hubs and services Anatomy of a service Spatial transformations 3.INCF Central 4.The current status: introduction to the demo 5.How do you join the INCF atlasing network 6.Summary, current limitations, future work

Summary We developed concepts of INCF-DAI, atlas hubs and services, INCF Central server, and communications between them We built service-oriented architecture prototype for distributed digital atlases of mouse brain, that relies, where possible: –On standard service descriptions and exchange schemas –On standardized spaces, and a registry of spaces –On a collection of coordinate translation services, with WHS at the center –On standard terminology, with term cross-walks where needed The translation services have performed well, and several tools were built for testing their performance Data from several atlas hubs can be spatially integrated for the first time, via services and service chains The integration has been demonstrated with a novel 3D Atlas Integration application The system is extensible

Limitations There are many other ways to link data to tell a neuroscience story –E.g. get genes from ABA, and check where these genes are expressed in EMAGE ( but not the focus of the prototype) Spatial integration methods: –From coordinate-based and anatomic feature-based, to integration by spatial placement rules WHS meshes are “fitted” into ABA volume space, for now –Because otherwise need to adjust 2D images in the view, and perhaps have their warped copies Spatial selection –Now by POI; in the future: by ROIs, transects, trajectories (e.g. along neuronal paths) The client does not yet integrate data returned by spatial requests: –It is still Alpha; client integration is planned for the complete version (for 2D images, in particular) – XML schemas are not yet standardized for must queries

Future work Additional atlasing hubs Additional data types and associated standard services (including simulation) XML schemas for the services Further coordinate space standardization Additional reference spaces and transformations (EMAP, in particular) A registration and metadata harvesting system A compelling neuroscience story Working closely with the ontology and metadata TFs on APIs and central registries Versioning of INCF-DAI

NeuroHub Web Service Atlas Web Service Neuro Catalog Service NIF Ontology Service SRS and transformati ons Service 2D and 3D Map Services Authorizatio n Service Other services Hub services

Hub

SRS Fiducials and Transformations Page

List of hub services Core capability descriptions: GetCapabilities and DescribeProcess (WPS-based). 2.Basic information about the content of the services, harvested by INCF central server: ListGenes, ListStructures, etc. 3.Descriptions of available spatial reference systems: ListSRSs, DescribeSRS (also, possibly, ListStructures and DescribeStructure, ListFiducials and DescribeFiducial, etc.). These methods are expected to be supported by an atlas hub, even if the locally-used Spatial Reference System (SRS) is not unique for the hub. The ListSRSs and DescribeSRS should be also supported at the central server. 4.Spatial transformations: ListTransformations, DescribeTransformation, GetTransformationChain, TransformPOI. These methods should be supported by the INCF central server and by those hubs that rely on a local coordinate system

List of hub services Structure Lookup: GetStructureNamesByPOI. This method should support structure lookup for canonical set of segmentations of each atlas. If there are multiple versions of the segmentations by the maintainer of the canonical space, the latest version is returned (unless another one is requested explicitly). 6.Other POI-based methods. Most of these methods are likely to be implemented at just one or a handful of hubs. They include: Get2DImagesByPOI; GetCorrelationMapByPOI; GetGenesByPOI. 7.Other methods, including those based on structure names, e.g. GetGenesByStructure. May be the only way to connect to atlases for which spatial transformations have not been or cannot be developed, but ontology cross-walks exist. 8.Registration, monitoring and subscription/notification services.

Hub deployment model

General Service Request /HostServiceController?Service=WPS&version=1.0.0 &request= &Identifier= & ResponseForm ={format} &DataInputs={Encoded Inputs} Web Processing Service 1.0.0; Waxholm Markup Language BaseServiceUrl:: /HostServiceController?Service=WPS {BaseServiceUrl}&request=GetCapabilities {BaseServiceUrl}&version=1.0.0&request= DescribeProcess&Identifier={ProcessToDescribe} {BaseService}request=Execute&Identifier={ProcessToExecute}

Service Conventions References: PropertyId; PropertyCode; PropertyName, PropertyDescription; also: PropertyURL Method names: –List: list objects without a query. These requests should support paging capabilities, and enable harvesting. –Describe: returns a single object using a persistent identifier –Get: returns one or more objects using a submitted query, or a pointer to a service Prefixed by hub and SRS/vocabulary, e.g. ABA:ABAVoxel:Hip Support for paging; authentication/authorization (at which level?) Services for metadata and data?

Service descriptions (for atlas services) GetCapabilities DescribeProcess Execute identifiers: –ListSRSs –DescribeSRS(SRSCode)…. Also, ListStructures, ListFiducials, etc. –ListTransformations –DescribeTransformation(transformationCode) –GetTransformationChain(fromSRS,toSRS,filter) –TransformPOI(POI,toSRS) –GetStructureNamesByPOI(POI, outputVocabulary, filter) (shall it be Feature or Structure?) –Get2DImagesByPOI (POI) –GetMap/GetCoverage (standard requests in WMS and WCS) –Also: GetCorrelationMapByPOI, GetGenesByPOI, etc.

Additional requests for generic INCF hubs and INCF Central ListHubs DescribeHub (HubCode) ListHubServices DescribeHubService (HubServiceCode) RegisterHubService, HarvestHubService, GetHubServiceStatus, GetCatalogUpdates Each HubService is expected to have its own capabilities method

Examples from spec {BaseService}request=Execute&Identifier=GetTransformationChain&DataInputs= inputSrsCode={Code};targetSrsCode={Code};filter={string}

Examples from spec {BaseService}request=Execute&identifier=GetStructureNamesByPOI&DataInputs=srs Code=ABAVoxel;x=295.0;y=113.0;z=146.0;vocabulary=ABAVoxel;filter=structureset:Fine

Examples from spec {BaseService}request=Execute&Identifier=TransformPOI&DataInputs=inputSRSCode= WHS;targetSRSCode=paxinos;x=280.0;y=112.0&z=162.0&filter=

Issues Scope and structure of WaxML, for short and long term WaxML specification for typical atlas responses (GetGenes? Get2dImageCollection? Others) How we separate data discovery from retrieval (what is metadata) Scope and structure of atlas service stack vs other services Ultimately – the issue of data models (for spaces, transformations, hubs, etc.); exposing them as API, and populating them via a community process  this is how the meeting is structured

Next steps with the specification (optimistic) Initial development and editing, with contributions from DAI members and INCF research staff (to be completed within 1 month after the meeting - by May 1); Pilot implementations demonstrating the services (ongoing) The document is posted on the INCF site, with a two- month RFC time (May-June) Responses to the comments and discussion within DAI (July-August) The specification is presented for a vote, as recommended standard, by the III Neuroscience Congress in Kobe (end of August – September).