Crystal25 Hunter Valley, Australia, 11 April 2007 Crystal25 Hunter Valley, Australia, 11 April 2007 JAINIS (JCU and Indiana Instrument Services): A Grid Enabled Remote Access and Data Management System for X-Ray Diffraction I. M. Atkinson 1, M. D. Wyatt 1, T. King 1, N. G. Sim 1, P. Turner, R. Quilici 2, D. F. McMullen 3 A collaboration between: James Cook University, the University of Sydney and Indiana University.

Crystal25 Hunter Valley, Australia, 11 April 2007 Crystal25 Hunter Valley, Australia, 11 April 2007 Goal: Develop a Grid enabled and Web services based portal system for collaborative remote access to scientific instruments and their data. Why? Maximise resource utilization and return on investment/s Outreach, diffusion, training and education Grid? Access to compute cycles for processing and modeling and flexible distributed storage management that may further facilitate remote access Multi-Site Collaboration Goal for production level service: Significant collaboration between USyd, JCU, IU (and many others: UAdel, Monash, UQ, DART…) Deployment and user focus issues (this is hard)

Crystal25 Hunter Valley, Australia, 11 April 2007 Crystal25 Hunter Valley, Australia, 11 April 2007 Integrate instruments and sensors (e.g. real-time data sources) into a grid environment with Web Services interfaces Abstract instrument capabilities and functions to reduce data acquisition and analysis applications dependence on specialized knowledge about particular instruments Move production of metadata as close to instruments as possible and facilitate the automatic production of metadata Develop a standard, reusable methodology for grid enabling instruments Rick McMullen, Knowledge Acquisition and Projection Laboratory, Indiana University Common Instrument Middleware Architecture (CIMA)

Crystal25 Hunter Valley, Australia, 11 April 2007 Crystal25 Hunter Valley, Australia, 11 April 2007

Crystal25 Hunter Valley, Australia, 11 April 2007 Crystal25 Hunter Valley, Australia, 11 April 2007

Crystal25 Hunter Valley, Australia, 11 April 2007 Crystal25 Hunter Valley, Australia, 11 April 2007

Crystal25 Hunter Valley, Australia, 11 April 2007 Crystal25 Hunter Valley, Australia, 11 April 2007 IU CIMA X-Ray system 4 Components i. Instrument Representative, ii.My Manager, iii.Data Manager, iv.Portal Interface ● Instrument Rep, My Manager, Data Manager Service implemented in c++, gSoap, Perl ● Storage – ext3 file-system, MySQL database ● Portal – jrs168 portlets, gridsphere Issues – My Manager & Portal implementations domain specific i.e. Crystallography vs oceanographic monitoring

Crystal25 Hunter Valley, Australia, 11 April 2007 Crystal25 Hunter Valley, Australia, 11 April 2007 Video feeds CIMA Instrument Representative Data Manager CIMA SOAP Interface GridSphere Portal SRB data repository (data and metadata) HSM Storage SRB MCAT Data Processing Applications, Annontation and visualisation Instruments Live data feeds CIMA core code from IU DART/JCU/UQ/Usyd/MU Implementation Workflow & comp pipelines. Automatic metadata generation. PGL: SRB access & schemas SOAP

Crystal25 Hunter Valley, Australia, 11 April 2007 Crystal25 Hunter Valley, Australia, 11 April 2007  NFS and MySQL replaced with Storage Resource Broker (SRB)  Kepler workflow used for MyManager  Ability to customise data storage via a workflow system  Highly extensible, not restricted to SRB as repository Data Management

Crystal25 Hunter Valley, Australia, 11 April 2007 Crystal25 Hunter Valley, Australia, 11 April 2007 Data Grid middleware from SDSC (San Diego Supercomputing Centre) Uniform interface to heterogeneous resources over a network. Logical View of data User defined metadata structures Works in conjunction with MCAT. Storage Resource Broker (SRB)

Crystal25 Hunter Valley, Australia, 11 April 2007 Crystal25 Hunter Valley, Australia, 11 April 2007  Use of Personal Grid Library (PGL) for SRB data manipulation / metadata display  Metadata schema definitions applied to experimental data  Stored experimental data is able to be easily retrieved, secured and annotated  Visual diagnostics for sensors (e.g. Graphs, Visual Indicators)  Live CCD image analysis (experimental status) User Facing Portal

Crystal25 Hunter Valley, Australia, 11 April 2007 Crystal25 Hunter Valley, Australia, 11 April 2007

Crystal25 Hunter Valley, Australia, 11 April 2007 Crystal25 Hunter Valley, Australia, 11 April 2007

Crystal25 Hunter Valley, Australia, 11 April 2007 Crystal25 Hunter Valley, Australia, 11 April 2007 A. Beitz, et al. Monash

Crystal25 Hunter Valley, Australia, 11 April 2007 Crystal25 Hunter Valley, Australia, 11 April 2007 J. Hunter, et al. UQ

Crystal25 Hunter Valley, Australia, 11 April 2007 Crystal25 Hunter Valley, Australia, 11 April 2007  Initial Instrument Representatives deployed at JCU, USyd, Monash & UQ Goal  Each site having their own Data Manager and SRB storage facility which is federated across sites  Shibbolith based AAA and Virtual Organisations  Federate/Replicate data into a a national SRB store… Data Federations

Crystal25 Hunter Valley, Australia, 11 April 2007 Crystal25 Hunter Valley, Australia, 11 April 2007 Future Developments –Data federations (SRB) –Security and Rights Management Shibboleth –Analysis tools (Portals/portlets) –Workflows Goal is an end-to-end workbench Extensions to larger and smaller instruments (e.g. sensors)

Crystal25 Hunter Valley, Australia, 11 April 2007 Crystal25 Hunter Valley, Australia, 11 April 2007 Acknowledgements: JCUSydney UniIndiana Uni Adelaide Uni S. NessD. du Boulay K. Chiu (SUNNY) A. Wendelborn M. Morgan C. Chee P. Coddington F. Eilert D. Zhang Department of Education Science and Training: Dataset Acquisition, Accessibility, and Annotation e-Research Technologies (DART) project Australian Research Council: e-Research Seed Funding Programme and the Research Networks Programme (MMSN: Molecular and Materials Structure Network).

Crystal25 Hunter Valley, Australia, 11 April 2007 Crystal25 Hunter Valley, Australia, 11 April 2007