1. ICAT- Experience and activities at ISIS
Tom Griffin, STFC ISIS Facility
4th PaN-Data & CRISP Harmonisation Meeting
XFEL/DESY
January 2013

3. Summary Data flows Authentication Authorisation Data policy Problems
Business data - Other systems
Experiment data
Authentication
Authorisation
Data policy
Problems
Future plans

6. Getting data into ICAT Pre-experiment C#
Creates empty investigations and datasets ready for data
Must update ICAT after changes

7. Getting data into ICAT During the experiment ‘ICATIngest’
Subscribes to file system events ReadDirectoryChangesW()
Invokes ‘WriteRaw’ and ‘NxIngest’
Caches groups of files for performance
Extracts ‘science’ metadata

8. Data files ISIS RAW files Nexus Custom binary format + custom reader
Outputs ICAT friendly XML
Nexus
Not really application definition compliant
Use mapping files to convert nexus to ICAT friendly XML

9. Getting Data out - Web Access
Main point of access for ISIS data
HTTP file serving – pluggable architecture
Zipped or non-zipped

10. TopCAT – basic web interface

18. Getting Data out - Mantid
More convenient for users
Must work on and off site
Derived data upload
Potential for provenance tracking

24. Authentication ICAT 4.x plugin architecture
Web services to ISIS user database
15 minutes to write
Single sign on version too
Much easier to reuse existing auth systems

25. Authentication -Umbrella
For specific permissions will require local account stub
Facilities may require local registration for ‘public’ data
At ISIS will all go through local user database
Single authentication provider in ICAT

26. Authorisation My implementation not complete
Increasing test coverage….
So far just under 100 rules
Not everything is possible
Limitations around devolving control

27. Data Policy ICAT implements a data policy in rules
Who can access what data when
ISIS policy published and implemented
ICAT is flexible
tiny.cc/isisdp

28. DOIs – data citation
When to issue?

29. Problems Tracking reference numbers numbers Linking calibration data
‘Fixing’ data that has been wrongly assigned
Migration from 3.3 -> 4.2.2

30. Future Complete migration to 4.2.2 / 4.3
Replace current data server with IDS
Continue integration with Mantid
Improve and share tools (fixing data)
Federation of catalogues
Not just ICATs
OAI-PMH etc

31. Summary 3.3 in use for several years Upgrading to 4.2.2
Data upload key to future success
Easy to link to user database and umbrella
Hardest part is integration with business and experiment data

32. Questions...