1. ICAT Integration at ISIS Tom Griffin, ISIS Facility ICAT Developer Workshop The Cosener’s House, Abingdon August 2009 tom.griffin@stfc.ac.uk

2. ISIS Neutrons and Muons Operating since 1984 1600 user base; ~200 staff Volume of data ~6TB ~5M datafiles (27 instruments, 500/hour) >6.7GB metadata, >33M rows

3. Current Pre-Experiment

4. Data.ISIS

5. Main point of access for ISIS data Maintains index of Run number -> Folder –GEM12345 –\\isis\inst$\NDXGEM\Instrument\data\cycle_01_1\GEM12345.raw\\isis\inst$\NDXGEM\Instrument\data\cycle_01_1\GEM12345.raw –.txt.log.sav0 Ranges Zip files

7. Data.ISIS Authentication Data Portal ICAT API Data.ISIS User performs search via application e.g. Data Portal Search is executed in ICAT Permitted results are returned to application 1 1 1 1 2 2 3 3 3 3 2 2 Results are displayed to the user 4 4 4 4 5 5 User performs request to download datafile, multiple datafiles or dataset 5 5 6 6 ICAT creates http GET link and passes to back to user (routed through application) sessionId email (optional) fileId(s) or datasetId action (i.e. download, zip, compressed) 6 6 User clicks http link 7 7 7 7 Data.ISIS call ICAT API to check permissions sessionId & datafileId(s) or datasetId 8 8 8 8 Return Exception on failure or DownloadObject on success - userId - array [filename, cycle, run number] 9 9 9 9 User gets their data! 10 1010 1010

8. Current Pre-Experiment

9. ‘To Be’ Pre-Experiment

10. Problems RB Numbers Business System Not released to users Replace DataPortal with TopCat

11. Related Projects Grid MP – distributed PC network Analysis and simulation New portal required anyway -> 1)Single point of access for Grid MP 2)Use ISIS data in Grid jobs 3)Store results and process in ICAT Where to store data?

12. Freddie Akeroyd (Data.ISIS) Chris Moreton-Smith (ISIS DFS/FRS) Hannah Guest (Facilities Business System & ERAS) Matt Clarke (SECI modifications) Damian Flannery Stuart Ansell (writeRaw) Laurent Lerusse (NXIngest) Database Services, Roger Downing, FBU IT Service Desk (servers and hosting) Core ICAT – Glen, Shoaib, Michael..... Acknowledgments (who did what at ISIS) ISIS Computing Group ISIS Data Analysis Group

13. Questions...