ICAT Integration at ISIS Tom Griffin, ISIS Facility ICAT Developer Workshop The Cosener’s House, Abingdon August 2009

Slides:

Advertisements

Similar presentations

Operating-System Structures

Advertisements

Towards an information model for I2S2

I2S2 - Infrastructure for Integration in Structural Sciences Cross-Institutional Pilot

Chapter 20 Oracle Secure Backup.

A new Network Concept for transporting and storing digital video…………

ICAT Integration at DLS. Alun Ashton. What were the requirements? Integrate with current business system Collect Data and Metadata relating to a proposal.

The Data Management Requirements at SNS Shelly Ren & Steve Miller Scientific Computing Group, SNS-ORNL December 11, 2006.

Cloud Computing Ed Lazowska Bill & Melinda Gates Chair in Computer Science & Engineering University of Washington August 2010.

Basic Computer Networks Configurations (cont.) School of Business Eastern Illinois University © Abdou Illia, Spring 2006 Week 2, Thursday 1/19/2006)

Report Distribution Report Distribution in PeopleTools 8.4 Doug Ostler & Eric Knapp 7264.

5.1 © 2004 Pearson Education, Inc. Exam Managing and Maintaining a Microsoft® Windows® Server 2003 Environment Lesson 5: Working with File Systems.

SQL Reporting Services Overview SSRS includes all the development and management pieces necessary to publish end user reports in  HTML  PDF 

Sharepoint Portal Server Basics. Introduction Sharepoint server belongs to Microsoft family of servers Integrated suite of server capabilities Hosted.

Overview What is SQL Server? Creating databases Administration Security Backup.

Accounting & Financial Services OOA & UCDHS Electronic Document Management System July 2008 Project website:

Windows Server MIS 424 Professor Sandvig. Overview Role of servers Performance Requirements Server Hardware Software Windows Server IIS.

Discussion and conclusion The OGC SOS describes a global standard for storing and recalling sensor data and the associated metadata. The standard covers.

V 0.1Slide 1 Security – Audit Trail How to maintain audit trail? Access Control Other Information Configuration Customization  audit trail  purge.

Cookies Set a cookie – setcookie() Extract data from a cookie - $_COOKIE Augment user authentication script with a cookie.

Taming the facility data explosion The ICAT system explained Damian Flannery NOBUGS 2008 Sydney ICAT.

AYAN MITRA CHRIS HOFFMAN JANA HUTCHINS Arizona Geospatial Data Sharing Web Application Development April 10th, 2013.

M i SMob i S Mob i Store - Mobile i nternet File Storage Platform Chetna Kaur.

Long Term Ecological Research Network Information System LTER Grid Pilot Study LTER Information Manager’s Meeting Montreal, Canada 4-7 August 2005 Mark.

UNIT - 1Topic - 2 C OMPUTING E NVIRONMENTS. What is Computing Environment? Computing Environment explains how a collection of computers will process and.

ICAT Overview Tom Griffin, ISIS Facility ICAT Developer Workshop The Cosener’s House, Abingdon August 2009

ITIS 1210 Introduction to Web-Based Information Systems Chapter 23 How Web Host Servers Work.

Monte Carlo Instrument Simulation Activity at ISIS Dickon Champion, ISIS Facility.

DDN & iRODS at ICBR By Alex Oumantsev History of ICBR  Campus wide Interdisciplinary Center for Biotechnology Research  Core Facility  Funded by the.

3rd June 2004 CDF Grid SAM:Metadata and Middleware Components Mòrag Burgon-Lyon University of Glasgow.

1 All-Hands Meeting 2-4 th Sept 2003 e-Science Centre The Data Portal Glen Drinkwater.

Wenjing Wu Computer Center, Institute of High Energy Physics Chinese Academy of Sciences, Beijing BOINC workshop 2013.

CBEO Portal Presentation 2/6/2008, 4:30pm EST SDSC Or link from

Grid MP at ISIS Tom Griffin, ISIS Facility. Introduction About ISIS Why Grid MP? About Grid MP Examples The future.

DB Installation and Care Carmine Cioffi Database Administrator and Developer ICAT Developer Workshop, The Cosener's House August

Mainframe (Host) - Communications - User Interface - Business Logic - DBMS - Operating System - Storage (DB Files) Terminal (Display/Keyboard) Terminal.

1 © State of New South Wales through the NSW Department of Education and Training, This work may be freely reproduced and distributed for personal,

Introduction to dCache Zhenping (Jane) Liu ATLAS Computing Facility, Physics Department Brookhaven National Lab 09/12 – 09/13, 2005 USATLAS Tier-1 & Tier-2.

IPlant Collaborative Hands-on Cyberinfrastructure Workshop – Part 2 R. Walls University of Arizona Biodiversity Information Standards (TDWG) Sep. 29, 2015,

GO-ESSP Workshop, LLNL, Livermore, CA, Jun 19-21, 2006, Center for ATmosphere sciences and Earthquake Researches Construction of e-science Environment.

CaBench-to-Bedside (caB2B) An easy to use tool for searching across the caGrid Mukesh Sharma Washington University School of Medicine.

Building the e-Minerals Minigrid Rik Tyer, Lisa Blanshard, Kerstin Kleese (Data Management Group) Rob Allan, Andrew Richards (Grid Technology Group)

The CERA2 Data Base Data input – Data output Hans Luthardt Model & Data/MPI-M, Hamburg Services and Facilities of DKRZ and Model & Data Hamburg,

Introduction to Internet. Chapter 1 Objectives Origins of the Internet Packets and Routers TCP/IP DNS HTTP URL Client-Server.

Nfs or ftp server Server 1 Server 2 ClinCapture web app SAS script SAS REST notification service [SAS RNS] (can be run as standalone app) upload / download.

 Registry itself is easy and straightforward in implementation  The objects of registry are actually complicated to store and manage  Objects of Registry.

1.Launch from PeopleSoft (7 or 8) Panels Plug-and-play Perfect fit for PeopleSoft 8.X No modification needed 2.Launch from CS Link Anywhere, anytime Good.

Metadata for structural science Workshop on research metadata in context Nijmegen, 7–8 September 2010 Simon Lambert STFC e-Science UK.

TopCAT Use Cases Priorities User Interface 1 ICAT developer workshop, August 2009 Laurent Lerusse – STFC

AliEn AliEn at OSC The ALICE distributed computing environment by Bjørn S. Nilsen The Ohio State University.

ICAT Schema Current Schema organization What’s there but not yet implemented What could we want in the future 1 ICAT developer workshop, August 2009.

29 March 2004 Steven Worley, NSF/NCAR/SCD 1 Research Data Stewardship and Access Steven Worley, CISL/SCD Cyberinfrastructure meeting with Priscilla Nelson.

Stream Spider Distributed Music Streaming Service with Sliced Music Files from Different Originating Hosts Justin Steffy and Michael D. Elder.

Using the ICAT API to ingest business and experiment metadata Tom Griffin, STFC ISIS Facility NOBUGS 2012 ICAT Workshop

The Research Data Archive at NCAR: A System Designed to Handle Diverse Datasets Bob Dattore and Steven Worley National Center for Atmospheric Research.

AHM04: Sep 2004 Nottingham CCLRC e-Science Centre eMinerals: Environment from the Molecular Level Managing simulation data Lisa Blanshard e- Science Data.

Oct HPS Collaboration Meeting Jeremy McCormick (SLAC) HPS Web 2.0 OR Web Apps and Databases (Oh My!) Jeremy McCormick (SLAC)

The GridPP DIRAC project DIRAC for non-LHC communities.

ICAT Status Alistair Mills Project Manager Scientific Computing Department.

1 (c) 2013 FabSoft. MOST Cloud Service What is a Cloud Service? A cloud service is internet-based, meaning that MOST is hosted on a server farm on the.

6/19/2016 أساسيات الأتصال و الشبكات Communication & Networks Fundamentals lab 4.

1 Design and Implementation of EarthScope Data Portal Chaitan Baru, Kai Lin San Diego Supercomputer Center.

Architecture Review 10/11/2004

Users and Administrators

ICAT- Experience and activities at ISIS

TYPES OF SERVER. TYPES OF SERVER What is a server.

Solving ETL Bottlenecks with SSIS Scale Out

API and Interface Tasks

Chapter 2: Operating-System Structures

Introducing NTFS Reliability Security Long file names Efficiency

Users and Administrators

Presentation transcript:

ICAT Integration at ISIS Tom Griffin, ISIS Facility ICAT Developer Workshop The Cosener’s House, Abingdon August 2009

ISIS Neutrons and Muons Operating since user base; ~200 staff Volume of data ~6TB ~5M datafiles (27 instruments, 500/hour) >6.7GB metadata, >33M rows

Current Pre-Experiment

Data.ISIS

Main point of access for ISIS data Maintains index of Run number -> Folder –GEM12345 –\\isis\inst$\NDXGEM\Instrument\data\cycle_01_1\GEM12345.raw\\isis\inst$\NDXGEM\Instrument\data\cycle_01_1\GEM12345.raw –.txt.log.sav0 Ranges Zip files

Data.ISIS Authentication Data Portal ICAT API Data.ISIS User performs search via application e.g. Data Portal Search is executed in ICAT Permitted results are returned to application Results are displayed to the user User performs request to download datafile, multiple datafiles or dataset ICAT creates http GET link and passes to back to user (routed through application) sessionId (optional) fileId(s) or datasetId action (i.e. download, zip, compressed) 6 6 User clicks http link Data.ISIS call ICAT API to check permissions sessionId & datafileId(s) or datasetId Return Exception on failure or DownloadObject on success - userId - array [filename, cycle, run number] User gets their data!

Current Pre-Experiment

‘To Be’ Pre-Experiment

Problems RB Numbers Business System Not released to users Replace DataPortal with TopCat

Related Projects Grid MP – distributed PC network Analysis and simulation New portal required anyway -> 1)Single point of access for Grid MP 2)Use ISIS data in Grid jobs 3)Store results and process in ICAT Where to store data?

Freddie Akeroyd (Data.ISIS) Chris Moreton-Smith (ISIS DFS/FRS) Hannah Guest (Facilities Business System & ERAS) Matt Clarke (SECI modifications) Damian Flannery Stuart Ansell (writeRaw) Laurent Lerusse (NXIngest) Database Services, Roger Downing, FBU IT Service Desk (servers and hosting) Core ICAT – Glen, Shoaib, Michael..... Acknowledgments (who did what at ISIS) ISIS Computing Group ISIS Data Analysis Group

Questions...