ECAS Curation John J. Tran. DATA “BLOB” –staging area ALVIN LIU WHI COLORADO SELDI, MALDI, & MISC PI BILL GRIZZLE Current Curation Process & Status *

Slides:

Advertisements

Similar presentations

IRRA DSpace April 2006 Claire Knowles University of Edinburgh.

Advertisements

NIMAC 2.0 Publisher Portal: Managing Inventory

4 Oracle Data Integrator First Project – Simple Transformations: One source, one target 3-1.

Advanced Metadata Usage Daan Broeder TLA - MPI for Psycholinguistics / CLARIN Metadata in Context, APA/CLARIN Workshop, September 2010 Nijmegen.

Tutorial 1: Developing a Basic Web site

EDRN’s Validation Study Information Management System Developed for EDRN by the DMCC Cancer Biomarkers Group Division of Cancer Prevention Jet Propulsion.

19-20 March 2003 IVOA Registry Workgroup LeSc Astrogrid Registry: Early Designs Elizabeth Auden Astrogrid Registry Workgroup Leader IVOA Registry Workgroup.

Dr Gordon Russell, Napier University Unit Data Dictionary 1 Data Dictionary Unit 5.3.

Transformations at GPO: An Update on the Government Printing Office's Future Digital System George Barnum Coalition for Networked Information December.

2009 Mid–Term Review El Verde Field Station June 4, 2009.

Digital Preservation Practices and Strategies at Colorado State University Libraries.

Aug. 20, JPL, SoCalBSI '091 The power of bioinformatics tools in cancer research Early Detection Research Network, JPL Mentors: Dr. Chris Mattmann,

Ingest and Loading DigiTool Version 3.0. Ingest and Loading 2 Ingest Agenda Ingest Overview and Introduction Ingest activity steps Transformers Task Chains.

Data Warehouse success depends on metadata

Geography 465 Overview Geoprocessing in ArcGIS. MODELING Geoprocessing as modeling.

Best Practices for Including Enumerated Value Domains in UML Models What are the mechanics of creating CDEs associated with enumerated value domains in.

Bookkeeping data Monitoring info Get jobs Site A Site B Site C Site D Agent Production service Monitoring service Bookkeeping service Agent © Andrei Tsaregorodtsev.

Mail Merge Mailing Labels using DSL Judy Meyer H/CCA Student Services.

Publishing Digital Content to a LOR Publishing Digital Content to a LOR 1.

SCIENCE-DRIVEN INFORMATICS FOR PCORI PPRN Kristen Anton UNC Chapel Hill/ White River Computing Dan Crichton White River Computing February 3, 2014.

DEV-07: Increasing Productivity with Tools for Business Logic Gikas Principal Software Engineer.

Case History: Library of Congress Audio-Visual Prototyping Project METS Opening Day October 27, 2003 Carl Fleischhauer Office of Strategic Initiatives.

METS-Based Cataloging Toolkit for Digital Library Management System Dong, Li Tsinghua University Library

® Cancer Data Standards Repository (caDSR) in the Context of Clinical Trials How is caDSR helping CCR collect and report clinical trials data? The case.

San Diego Supercomputer CenterUniversity of California, San Diego Preservation Research Roadmap Reagan W. Moore San Diego Supercomputer Center

5-7 November 2014 DR Workflow Practical Digital Content Management from Digital Libraries & Archives Perspective.

ECHO DEPository Project: Highlight on tools & emerging issues The ECHO DEPository Project is a 3-year digital preservation research and development project.

Title, meta, link, script.  The title looks like:  The tag defines the title of the document in the browser toolbar.  It also: ◦ Provides a title for.

A Remarkable Record of Science for Change Since 1967.

DMPf – USGS Chesapeake Bay -Cassandra Ladino 02/04/14.

Presentation on SubmissionTrackingTool: by Anjan Sharma.

Introduction to Apache OODT Yang Li Mar 9, What is OODT Object Oriented Data Technology Science data management Archiving Systems that span scientific.

Page 1 Informatics Pilot Project EDRN Knowledge System Working Group San Antonio, Texas January 21, 2001 Steve Hughes Thuy Tran Dan Crichton Jet Propulsion.

1 A National Virtual Specimen Database for Early Cancer Detection June 26, 2003 Daniel Crichton NASA Jet Propulsion Laboratory Sean Kelly NASA Jet Propulsion.

0 A Workable Solution for Basic Metadata January 9, 2006.

Digital Library of the Caribbean Creating Single Items with mydLOC and Editing Materials with the Curator Dashboard

Execute Workflow. Home page To execute a workflow navigate to My Workflows Page.

Integrating a Statewide Web Gateway With Digital Collections ______________________ Eric Weig and Beth Kraemer University of Kentucky and KCVL.

Mind Your Metadata Geri Miller. Metadata in ArcGIS ArcGIS metadata goals Editing metadata Setting your metadata style Leveraging metadata in ArcGIS Importing.

BIEN Confederated DB (S) Analytical DB(s) Heterogeneous source database(s) of Plots/Specimens/Occurrences Synonymy Names Reference taxonomy *** *** Feedback.

1 Metadata –Information about information – Different objects, different forms – e.g. Library catalogue record Property:Value: Author Ian Beardwell Publisher.

Data Curation Workflow

Hampshire Hub Data Platform Progress update 1 October Bill Roberts Swirrl.

David Adams ATLAS DIAL/ADA JDL and catalogs David Adams BNL December 4, 2003 ATLAS software workshop Production session CERN.

Metadata Extraction & Web Archives: Automating the Record Creation Process Abbie Grotke / Gina Jones /

ARROW Institutional Repositories for Managing e-Theses Presentation to ETD September 2005 Geoff Payne, ARROW Project Manager.

Hussein Suleman University of Cape Town Department of Computer Science Digital Libraries Laboratory February 2008 Data Curation Repositories:

Metadata Input Tool for CADIS Scientists and Data Managers by D. Stott August 8, 2007.

Scratchpads and the new Biodiversity Data Journal Biodiversity Data Publishing made… easier Dimitris Koureas Natural History Museum London.

EDRN Biomarker Database Curation Web Interface and Model.

FRErator – the Bridge between FRE and Curator DB.

Writing Metadata Working Towards Best Practices for SEFSC.

The library is open Digital Assets Management & Institutional Repository Russian-IUG November 2015 Tomsk, Russia Nabil Saadallah Manager Business.

Collections Management Museums What’s new in EMu ? Part II Bernard Marshall Chief Technology Officer KE Software.

Application Web Service Toolkit Allow users to quickly add new applications GGF5 Edinburgh Geoffrey Fox, Marlon Pierce, Ozgur Balsoy Indiana University.

Developer Exam Preparation Thom Robbins Bryan Soltis

What problems are we trying to solve? Hannes Tschofenig.

Git How to 1. Why Git To resolve problems in lab exams (accidental deletions) Use existing Libraries with ease (Statistics and Computer) Prepare undergraduates.

National Cancer Institute caDSR Briefing for Small Scale Harmonication Project Denise Warzel Associate Director, Core Infrastructure caCORE Product Line.

Preservation Data Services Persistent Archive Research Group Reagan W. Moore October 1, 2003.

Fitting into an Appraisal, Accessioning, Processing, Discovery, and Delivery Workflow Chris Prom, University of Illinois at Urbana Champaign.

University of Colorado at Denver and Health Sciences Center Department of Preventive Medicine and Biometrics Contact:

Summary : * Digital photography * Project overview * digiKam 1.x: - Main interface - Camera interface - Image Editor - Plugin interfaces - Light Table.

Stanford University, Stanford, CA, USA

Moving on : Repository Services after the RAE

Fernando Aguilar, IFCA-CSIC

EDRN’s Validation Study Information Management System

Using CuCMS: a workshop

Overview of Curriki Site and Features

Presentation transcript:

eCAS Curation John J. Tran

DATA “BLOB” –staging area ALVIN LIU WHI COLORADO SELDI, MALDI, & MISC PI BILL GRIZZLE Current Curation Process & Status * Working with DMCC -- Combining knowledge from ESIS & eCAS into user-friendly interface -- Strategically selection of relevant meta data requirements for eCAS curation interface * Curation process is manual * We’ve collected data from these sites * Holy Grail: semi-automated curation work flow from data to archive

eCAS high level system design dB & file storage policy repository eCAS system web interface batch cmd-line interface meta-data & policy manager

ECAS Policy and Catalog Management RDF Process CDEs Protocols Publications Biomarkers REPOS XML/MysSQL CURATION PROCESS Repository Manager A. Create/Update/Delete Repository = collection of dataset B. Manage/Update Data Set 1. Create CDEs if no CDEs exist 2. Create map CDEs  Product Type Populate Data Elements in Data Set 1. Map to protocol ID (RDF/bmdb) 2. Map to publication 3. Map to biomarker 4. Set policy information for data set Curation Policy Builder CurationPolicy.XML Curation & Pre-loading Stage 1. Download Data to staging area 2. Define product composition (files organization) 3. Build policy file (CurationPolicy.XML) 4. Make policy file ready for use DATA “BLOB” –staging area DATA LOADING PROCESS ALVIN LIU DMCC BILL GRIZZLE ETC Load Data 1. Point to data local or remote 2. For each product in data set a. generate metadata b. validate metadata c. catalog product + metada d. write product metadata to repository e. hand transform data f. roll back on errors 3. Commit process Load & Manage Data EDRN ECAS ARCHIVE

Curation & Policy/FM Workflow loginmain menu 1. curate data 2. manage policy & metadata policy manager 1. list policies 2. add policy 3. policy wizard data manager 1. index data 2. search/modify data 3. add data policy wizard 1. build using existing templates 2. start from scratch list policy templates > define data elements input key-pair values elements define product types input values for product types define product type map map relationship b/w product types & elements

CURATION DOCUMENT FLOW Select An Action * List all CDEs * List all EDRN data set * Associate CDE/Data set Please select a repository * Core * Seldi * Maldi * Misc PI - Add new Repository breadcrumb: home > New Repository Creator: Description: breadcrumb: home > repo: create Repository: Core * List all CDEs * List all EDRN data set * Associate CDE/Data set breadcrumb: home > repo: core CDE: Core breadcrumb: home > repo: core > CDE IdNameDesc EDRN Data Set: Core breadcrumb: home > repo: core > Data Set IdNameDesc MAP: Core breadcrumb: home > repo: core > MAP IdParentDesc breadcrumb: home > repo: core > CDE > edit CDE : edit ID: Name: Description: breadcrumb: home > repo: core > EDRN Data Set > edit EDRN Data Set: edit ID: Name: Description: breadcrumb: home > repo: core > MAP > edit Associate CDE/Data set ID: Parent: Elements: abc… cde… CDEs Abc Def Hij javascript Metadata K/V K/V K/V javascript

Sneak Preview of Tool in the Pipe-line