P l a n e t a r y D a t a S y s t e m 1 Data Integrity Compilation Data Integrity Compilation Presented by: Al Schultz Location: PDS Management Council.

Slides:

Advertisements

Similar presentations

Business Continuity/Disaster Recovery: Solutions For Firms Of All Sizes Atlas Lee, CBCP Director Of Business Continuity Atlas Lee, CBCP Director Of Business.

Advertisements

Computing Infrastructure

Internet Contracting Estimating and Accounting System ICEAS Multi-Client Software Presented By: I C E A S.

NetApp Confidential - Limited Use

Harris LiveVault® Online Backup System. Harris LiveVault 2  What is Harris LiveVault?  Why Harris LiveVault?  How Harris LiveVault works  Harris LiveVault.

Backup Strategy. An Exam question will ask you to describe a backup strategy. Be able to explain: Safe, secure place in different location. Why? – For.

Backups Rob Limbaugh March 2, Agenda  Explain of a Backup and purpose  Habits  Discuss Types  Risk/Scope  Disasters and Recovery.

Shared File Service VM Forum January, SFS Topics Targeted Usage Security Accessing CIFS Shares Availability & Protection Monitoring Pricing.

Barracuda Backup Service Data Backup and Disaster Recovery.

S. Gadomski, "ATLAS computing in Geneva", journee de reflexion, 14 Sept ATLAS computing in Geneva Szymon Gadomski description of the hardware the.

Dawei Lin, Ph.D. Director, Bioinformatics Core UC Davis Genome Center July 20, 2008, SLIMS (Solexa sequencing.

Servers Redundant Array of Inexpensive Disks (RAID) –A group of hard disks is called a disk array FIGURE Server with redundant NICs.

Data Center Services Facilities Management Production Support Operations.

P l a n e t a r y D a t a S y s t e m 1 Data Integrity Compilation Data Integrity Compilation Presented by: Al Schultz Location: PDS Management Council.

Costing. Life 2 Model Research (JISC Project) Application to whole operation of Library Attempts to cost common factors Does not take into account certain.

Enterprise Storage Our Journey Thus Far John D. Halamka MD CIO, Harvard Medical School and Beth Israel Deaconess Medical Center.

November 2009 Network Disaster Recovery October 2014.

U3A 1 How To Backup CD/DVD, Memory Stick, Memory Card, 2nd hard drive, auto.

Physical Media WG Report Physical Media Working Group August 13, 2007.

Global Distribution Software Modules AWR Worldwide Enterprise Network.

Preserving Electronic Mailing Lists: The H-Net Archive H-Net Mapped to the OAIS Model Preservation AssessmentPreservation improvementsOverview How H-Net.

Johannes Spitzbart Phonogrammarchiv, Austrian Academy of Sciences Österreichische Tage der Digitalen Geisteswissenschaften save the data - workshop on.

Access Across Time: How the NAA Preserves Digital Records Andrew Wilson Assistant Director, Preservation.

D0 SAM – status and needs Plagarized from: D0 Experiment SAM Project Fermilab Computing Division.

One year experience and some statistics Russian mirror of PhysNet: one year experience and some statistics Serge Krashakov Landau Institute.

Data goodness Mostly in black and white By Dom. You must love your data! Lost data : Current imaging data in BRIC cost ~£5.1M, just for scanning costs!

Update from the Data Integrity & Tracking WG Management Council F2F UCLA Los Angles, CA August 13-14, 2007

Managed End to End Data Solutions Europe and Africa.

Services for Object Storage and Preservation March 2008 All content in these slides is considered work in progress. In no way does it represent an absolute.

NSSDCA Status Dave Williams PDS Management Council Berkeley, CA 19 November 2014.

1618Tb SDDS storage BE-CO-IN. Proposal Upgrade operational LHC_DATA to 1.1 Tbytes with the same operational level : – New faster technology (SAS disks.

TATII ITS Network (Fiber ) Portal Server Fourth Avenue Building Database Server Dual Sparc SAN (RAID) 1.2 TB Direct Connection backup_tables raw_data_files.

Scientific Advisory Committee – September 2011COLA Information Systems COLA’s Information Systems 2011.

Cosc 4750 Backups Why Backup? In case of failure In case of loss of files –User and system files Because you will regret it, if you don’t. –DUMB = Disasters.

29 Nov 2006PDS MC NSSDC MOU history PDS-NSSDC MOU circa 1994 Reviewed in Jan 2003, June 2004, Oct 2005, Nov 2006 Add words to remove HQ changes Change.

Security and Backup. Introduction A back-up strategy must cover all eventualities: Accidental damage Equipment failure Deliberate damage It must consider:

06-1L ASTRO-E2 ASTRO-E2 User Group - 14 February, 2005 Astro-E2 Archive Lorella Angelini/HEASARC.

Operations Report PDS Management Council Meeting Flagstaff, Arizona August 2006

PDS/NSSDC Deliveries Dave Williams PDS Management Council Columbia, MD 3 April 2013.

PDS Geosciences Node Page 1 Archive Status MRO and Phoenix Ed Guinness PDS Management Council August 13, 2007 Ed Guinness.

CERN - IT Department CH-1211 Genève 23 Switzerland t High Availability Databases based on Oracle 10g RAC on Linux WLCG Tier2 Tutorials, CERN,

Archiving and Preservation Michele Kimpton CEO, DuraSpace Bryan Beecher Director, ICPSR DuraSpace Webinar November 2, 2011.

Preserving Electronic Mailing Lists as Scholarly Resources: The H-Net Archives Lisa M. Schmidt

San Diego Super Computing (SDSC) Testing Summary and Recommendations Physical Media Working Group March 2008.

Digital Library Storage Strategies Robert Cartolano, Director Library Information Technology Office November 14, 2008.

Backups Discussion document. Backup – End State Utilize Netapp -SnapManager for Virtual Infrastructure (SMVI) – currently deployed -All VMDKs will be.

1 The National Environmental Information Exchange Network History and Overview Office of Water National STORET/WQX User Meeting November 27, 2007.

Physical Media WG Update November 30, Fall 2006 WG Status Initial technology plan Use Cases Decision Matrix Media Testing NMSU Website Physical.

Data Integrity Management Council F2F Washington, D.C. November 29-30, 2006

NSSDC Deep Archive / PDS4/ CDs-DVDs/Hard-Copy Archive 28 August 2012 Dave Williams.

ERAD PACS Diagram Overview. Network Architecture Ortho Center of Palm Beach SITE 2 MAIN SITE SITE 3 Off-site OR Onsite Disaster Recovery Online Storage.

NSSDC Status Dave Williams, Pat McCaslin, Stef McLaughlin PDS Management Council UCLA 27 August 2014.

Enw / Name. What is a on-line / paper based data capture form Can you give an example where each are used? Automated data capture systems are used around.

Microsoft Research San Francisco (aka BARC: bay area research center) Jim Gray Researcher Microsoft Research Scalable servers Scalable servers Collaboration.

A Solution for Maintaining File Integrity within an Online Data Archive Dan Scholes PDS Geosciences Node Washington University 1.

Oracle Database High Availability

Backups for Azure SQL Databases and SQL Server instances running on Azure Virtual Machines Session on backup to Azure feature (manual and managed) in SQL.

Open-E Data Storage Software (DSS V6)

22 September 2017, ESA/ESRIN - Frascati

Atlas Lee, CBCP Director Of Business Continuity

Session XIa - halFILE Disaster Recovery

Cluster Active Archive

Oracle Database High Availability

2018 Real Dell EMC E Exam Questions Killtest

XenData SX-550 LTO Archive Servers

JDAT Production Hardware

Atlas Lee, CBCP Director Of Business Continuity

Atlas Lee, CBCP Director Of Business Continuity

OnBase Training Speaker: Dora Compis Disaster Recovery.

IBM Tivoli Storage Manager

Presentation transcript:

P l a n e t a r y D a t a S y s t e m 1 Data Integrity Compilation Data Integrity Compilation Presented by: Al Schultz Location: PDS Management Council Meeting UMUC Adelphi, MD Date: November 20-21, 2008

P l a n e t a r y D a t a S y s t e m 2 Primary Copy of Data Availability & Data Integrity ATMGEOImgPPISBN, PSI RingsENNAIF Data Volume (TB) 1.0 TB22.8 TB 3.0 TB1.0 TB, 0.13 TB 1.0 TB0.02 TB0.7 TB Online Capacility (TB) 15 TB35 TB47.3 TB18 TB1.4 TB, 0.2 TB 2.0 TB8.0 TB0.2 TB % of Archive which is Online 90%100% 100%, 100% 100% Online Archive Method (e.g., RAID System) RAIDRAID 5RAID, Storage cluster RAIDRAID, RAID 5 RAID 5+0RAIDRAID-DP Schedule to perform Online Archive Data Integrity check (e.g., daily, weekly) Monthly DailyMonthlyNot scheduled, None MonthlyPer buildMonthly Online Archive Data Integrity Checksum Type(s) (e.g., MD5) MD5deepMD5 MD5, MD5 MD5No MD5/CRC32 used (catalog) MD5

P l a n e t a r y D a t a S y s t e m 3 2 nd Copy of Data (Backup) Availability & Data Integrity ATMGEOImgPPISBN, PSI RingsENNAIF Online Archive Backup Location SDSC, NMSU, JPL Server Room 2 USGS/JPL (MM), EN, GEO (Mer,Mex), Rings (Voy) SDSCUMCP, on-site Redwood, CA, Big Stone Gap, VA UMD / EN Iron Mountain, Co Online Archive Backup Method (e.g., SDSC) SDSC transfer, DVD/CD, JPL copies CopyMedia or Internet Exchange SDSC transfer Rsync, rsync 1TB Data Bricks Rsync / tape AIT Online Archive Backup Schedule MonthlyWith data release As receivedAs neededNightly & weekly, Nightly & weekly Bi-weeklyWeekly / daily Monthly & yearly % Online Archive been Backed UP 100% 93%90%100%, 100% 100% Schedule to perform Online Archive Backup copy Data Integrity check YearlyMonthlyRedundant with primary copy MonthlyNone, None MonthlyOnly at primary None Online Archive Back up copy Data Integrity Checksum Type(s) MD5deepMD5Redundant with primary copy MD5MD5, MD5 MD5N/A

P l a n e t a r y D a t a S y s t e m 4 3 nd Copy of Data (Backup) Availability & Data Integrity ATMGEOImgPPISBN, PSI RingsENNAIF Deep Archive Location NSSDCOffsite, St. Louis NSSDC NSSDC, NSSDC NSSDCN/ANSSDC % of Deep Archive Unk100%1%50%50-90%, Unk 95%N/A100% Deep Archive Method (e.g., NSSDC Tape System) NSSDC transfer, DVD/CD LTD4 tape CD/DVD copies NSSDC on-line Unk, NSSDC NSSDC tape system N/ANSSDC tape system Schedule to perform Deep Archive Data Integrity check Yearly4x per year Per NSSDC agreement As neededNone, N/A Semi-annualN/AOnce per 6 months Deep Archive Checksum Types(s) MD5deepLTD4 built in Per NSSDC agreement MD5MD5 & Unk, N/A MD5N/AMD5 % Deep Archive Data Integrity Checked 0%100%Per NSSDC agreement 0%Unk, 0% 0%N/A0%

P l a n e t a r y D a t a S y s t e m 5 PDS Node System ATMGEOImgPPISBN, PSIRingsENNAIF Primary Location NMSUWUSTLJPL/USGSUCLAUMCP, Tucson Mountainvie w, CA, Big Stone Gap, VA JPL-ENJPL Bld 230 Backup Location NMSU, SDSC, JPL WUSTL & offsite JPL/USGSUCLAUMCP, on-siteRedwood City, Ca JPL-ENJPL Bld 230 Backup Method SDSC transfer, DVD/CD tapeTransfer media/internet tapeRsync, rsync1 TB Data Bricks tapeRAID-DP Backup Schedule MonthlyDaily & 4x per year as receivedMonthl y Weekly/nightly, Weekly/bi- nightly Bi-weeklyDailyN/A Mirror Site Location SDSCN/APendingNoneNone, N/A N/AUMDN/A Mirror Test Schedule Transfer in progress N/ATBDNoneNone, N/A N/AEvery 6 months N/A Mirror Backup Method UnkN/ATBDNoneNone, N/A N/ARsyncN/A