LBA Correlator & Operations LBA’05 Ops mtg Hobart 8 Feb 2005 Tasso Tzioumis.

Slides:



Advertisements
Similar presentations
Data Quality and Related Issues – A Discussion Dave De Young NOAO.
Advertisements

VLBI progress Down-under Tasso Tzioumis Australia Telescope National Facility (ATNF) 25 September 2008.
Archived File The file below has been archived for historical reference purposes only. The content and links are no longer maintained and may be outdated.
Introduction to open access – Creative Commons licensing Where can I find free educational materials – Shared content sites – Creative Commons searches.
Fox Scientific, Inc. ONLINE ORDERING 101. Welcome to our website On our main page you can find current promotions, the vendors we offer, technical references.
Lecture 13 Page 1 CS 111 Online File Systems: Introduction CS 111 On-Line MS Program Operating Systems Peter Reiher.
RETHINK BACKUP & ARCHIVE. 2 Backup and Archive are Top IT Priorities Which of the following would you consider to be your org’s most important IT priorities.
Backing Up a Hard Disk CGS2564. Why Backup Programs? Faster Optimized to copy files Can specify only files that have changed Safer Can verify backed up.
Cloud Computing Resource provisioning Keke Chen. Outline  For Web applications statistical Learning and automatic control for datacenters  For data.
ABSTRACT WormBase is a freely available information resource primarily for the nematode Caenorhabditis elegans but which progressively includes data from.
Summary previous session 1 3 D:\ tools models add meta information netCDF on web server transform to netCDF netCDF on OPeNDAP server data.
January 11, Csci 2111: Data and File Structures Week1, Lecture 1 Introduction to the Design and Specification of File Structures.
Database Management Systems, R. Ramakrishnan and J. Gehrke1 Query Evaluation Chapter 11 External Sorting.
NOAO/Gemini Data workshop – Tucson,  Hosted by CADC in Victoria, Canada.  Released September 2004  Gemini North data from May 2000  Gemini.
File Transfer Methods : A Security Perspective. What is FTP FTP refers to the File Transfer Protocol, one of the protocols within the TCP/IP protocol.
CS 333 Introduction to Operating Systems Class 18 - File System Performance Jonathan Walpole Computer Science Portland State University.
NLC - The Next Linear Collider Project Lee Ann Yasukawa 05/25/99 NLC Archiving Requirements (Preliminary)
Baseband processing, e-VLBI, and SKA simulations Steven Tingay Swinburne University of Technology MNRF Symposium, 8 June, 2004.
1 ADASTRIP-7/01 ADASTRIPADASTRIP Presented by Treehouse Software.
What is Data Storage? ‘Storing’ data, we mean putting the data in a known place. ‘ Writing ’ data or ‘ saving ’ data are other ways of saying ‘storing’
VLBI Report LBA Operations & eVLBI Tasso Tzioumis ATNF May 2008.
Utility Programs  A type of system software that is used to solve a particular problem is called utility program. Many operating system provides different.
Transactions and Reliability. File system components Disk management Naming Reliability  What are the reliability issues in file systems? Security.
October 16-18, Research Data Set Archives Steven Worley Scientific Computing Division Data Support Section.
Field Project Planning, Operations and Data Services Jim Moore, EOL Field Project Services (FPS) Mike Daniels, EOL Computing, Data and Software (CDS) Facility.
KVN Current Status of Daejeon Correlator for EAVN Se-Jin Oh and staff of Correlator team of KASI/NAOJ 8 th East Asian VLBI Workshop, 8~10 July
The State of Kansas Data Access and Support Center (DASC)
Operations Specialists Paranal [calChecker] [HC monitor]calCheckerHC monitor September 2013 The big picture I Why do we take calibrations? SCIENCE observations.
Basic concepts of radio interferometric (VLBI) observations Hiroshi Imai Department of Physics and Astronomy Graduate School of Science and Engineering.
Privacy-Preserving Public Auditing for Secure Cloud Storage
E-VLBI at ≥ 1 Gbps -- “unlimited” networks? Tasso Tzioumis Australia Telescope National Facility (ATNF) 4 November 2008.
Oracle Advanced Compression – Reduce Storage, Reduce Costs, Increase Performance Session: S Gregg Christman -- Senior Product Manager Vineet Marwah.
EASI a free web database application for collecting and managing monitoring records.
2003 LBA Ops meeting Tasso Tzioumis. Previous LBA ops meeting 3 April 2003, Hobart Minutes on
Stanford Linear Accelerator Center R. D. Hall1 EPICS Collaboration Mtg Oct , 2007 Oracle Archiver Past Experience Lessons Learned for Future EPICS.
Netherlands Institute for Radio Astronomy 1 ASTRON is part of the Netherlands Organisation for Scientific Research (NWO) LOFAR Operations and Schedule.
VLA Archive Image Pilot Pilot to create images from VLA archive data Loránt Sjouwerman National Radio Astronomy Observatory.
Z-Books: Hunting Down Zombie Ebooks Hiding in your Catalog Kathryn OVGTSL 2013#ovgtsl2013 May 17, 2013.
Log-Structured File Systems
DiFX Performance Testing Chris Phillips eVLBI Project Scientist 25 June 2009.
An FX software correlator for VLBI Adam Deller Swinburne University Australia Telescope National Facility (ATNF)
VLBI Facility report - Mar 2003 Tasso Tzioumis ATNF CSIRO.
GIST 19: GGSPS status Status of GGSPS development and operations Andy Smith GGSPS software project manager.
Croydon Libraries on the Web by Heather Kirby Croydon Libraries.
CS333 Intro to Operating Systems Jonathan Walpole.
Accounts Payable Automation. Copyright iPayables 2012 Background iPayables, founded 1999, creates invoice and payment platform. Success based on solid.
Online monitor for L2 CAL upgrade Giorgio Cortiana Outline: Hardware Monitoring New Clusters Monitoring
TOW 2003 Correlator Class Kerry Kingham US Naval Observatory Arno Mueskens University of Bonn Mike Titus MIT Haystack Observatory.
Gregory Mendell, LIGO Hanford Observatory LIGO-G WLIGO-G WLIGO-G W LIGO S5 Reduced Data Set Generation March 2007.
EVLA Data Processing PDR E2E Data Archive System John Benson, NRAO July 18, 2002.
Establish consultant availability Establish a single Appointment Centre Model clinical service delivery Focus on follow ups Focus on patient information.
Atacama Large Millimeter/submillimeter Array Expanded Very Large Array Robert C. Byrd Green Bank Telescope Very Long Baseline Array Emmanuel Momjian (NRAO)
03/09/2007http://pcalimonitor.cern.ch/1 Monitoring in ALICE Costin Grigoras 03/09/2007 WLCG Meeting, CHEP.
Objective of this lesson Understand data archives and back-ups.
CMS: T1 Disk/Tape separation Nicolò Magini, CERN IT/SDC Oliver Gutsche, FNAL November 11 th 2013.
Scenario use cases Szymon Mueller PSNC. Agenda 1.General description of experiment use case. 2.Detailed description of use cases: 1.Preparation for observation.
Fox Scientific, Inc. ONLINE ORDERING 101. Welcome to our website On our main page you can find current promotions, the vendors we offer, technical references.
File-System Management
Input from DMG (and SsS) requirements that would help to streamline the data reduction workflow E. Villard.
Jonathan Walpole Computer Science Portland State University
Computer Systems Nat 4/5 Computing Science
NRAO VLA Archive Survey
Experiences and Outlook Data Preservation and Long Term Analysis
مفاهیم بهره وري.
LBA05 ops meeting Tasso Tzioumis
Gustaaf van Moorsel September 9, 2003
Radio Observatory Report
Observing with Modern Observatories (the data flow)
EVN Amplitude Calibration Jun Yang, JIVE
WSAQDW planning documents
Presentation transcript:

LBA Correlator & Operations LBA’05 Ops mtg Hobart 8 Feb 2005 Tasso Tzioumis

Correlator Operations Part-time (3 days/week) correlator operator –Gary Scott ( 3 year term appointment) –Flexible hours; present most week days –Absentee correlation - most Observns –PIs still encouraged to come and correlate, especially for “special” observations –Also assists with tape handling –Preliminary check of data in AIPS ATLOD - load data Plot and inspect visibilities and spectra (VPLOT, POSSM) Plots on experiment FTP area - web accessible –Further checks? (FRING, ANTAB….)

S2 Tapes ~ 1600 tapesets on web tape list ~ 1200 tapesets are LBA (+ CA VSOP) ~ 1100 tapesets in unreleased observations!!!! 30-50% in pulsars (v041,v088,…) Only 16 observations never correlated (100 sets) + 10 partially correlated. Problems or specials? –Will contact PIs but many obs are old Uncorrelated backlog mainly eliminated Recent observations correlated quickly

Tape release? Need tapesets /session Re-using recent observations (e.g. V131) Need to “free” more tapesets PIs need to inform us when release ok Data Quality checks at correlator. –Sufficient to release tapes?? –What other checks PIs want? –Define time limits?

Correlator issues Tape release S2 transport/tapes unreliable –Synch lost more often –Old tapes worse –Transports aging –S2 Electronics maintenance problematic –  Correlation more “labour-intensive” and slower –Tape aging --> unplayable old observations?

Correlator lifetime? LBA correlator and S2s aging fast No S2 support from Canadians May need to “cannibalise” units to fix others –Limited correlator inputs? –Replace S2s at observatories with disks? –Develop disk-->tape copying? (eVLBI project) Will limit flexibility and efficiency Projected S2 correlator life ? –~ 3 years? to match eVLBI developments?

Online archive LBA archive ~ 650 CDs –Total content < 400 GB - one disk! –Need to “weed” recorrelations Online availability? –“raw” rpfits files? (weeding still needed) –Metadata required? (logs, calfiles?, corr model?) –Online as AIPS multisource fits? FRING, Cal files? Usefulness of online archive?? –Worth the effort and how much? –Propose project?