DiFX New Features. Disk based file specification Improved specification of files to correlate Wild cards with time of scan name filtering? External reference.

Slides:



Advertisements
Similar presentations
ELBA – Current Developments Chris Phillips eVLBI Project Scientist 16 June 2008.
Advertisements

Running DiFX with SGE/OGE Helge Rottmann Max-Planck-Institut für Radioastronomie Bonn, Germany DiFX Meeting Sydney.
1. Topics Is Cloud Computing the way to go? ARC ABM Review Configuration Basics Setting up the ARC Cloud-Based ABM Hardware Configuration Software Configuration.
Setting up of condor scheduler on computing cluster Raman Sehgal NPD-BARC.
Mark 6: design and status update VLBI Technical Meeting MIT Haystack Observatory Roger Cappallo Alan Whitney Chet Ruszczyk.
MCDST : Supporting Users and Troubleshooting a Microsoft Windows XP Operating System Chapter 9: Troubleshooting Power Management and I/O Devices.
70-290: MCSE Guide to Managing a Microsoft Windows Server 2003 Environment Chapter 8: Implementing and Managing Printers.
DBBC3 Development - Digital Base-Band Converter 3
GLAST LAT ProjectNovember 18, 2004 I&T Two Tower IRR 1 GLAST Large Area Telescope: Integration and Test One and Two Tower Integration Readiness Review.
Operating Systems.
Chapter Ten Performance Tuning. Objectives Create a performance baseline Create a performance baseline Understand the performance and monitoring tools.
–Streamline / organize Improve readability of code Decrease code volume/line count Simplify mechanisms Improve maintainability & clarity Decrease development.
VLBI Report LBA Operations & eVLBI Tasso Tzioumis ATNF May 2008.
2012 DiFX Users Meeting, Sydney John Spitzak Purpose: Provide a User Interface for specific needs of USNO o Run and organize repetitive DiFX jobs (daily.
Linux Deploy This application is open source software for quick and easy installation of the operating system (OS) GNU/Linux on your Android device. The.
Windows.Net Programming Series Preview. Course Schedule CourseDate Microsoft.Net Fundamentals 01/13/2014 Microsoft Windows/Web Fundamentals 01/20/2014.
6/1/2001 Supplementing Aleph Reports Using The Crystal Reports Web Component Server Presented by Bob Gerrity Head.
Configuring the MagicInfo Pro Display
Current LBA Developments Chris Phillips CSIRO ATNF 13/7/2005.
SSI-OSCAR A Single System Image for OSCAR Clusters Geoffroy Vallée INRIA – PARIS project team COSET-1 June 26th, 2004.
DiFX: Software Correlation at Swinburne for the LBA Adam Deller Swinburne University/CSIRO Australia Telescope National Facility Supervisors: A/Prof Steven.
Christopher Jeffers August 2012
©Kwan Sai Kit, All Rights Reserved Windows Small Business Server 2003 Features.
Online Data Challenges David Lawrence, JLab Feb. 20, /20/14Online Data Challenges.
E-VLBI Software Suite & DRAGON Interoperability Chet Ruszczyk, Jason Soohoo.
Artdaq Introduction artdaq is a toolkit for creating the event building and filtering portions of a DAQ. A set of ready-to-use components along with hooks.
OFED 1.2 Lessons, 1.3 Planning and Field Support May 07 Tziporet Koren.
Update on LBA Technical Developments CASS Chris Phillips 11 October 2013.
Debugging and Profiling GMAO Models with Allinea’s DDT/MAP Georgios Britzolakis April 30, 2015.
Marcelo R.N. Mendes. What is FINCoS? A Java-based set of tools for data generation, load submission, and performance measurement of event processing systems;
E-VLBI: Creating a Global Radio Telescope via High-Speed Networks Alan R. Whitney MIT Haystack Observatory SLAC Data Management Workshop 17 March 2004.
NMS Case Study HP OpenView Network Node Manager Hong-taek Ju DP&NM Lab. Dept. of Computer Science and Engineering POSTECH, Pohang Korea Tel:
Software Status Sonja Vrcic Socorro,
VTP: VDIF Transport Protocol Chris Phillips, Alan Whitney, Mamoru Sekido & Mark Kettenis November 2011.
Capabilities of Software. Object Linking & Embedding (OLE) OLE allows information to be shared between different programs For example, a spreadsheet created.
DiFX Performance Testing Chris Phillips eVLBI Project Scientist 25 June 2009.
An FX software correlator for VLBI Adam Deller Swinburne University Australia Telescope National Facility (ATNF)
Update on the Software Correlator Nico Kruithof, Huseyin Özdemir, Yurii Pydoprihora, Ruud Oerlemans, Mark Kettenis, JIVE.
AGVCR AccessGrid Video “Cassette” Recorder © 2005 Derek Piper and the Indiana University School of Informatics 02/07/2006.
Configuration Mapper Sonja Vrcic Socorro,
Distributed FX software correlation Adam Deller Swinburne University/CSIRO Australia Telescope National Facility Supervisors: A/Prof Steven Tingay, Prof.
DiFX Overview Adam Deller NRAO 3rd DiFX workshop, Curtin University, Perth.
Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Tools and techniques for managing virtual machine images Andreas.
August 2003 At A Glance The IRC is a platform independent, extensible, and adaptive framework that provides robust, interactive, and distributed control.
Marcelo R.N. Mendes. What is FINCoS? A set of tools for data generation, load submission, and performance measurement of CEP systems; Main Characteristics:
Demonstration of 16Gbps/station broadband-RF VLBI system Alan Whitney for the VLBI2010 development team MIT Haystack Observatory 22 October st Intl.
Ryan Rasmussen Maggie Krause Jiajun Yang. Hardware Progress Mechanical assembly complete Received APM case and power module last week Connected wi-fi.
SPI NIGHTLIES Alex Hodgkins. SPI nightlies  Build and test various software projects each night  Provide a nightlies summary page that displays all.
Development of DiFX Adam Deller NRAO 3rd DiFX workshop, Curtin University, Perth.
Isecur1ty training center Presented by : Eng. Mohammad Khreesha.
1 Masaki Hirabaru and Yasuhiro Koyama PFLDnet 2006 Febrary 2, 2006 International e-VLBI Experience.
Gorman, Stubbs, & CEP Inc. 1 Introduction to Operating Systems Lesson 8 Linux.
Software Requirements for the Testing of Prototype Correlator Sonja Vrcic Socorro, December 11, 2007.
TOG, Yebes, November 2007, A. Szomoru, JIVE Arpad Szomoru, JIVE EXPReS progress, Mark5 and e-VLBI.
Marcelo R.N. Mendes. What is FINCoS? A Java-based set of tools for data generation, load submission, and performance measurement of event processing systems;
Scenario use cases Szymon Mueller PSNC. Agenda 1.General description of experiment use case. 2.Detailed description of use cases: 1.Preparation for observation.
Introduction to Fabric Kiwi Team PSNC. E-VLBI system – general idea.
WP7: Computation in a shared infrastructure Mark Kettenis JIVE Presented by Norbert Meyer PSNC.
TOG, Torun, April 2009 Arpad Szomoru, JIVE EVN/JIVE Technical developments.
Creation and Visualization of 3D Scenes with the MRPT library January, 2007 Jose Luis Blanco Claraco Dept. of Automation and System Engineering University.
What is FABRIC? Future Arrays of Broadband Radio-telescopes on Internet Computing Huib Jan van Langevelde, JIVE Dwingeloo.
Tech Level Cyber Security Lesson 7
Packaging and Deploying Windows Applications
The Distributed Application Debugger (DAD)
Getting Started with R.
SOFTWARE DESIGN AND ARCHITECTURE
Microsoft Windows Technical Support Setup,Install,Activate
Programmable Logic Controllers (PLCs) An Overview.
DiFX Python Interface John Spitzak (USNO).
Building a Full-Product Installer Using p2
Presentation transcript:

DiFX New Features

Disk based file specification Improved specification of files to correlate Wild cards with time of scan name filtering? External reference – Ie remove from.input file

Threads/machine files Generate threads and machine files Could use generalized cluster setup file DataStream cave Threads 7 Core cave Threads 5 Core cave09 cave10..13

FITS-IDI Pulsar binning Pulsar binning support in FITS-IDI needed?

Monitoring and Diagnosis What is needed….?

Tk-errmon? GUI for multicast message/error distrobution Tab per machine Filter on type etc

User Interface Common GUI?

Geodesy Output Route into HOPS

Space VLBI Presently not feasible in difx (??) – Acceleration needed as well as rates Radioastron scheduled to launch in spring ’10 VSOP-2 to launch soon

DiFX Installation What more is needed? Distribution packages ? – E.g. Debian, Ubuntu

IF distribution Allow parallelization via subband as well as timeslicing – Specifically for widebandwidth, with small number of telescopes – E.g 8 Gbps, single baseline “Eliminate” DataStream – Allow eVLBI recorder to send data to core nodes

eVLBI disconnection Allow telescopes to disconnect (& reconnect) in eVLBI mode – Need some method to determine time of other DataStreams and create fake (but invalid) data stream.

Unified DataStream Helper eVLBI requires starting of remote Mark5/LBADR – startcorr.pl will start LBADR recorder – needs extra information which is not in.input file Mark5? Remote playback could be done the same way Method to (optionally) invoke DataStream “helper” application

Realtime DiFX Better support observing without predetermined schedule (e.g. connected element interferometry)

Remove IPP dependencies Isolate all IPP code to a well contained set of routines – Develop non-IPP versions – Version for non-Intel maths libraries

libDiFX Break ALL key components into individual modules – E.g. no mpi dependencies Allow cutdown correlator to be assembled easily (e.g. nonDiFX) Replace specific parts optimised for a particular setup (e.g. lots of telescopes, remove fringe rotation)

DiFX++ When should we abandon DiFX and start a newer better faster…..