Figure 1. Typical QLWFPC2 performance results with two WFPC2 observations of a Local Group globular cluster running on a 5-node Beowulf cluster with 1.8.

Slides:



Advertisements
Similar presentations
The Australian Virtual Observatory e-Science Meeting School of Physics, March 2003 David Barnes.
Advertisements

Distributed Systems CS
Beowulf Supercomputer System Lee, Jung won CS843.
Types of Parallel Computers
Presented by: Yash Gurung, ICFAI UNIVERSITY.Sikkim BUILDING of 3 R'sCLUSTER PARALLEL COMPUTER.
Long-Term Preservation of Astronomical Research Results Robert Hanisch US National Virtual Observatory Space Telescope Science Institute Baltimore, MD.
A Comparative Study of Network Protocols & Interconnect for Cluster Computing Performance Evaluation of Fast Ethernet, Gigabit Ethernet and Myrinet.
1.1 Installing Windows Server 2008 Windows Server 2008 Editions Windows Server 2008 Installation Requirements X64 Installation Considerations Preparing.
Understanding Operating Systems 1 Overview Introduction Operating System Components Machine Hardware Types of Operating Systems Brief History of Operating.
Operating Systems.
GIANT TO DWARF RATIO OF RED-SEQUENCE GALAXY CLUSTERS Abhishesh N Adhikari Mentor-Jim Annis Fermilab IPM / SDSS August 8, 2007.
Chapter 2: Hardware Basics Section I: Preparing to Use Technology.
2002 January 28 AURA Software Workshop The MATPHOT Algorithm for Digital Point Spread Function CCD Stellar Photometry Kenneth J. Mighell National.
07/14/08. 2 Points Introduction. Cluster and Supercomputers. Cluster Types and Advantages. Our Cluster. Cluster Performance. Cluster Computer for Basic.
1b.1 Types of Parallel Computers Two principal approaches: Shared memory multiprocessor Distributed memory multicomputer ITCS 4/5145 Parallel Programming,
Cache Memory By Sean Hunter.
Measuring zSeries System Performance Dr. Chu J. Jong School of Information Technology Illinois State University 06/11/2012 Sponsored in part by Deer &
Digital Graphics and Computers. Hardware and Software Working with graphic images requires suitable hardware and software to produce the best results.
Computers: Software Patrice Koehl Computer Science UC Davis.
Edge Based Cloud Computing as a Feasible Network Paradigm(1/27) Edge-Based Cloud Computing as a Feasible Network Paradigm Joe Elizondo and Sam Palmer.
How Computers Work. A computer is a machine f or the storage and processing of information. Computers consist of hardware (what you can touch) and software.
NEO Research Project in Korea Wonyong Han 1, Yong-Ik Byun 2, Hong-Suh Yim 1, Young-Jun Choi 1, Hong-Kyu Moon 1 & NESS Team 1 Korea Astronomy and Space.
System Software System software deals with the physical complexities of how the hardware works. System software generally consists of four kinds of programs:
Elements of a Computer System Dr Kathryn Merrick Thursday 4 th June, 2009.
Operational computing environment at EARS Jure Jerman Meteorological Office Environmental Agency of Slovenia (EARS)
Chapter 4 COB 204. What do you need to know about hardware? 
Translate the following message:
Slide 1 DESIGN, IMPLEMENTATION, AND PERFORMANCE ANALYSIS OF THE ISCSI PROTOCOL FOR SCSI OVER TCP/IP By Anshul Chadda (Trebia Networks)-Speaker Ashish Palekar.
1b.1 Types of Parallel Computers Two principal approaches: Shared memory multiprocessor Distributed memory multicomputer ITCS 4/5145 Parallel Programming,
Classification of Computers
At A Glance VOLT is a freeware, platform independent tool set that coordinates cross-mission observation planning and scheduling among one or more space.
March 3rd, 2006 Chen Peng, Lilly System Biology1 Cluster and SGE.
The Cluster Computing Project Robert L. Tureman Paul D. Camp Community College.
Introduction, background, jargon Jakub Yaghob. Literature T.G.Mattson, B.A.Sanders, B.L.Massingill: Patterns for Parallel Programming, Addison- Wesley,
EÖTVÖS UNIVERSITY BUDAPEST Department of Physics of Complex Systems VO Spectroscopy Workshop, ESAC Spectrum Services 2007 László Dobos (ELTE)
CMAQ Runtime Performance as Affected by Number of Processors and NFS Writes Patricia A. Bresnahan, a * Ahmed Ibrahim b, Jesse Bash a and David Miller a.
2004 January 27Mathematical Challenges of Using Point Spread Function Analysis Algorithms in Astronomical ImagingMighell 1 Mathematical Challenges of Using.
Frontiers in Massive Data Analysis Chapter 3.  Difficult to include data from multiple sources  Each organization develops a unique way of representing.
The Guts. CPU CPU Socket The CPU is generally a 2 inch ceramic square with a silicon chip located inside. The chip usually about the size of a thumbnail.
Yerkes Astrophysics Academy for Young Scientists (YAAYS) Summer Institute 2007 Hubble data-mining project: The Ring Nebula M57 Vivian Hoette and Max Mutchler.
PARALLEL COMPUTING overview What is Parallel Computing? Traditionally, software has been written for serial computation: To be run on a single computer.
Parts of the computer.
S. B. Holmes, P. Massey (Lowell Observatory) Detections Using data obtained with the MOSAIC 8k x 8k CCD at the 4-m Mayall telescope at KPNO, we have been.
COMPUTERS in the CLASSROOM
By Jeff Dean & Sanjay Ghemawat Google Inc. OSDI 2004 Presented by : Mohit Deopujari.
AstroGrid NAM 2001 Andy Lawrence Cambridge NAM 2001 Andy Lawrence Cambridge Belfast Cambridge Edinburgh Jodrell Leicester MSSL.
Memory Hierarchy: Terminology Hit: data appears in some block in the upper level (example: Block X)  Hit Rate : the fraction of memory access found in.
By Chi-Chang Chen.  Cluster computing is a technique of linking two or more computers into a network (usually through a local area network) in order.
COMP381 by M. Hamdi 1 Clusters: Networks of WS/PC.
PROOF tests at BNL Sergey Panitkin, Robert Petkus, Ofer Rind BNL May 28, 2008 Ann Arbor, MI.
Week1: Introduction to Computer Networks. Copyright © 2012 Cengage Learning. All rights reserved.2 Objectives 2 Describe basic computer components and.
Slide 6-1 Chapter 6 System Software Considerations Introduction to Information Systems Judith C. Simon.
CIP HPC CIP - HPC HPC = High Performance Computer It’s not a regular computer, it’s bigger, faster, more powerful, and more.
CRBLASTER: Benchmarking a Cosmic-Ray Rejection Application on the Tilera 64-core TILE64 Processor Kenneth Mighell National Optical Astronomy Observatory.
Background Computer System Architectures Computer System Software.
Constructing a system with multiple computers or processors 1 ITCS 4/5145 Parallel Programming, UNC-Charlotte, B. Wilkinson. Jan 13, 2016.
PERFORMANCE OF THE OPENMP AND MPI IMPLEMENTATIONS ON ULTRASPARC SYSTEM Abstract Programmers and developers interested in utilizing parallel programming.
Information Technology (IT). Information Technology – technology used to create, store, exchange, and use information in its various forms (business data,
DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING CLOUD COMPUTING
Introduction to Parallel Computing: MPI, OpenMP and Hybrid Programming
Constructing a system with multiple computers or processors
Drill Translate the following message:
Hadoop Clusters Tess Fulkerson.
Chapter 3 Hardware and software 1.
Constructing a system with multiple computers or processors
Constructing a system with multiple computers or processors
Constructing a system with multiple computers or processors
Chapter 3 Hardware and software 1.
Types of Parallel Computers
L. Glimcher, R. Jin, G. Agrawal Presented by: Leo Glimcher
Presentation transcript:

Figure 1. Typical QLWFPC2 performance results with two WFPC2 observations of a Local Group globular cluster running on a 5-node Beowulf cluster with 1.8 GHz CPUs and Gigabit Ethernet. The blue points show actual run times for between 1 and 5 processors. The thin line shows a simple performance model based on measured cluster performance metrics (network bandwidth, disk drive bandwidth, and performance with a single CPU). The thick line shows the theoretical limit of performance. Note the current version of the QLWFPC2 algorithm already meets the ideal performance values for 1, 2, and 4 processors. QLWFPC2: Parallel-Processing Quick-Look WFPC2 Stellar Photometry based on the Message Passing Interface Kenneth John Mighell National Optical Astronomy Observatory MOTIVATION Software tools which provide quick-look data analysis with moderate accuracy (say 3 to 6 percent relative precision) could prove to be very powerful data mining tools for researchers using the U.S. National Virtual Observatory (NVO). The NVO data server may also find quick-look analysis tools to be very useful from a practical operational perspective. While quick-look stellar photometry codes are excellent tools to create metadata about the contents of CCD image data in the NVO archive, they also can provide the user with real-time analysis of NVO archival data. It is significantly faster to transmit to the NVO user a quick-look color-magnitude diagram (consisting of a few kilobytes of graphical data) than it is to transmit the entire observational data set which may consist of 10, 100, or more megabytes of data. By judiciously expending a few CPU seconds at the NVO data server, an astronomer using the NVO might well be able to determine whether a given set of observations is likely to meet their scientific needs. Quick-look analysis tools thus could not only provide a better user experience for NVO researchers but it could simultaneously allow the NVO data servers to perform their role more efficiently with better allocation of scarce computational resources. Successful quick-look analysis tools must be fast. Such tools must provide useful information in just a few seconds in order to be capable of improving the user experience with the NVO archive. QDPHOT The MXTOOLS package for IRAF ( has a fast stellar photometry task called QDPHOT (Quick & Dirty PHOTometry) which quickly produces good (~5% relative precision) CCD stellar photometry from 2 CCD images of a star field. For example, QDPHOT takes a few seconds to analyze 2 Hubble Space Telescope WFPC2 frames containing thousands of stars in Local Group star clusters (Mighell 2000: ADASS IX). Instrumental magnitudes produced by QDPHOT are converted to standard colors using the MXTOOLS task WFPC2COLOR. QLWFPC2 I have recently implemented a parallel-processing version of the combination of the QDPHOT and WFPC2COLOR tasks using the MPICH implementation of the Message Passing Interface (MPI) from the Argonne National Laboratory ( This new stand-alone multi-processing WFPC2 stellar photometry task is called QLWFPC2 (Quick Look WFPC2) and is designed to analyze two complete WFPC2 observations of Local Group star clusters in less than 5 seconds on a 5- node Beowulf cluster of Linux-based PCs with a local network based on a Gigabit Ethernet router. QLWFPC2 is written in ANSI/ISO C and uses the CFITSIO library ( software/fitsio), from NASA’s Goddard Space Flight Center, to read FITS images and the Parameter Interface Library ( std.cgi?Soft/isdc_releases_public#osa-2.0), from the INTEGRAL Science Data Center, for the IRAF parameter-file user interface. ACKNOWLEDGEMENT This work is supported by a grant from the National Aeronautics and Space Administration (NASA), Interagency Order No. S G, which was awarded by the Applied Information Systems Research Program (AISRP) of NASA’s Office of Space Science (NRA 01-OSS-01). QLWFPC2 PERFORMANCE The current implementation of QLWFPC2 was tested on a Beowulf cluster composed of 5 single 1.8-GHz AMD Athalon CPUs with 3 GB total memory connected with a Gigabit Ethernet router with 40 GB of local disk and 120 GB of NFS- mounted disk. QLWFPC2 running on 4 processors takes ~2.4 seconds (see Figure 1) to analyze WFPC2 archive data sets u37ga407r.c0.fits (F555W; 300 s) and u37ga401r.c0.fits (F814W; 300 s) of M54 which is the bright massive globular cluster near the center of the Sagittarius dwarf spheroidal galaxy. QLWFPC2 analyzed over 50,000 point source candidates and reported V, I, F555W and F814W photometry of 14,611 stars with signal-to-noise ratios of 8 or better. The analysis of these HST observations of M54 lead to the serendipitous discovery of more than 50 new bright variable stars in the central region of M54 where no variables have been reported by previous ground-based studies of variables in M54. ABSTRACT I describe a new parallel-processing stellar photometry code called QLWFPC2 which is designed to do quick-look analysis of two entire WFPC2 observations from the Hubble Space Telescope in under 5 seconds using a fast Beowulf cluster with a Gigabit Ethernet local network. QLWFPC2 running on 4 processors takes about 2.4 seconds to analyze HST WFPC2 archive observations of M54 (NGC 6715) which is the bright massive globular cluster near the center of the nearby Sagittarius dwarf spheroidal galaxy. The analysis of these HST observations of M54 lead to the serendipitous discovery of more than 50 new bright variable stars in the central region of M54 where no variables have been reported by previous ground-based studies of variables in M54. This discovery is an example of how QLWFPC2 can be used to quickly explore the time domain of observations in the HST Data Archive. Further information about QLWFPC2, including documentation and source code may be found at the following web site: RECOMMENDATIONS:   Buy fast machines. QLWFPC2 almost met the design goal of 5 seconds with a single CPU. Note that an infinite number of machines operating at < 1 GHz could not meet the 5-s design goal.   Buy fast networks. Gigabit Ethernet is ideally suited for today’s GHz-class CPUs and is now very affordable. Old networks operating at Fast Ethernet speeds will be bandwidth-bound for tasks requiring large (>1 MB) messages. The test Beowulf cluster has a latency of 90 microseconds and a sustained bandwidth of 33 MB/s for large messages.   Buy fast disks. The main disk of the test Beowulf cluster can read large FITS files at a respectable ~30 MB/s with 7200 rpm disks. Nevertheless, reading 2 WFPC2 images still takes 0.6 s to read which is a significant fraction of the measured total execution times.