Image Builder/Data Handling System Design Tao Qian UIUC April 10, 2007.

Slides:

Advertisements

Similar presentations

Exercises and Solutions Lecture 1

Advertisements

1 Communication in Distributed Systems REKs adaptation of Tanenbaums Distributed Systems Chapter 2.

Multiple Processor Systems Chapter Multiprocessors 8.2 Multicomputers 8.3 Distributed systems.

An 8051 Based Web Server Project by Mason Kidd Advised by Dr. Schertz.

Multiple Processor Systems 8.1 Multiprocessors 8.2 Multicomputers 8.3 Distributed systems.

Bob Baker Communications Bob Baker September 1999.

Aparna SrikantaswamyAPI for Multipath Routing1 An API for Multipath Routing Aparna Srikanta Swamy CS522 Semester Project 12/05/01.

COMPUTER NETWORKS.

© 2007 Cisco Systems, Inc. All rights reserved.Cisco Public 1 Version 4.0 Communicating over the Network Network Fundamentals – Chapter 2.

Ch. 31 Q and A IS 333 Spring 2015 Victor Norman. SNMP, MIBs, and ASN.1 SNMP defines the protocol used to send requests and get responses. MIBs are like.

Network File System (NFS) in AIX System COSC513 Operation Systems Instructor: Prof. Anvari Yuan Ma SID:

TCP/IP protocols Communication over Internet is mostly TCP/IP (Transmission Control Protocol over Internet Protocol) TCP/IP "stack" is software which allows.

Networking and the Internet © 2007 Pearson Addison-Wesley. All rights reserved.

Lecture 2 TCP/IP Protocol Suite Reference: TCP/IP Protocol Suite, 4 th Edition (chapter 2) 1.

1 Chapter Client-Server Interaction. 2 Functionality  Transport layer and layers below  Basic communication  Reliability  Application layer.

Straw electronics Straw Readout Board (SRB). Full SRB - IO Handling 16 covers – Input 16*2 links 400(320eff) Mbits/s Control – TTC – LEMO – VME Output.

Fayoum University Faculty of Engineering Electrical Engineering Department E-voting system Using Smart Card Under the supervision Of: Dr. Magdy Amer.

Socket Swapping for efficient distributed communication between migrating processes MS Final Defense Praveen Ramanan 12 th Dec 2002.

Internet Addresses. Universal Identifiers Universal Communication Service - Communication system which allows any host to communicate with any other host.

Three fundamental concepts in computer security: Reference Monitors: An access control concept that refers to an abstract machine that mediates all accesses.

LiveCycle Data Services Introduction Part 2. Part 2? This is the second in our series on LiveCycle Data Services. If you missed our first presentation,

Jozef Goetz, Application Layer PART VI Jozef Goetz, Position of application layer The application layer enables the user, whether human.

Cluster Computers. Introduction Cluster computing –Standard PCs or workstations connected by a fast network –Good price/performance ratio –Exploit existing.

Follow the Data Data (and information) move from place to place in computer systems and networks. As it moves it changes form frequently. This story describes.

CS4273: Distributed System Technologies and Programming Lecture 13: Review.

Outline Overview Video Format Conversion Connection with An authentication Streaming media Transferring media.

Copyright 2002, S.D. Personick. All Rights Reserved.1 Telecommunications Networking II Topic 24 Internet Wrap-up Dr. Stewart D. Personick Drexel University.

Electrical and Computer Engineering Dynamic Advertising System Midway Design Review – December 7, 2009 Team Zink Nicholas Cipriano, Ali Jameel, Jeremy.

Project 1. Goals  Write a simple TCP/IP client that supports a specific protocol  The server is running right now on login.ccs.neu.edu:27993  If your.

Chapter 10 Connecting Computers. FIGURE 10.0.F01: The Xerox Alto. Photographed by Dr. Richard Smith at the Computer History Museum, California.

ECE200 – Computer Organization Chapter 9 – Multiprocessors.

1 Version 3.1 Module 1 WANs and Routers. 2 Version 3.1 WANS WAN operates at the physical layer and the data link layer of the OSI reference model. Provide.

Chapter 15 – Part 2 Networks The Internal Operating System The Architecture of Computer Hardware and Systems Software: An Information Technology Approach.

Application Block Diagram III. SOFTWARE PLATFORM Figure above shows a network protocol stack for a computer that connects to an Ethernet network and.

802.11n Sniffer Design Overview Vladislav Mordohovich Igor Shtarev Luba Brouk.

Copyright Jim Martin The World Wide Web Dr. Jim Martin

Registers are always 32 bit / 4 byte memory is accessed directly when memory cannot be accessed kind of flag is on.

CE Operating Systems Lecture 13 Linux/Unix interprocess communication.

Robin HJ & R. Hughes-Jones Manchester Sep 1999 Gigabit Ethernet in Ptolemy Status Sep 99 : Stars that exist : –GigEChipTranslate between GigEPacket and.

1 Client-Server Interaction. 2 Functionality Transport layer and layers below –Basic communication –Reliability Application layer –Abstractions Files.

Image Builder Design Tao Qian UIUC April 10, 2007.

18-19 July, 2002Correlator Backend System OverviewTom Morgan 1 Correlator Backend System Overview Tom Morgan, NRAO.

Field Programmable Port Extender (FPX) 1 NCHARGE: Remote Management of the Field Programmable Port Extender (FPX) Todd Sproull Washington University, Applied.

Source Controller software Ianos Schmidt The University of Iowa.

“OpenCALEA” Pragmatic Cost Effective CALEA Compliance Manish Karir, Merit - Research and Development.

Linux Operations and Administration

Status & development of the software for CALICE-DAQ Tao Wu On behalf of UK Collaboration.

Simple Socket Server m Yumiko Kimezawa September 19, 20121RPS.

Networks and the Internet Topic 3. Three Important Networking Technologies Networks, Internet, WWW.

438 labs. Labs Lab 1: Refreshing C and Linux Programming 2 Lab 1: Refreshing C and Linux Programming 2 – First: Getting familiar with development.

Cluster Computers. Introduction Cluster computing –Standard PCs or workstations connected by a fast network –Good price/performance ratio –Exploit existing.

FTOP: A library for fault tolerance in a cluster R. Badrinath Rakesh Gupta Nisheeth Shrivastava.

COMPASS DAQ Upgrade I.Konorov, A.Mann, S.Paul TU Munich M.Finger, V.Jary, T.Liska Technical University Prague April PANDA DAQ/FEE WS Игорь.

iperf a gnu tool for IP networks

Introduction to Networks

Tiny http client and server

Chapter 21 Address Mapping

“OpenCALEA” Pragmatic Cost Effective CALEA Compliance

Distributed Systems.

Lecture 6: TCP/IP Networking By: Adal Alashban

Introduction to Networks

Ethernet : Framing and Addressing

Introduction to Opnet Mobile Networks Introduction to Opnet

Correlator – Backend System Overview

Transport layer API: Socket Programming

Follow the Data Data (and information) move from place to place in computer systems and networks. As it moves it changes form frequently. This story.

Network and the internet

Designing a PC Farm to Simultaneously Process Separate Computations Through Different Network Topologies Patrick Dreher MIT.

Single path routing in most of the servers

Cluster Computers.

Presentation transcript:

Image Builder/Data Handling System Design Tao Qian UIUC April 10, 2007

SISPI Workshop Tao Qian 2 Basic Structure Image Builder/Data Handling System will consist of a Linux farm with a supervisor layer: DHS supervisor layer Coordinate the connections between PANs and DHS machine DHS Collect pixel data from PANs Combine pixel data with metadata Form FITS file and deliver file to downstream

SISPI Workshop Tao Qian 3 Basic Idea 1.DHS will consist of a set of Linux boxes 2.At any time, only one DHS machine connects with PANs 3.Connection between PANs and DHS are through sockets 4.DHS machine is server 5.PANs are clients 6.Clients push the pixel data and metadata into DHS 7.DHS stuffs the data into the shared memory

SISPI Workshop Tao Qian 4 Primary Processes in DHS 1.collector: Listens for connection from clients (PANs) Stuffs the pixel data and configuration into the shared memory Listens for commands from supervisor layer, reporting the current status (may need message bus) 2.sharedMemoryManager: Organize the shared memory cache Provide access pointer for other processes 3.pixelFeed Gather global metadata from database Assemble image from the shared memory into FITS format Send file to DTS through sockets

SISPI Workshop Tao Qian 5 Data Flow Testing (1) A toy model has been developed Client on (storm.des.hep.uiuc.edu) Pentium 4 Linux machine Server on (cyclone.des.hep.uiuc.edu) Pentium 4 Linux machine Transferring data from a full PAN: numCcd = 9; nx = 2248; ny = integer = 4 bytes Total pixels = 9 * 2248 * 4146 *4 = 83,881,872 * 4 = 335, 527, 488 bytes The data is equivalent to 9 CCD pixel data with 4 bytes per pixel.

SISPI Workshop Tao Qian 6 Data Flow Testing (2) Packet size (bytes) ActionTrial 1 (sec) Trial 2 (sec) Trial 3 (sec) Averag e (sec) Average Rate 4096 Recv MBytes Mbits &Saving Recv Mbytes 90.7 Mbits &Saving Ethernet card 10Mb/100Mb on both ends Receive data : 335, 527, 488 bytes Total time = (30*2)/PAN x 5 PANs = 300 seconds Need faster Ethernet switch !!!

SISPI Workshop Tao Qian 7 Next Step Modify the client code and implant it into MONSOON –Link the correct dhsUtil.lib in MONSOON which will send data to DHS (may need to modify the monsoon library?) Develop the server –Develop based on DHS software from NEWFIRM Design DHS supervisor layer