The Design of a Grid Computing System for Drug Discovery and Design

Slides:



Advertisements
Similar presentations
Topology The physical topology of a network refers to the configuration of cables, computers and other peripherals. The main types of network topologies.
Advertisements

Lecture 12: MapReduce: Simplified Data Processing on Large Clusters Xiaowei Yang (Duke University)
Computer networks Fundamentals of Information Technology Session 6.
Chapter 5: Server Hardware and Availability. Hardware Reliability and LAN The more reliable a component, the more expensive it is. Server hardware is.
Clydesdale: Structured Data Processing on MapReduce Jackie.
Dr. David Wallom Use of Condor in our Campus Grid and the University September 2004.
A P2P Distributed Computing Platform: (Wed) C.Y. ‘Connor’ Park KISTI Supercomputing Center.
P2P Distributed Computing Platform: C.Y. ‘Connor’ Park KISTI Supercomputing Center.
Rheeve: A Plug-n-Play Peer- to-Peer Computing Platform Wang-kee Poon and Jiannong Cao Department of Computing, The Hong Kong Polytechnic University ICDCSW.
Efficient Content Location Using Interest-based Locality in Peer-to-Peer Systems Presented by: Lin Wing Kai.
Rutgers PANIC Laboratory The State University of New Jersey Self-Managing Federated Services Francisco Matias Cuenca-Acuna and Thu D. Nguyen Department.
MASPLAS ’02 Creating A Virtual Computing Facility Ravi Patchigolla Chris Clarke Lu Marino 8th Annual Mid-Atlantic Student Workshop On Programming Languages.
Introduction  What is an Operating System  What Operating Systems Do  How is it filling our life 1-1 Lecture 1.
CPS216: Advanced Database Systems (Data-intensive Computing Systems) How MapReduce Works (in Hadoop) Shivnath Babu.
07/14/08. 2 Points Introduction. Cluster and Supercomputers. Cluster Types and Advantages. Our Cluster. Cluster Performance. Cluster Computer for Basic.
Chapter 2 Computer Clusters Lecture 2.1 Overview.
A Workflow-Aware Storage System Emalayan Vairavanathan 1 Samer Al-Kiswany, Lauro Beltrão Costa, Zhao Zhang, Daniel S. Katz, Michael Wilde, Matei Ripeanu.
Distributed Systems Early Examples. Projects NOW – a Network Of Workstations University of California, Berkely Terminated about 1997 after demonstrating.
I-Cluster ACM-SIGOPS JTE Cluster Computing Bruno Richard Research Program Manager HP Labs Grenoble.
RUNNING PARALLEL APPLICATIONS BEYOND EP WORKLOADS IN DISTRIBUTED COMPUTING ENVIRONMENTS Zholudev Yury.
Test Of Distributed Data Quality Monitoring Of CMS Tracker Dataset H->ZZ->2e2mu with PileUp - 10,000 events ( ~ 50,000 hits for events) The monitoring.
Service Architecture of Grid Faults Diagnosis Expert System Based on Web Service Wang Mingzan, Zhang ziye Northeastern University, Shenyang, China.
FESR Consorzio COMETA Grid Introduction and gLite Overview Corso di formazione sul Calcolo Parallelo ad Alte Prestazioni (edizione.
Grid Appliance – On the Design of Self-Organizing, Decentralized Grids David Wolinsky, Arjun Prakash, and Renato Figueiredo ACIS Lab at the University.
2: Application Layer1 Chapter 2 outline r 2.1 Principles of app layer protocols r 2.2 Web and HTTP r 2.3 FTP r 2.4 Electronic Mail r 2.5 DNS r 2.6 Socket.
Evaluation of Agent Teamwork High Performance Distributed Computing Middleware. Solomon Lane Agent Teamwork Research Assistant October 2006 – March 2007.
Mainframe (Host) - Communications - User Interface - Business Logic - DBMS - Operating System - Storage (DB Files) Terminal (Display/Keyboard) Terminal.
Grid Computing at Yahoo! Sameer Paranjpye Mahadev Konar Yahoo!
What is Triana?. GAPGAP Triana Distributed Work-flow Network Action Commands Workflow, e.g. BPEL4WS Triana Engine Triana Controlling Service (TCS) Triana.
Transparently Gathering Provenance with Provenance Aware Condor Christine Reilly and Jeffrey Naughton Department of Computer Sciences University of Wisconsin.
Virtual Private Grid (VPG) : A Command Shell for Utilizing Remote Machines Efficiently Kenji Kaneda, Kenjiro Taura, Akinori Yonezawa Department of Computer.
Scheduling Interactive Tasks in the Grid-based Systems M. Okoń, M. Lawenda, N. Meyer, D. Stokłosa, T. Rajtar, D. Kaliszan, M. Stroiński TERENA Networking.
Computer Networking P2P. Why P2P? Scaling: system scales with number of clients, by definition Eliminate centralization: Eliminate single point.
Research of P2P Architecture based on Cloud Computing Speaker : 吳靖緯 MA0G0101.
Supporting Privacy Protection in Personalized Web Search.
DynamicMR: A Dynamic Slot Allocation Optimization Framework for MapReduce Clusters Nanyang Technological University Shanjiang Tang, Bu-Sung Lee, Bingsheng.
Timeshared Parallel Machines Need resource management Need resource management Shrink and expand individual jobs to available sets of processors Shrink.
Data Communications and Networks Chapter 9 – Distributed Systems ICT-BVF8.1- Data Communications and Network Trainer: Dr. Abbes Sebihi.
Super Scaling PROOF to very large clusters Maarten Ballintijn, Kris Gulbrandsen, Gunther Roland / MIT Rene Brun, Fons Rademakers / CERN Philippe Canal.
+ Support multiple virtual environment for Grid computing Dr. Lizhe Wang.
Load Rebalancing for Distributed File Systems in Clouds.
Computational Sciences at Indiana University an Overview Rob Quick IU Research Technologies HTC Manager.
1 An unattended, fault-tolerant approach for the execution of distributed applications Manuel Rodríguez-Pascual, Rafael Mayo-García CIEMAT Madrid, Spain.
Distributed Caching and Adaptive Search in Multilayer P2P Networks Chen Wang, Li Xiao, Yunhao Liu, Pei Zheng The 24th International Conference on Distributed.
Instrument Elements at Work Imperial Brunel Elettra IBM CNIT IASA IMAA GRNet UniUD INFN Monitor GUI Network Information Provider Instrument Element Network.
A Web Based Job Submission System for a Physics Computing Cluster David Jones IOP Particle Physics 2004 Birmingham 1.
INFSO-RI Enabling Grids for E-sciencE EGEE is a project funded by the European Union under contract IST Report from.
Lecture 3 – MapReduce: Implementation CSE 490h – Introduction to Distributed Computing, Spring 2009 Except as otherwise noted, the content of this presentation.
A very short introduction to Project Supercomputing Center, KISTI Chan Yeol (Connor) Park
Consulting Services JobScheduler Architecture Decision Template
An example of peer-to-peer application
Volunteer Computing for Science Gateways
Introduction to Distributed Platforms
Grid Computing.
Introduction to HDFS: Hadoop Distributed File System
NGS computation services: APIs and Parallel Jobs
MapReduce Simplied Data Processing on Large Clusters
Patrick Dreher Research Scientist & Associate Director
Comparison of LAN, MAN, WAN
Cluster Resource Management: A Scalable Approach
Peer to Peer Information Retrieval
Chapter 17: Database System Architectures
CGS 3763 Operating Systems Concepts Spring 2013
Basic Grid Projects – Condor (Part I)
CLUSTER COMPUTING.
Tiers vs. Layers.
Distributed computing deals with hardware
User interaction and workflow management in Grid enabled e-VLBI experiments Dominik Stokłosa Poznań Supercomputing and Networking Center, Supercomputing.
Decentralised Middleware and Workflow Enactment
GGF10 Workflow Workshop Summary
Presentation transcript:

The Design of a Grid Computing System for Drug Discovery and Design David Gobaud Computational Drug Discovery Stanford University 14 March 2006

Outline Overview Network Topology Structure Summary Resource Management Agent Job Submission Client Scheduler Node Execution Node Experimental Results Questions

Overview Novel Grid computing system for drug discovery and design (DDG) Use idle resources donated by nodes over the internet P2P technology Hybrid resource management architecture Avoid master/slave problems

Network Topology Centralized + Decentralized P2P Clusters that wish to compute join DDG and become execution nodes Grouped into Virtual Organizations (VO) by structures and interests Communicate via Resource Management Agents (RMA) in P2P mode One peer fails – other peers can compensate Each local RMA captures and maintains resource info of a VO

Structure Summary End-users submit protein molecule files to DDG via web portal End-users create job request template for a new application Specify QoS requirements Job submitted to Scheduler Node by Submission Node DDG provides a distributed repository where execution nodes publish their real-time workload info

Structure Summary Scheduler Node Divide job into many parallel sub-jobs Query repository to find and allocate execution nodes for sub-jobs to satisfy QoS requirements Combine results from sub-jobs into whole docking result Return result to submission node which informs the user the job is complete

High-Level Workflow of DDG

Experimental Results Feasibility and performance evaluated by protein molecules docking experiments Many DBs used – Specs, ACD, CNPP, and NCBI 20 PC LAN 1Ghz Pentium IVs 256 Mb RAM 40 Gb hard drives 100 Mb/sec Ethernet SGI Origin 3800 cluster of 64 processors SUNWAY cluster of 32 processors SUNWAY cluster of 64 processors SUNWAY cluster of 256 processors

Experimental Results – Running Time

Experimental Results – Robustness 20 PCs each receiving 100 protein molecules to dock Some PCs randomly shut down After P2P network stabilized fraction of molecules that could not be processed measures Molecule docking failure rate almost equal to nodes failure rate as expected No significant resource scheduling failure -> Robust

Questions Why are failed docking computations not rescheduled? What type of network topology does Folding@Home use?