Alain Roy Computer Sciences Department University of Wisconsin-Madison ClassAds: Present and Future.

Slides:

Advertisements

Similar presentations

Greg Thain Computer Sciences Department University of Wisconsin-Madison Condor Parallel Universe.

Advertisements

Matchmaking in the Condor System Rajesh Raman Computer Sciences Department University of Wisconsin-Madison

WP 1 Grid Workload Management Massimo Sgaravatto INFN Padova.

Dr. David Wallom Use of Condor in our Campus Grid and the University September 2004.

1 Draft of a Matchmaking Service Chuang liu. 2 Matchmaking Service Matchmaking Service is a service to help service providers to advertising their service.

Advanced Topics COMP163: Database Management Systems University of the Pacific December 9, 2008.

Alain Roy Computer Sciences Department University of Wisconsin-Madison 23-June-2002 Introduction to Condor.

DataGrid Kimmo Soikkeli Ilkka Sormunen. What is DataGrid? DataGrid is a project that aims to enable access to geographically distributed computing power.

The Condor Data Access Framework GridFTP / NeST Day 31 July 2001 Douglas Thain.

Jaeyoung Yoon Computer Sciences Department University of Wisconsin-Madison Virtual Machine Universe in.

Jim Basney Computer Sciences Department University of Wisconsin-Madison Managing Network Resources in.

Derek Wright Computer Sciences Department, UW-Madison Lawrence Berkeley National Labs (LBNL)

Condor Project Computer Sciences Department University of Wisconsin-Madison Virtual Machines in Condor.

Zach Miller Computer Sciences Department University of Wisconsin-Madison What’s New in Condor.

QCDgrid Technology James Perry, George Beckett, Lorna Smith EPCC, The University Of Edinburgh.

Miron Livny Computer Sciences Department University of Wisconsin-Madison Harnessing the Capacity of Computational.

Alain Roy Computer Sciences Department University of Wisconsin-Madison An Introduction To Condor International.

Todd Tannenbaum Computer Sciences Department University of Wisconsin-Madison What’s New in Condor.

1 Todd Tannenbaum Department of Computer Sciences University of Wisconsin-Madison

An Introduction to High-Throughput Computing Rob Quick OSG Operations Officer Indiana University Some Content Contributed by the University of Wisconsin.

An Introduction to High-Throughput Computing Monday morning, 9:15am Alain Roy OSG Software Coordinator University of Wisconsin-Madison.

Workload Management WP Status and next steps Massimo Sgaravatto INFN Padova.

Installing and Managing a Large Condor Pool Derek Wright Computer Sciences Department University of Wisconsin-Madison

1 HawkEye A Monitoring and Management Tool for Distributed Systems Todd Tannenbaum Department of Computer Sciences University of.

Hao Wang Computer Sciences Department University of Wisconsin-Madison Security in Condor.

Peter Keller Computer Sciences Department University of Wisconsin-Madison Quill Tutorial Condor Week.

Grid Computing I CONDOR.

QCDGrid Progress James Perry, Andrew Jackson, Stephen Booth, Lorna Smith EPCC, The University Of Edinburgh.

Grid Workload Management & Condor Massimo Sgaravatto INFN Padova.

DataGrid WP1 Massimo Sgaravatto INFN Padova. WP1 (Grid Workload Management) Objective of the first DataGrid workpackage is (according to the project "Technical.

1 The Roadmap to New Releases Todd Tannenbaum Department of Computer Sciences University of Wisconsin-Madison

Hunter of Idle Workstations Miron Livny Marvin Solomon University of Wisconsin-Madison URL:

Miron Livny Computer Sciences Department University of Wisconsin-Madison Welcome and Condor Project Overview.

Condor Project Computer Sciences Department University of Wisconsin-Madison A Scientist’s Introduction.

- Distributed Analysis (07may02 - USA Grid SW BNL) Distributed Processing Craig E. Tull HCG/NERSC/LBNL (US) ATLAS Grid Software.

Condor: High-throughput Computing From Clusters to Grid Computing P. Kacsuk – M. Livny MTA SYTAKI – Univ. of Wisconsin-Madison

Nick LeRoy & Jeff Weber Computer Sciences Department University of Wisconsin-Madison Managing.

Alain Roy Computer Sciences Department University of Wisconsin-Madison I/O Access in Condor and Grid.

1 The Roadmap to New Releases Todd Tannenbaum Department of Computer Sciences University of Wisconsin-Madison

Derek Wright Computer Sciences Department University of Wisconsin-Madison MPI Scheduling in Condor: An.

Todd Tannenbaum Computer Sciences Department University of Wisconsin-Madison Quill / Quill++ Tutorial.

Alain Roy Computer Sciences Department University of Wisconsin-Madison Packaging & Testing: NMI & VDT.

Tevfik Kosar Computer Sciences Department University of Wisconsin-Madison Managing and Scheduling Data.

Derek Wright Computer Sciences Department University of Wisconsin-Madison Condor and MPI Paradyn/Condor.

Derek Wright Computer Sciences Department University of Wisconsin-Madison New Ways to Fetch Work The new hook infrastructure in Condor.

Alain Roy Computer Sciences Department University of Wisconsin-Madison Condor & Middleware: NMI & VDT.

Nick LeRoy Computer Sciences Department University of Wisconsin-Madison Hawkeye.

Peter F. Couvares Computer Sciences Department University of Wisconsin-Madison Condor DAGMan: Managing Job.

Greg Thain Computer Sciences Department University of Wisconsin-Madison Configuring Quill Condor Week.

An Introduction to High-Throughput Computing With Condor Tuesday morning, 9am Zach Miller University of Wisconsin-Madison.

Scheduling & Resource Management in Distributed Systems Rajesh Rajamani, May 2001.

Nicholas Coleman Computer Sciences Department University of Wisconsin-Madison Distributed Policy Management.

Peter Couvares Computer Sciences Department University of Wisconsin-Madison Condor DAGMan: Introduction &

Jaime Frey Computer Sciences Department University of Wisconsin-Madison What’s New in Condor-G.

GIIS Implementation and Requirements F. Semeria INFN European Datagrid Conference Amsterdam, 7 March 2001.

Matthew Farrellee Computer Sciences Department University of Wisconsin-Madison Condor and Web Services.

Douglas Thain, John Bent Andrea Arpaci-Dusseau, Remzi Arpaci-Dusseau, Miron Livny Computer Sciences Department, UW-Madison Gathering at the Well: Creating.

Grid Workload Management (WP 1) Massimo Sgaravatto INFN Padova.

An Introduction to High-Throughput Computing Monday morning, 9:15am Alain Roy OSG Software Coordinator University of Wisconsin-Madison.

Todd Tannenbaum Computer Sciences Department University of Wisconsin-Madison Condor NT Condor ported.

Condor Project Computer Sciences Department University of Wisconsin-Madison Running Interpreted Jobs.

Jaime Frey Computer Sciences Department University of Wisconsin-Madison Condor and Virtual Machines.

Condor Project Computer Sciences Department University of Wisconsin-Madison Condor Introduction.

Condor on Dedicated Clusters Peter Couvares and Derek Wright Computer Sciences Department University of Wisconsin-Madison

Chuang Liu, Lingyun Yang, Dave Angulo, Ian Foster

A Distributed Policy Scenario

Basic Grid Projects – Condor (Part I)

Wide Area Workload Management Work Package DATAGRID project

Frieda meets Pegasus-WMS

Production Manager Tools (New Architecture)

Presentation transcript:

Alain Roy Computer Sciences Department University of Wisconsin-Madison ClassAds: Present and Future

What Are ClassAds? › A ClassAd maps attributes to expressions › Expressions  Constants: strings, numbers, etc.  Expressions: other.Memory > 600M  Lists: { “roy”, “pfc”, “melski” }  Other ClassAds › Powerful tool for grid computing  Semi-structured (you pick your structure)  Matchmaking

ClassAd Example [ Type = “Job”; Owner = “roy”; Universe = “Standard”; Requirements = (other.OpSys == “Linux” && other.DiskSpace > 140M); Rank = (other.DiskSpace > 300M ? 10 : 1); ClusterID = 12314; JobID = 0; Env = “”; … ] (To see a real ClassAd, try: condor_q –l or condor_status –l )

ClassAd Matchmaking [ Type = “Job”; Owner = “roy”; Requirements = (other.OpSys == “Linux” && other.DiskSpace > 140M); Rank = (other.DiskSpace > 300M ? 10 : 1); ] [ Type = “Machine”; OpSys = “Linux”; DiskSpace = 500M; AllowedUsers = {“roy”, “melski”, “pfc”}; Requirements = (IsMember(other.Owner, AllowedUsers); ]

ClassAds Are Open Source › Library GNU Public License (LGPL) › Complete source code included  Library code  Test program › Available from: › Version 0.9.3

Who Uses ClassAds? › Condor › European Data Grid › NeST › Web site › …You?

ClassAd User: Condor › ClassAds describe jobs and machines › Matchmaking figures out what jobs run on which machines › DAGMan will soon internally represent DAGs as ClassAds

ClassAd User: EU Datagrid › JDL: ClassAd schema to describe jobs/machines › ResourceBroker: matches jobs to machines

ClassAd User: NeST › NeST is a storage appliance › NeST uses ClassAd collections for persistent storage of:  User Information  File meta-data  Disk Information  Lots (storage space allocations)

ClassAd User: Web Site › Web-based application in Germany › User actions (transitions) are constrained › Constraints expressed through ClassAds

What’s New? › ClassAd collections › XML representation

New: ClassAd Collections › A collection is a “bag” of ClassAds › Can have views on a collection:  Sub view: contains ClassAds matching a constraint, like (memory > 300M)  Partition view: contains sub view for each value of a constraint

Collection Views Root View (all machine ClassAds)OpSys == “Linux”Memory > 600MOwner: royOwner: pfcOwner: ned

New: XML Representation › Does not replace original representation › Sample: Job Roy (other.Memory > 600M)

What’s Coming? › 1.0—real soon now  Smoothing out minor issues  Updated documentation › After 1.0  Set-extensions (with University of Chicago)  Gang-matching, indexing  Recoverable transmission of ClassAds (two- phase commit)  Access from multiple languages (SWIG)

Cool! Tell Me More! Q&A: 9:00am-noon, Wed., Room ClassAd users mailing list