Introduction to EMF Server Communication and Cases Beta Testing November 4, 2009.

Slides:



Advertisements
Similar presentations
Parallel Scripting on Beagle with Swift Ketan Maheshwari Postdoctoral Appointee (Argonne National.
Advertisements

1 OBJECTIVES To generate a web-based system enables to assemble model configurations. to submit these configurations on different.
The Web Warrior Guide to Web Design Technologies
Bookshelf.EXE - BX A dynamic version of Bookshelf –Automatic submission of algorithm implementations, data and benchmarks into database Distributed computing.
Lesson 10-Controlling User Processes. Overview Managing and processing processes. Managing jobs. Exiting/quitting when jobs have been stopped.
Oasys 9.2. Oasys 9.2 – What is new? Multiple Models and Automatic Processes –Generate a series of models in Primer in one operation –Submit list of models.
UNIX Chapter 01 Overview of Operating Systems Mr. Mohammad A. Smirat.
A simple PHP application We are going to develop a simple PHP application with a Web interface. The user enters two numbers and the application returns.
Spatial Analysis, Geoprocessing,
Guide To UNIX Using Linux Third Edition
Asynchronous Solution Appendix Eleven. Training Manual Asynchronous Solution August 26, 2005 Inventory # A11-2 Chapter Overview In this chapter,
Guide to Linux Installation and Administration, 2e1 Chapter 6 Using the Shell and Text Files.
70-290: MCSE Guide to Managing a Microsoft Windows Server 2003 Environment, Enhanced Chapter 8: Implementing and Managing Printers.
70-290: MCSE Guide to Managing a Microsoft Windows Server 2003 Environment Chapter 8: Implementing and Managing Printers.
Simple PHP application. A simple application We are going to develop a simple PHP application with a Web interface. The user enters two numbers and the.
CGI Programming: Part 1. What is CGI? CGI = Common Gateway Interface Provides a standardized way for web browsers to: –Call programs on a server. –Pass.
Testing Processes of Web Applications Susan Cohen.
Jaeyoung Yoon Computer Sciences Department University of Wisconsin-Madison Virtual Machine Universe in.
DIRAC API DIRAC Project. Overview  DIRAC API  Why APIs are important?  Why advanced users prefer APIs?  How it is done?  What is local mode what.
ALEPH version Services / Task Manager South Dakota Library Network 1200 University, Unit 9672 Spearfish, SD © South Dakota Library.
National Alliance for Medical Image Computing Grid Computing with BatchMake Julien Jomier Kitware Inc.
1 port BOSS on Wenjing Wu (IHEP-CC)
Rsv-control Marco Mambelli – Site Coordination meeting October 1, 2009.
HTCondor and BOINC. › Berkeley Open Infrastructure for Network Computing › Grew out of began in 2002 › Middleware system for volunteer computing.
A Distributed Computing System Based on BOINC September - CHEP 2004 Pedro Andrade António Amorim Jaime Villate.
Guide To UNIX Using Linux Fourth Edition
Robert Fourer, Jun Ma, Kipp Martin Copyright 2006 An Enterprise Computational System Built on the Optimization Services (OS) Framework and Standards Jun.
Business Unit or Product Name © 2007 IBM Corporation Introduction of Autotest Qing Lin.
Copyright © 2007, Oracle. All rights reserved. Managing Concurrent Requests.
Java CGI Lecture notes by Theodoros Anagnostopoulos.
Unix Tutorial for FreeSurfer Users. Helpful To Know FreeSurfer Tutorial Wiki:
Galaxy for Bioinformatics Analysis An Introduction TCD Bioinformatics Support Team Fiona Roche, PhD Date: 31/08/15.
COMP3019 Coursework: Introduction to GridSAM Steve Crouch School of Electronics and Computer Science.
1 Dryad Distributed Data-Parallel Programs from Sequential Building Blocks Michael Isard, Mihai Budiu, Yuan Yu, Andrew Birrell, Dennis Fetterly of Microsoft.
SCI-BUS is supported by the FP7 Capacities Programme under contract nr RI Creating the Autodock gateway from WS-PGRADE/gUSE and making it cloud-enabled.
NGS data analysis CCM Seminar series Michael Liang:
The Professional Open Source™ Company CLI Shell JBossNetwork Enterprise Manager Command Line Interface.
INFSO-RI Enabling Grids for E-sciencE Workload Management System Mike Mineter
Unix Tutorial for FreeSurfer Users. Helpful To Know FreeSurfer Tutorial Wiki:
1 The New Emissions Modeling Framework System Emissions Modeling Team Marc Houyoux, Madeleine Strum, Rich Mason, Norm Possiel David Misenheimer, Darryl.
Basic Shell Scripting - Part 1 Objective - Learn to: Read Start-up Files Edit Start-up Files Modify Your User Environment Communicate with Users Write.
The european ITM Task Force data structure F. Imbeaux.
A Brief Documentation.  Provides basic information about connection, server, and client.
ADTs and C++ Classes Classes and Members Constructors The header file and the implementation file Classes and Parameters Operator Overloading.
Using the EMF for Emissions Sensitivities and for CMAQ Alison M. Eyth Alexis Zubrow Qun He UNC Institute for the Environment October, 2008.
Lesson 1 Operating Systems, Part 1. Objectives Describe and list different operating systems Understand file extensions Manage files and folders.
C o n f i d e n t i a l 1 Course: BCA Semester: III Subject Code : BC 0042 Subject Name: Operating Systems Unit number : 1 Unit Title: Overview of Operating.
Getting started DIRAC Project. Outline  DIRAC information system  Documentation sources  DIRAC users and groups  Registration with DIRAC  Getting.
1 Chapter Overview Defining Operators Creating Jobs Configuring Alerts Creating a Database Maintenance Plan Creating Multiserver Jobs.
LING 408/508: Programming for Linguists Lecture 18 November 2 nd.
EGEE-III INFSO-RI Enabling Grids for E-sciencE Ricardo Rocha CERN (IT/GS) EGEE’08, September 2008, Istanbul, TURKEY Experiment.
Lesson 3-Touring Utilities and System Features. Overview Employing fundamental utilities. Linux terminal sessions. Managing input and output. Using special.
Shell Interface Shell Interface Functions Data. Graphical Interface Graphical Interface Command-line Interface Command-line Interface Experiments Private.
4.3 Functions. Functions Last class we talked about the idea and organization of a function. Today we talk about how to program them.
LSF Universus By Robert Stober Systems Engineer Platform Computing, Inc.
Recent Enhancements to Quality Assurance and Case Management within the Emissions Modeling Framework Alison Eyth, R. Partheepan, Q. He Carolina Environmental.
BIF713 Introduction to Linux. Agenda Getting Started: Using Linux Unix and Linux - Structure / Features Elements of the Linux Philosophy Linux Command.
Copyright 2007, Information Builders. Slide 1 iWay Web Services and WebFOCUS Consumption Michael Florkowski Information Builders.
EOVSA Data and Database System J. McTiernan EOVSA Technical DesignMeeting 7-Nov-2011.
Mandatory Assignment INF3190. Part 1: Client-server communication via TCP Develop a client-server application in C which allows a client to send UNIX.
9 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Scheduling and Bursting Reports.
Lesson 8-Specifying Instructions to the Shell. Overview An overview of shell. Execution of commands in a shell. Shell command-line expansion. Customizing.
Mobile Analyzer A Distributed Computing Platform Juho Karppinen Helsinki Institute of Physics Technology Program May 23th, 2002 Mobile.
Using Galaxy to build and run data processing pipelines Jelle Scholtalbers / Charles Girardot GBCS Genome Biology Computational Support.
Chapter 4: server services. The Complete Guide to Linux System Administration2 Objectives Configure network interfaces using command- line and graphical.
ScienceTools Linux Instructions Windows Instructions.
Geant4 GRID production Sangwan Kim, Vu Trong Hieu, AD At KISTI.
University of Kansas Department of Electrical Engineering and Computer Science Dr. Susan Gauch April 21, 2005 I T T C Introduction to Web Technologies.
PHP and Forms.
Presentation transcript:

Introduction to EMF Server Communication and Cases Beta Testing November 4, 2009

2 EMF Case Manager  The Case Manager uses SMOKE programs to create emissions modeling outputs.  Case: Set of Jobs, Inputs, and Parameters that fully describe a SMOKE or other model run

3 EMF Cases – Summary tab Sector: Emissions modeling sector(e.g., ptipm, ptnonipm)

4 EMF Cases – Jobs tab Job: Program or script with settings and arguments needed to run a script

5 EMF Cases – Inputs tab Input: Dataset used by one or more Jobs

6 EMF Cases – Parameters tab Parameter: Setting, usually an environment variable, used by one or more Jobs

7 Key EMF Definitions  Case: Set of Jobs, Inputs, and Parameters that fully describe a SMOKE run  Job: Program or script with settings and arguments needed to run a script  Input: Dataset used by one or more Jobs  Parameter: Setting, usually an environment variable, used by one or more Jobs

8 Case Management Overview Run scripts Dataset Manager Case Manager SMOKE datasets in PostgreSQL Case Inputs: Datasets to use for runs Jobs: Run scripts Parameters: SMOKE and script settings SMOKE Tell Dataset Manager to export Inputs as ASCII SMOKE files Generate wrapper scripts with Inputs and Parameters defined as Environment Variables Submit Job to a run queue on a UNIX/Linux computer Run Outputs: List of results (e.g., AQ model inputs) EMF utility to register outputs History: Who, when, what run & status Completed

9 Running a Job in EMF  User selects job(s) and clicks run (EMF client)  EMF exports inputs (EMF server)  EMF writes a job wrapper script (EMF server) EMF wrapper scripts are specific for a CASE and JOB and specific run (date-stamped) Include all inputs and parameters needed by that job EMF wrapper scripts call model specific scripts (e.g., SMOKE sector specific script)

10 Running a Job in EMF (2)  EMF submits job to the SMOKE server After exports are completed and dependencies are satisfied  Job executable runs (SMOKE server) SMOKE scripts and ASSIGNS file have been modified to be more generic SMOKE scripts communicate status, message and imports files via EMF command-line client

11 Communication between Servers  EMF to SMOKE server: EMF uses ssh-keys to start running programs on SMOKE server and for authentication On EMF server the Linux user “emf” ssh into SMOKE server as the SMOKE user and starts the EMF wrapper script  SMOKE to EMF server: SMOKE scripts call EMF command-line client Communicate changes in status, messages, or imports outputs generated by SMOKE EMF ServerSMOKE Server ssh EMF cmd-line client

12 Delivered Cases 1. Test messages and imports 2. Spatial Surrogate SMOKE platform Each delivery will include: Case to import into EMF Executable scripts Potentially one or more datasets to import into EMF