SLA Management in AssessGrid Dominic Battré, TU Berlin

Dominic Battré – SLA Management in AssessGrid 2 AssessGrid in a Nutshell Requirement for Service Level Agreements from users Reluctance to sign SLAs by providers

Dominic Battré – SLA Management in AssessGrid 3 AssessGrid in a Nutshell

Dominic Battré – SLA Management in AssessGrid 4 AssessGrid in a Nutshell

Dominic Battré – SLA Management in AssessGrid 5

6 AssessGrid in a Nutshell

Dominic Battré – SLA Management in AssessGrid 7 AssessGrid in a Nutshell TeraGrid Grid 3 DAS-2 … * statistics from 2005/2006! failed jobs succ. jobs

Dominic Battré – SLA Management in AssessGrid 8 AssessGrid in a Nutshell

Dominic Battré – SLA Management in AssessGrid 9 AssessGrid in a Nutshell User: - Which provider is reliable? - How reliable is a provider? - Does a provider lie? Provider: - How reliable am I? - Can I sign SLAs? - Can I improve my reliability?

Dominic Battré – SLA Management in AssessGrid 10 Agenda AssessGrid in a Nutshell Content of SLAs Demo - Job submission and provider selection - Fault Tolerance Underlying technology - Negotiation Manager - Risk Assessment and Management Content of SLAs as WS-Agreement Future Challenges

Dominic Battré – SLA Management in AssessGrid 11 Content of SLAs time Job 1 Job 2 Job 3 Job 5 Job 4 Job 7 Job 6 nodes Job 1 Each job specified with Job 1 runtime nr. nodes Earliest start timeLatest finish time Schedule Participating parties Job Definition - Scheduling - Executable - File Staging - Acceptable Probability of Failure Price and penalty

Dominic Battré – SLA Management in AssessGrid 12

Dominic Battré – SLA Management in AssessGrid 13 Job Submission and Provider Selection Specify Job End-UserBrokerProviders Program, Input, Output Acceptable PoF Penalty in case of failure Deadline

Dominic Battré – SLA Management in AssessGrid 14 Job Submission and Provider Selection Get Quotes End-UserBrokerProviders

Dominic Battré – SLA Management in AssessGrid 15 Job Submission and Provider Selection Get Quotes End-UserBrokerProviders Forwarding based on Matching of templates to request Quotes created in the past Performance in the past

Dominic Battré – SLA Management in AssessGrid 16 Job Submission and Provider Selection Generate Quotes End-UserBrokerProviders Calculate Probability of Failure (PoF) Calculate required number of spare nodes, extra time Calculate price Check available resources in schedule

Dominic Battré – SLA Management in AssessGrid 17 Job Submission and Provider Selection Quotes End-UserBrokerProviders

Dominic Battré – SLA Management in AssessGrid 18 Job Submission and Provider Selection Enhance Quotes End-UserBrokerProviders Own estimation of PoF in case of unreliable providers Perform ranking respecting user’s desire

Dominic Battré – SLA Management in AssessGrid 19 Job Submission and Provider Selection Quotes End-UserBrokerProviders

Dominic Battré – SLA Management in AssessGrid 20 Job Submission and Provider Selection Select Provider End-UserBrokerProviders Criteria: Price, PoF, Adjusted PoF AHP-Ranking

Dominic Battré – SLA Management in AssessGrid 21 Job Submission and Provider Selection Get Reputation End-UserBrokerProviders

Dominic Battré – SLA Management in AssessGrid 22 DS Analytical Hierarchy Process Past Performance Maintenance Security Customer Support Infrastructure Experience Staff 24/7 Staff training/yr Staff experience Red. Power Red. Storage Storage Age … Maintenance Infrastructure

Dominic Battré – SLA Management in AssessGrid 23 Job Submission and Provider Selection End-UserBrokerProviders Create Agreement

Dominic Battré – SLA Management in AssessGrid 24

Dominic Battré – SLA Management in AssessGrid 25 Demonstration of Fault Tolerance

Dominic Battré – SLA Management in AssessGrid 26

Dominic Battré – SLA Management in AssessGrid 27 Negotiation Manager Globus Toolkit 4 Apache 2 License 2 Flavours - Simple Framework - AssessGrid Implementation (OpenCCS, Risk Assessment, …) Features - Template Store - Access Control, Credential Delegation - State Management - Staging by GridFTP - Simple Validation of CreationConstraints - Extensible - WS-Notification - Optional: Quote Mechanism - Optional: Cheap Cancellation Extension

Dominic Battré – SLA Management in AssessGrid 28 Template Store Optional component Templates stored persistently in RDBMS Get, Insert, Delete by WS-RF Monitoring by WS-Notification Access policies: - Everybody can read - Admin(s) can modify Templates used in AssessGrid - Regular Job (POSIX and SPMD) - Out-sourced Job with checkpoint data-set

Dominic Battré – SLA Management in AssessGrid 29 Access Control Default: - 3 User Groups Admins, Owners, Users - Admin has access to anything - Owner is legally responsible - Users have read access - Owner and Users are different in case of SLA outsourcing Overwriteable Option to delegate credentials

Dominic Battré – SLA Management in AssessGrid 30 State Management Asynchronous, multi-threaded, persistent state management Wait for stage-in StartDo stage-in Wait for execution Stage-in done Wait for termination Do stage-out Stage-out done Cleanup

Dominic Battré – SLA Management in AssessGrid 31 File-staging Files specified by JSDL User delegates credentials User estimates duration - Shorter duration triggers earlier execution - Longer duration triggers later execution Staging by GridFTP

Dominic Battré – SLA Management in AssessGrid 32 CreationConstraints Difficult to support Namespaces: //wsag:…/assessgrid:… - prefixes are just strings Very difficult to support structural information xs:group, xs:all, xs:choice, xs:sequence Possible but difficult to support xs:restriction xs:simple - Check for enumeration (xs:restriction of xs:string) - Check for valid dates (xs:restriction of xs:date) - Everything else close to impossible {min,max}{In,Ex}clusive totalDigits, fractionDigits, length, … probably useless Context Terms Creation Constraints

Dominic Battré – SLA Management in AssessGrid 33 Optional Quote Mechanism UserProvider Get Template Fill Template Create Agreement Yes / Nobound Create Quote modify

Dominic Battré – SLA Management in AssessGrid 34 Extensible Not: Black Box Interface Domain specific Implementation deployed But: WSDL Domain specific Implementation NegMgr WSDL deployed

Dominic Battré – SLA Management in AssessGrid 35 Cancellation Policy Motivation: - Serious issues of 3-way commit protocol (reservations) Goal: Cheap Cancellation Policy - “Full refund if product bought online is returned online within 14 days” (German law) - “Cancellation before first day of validity: 15 EUR, after that: not possible” (Deutsche Bahn) - “less than 24 hours before scheduled stay: 50% of first day for cancellation” (hotels)

Dominic Battré – SLA Management in AssessGrid 36 Cancellation Policy - Rules Ends of periods: Price: createQuotecreateAgreementEarliest Start +5min-1d 1 EUR - 80%

Dominic Battré – SLA Management in AssessGrid 37 Cancellation Policy - Combination createQuotecreateAgreementEarliest Start +5min -1d Full price -50% 0.50 EUR time price Used in Broker for roll-back of unsuccessful workflow mappings

Dominic Battré – SLA Management in AssessGrid 38 Context … /C=DE/O=… … DN … Context Terms Creation Constraints

Dominic Battré – SLA Management in AssessGrid 39 Terms, SDTs Conjunction of terms - Common structure of templates - WS-AG too powerful/difficult to fully support Service Description Term (one) - assessgrid:ServiceDescription (extension of abstract ServiceTermType) jsdl:POSIXExecutable / SPMD (executable, arguments, environment) jsdl:Resources jsdl:DataStaging * assessgrid:PoF (upper bound) Context Terms Creation Constraints

Dominic Battré – SLA Management in AssessGrid 40 Terms, GuaranteeTerms No hierarchy but two meta guarantees - ProviderFulfillsAllObligations e.g. Reward: 1000 EUR, Penalty 1000 EUR - ConsumerFulfillsAllObligations e.g. Reward: 0 EUR, Penalty 1000 EUR First violation is responsible for failure No hardware problem, then User fault Other Guarantees - Execution Time Any start time (best effort) Exact start time Earliest start time, latest finish time - Maximum StageIn/Out time - No Cancellation Context Terms Creation Constraints No timely execution No stage-out

Dominic Battré – SLA Management in AssessGrid 41 Stuff I did not talk about Risk Assessment Risk Management - Checkpointing details, runtime extension, spare nodes, … Confidence and Reputation Service Workflows - Description in WS-Agreement - Mapping to individual SLAs Simulation tools

Dominic Battré – SLA Management in AssessGrid 42 Future Challenges Failure detection and analysis (Re)negotiation Risk Assessment Interoperability of WS-Agreement implementations by micro-specs – or even common template structures Automatic evaluation of CreationConstraints Posthumous resolving of disagreements Third party blaming Persisting Problems - Dependencies of violated guarantees - Violation caused by third party or unknown cause - Failure/success of entire SLA

Dominic Battré – SLA Management in AssessGrid 43