Distributed Systems Fall 2009 Course introduction

Fall 20095DV0203 Defining distributed systems “A distributed system is one in which components located at networked computers communicate and coordinate their actions by passing messages.” (Coulouris, Dollimore, Kindberg, 2005) ”A distributed system is one in which the failure of computer you didn't even know existed can render your own computer unusable.” (Leslie Lamport, 1987)

Fall 20095DV0204 Outline Staff presentation Course presentation Lessons from last year This year's course Basics and challenges of distributed systems Summary

Fall 20095DV0205 Staff presentation Daniel Henriksson Lars Larsson Questions about the assignment? –Always send to both teachers, even replies to replies! Questions about lectures? –Send to the appropriate teacher!

Fall 20095DV0206 Course presentation Students should obtain: –Knowledge of theoretical models for distributed systems –Knowledge of problems and solutions in designing and implementation of distributed systems

Course presentation The course covers: –Architectural models of distributed systems –Client-Server, peer-to-peer, transactions, transparency, naming, error handling, resource management, and synchronization –Computer security in a broad perspective –Distributed programming and middlewares

Fall 20095DV0208 Course presentation Theoretical part (4.5 ECTS) –Theory, methods, algorithms, and principles Practical part (3 ECTS) –Practical obligatory assignment

Fall 20095DV0209 Lessons from last year (positive) Students were very happy with the level of assistance with the assignment The anonymous feedback form was appreciated, as was the web site itself Students acquired a lot of new knowledge The average grade given to the course was “very good” The practical assignment was rewarding, albeit hard

Fall 20095DV02010 Lessons from last year (negative) The practical assignment was hard, albeit rewarding The book was hard to read, and did not cover ordering algorithms sufficiently Java RMI was hard to use First set of theory lectures should be allocated more time Lectures should all have custom made slides Exam too focused on details Interface used for practical was confusing

Fall 20095DV02011 This year's course Same practical assignment, but: –No potentially confusing interface –Any programming language allowed Added a group exercise on Java RMI All lectures will have custom made slides More time allocated to key topics Exam more focused on “broader” questions Keep up good work with assistance, listening to feedback, lecturing style, and web page

Fall 20095DV02012 The practical assignment Group communication middleware –Group membership handling –Message ordering guarantees –(Reliable) Multicast communication Presentation of working implementation at the end of the course Solved in pairs (2 students per team) Deals with theory from the first set of lectures

Fall 20095DV02013 What to learn? Book is dense with information –See reading guide on web page – it is actually accurate –Extremely good, but no easy read Start now! You will be busy later... Understand the problems and solutions –Learn the general ideas of algorithms and how/why they work, not every minute step Definitions are very important!

Fall 20095DV02014 Schedule is subject to change We want to be able to adapt the schedule to fit your needs If our in-class discussion of topics take longer time, that’s fine (within reason) Consider the topics for the lectures to be a reading guide, not a guarantee If this poses a problem, please give us feedback, and we will try to remedy the situation

Fall 20095DV02015 Basics of distributed systems Architectural model –Placement of components across the network and their relationships, and the network itself System model –Client/Server HTTP, FTP, … countless others! –Peer-to-Peer (P2P) BitTorrent, Freenet, Direct Connect, … –Combinations: multiple servers, mobile code, mobile agents, thin clients

Fall 20095DV02016 Design of distributed systems Performance issues –Responsiveness –Throughput –Load balancing Quality of Service (QoS) –Reliability, security, performance (time-critical) –Adaptability

Design of distributed systems Caching and replication –Web caching and Content Distribution Networks Dependability –Correctness, security, and fault tolerance

Fall 20095DV02018 Distributed systems challenges Heterogeneous systems –Networks, hardware, OS, languages, protocols Openness Security Scalability –Control physical resource costs and performance loss, resource conservation, bottleneck avoidance

Distributed systems challenges Failure handling –Detection, masking, tolerance, recovery, redundancy Concurrency –No global time, simultaneously running processes Transparency –Access, location, concurrency, replication, failure, mobility, performance, scaling

Fall 20095DV02020 Distributed systems challenges How are parameters passed and how is data converted? –Marshalling, unmarshalling How are distributed resources (functions, methods, objects) published and discovered? How are errors handled? Security! Memory handling – distributed garbage collection?

Fall 20095DV02021 Middleware Distributed systems often utilize middleware to aid development Offers layer of abstraction Extends upon traditional programming models: –Local procedure call → Remote procedure call –OOP → Remote Method Invocation –Event-based programming model

Fall 20095DV02022 Middleware Applications, Services RMI, RPC Request/Reply protocol Marshalling, Unmarshalling UDP, TCP Middleware

Fall 20095DV02023 Operation invocation Data structures must be “flattened” and serialized for transport –External formats, e.g. XML, JSON, Java Object Serialization,... Use interface –Procedures having either input, output, or both –No pointers –Service interface: provided services –Remote interface: operations accessible from other processes –Cross-language/platform interfaces: IDL, WSDL

Fall 20095DV02024 Semantics (Local call = exactly once) Maybe once –Omission failures (lost packets, crashes) At-least-once –Crash failures, arbitrary failures (multiple executions) –Used by Sun RPC At-most-once –Executed exactly once or not at all –Used by Java RMI, Corba

Fall 20095DV02025 Security Distributed system = increased exposure Client- and Server-authentication Client authorization –Is the client allowed to perform X? Proof of execution –Server must be able to prove that something has been executed –Also, non-repudiation: it should not be possible to claim that something did not happen if it did

Fall 20095DV02026 Distributed systems: a mess! Communication performance variations –Latency (delay), bandwidth (throughput), jitter (variation in time) Clocks and timing –Clock drift Interaction models –Asynchronous, synchronous Event ordering –Delays cause replies to arrive to some process before the request

Distributed systems: a mess! Failures –Distributed systems are much more likely to fail unexpectedly –Lost packets, bit errors, local failures, no response, method does not exist, etc … If you can write stable programs in spite of these difficulties, you are a great programmer!

Fall 20095DV02028 Summary Introduction to the course What are distributed systems? Consequences of distributed systems –Concurrency –No global time –Unrelated errors

Fall 20095DV02029 Summary Challenges –Heterogeneity –Openness –Security –Scalability –Error handling –Concurrency –Transparency Middlewares

Fall 20095DV02030 Next lecture Time, and the lack thereof –No global time, and its implications –Algorithms for (roughly) synchronizing processes