1. CLOUD COMPUTING ARCHITECTURES & APPLICATIONS LECTURERS LAZAR KIRCHEV, PhD ILIYAN NENOV KRUM BAKALSKY 7 March, 2011 LECTURE #1 INTRODUCTION. CONTEMPORARY LARGE-SCALE PROBLEMS.

2. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications2 OUTLINE Examples of Contemporary Large-Scale Problems Introduction  Introduction to Distributed Computing  Parallel vs. Distributed Computing  History of Distributed Computing  Parallelization and Synchronization  Networking Basics

3. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications3 Moor’s Law and the Limits of a Single Computer The density of transistors on the CPU chip will double every 18 months for approximately the same cost of CPU Gordon Earle Moore (born 3 January 1929) co-founder and Chairman Emeritus of Intel Corporation and the author of Moor’s Law (published in Electronics Magazine 19/04/1965)  From 1965 to 2002 the CPU speed doubled roughly every two years. But between 2003 and 2005  transistors reached physical limits (smallest transistor is 7 atoms large)  memory bandwidth reached its limits (6 GHz processor will turn the fastest RAM in bottleneck)  Therefore, ‘faster’ CPUs is not so justified from business perspective  So, the scope of problems for a single computer will not get dramatically larger any time in the next years.

4. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications4 The Need for Computing Grows ‘Dot Com’ crisis started Moor’s law was broken Overall digital storage capacity overtook analog one Outsourcing boom started What is so special about year 2002? RESEARCH Astrophysics Genome Encoding Particles Physics Recommended Readings RAY KURZWEIL “The age of Spiritual Machines” “Singularity is Near” THOMAS L FRIEDMAN “The World is Flat” BUSINESS Outsourcing Business Analytics Real-Time Decisions Mobile Movies Satellite Image Processing CONSUMER Digital Photos Streaming Content Social Networking Weather Forecast

5. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications5 Scope of Problems What can you do with 1 computer? What can you do with 100 computers? What can you do with a data center?

6. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications6 Examples of Data Centeres Google One of the most scalable supercomputer in the corporate world  http://www.youtube.com/watch?v=zRwPSFpLX8I&feature=related http://www.youtube.com/watch?v=zRwPSFpLX8I&feature=related Intel One of the fastest super computers in the corporate world  http://www.youtube.com/watch?v=FILhtOfLMzI&feature=related http://www.youtube.com/watch?v=FILhtOfLMzI&feature=related

7. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications7 Examples of Distributed Problems (1/2) Rendering multiple frames of high-quality animation Avatar © Lightstorm Entertainment Shrek © DreamWorks Animation; Lord of the Rings © New Line Cinema; Happy Feet © Kingdom Feature Productions Simulating several hundred or thousand characters

8. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications8 Examples of Distributed Problems (2/2) Indexing Internet Content (Google) Simulating an Internet-sized network for network experiments (PlanetLab) Speeding up content delivery (Akamai) Data mining over large data sets (SAP) Encoding Human Genome (genome.gov) Searching for new particles (CERN) Analyzing space satellite images (NASA) What is the key attribute of all these examples?

9. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications9 Patterns for Solving Large-Scale Distributed Problems Large scale problem  Can be broken into smaller problems  Smaller problems can be independently solved  Final solution can be constructed from the solutions of smaller problems Big Problem Big Solution

10. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications10 Parallel vs. Distributed Parallel computing can mean: Vector Processing of Data Multiple CPUs in a single computer Distributed computing is multiple CPUs across many computers over a network  Same copy of memory  Same copy of storage

11. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications11 Brief History of Distributed Computing 1975-1985 Parallel computing was favored in the early years Primary vector-based at first Gradually more thread based parallelism was introduced Supercomputer: Cray 2

12. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications12 Brief History of Distributed Computing 1985-1995  “Massively parallel architectures” start rising in prominence  Message Passing Interface (MPI) and other libraries developed  Bandwidth was a big problem

13. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications13 Brief History of Distributed Computing 1995 - 2010  Cluster/grid architecture increasingly dominant  Special node machines eschewed in favour of COTS (Commercial, Off-The- Shelf) technologies  Web-wide cluster software  Companies like Google take this to the extreme

14. Parallelization and Synchronization

15. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications15 Parallelization Idea (1/4) Parallelization is “easy” if processing can be cleanly split into finite number of units: Work W2 W3 W1 W4 Partition Problem

16. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications16 Parallelization Idea (2/4) In a parallel computation, we would like to have as many threads as we have processors. e.g., a four-processor computer would be able to run four threads at the same time W2 W3W1 W4 Spawn Worker Threads

17. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications17 Parallelization Idea (3/4) Workers Process Data W2 W3W1 W4

18. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications18 Parallelization Idea (4/4) W2 W3W1 W4 Results

19. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications19 Parallelization Challenges and Pitfalls (1/2) How do we assign work to these threads What happens if we have more work than threads How do we aggregate the results the end How do we know all the workers have finished What if the work can not be divided into completely separate tasks > What is the common theme of all these problems?

20. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications20 Parallelization Challenges and Pitfalls (2/2) Each of these problems represents a point at which multiple threads must communicate with one another, or access a shared resource. Golden rule: Any memory that can be used by multiple threads must have an associated synchronization system!

21. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications21 What is wrong with this source? Thread 1: void foo() { x++; y = x; } Thread 2: void bar() { y++; x+=3; } If the initial state is y = 0, x = 6, what happens after these threads finish running? There are multiple possible final states. Y is definitely a problem, because we don’t know if it will be “1” or “7”… but X can also be 7, 10, or 11!

22. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications22 Multithreaded = Unpredictability Thread 1: void foo() { eax = mem[x]; inc eax; mem[x] = eax; ebx = mem[x]; mem[y] = ebx; } Thread 2: void bar() { eax = mem[y]; inc eax; mem[y] = eax; eax = mem[x]; add eax, 3; mem[x] = eax; } Many things that look like “one step” operations actually take several steps under the hood: When we run a multithreaded program, we don’t know what order threads run in, nor do we know when they will interrupt one another.

23. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications23 Multithreaded = Unpredictability This applies to more than just integers: Pulling work units from a queue Reporting work back to master unit Telling another thread that it can begin the “next phase” of processing … All require synchronization!

24. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications24 Synchronization Primitives A synchronization primitive is a special shared variable that guarantees that it can only be accessed atomically. Hardware support guarantees that operations on synchronization primitives only ever take one step

25. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications25 Semaphores A semaphore is a flag that can be raised or lowered in one step Semaphores were flags that railroad engineers would use when entering a shared track  Only one side of the semaphore can ever be red! (Can both be green?)

26. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications26 Semaphores set() and reset() can be thought of as lock() and unlock() Calls to lock() when the semaphore is already locked cause the thread to block. Pitfalls: Must “bind” semaphores to particular objects; must remember to unlock correctly

27. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications27 The “corrected” example Thread 1: void foo() { sem.lock(); x++; y = x; sem.unlock(); } Thread 2: void bar() { sem.lock(); y++; x+=3; sem.unlock(); } Global variable:  Semaphore sem = new Semaphore()

28. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications28 Condition Variables A condition variable notifies threads that a particular condition has been met Inform another thread that a queue now contains elements to pull from (or that it’s empty – request more elements!) Pitfall: What if nobody’s listening?

29. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications29 The Final Example Thread 1: void foo() { sem.lock(); x++; y = x; fooDone = true; sem.unlock(); fooFinishedCV.notify(); } Thread 2: void bar() { sem.lock(); if(!fooDone) fooFinishedCV.wait(sem); y++; x+=3; sem.unlock(); } Global Variables:  Semaphore sem = new Semaphore();  ConditionVar fooFinishedCV = new ConditionVar();  boolean fooDone = false;

30. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications30 Too Much Synchronization Leads to Deadlock Synchronization becomes even more complicated when multiple locks can be used Can cause entire system to “get stuck”

31. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications31 The Moral of the Story Synchronization is hard  Need to consider all possible shared state  Must keep locks organized and use them consistently and correctly Knowing there are bugs may be tricky; fixing them can be even worse! Keeping shared state to a minimum reduces total system complexity

32. Fundamentals of Networking

33. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications33 Sockets: The Internet = tubes? A socket is the basic network interface Provides a two-way “pipe” abstraction between two applications Client creates a socket, and connects to the server, who receives a socket representing the other side

34. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications34 Ports Within an IP address, a port is a sub-address identifying a listening program Allows multiple clients to connect to a server at once

35. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications35 What makes this work? Underneath the socket layer are several more protocols Most important are TCP and IP (which are used hand-in-hand so often, they’re often spoken of as one protocol: TCP/IP) Even more low-level protocols handle how data is sent over Ethernet wires, or how bits are sent through the air using 802.11 wireless…

36. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications36 Why is This Necessary? Not actually tube-like “underneath the hood” Unlike phone system (circuit switched), the packet switched Internet uses many routes at once

37. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications37 Networking Issues If a party to a socket disconnects, how much data did they receive? … Did they crash? Or did a machine in the middle? Can someone in the middle intercept/modify our data? Traffic congestion makes switch/router topology important for efficient throughput

38. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications38 Conclusions Processing more data means using more machines at the same time Cooperation between processes requires synchronization Designing real distributed systems requires consideration of networking topology

39. END OF LECTURE #1

40. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications40 The information in this document is compiled using varous public sources, freely available in internet. These sources include:  http://www.scribd.com/doc/17929394/Cloud-Computing-Use-Cases-Whitepaperhttp://www.scribd.com/doc/17929394/Cloud-Computing-Use-Cases-Whitepaper  http://www.enisa.europa.eu/act/rm/files/deliverables/cloud-computing-risk-assessmenthttp://www.enisa.europa.eu/act/rm/files/deliverables/cloud-computing-risk-assessment  http://code.google.com/edu/parallel/index.html http://code.google.com/edu/parallel/index.html  Google: Cluster Computing and MapReduce: http://code.google.com/edu/submissions/mapreduce-minilecture/listing.htmlhttp://code.google.com/edu/submissions/mapreduce-minilecture/listing.html  Google Course: MapReduce in a Week http://code.google.com/edu/submissions/mapreduce/listing.htmlhttp://code.google.com/edu/submissions/mapreduce/listing.html  Intensive MapReduce course at MIT http://mr.iap.2008.googlepages.comhttp://mr.iap.2008.googlepages.com  Hadoop Virtual Image Documentation http://code.google.com/edu/parallel/tools/hadoopvm/index.htmlhttp://code.google.com/edu/parallel/tools/hadoopvm/index.html  http://www.umiacs.umd.edu/~jimmylin/cloud-computinghttp://www.umiacs.umd.edu/~jimmylin/cloud-computing  Colby Ranger, Ramanan Raghuraman, Arun Penmetsa, Gary Bradski, Christos Kozyrakis,  Evaluating MapReduce for Multi-core and Multiprocessor Systems, http://csl.stanford.edu/~christos/publications/2007.cmp_mapreduce.hpca.pdfhttp://csl.stanford.edu/~christos/publications/2007.cmp_mapreduce.hpca.pdf  http://www.dbms2.com/2008/08/26/why-mapreduce-matters-to-sql-data-warehousinghttp://www.dbms2.com/2008/08/26/why-mapreduce-matters-to-sql-data-warehousing  Bingsheng He, Wenbin Fang, Qiong Luo, Mars: A MapReduce Framework on Graphics Processors http://www.cse.ust.hk/catalac/users/saven/GPGPU/MapReduce/PACT08/171.pdfhttp://www.cse.ust.hk/catalac/users/saven/GPGPU/MapReduce/PACT08/171.pdf  Hung-chih Yang, Ali Dasdan, Map-reduce-merge: simplified relational data processing on large clusters http://portal.acm.org/citation.cfm?doid=1247480.1247602http://portal.acm.org/citation.cfm?doid=1247480.1247602  Foto N. Afrati, Jeffrey D. Ullman, A New Computation Model for Rack-Based Computing http://infolab.stanford.edu/~ullman/pub/mapred.pdfhttp://infolab.stanford.edu/~ullman/pub/mapred.pdf  Ralf Lammel, Google’s MapReduce Programming Model Revisite http://www.cs.vu.nl/~ralf/MapReduce/paper.pdfhttp://www.cs.vu.nl/~ralf/MapReduce/paper.pdf  http://www.baselinemag.com/c/a/Infrastructure/How-Google-Works-1http://www.baselinemag.com/c/a/Infrastructure/How-Google-Works-1  Joe Hellerstein, Parallel Programming in the Age of Big Data http://gigaom.com/2008/11/09/mapreduce-leads-the-way-for-parallel-programminghttp://gigaom.com/2008/11/09/mapreduce-leads-the-way-for-parallel-programming  Jeffrey Dean and Sanjay Ghemawat, MapReduce: Simplified Data Processing on Large Clusters https://sites.google.com/a/colgate.edu/cloudintro/Homehttps://sites.google.com/a/colgate.edu/cloudintro/Home © 2011 COPYRIGHTS DISCLAIMER The information in this document is proprietary to Sofia University “Sv. Kliment Ohridski” (called THE UNIVERSITY bellow) http://uni-sofia.bg THE UNIVERSITY assumes no responsibility for errors or omissions in this document. THE UNIVERSITY does not warrant the accuracy or completeness of the information, text, graphics, links, or other items contained within this material. This document is provided without a warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. This document is used only for educational purposes related to the masters programs of THE UNIVERSITY, Faculty of Mathematics and Informatics. This document is compiled using various public sources freely available in internet or offered by SAP AG. This document is not used directly or indirectly for any type of commercial use. http://fmi.uni-sofia.bg THE UNIVERSITY shall have no liability for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials. This limitation shall not apply in cases of intent or gross negligence. The statutory liability for personal injury and defective products is not affected. THE UNIVERSITY has no control over the information that you may access through the use of hot links contained in these materials and does not endorse your use of third-party Web pages nor provide any warranty whatsoever relating to third-party Web pages.

41. 2011 Sofia University “Sv. Kliment Ohridski” > Faculty of Mathematics and Informatics > Cloud Computing Architecture and Applications41 Headline area Drawing area White space The Grid