1 Cloud Computing Research at TU Delft – A. Iosup Alexandru Iosup Parallel and Distributed Systems Group Delft University of Technology The Netherlands.

Slides:

Advertisements

Similar presentations

1 Cloud Futures Workshop 2010 – Cloud Computing Support for Massively Social Gaming Alexandru Iosup Parallel and Distributed Systems Group Delft University.

Advertisements

The First 16 Years of the Distributed ASCI Supercomputer Henri Bal Vrije Universiteit Amsterdam COMMIT/

7 april SP3.1: High-Performance Distributed Computing The KOALA grid scheduler and the Ibis Java-centric grid middleware Dick Epema Catalin Dumitrescu,

Computer Organization TI1400 Alexandru Iosup (lecturer) Henk Sips (original slides) Parallel and Distributed Systems

19 November 2013 Exploring Portfolio Scheduling for Long-term Execution of Scientific Workloads in IaaS Clouds Alexandru Iosup Delft University of Technology.

June 1, Inter-Operating Grids through Delegated MatchMaking Alexandru Iosup, Dick Epema PDS Group, TU Delft, NL Todd Tannenbaum, Matt Farrellee,

June 2, GrenchMark : A Framework for Analyzing, Testing, and Comparing Grids CCGrid 2006 A. Iosup, D.H.J. Epema PDS Group, ST/EWI, TU Delft.

June 3, 2015 Synthetic Grid Workloads with Ibis, K OALA, and GrenchMark CoreGRID Integration Workshop, Pisa A. Iosup, D.H.J. Epema Jason Maassen, Rob van.

The Performance of Bags-Of-Tasks in Large-Scale Distributed Computing Systems Alexandru Iosup, Ozan Sonmez, Shanny Anoep, and Dick Epema ACM/IEEE Int’l.

Inter-Operating Grids through Delegated MatchMaking Alexandru Iosup, Dick Epema, Hashim Mohamed,Mathieu Jan, Ozan Sonmez 3 rd Grid Initiative Summer School,

1 NetGames 2010 – CAMEO: Continuous Analytics for Massively Multiplayer Online Games CAMEO : Enabling Social Networks for Massively Multiplayer Online.

1 Google Workshop at TU Delft, 2010 – Online Games and Clouds Cloudifying Games: Rain for the Thirsty Alexandru Iosup Parallel and Distributed Systems.

1 A Performance Study of Grid Workflow Engines Alexandru Iosup and Dick Epema PDS Group Delft University of Technology The Netherlands Corina Stratan Parallel.

1 Trace-Based Characteristics of Grid Workflows Alexandru Iosup and Dick Epema PDS Group Delft University of Technology The Netherlands Simon Ostermann,

June 25, GrenchMark: A synthetic workload generator for Grids KOALA Workshop A. Iosup, H. Mohamed, D.H.J. Epema PDS Group, ST/EWI, TU Delft.

June 25, GrenchMark: Synthetic workloads for Grids First Demo at TU Delft A. Iosup, D.H.J. Epema PDS Group, ST/EWI, TU Delft.

June 28, Resource and Test Management in Grids Rapid Prototyping in e-Science VL-e Workshop, Amsterdam, NL Dick Epema, Catalin Dumitrescu, Hashim.

1 ASCI, 2010 – Analysis of BBO Fans Social Networks Analysis of BBO Fans, an Online Social Gaming Community Alexandru Iosup Parallel and Distributed Systems.

University of Dortmund June 30, On Grid Performance Evaluation using Synthetic Workloads JSSPP 2006 Alexandru Iosup, Dick Epema PDS Group, ST/EWI,

July 13, “How are Real Grids Used?” The Analysis of Four Grid Traces and Its Implications IEEE Grid 2006 Alexandru Iosup, Catalin Dumitrescu, and.

Euro-Par 2008, Las Palmas, 27 August DGSim : Comparing Grid Resource Management Architectures Through Trace-Based Simulation Alexandru Iosup, Ozan.

2011 / 01 / 13 Andy Wang.  Thesis Subject  Paper Reading  Current Works  Future Works.

1 Efficient Management of Data Center Resources for Massively Multiplayer Online Games V. Nae, A. Iosup, S. Podlipnig, R. Prodan, D. Epema, T. Fahringer,

1 TI1100A – Internet-Based Applications – A. Iosup Alexandru Iosup Parallel and Distributed Systems Group Delft University of Technology Our team: Undergrad.

Course Announcements Tomorrow, Jan 11, at 09:45, Lecture Hall H: “GPU Programming: Tips and Tricks” Ana Varbanescu Exam-related questions: contact Hai.

August 28, Performance Analysis of Cloud Computing Services for Many-Tasks Scientific Computing Berkeley, CA, USA Alexandru Iosup, Nezih Yigitbasi,

Introduction. Readings r Van Steen and Tanenbaum: 5.1 r Coulouris: 10.3.

August 29, Our team: Undergrad Nassos Antoniou, Thomas de Ruiter, Ruben Verboon, … Grad Siqi Shen, Nezih Yigitbasi, Ozan Sonmez Staff Henk Sips,

Euro-Par 2007, Rennes, 29th August 1 The Characteristics and Performance of Groups of Jobs in Grids Alexandru Iosup, Mathieu Jan *, Ozan Sonmez and Dick.

: Massivizing Online Games using Cloud Computing Alexandru Iosup Parallel and Distributed Systems Group, Delft University of Technology, The Netherlands.

1 EIT ICT Labs Workshop at TU Delft, May 2011 – Cloud Computing Parallel and Distributed Systems Group Delft University of Technology The Netherlands Our.

On the Gamification of a Graduate Course on Cloud Computing Alexandru Iosup and Dick Epema (Delft University of Technology, the Netherlands) [1] (Blackburg.

1 TUD-PDS A Periodic Portfolio Scheduler for Scientific Computing in the Data Center Kefeng Deng, Ruben Verboon, Kaijun Ren, and Alexandru Iosup Parallel.

A Performance Evaluation of Azure and Nimbus Clouds for Scientific Applications Radu Tudoran KerData Team Inria Rennes ENS Cachan 10 April 2012 Joint work.

May 25, Our team: Undergrad Tim Hegeman, Stefan Hugtenburg, Jesse Donkevliet … Grad Siqi Shen, Guo Yong, Nezih Yigitbasi Staff Henk Sips, Dick Epema,

1 EuroPar 2009 – POGGI: Puzzle-Based Online Games on Grid Infrastructures POGGI: Puzzle-Based Online Games on Grid Infrastructures Alexandru Iosup Parallel.

The Cloud Workloads Archive: A Status Report

1 TUD-CH-Informatica Gamification: Playful Teaching for Generation-X/-Y/-Z/… Alexandru Iosup Parallel and Distributed Systems Group.

Marcos Dias de Assunção 1,2, Alexandre di Costanzo 1 and Rajkumar Buyya 1 1 Department of Computer Science and Software Engineering 2 National ICT Australia.

1 Massivizing Social Games: High Performance Computing and High Quality Time – A. Iosup Alexandru Iosup Parallel and Distributed Systems Group Delft University.

1 Massivizing Social Games: Distributed Computing Challenges and High Quality Time – A. Iosup Alexandru Iosup Parallel and Distributed Systems Group Delft.

10/6/2015B.Ramamurthy110/6/2015B.Ramamurthy1 Midterm Review CSE4/587 B.Ramamurthy.

An Autonomic Framework in Cloud Environment Jiedan Zhu Advisor: Prof. Gagan Agrawal.

1 A Framework for Data-Intensive Computing with Cloud Bursting Tekin Bicer David ChiuGagan Agrawal Department of Compute Science and Engineering The Ohio.

October 18, Our team: Undergrad Anand Sawant, Ruben Verboon, Gargi Prasad, Arnoud Bakker, Nassos Antoniou, Thomas de Ruiter, … Grad Siqi Shen, Nezih.

1 Challenge the future KOALA-C: A Task Allocator for Integrated Multicluster and Multicloud Environments Presenter: Lipu Fei Authors: Lipu Fei, Bogdan.

1 ROIA 2009 – CAMEO: Continuous Analytics for Massively Multiplayer Online Games CAMEO: Continuous Analytics for Massively Multiplayer Online Games Alexandru.

October 23, Grid Computing: From Old Traces to New Applications Fribourg, Switzerland Alexandru Iosup, Ozan Sonmez, Nezih Yigitbasi, Hashim Mohamed,

October 23, Our team: Undergrad Nassos Antoniou, Thomas de Ruiter, Ruben Verboon, … Grad Siqi Shen, Nezih Yigitbasi, Ozan Sonmez Staff Henk Sips,

Experiments in computer science Emmanuel Jeannot INRIA – LORIA Aleae Kick-off meeting April 1st 2009.

October 27, Our team: Undergrad Nassos Antoniou, Thomas de Ruiter, Ruben Verboon, … Grad Siqi Shen, Nezih Yigitbasi, Ozan Sonmez Staff Henk Sips,

VENI – Massively Multiplayer Online Games: Near Zero-Cost and Near-Infinite Capability 1 Massively Multiplayer Online Games Near-Zero Cost and Near-Infinite.

1 LSAP, 2011 – Analysis of Online and Face-to-Face Bridge Communities An Analysis of Social Networks Analysis in Online and Face-to-Face Bridge Communities.

November 29, Our team: Undergrad Thomas de Ruiter, Anand Sawant, Ruben Verboon, … Grad Siqi Shen, Guo Yong, Nezih Yigitbasi Staff Henk Sips, Dick.

: Massivizing Online Games using Cloud Computing Alexandru Iosup Parallel and Distributed Systems Group, Delft University of Technology, The Netherlands.

1 Massivizing Social Games: Distributed Computing Challenges and High Quality Time – A. Iosup Alexandru Iosup Parallel and Distributed Systems Group Delft.

Ocean Observatories Initiative Serving Ocean Model Data on the Cloud M. Meisinger, C. Farcas, E. Farcas, C. Alexander, M. Arrott, J. de La Beaujardière,

Euro-Par, HASTE: An Adaptive Middleware for Supporting Time-Critical Event Handling in Distributed Environments ICAC 2008 Conference June 2 nd,

Directions in eScience Interoperability and Science Clouds June Interoperability in Action – Standards Implementation.

1 Views of Cloud Computing Prof. Ravi Sandhu Executive Director and Endowed Chair March 25, © Ravi Sandhu.

Alexandru Iosup Parallel and Distributed Systems Group Delft University of Technology The Netherlands Cloud Computing : Open Research Questions.

Cloud Benchmarking, Tools, and Challenges

Department of CSE CLOUD COMPUTING UNIT-V.

(Parallel and) Distributed Systems Group

Vlad Nae, Radu Prodan, Thomas Fahringer Institute of Computer Science

Cloud Computing Applications

Cloud Performance Evaluation at TU Delft (2008—)

On Dynamic Resource Availability in Grids

IaaS Cloud Benchmarking: Approaches, Challenges, and Experience

The Performance of Big Data Workloads in Cloud Datacenters

Presentation transcript:

1 Cloud Computing Research at TU Delft – A. Iosup Alexandru Iosup Parallel and Distributed Systems Group Delft University of Technology The Netherlands Our team: Undergrad Gargi Prasad, Arnoud Bakker, Nassos Antoniou, Thomas de Ruiter, … Grad Siqi Shen, Nezih Yigitbasi, Ozan Sonmez Staff Henk Sips, Dick Epema, Alexandru Iosup Collaborators Ion Stoica and the Mesos team (UC Berkeley), Thomas Fahringer, Radu Prodan (U. Innsbruck), Nicolae Tapus, Mihaela Balint, Vlad Posea (UPB), Derrick Kondo, Emmanuel Jeannot (INRIA),... Cloud Computing Research at TU Delft (and TU/e) 3TU. =++ EIT Meeting, Berlin, 24 February 2011

Cloud Computing Research at TU Delft – A. Iosup 2 PDS Group: Past Research in Grids The Koala Grid Scheduler Co-allocation Deployed since 2005 DAS-2,-3,-4 A Grid Research Toolbox The Grid Workloads Archive GrenchMark DGSim Performance models Availability models DGSim

Cloud Computing Research at TU Delft – A. Iosup 3 Cloud Futures Workshop 2010 – Cloud Computing Support for Massively Social Gaming 3 A View on Cloud Computing “The path to abundance” On-demand capacity Pay what you use Great for web apps (EIP, web crawl, DB ops, I/O) “The killer cyclone” Not so great performance for compute- or data- intensive e-Science apps 1 Performance variability 2 Tropical Cyclone Nargis (NASA, ISSS, 04/29/08) 1- Iosup et al., Performance Analysis of Cloud Computing Services for Many Tasks Scientific Computing, IEEE TPDS, 2011 (in print) Iosup et al., On the Performance Variability of Production Cloud Services, CCGrid 2011, pds.twi.tudelft.nl/reports/2010/PDS pdfpds.twi.tudelft.nl/reports/2010/PDS pdf VS

Cloud Computing Research at TU Delft – A. Iosup 4 Cloud Performance Studies Many-Tasks Scientific Computing Quantitative definition: J jobs and B bags-of-tasks Extracted proto-MT users from grid and parallel production environments Performance Evaluation of Four Commercial Clouds Amazon EC2, GoGrid, Elastic Hosts, Mosso Resource acquisition, Single- and Multi-Instance benchmarking Low compute and networking performance Clouds vs Other Environments Order of magnitude better performance needed for clouds Clouds already good for short-term, deadline-driven scientific computing 1- Iosup et al., Performance Analysis of Cloud Computing Services for Many Tasks Scientific Computing, IEEE TPDS, 2011 (in print) Iosup et al., On the Performance Variability of Production Cloud Services, CCGrid 2011, pds.twi.tudelft.nl/reports/2010/PDS pdfpds.twi.tudelft.nl/reports/2010/PDS pdf

Cloud Computing Research at TU Delft – A. Iosup 5 The Cloud Workloads Archive Looking for invariants Wr [%] ~40% Total IO, but absolute values vary # Tasks/Job, ratio M:(M+R) Tasks, vary Understanding workload evolution Trace IDTotal IO [MB]Rd. [MB]Wr [%] HDFS Wr[MB] CWA-0110,9346,80538%1,538 CWA-0275,54647,53937%8,563

Cloud Computing Research at TU Delft – A. Iosup 6 More Research on Clouds at TU Delft Scheduling cloud workloads MapReduce Scientific applications Cloud-based gaming infrastructure Massive Gaming is multi-billion business Driver for cloud resource management research Driver for distributed systems research CAMEO: MMOG Analytics POGGI: Game Content Generation A. Iosup et al., CAMEO: Enabling Social Networks for Massively Multiplayer Online Games through Continuous Analytics and Cloud Computing, ACM NetGames (2010) A. Iosup, POGGI: generating puzzle instances for online games on grid infrastructures. CCPE 23(2): (2011)

Cloud Computing Research at TU Delft – A. Iosup 7 Take Home Message: 3-TU Research in Clouds Understanding how real clouds work Modeling cloud infrastructure (performance, availability) and workloads Compare clouds with other platforms (grids, parallel production env., p2p,…) The Cloud Workloads Archive: easy to share cloud workload traces and research associated with them Complement the Grid Workloads Archive Scheduling: making clouds work eScience and gaming applications MapReduce Massive Gaming: services on clouds CAMEO: Massive Game Analytics Toolkit for Online Social Network analysis POGGI: game content generation at scale  See Pascal van Eck’s presentation Publications 2008: ACM SC 2009: ROIA, CCGrid, NetGames, EuroPar (Best Paper Award) 2010: IEEE TPDS, Elsevier CCPE,… 2011: ICPE, CCGrid, Book Chapter CAMEO, IEEE TPDS, IJAMC, …

Cloud Computing Research at TU Delft – A. Iosup 8 Thank you for your attention! Questions? Suggestions? Observations? Alexandru Iosup (or google “iosup”) Parallel and Distributed Systems Group Delft University of Technology More Info: Do not hesitate to contact me…