Andrea Righi – LinuxDay 2007 (Oct 27) Installare GNU/Linux su un ampio numero di client con SystemImager e BitTorrent Andrea Righi

Slides:



Advertisements
Similar presentations
WSUS Presented by: Nada Abdullah Ahmed.
Advertisements

Building on the BIRN Workshop BIRN Systems Architecture Overview Philip Papadopoulos – BIRN CC, Systems Architect.
Chapter 9 Designing Systems for Diverse Environments.
Novell Server Linux vs. windows server 2008 By: Gabe Miller.
Module 8: Concepts of a Network Load Balancing Cluster
Spotlighting Decentralized P2P File Sharing Archie Kuo and Ethan Le Department of Computer Science San Jose State University.
Presented by Stephen Kozy. Presentation Outline Definition and explanation Comparison and Examples Advantages and Disadvantages Illegal and Legal uses.
A. Frank 1 Internet Resources Discovery (IRD) Peer-to-Peer (P2P) Technology (1) Thanks to Carmit Valit and Olga Gamayunov.
Introduction  What is an Operating System  What Operating Systems Do  How is it filling our life 1-1 Lecture 1.
High Performance Cooperative Data Distribution [J. Rick Ramstetter, Stephen Jenks] [A scalable, parallel file distribution model conceptually based on.
Chapter 3 : Distributed Data Processing
Introduction to client/server architecture
DATABASE MANAGEMENT SYSTEMS 2 ANGELITO I. CUNANAN JR.
Virtual Network Servers. What is a Server? 1. A software application that provides a specific one or more services to other computers  Example: Apache.
1 MASTERING (VIRTUAL) NETWORKS A Case Study of Virtualizing Internet Lab Avin Chen Borokhovich Michael Goldfeld Arik.
Installing and Setting up mongoDB replica set PREPARED BY SUDHEER KONDLA SOLUTIONS ARCHITECT.
P2P File Sharing Systems
System Architecture & Hardware Configurations Dr. D. Bilal IS 592 Spring 2005.
Windows Server MIS 424 Professor Sandvig. Overview Role of servers Performance Requirements Server Hardware Software Windows Server IIS.
Cloud Computing for the Enterprise November 18th, This work is licensed under a Creative Commons.
Stuart Cunningham - Computer Platforms COMPUTER PLATFORMS Network Operating Systems Week 9.
About the Presentations The presentations cover the objectives found in the opening of each chapter. All chapter objectives are listed in the beginning.
INTRODUCTION TO CLOUD COMPUTING CS 595 LECTURE 7 2/23/2015.
Components of Windows Azure - more detail. Windows Azure Components Windows Azure PaaS ApplicationsWindows Azure Service Model Runtimes.NET 3.5/4, ASP.NET,
TRANSFORMING IT.  The cloud is a group of servers.  A user interacts with the cloud without worrying about how it is implemented. Cloud computing describes.
Virtualization. Virtualization  In computing, virtualization is a broad term that refers to the abstraction of computer resources  It is "a technique.
Layered Protocol. 2 Types of Networks by Logical Connectivity Peer to Peer and Client-Server Peer-to-peer Networks  Every computer can communicate directly.
High Performance Computing Cluster OSCAR Team Member Jin Wei, Pengfei Xuan CPSC 424/624 Project ( 2011 Spring ) Instructor Dr. Grossman.
Client-Server & Peer-to-Peer Networks
Networking Basics Lesson 1 Introduction to Networks.
Computers Are Your Future Tenth Edition Chapter 8: Networks: Communicating & Sharing Resources Copyright © 2009 Pearson Education, Inc. Publishing as Prentice.
NETWORK SERVERS Oliver Topping (with a little help from my Mum)
Bit Torrent A good or a bad?. Common methods of transferring files in the internet: Client-Server Model Peer-to-Peer Network.
Large Scale Parallel File System and Cluster Management ICT, CAS.
ITEP computing center and plans for supercomputing Plans for Tier 1 for FAIR (GSI) in ITEP  8000 cores in 3 years, in this year  Distributed.
A P2P-Based Architecture for Secure Software Delivery Using Volunteer Assistance Purvi Shah, Jehan-François Pâris, Jeffrey Morgan and John Schettino IEEE.
VMware vSphere Configuration and Management v6
GridNEWS: A distributed Grid platform for efficient storage, annotating, indexing and searching of large audiovisual news content Ioannis Konstantinou.
Communications & Networks National 4 & 5 Computing Science.
Hands-On Virtual Computing
1 Reforming Software Delivery Using P2P Technology Purvi Shah Advisor: Jehan-François Pâris Department of Computer Science University of Houston Jeffrey.
Bit Torrent Nirav A. Vasa. Topics What is BitTorrent? Related Terms How BitTorrent works Steps involved in the working Advantages and Disadvantages.
Geethanjali College Of Engineering and Technology Cheeryal( V), Keesara ( M), Ranga Reddy District. I I Internal Guide Mrs.CH.V.Anupama Assistant Professor.
Network - definition A network is defined as a collection of computers and peripheral devices (such as printers) connected together. A local area network.
Automating Xen Virtual Machine Deployment Kris Buytaert
rain technology (redundant array of independent nodes)
Telecommunications and Networks
Let's talk about Linux and Virtualization in 'vLAMP'
Instructor Materials Chapter 10: Application Layer
NGS data transmission, A point view from a user
Netscape Application Server
System Architecture & Hardware Configurations
Distributed Systems.
Heterogeneous Computation Team HybriLIT
Virtual Network Computing
Welcome! Thank you for joining us. We’ll get started in a few minutes.
3.1 Types of Servers.
The Client/Server Database Environment
System Architecture & Hardware Configurations
CHAPTER 1 INTRODUCTION:
Introduction to client/server architecture
Networks Intro to Digital Technology
Computing in the Modern World Unit 4
CompTIA Server+ Certification (Exam SK0-004)
Comparison of LAN, MAN, WAN
Department of Computer Science Course : Pre Computer Skills
Peer-to-Peer (p2p) Systems
Software models - Software Architecture Design Patterns
Client/Server and Peer to Peer
Sending data to EUROSTAT using STATEL and STADIUM web client
Presentation transcript:

Andrea Righi – LinuxDay 2007 (Oct 27) Installare GNU/Linux su un ampio numero di client con SystemImager e BitTorrent Andrea Righi

Andrea Righi – LinuxDay 2007 (Oct 27) The problem You have to install a huge number of computers: ✔ PC labs, ✔ Server farms, ✔ HPC clusters, ✔ Complex grid-computing environments, ✔ Etc. But you don't have enough time!

Andrea Righi – LinuxDay 2007 (Oct 27) Some practical examples

Andrea Righi – LinuxDay 2007 (Oct 27) The tool SystemImager is a software which automates GNU/Linux installs, software distributions and production deployment

Andrea Righi – LinuxDay 2007 (Oct 27) Major Goals Support all Linux distributions Support a large number of architectures Make it solve massive installation problems Create a centralized point of installation and maintenance

Andrea Righi – LinuxDay 2007 (Oct 27) What does it do for me? System Installation System Updates Build replicants of machines File system or block device migration

Andrea Righi – LinuxDay 2007 (Oct 27) Advantage of SystemImager It's open-source (GNU GPL v2) File-oriented approach: ✔ Distribution agnostic ✔ Hardware independence ✔ Filesystem independence ✔ Plain filesystem dump: exclude swap space or unused partitions ✔ Block device independence ✔ Live customization (manipulate cloned filesystems directly)

Andrea Righi – LinuxDay 2007 (Oct 27) Basic concepts

Andrea Righi – LinuxDay 2007 (Oct 27) Basic concept: image Image: ✔ Live snapshot of a machine containing files and directories from the root of that machine's filesystem ✔ chroot-able filesystem stored in /var/lib/systemimager/images/$NAME ✔ Examples: ✔ /var/lib/systemimager/images/RHEL4 ✔ /var/lib/systemimager/images/Ubuntu_gutsy ✔ /var/lib/systemimager/images/HPC_1.0 ✔...

Andrea Righi – LinuxDay 2007 (Oct 27) Basic concept: image server Image Server: ✔ a server that has all the images available for the installation ✔ “Jukebox” of images

Andrea Righi – LinuxDay 2007 (Oct 27) Basic concept: client Client: ✔ a machine to be auto-installed with a (single) pre- selected image ✔ Example: the dancing penguins are the clients :-)

Andrea Righi – LinuxDay 2007 (Oct 27) Basic concept: transport Transport: ✔ the protocol used to distribute images from the image server to the clients ✔ push/pull/p2p approach ✔ Examples: ✔ rsync, multicast, SSL, BitTorrent,...

Andrea Righi – LinuxDay 2007 (Oct 27) Application stack (SIS => SystemImager) Image Server (SystemImager) HPC clusters HA- clusters Web farms PC labs Clients

Andrea Righi – LinuxDay 2007 (Oct 27) Application stack (SIS => SystemImager) Image Server (SystemImager) Golden client HPC clusters HA- clusters Web farms PC labs Software (Debootstrap, YaST, yum, SystemInstaller,...) Clients

Andrea Righi – LinuxDay 2007 (Oct 27) Application stack (SIS => SystemImager) Image Server (SystemImager) Golden client HPC clusters HA- clusters Web farms PC labs Software (Debootstrap, YaST, yum, SystemInstaller,...) Clients RHEL4 HPC- 1.0 SUSE1 0 Debian 4

Andrea Righi – LinuxDay 2007 (Oct 27) Application stack (SIS => SystemImager) Image Server (SystemImager) Golden client HPC clusters HA- clusters Web farms PC labs Software (Debootstrap, YaST, yum, SystemInstaller,...) Clients RHEL4 HPC- 1.0 SUSE1 0 Debian 4 SystemImager transports + SystemConfigurato r

Andrea Righi – LinuxDay 2007 (Oct 27) Application stack (SIS => SystemImager) Image Server (SystemImager) Golden client HPC clusters HA- clusters Web farms PC labs Software (Debootstrap, YaST, yum, SystemInstaller,...) Clients RHEL4 HPC- 1.0 SUSE1 0 Debian 4 SystemImager transports + SystemConfigurato r

Andrea Righi – LinuxDay 2007 (Oct 27) Transports

Andrea Righi – LinuxDay 2007 (Oct 27) rsync transport rsync (plain / SSL encrypted): ✔ Client-server approach ✔ Limited in scalability ✔ Limited in reliability with a lot of clients ✔ Max Theoretical Bandwidth: Us / N

Andrea Righi – LinuxDay 2007 (Oct 27) rsync diagram Image server Client 1 Client 2 Client N... Upload bandwidth = U s Download bandwidth = U s / N

Andrea Righi – LinuxDay 2007 (Oct 27) Multicast (Flamethrower) transport Flamethrower: ✔ Multicast approach: ✔ Perfect scalability ✔ But limited in reliability ✔ Max Theoretical Bandwidth: Us

Andrea Righi – LinuxDay 2007 (Oct 27) Multicast (Flamethrower) diagram Image server Client 1 Client 2 Client N... Upload bandwidth = U s Download bandwidth = U s

Andrea Righi – LinuxDay 2007 (Oct 27) BitTorrent overview BitTorrent is a TCP/IP p2p oriented protocol designed for transferring files Peers connect to each other directly to send and receive chunks of data There is a central server (tracker) which coordinates the action of all such peers The tracker does not have any knowledge of the contents of the files being distributed Users upload (transmit outbound) at the same time they are downloading (receiving inbound)

Andrea Righi – LinuxDay 2007 (Oct 27) BitTorrent transport BitTorrent: ✔ p2p approach: scalability && reliability ✔ Qiu and Srikant model ✔ Total upload rate: ✔ Steady state: ✔ x(t) downloaders => ✔ y(t) uploaders => ✔ Max Theoretical Bandwidth: ✔ Us (steady state)

Andrea Righi – LinuxDay 2007 (Oct 27) BitTorrent diagram Image server (+tracker) Client 1 Client 2 Client N... Upload bandwidth = U s Download bandwidth = U s (in steady state)

Andrea Righi – LinuxDay 2007 (Oct 27) Testbed environment

Andrea Righi – LinuxDay 2007 (Oct 27) BCX/5120 IBM BCX/5120, with 5120 cores, is the largest computer in Italy for Scientific Computing 2 dual-core AMD Opteron(tm) 2.4GHz, 8GB RAM per node It is the 44th most powerful computer in the world (TOP500).

Andrea Righi – LinuxDay 2007 (Oct 27) BCX network topology Cisco Catalyst 6513 IBM Bladecenter H (from 1 up to 11) IBM LS21 switch IBM Blade LS21 (x 14) BladeCenter H internal bus (1Gb/s) Gb/s x 4 uplinks IBM e326m (image server) 1 Gb/s link

Andrea Righi – LinuxDay 2007 (Oct 27) Experimental results

Andrea Righi – LinuxDay 2007 (Oct 27) Deployment time

Andrea Righi – LinuxDay 2007 (Oct 27) Rsync: 50 clients (download rate)

Andrea Righi – LinuxDay 2007 (Oct 27) Multicast: 50 clients (download rate)

Andrea Righi – LinuxDay 2007 (Oct 27) BitTorrent: 50 clients (download rate)

Andrea Righi – LinuxDay 2007 (Oct 27) Rsync: 50 clients (aggregated bandwidth)

Andrea Righi – LinuxDay 2007 (Oct 27) Multicast: 50 clients (aggregated bandwidth)

Andrea Righi – LinuxDay 2007 (Oct 27) BitTorrent: 50 clients (aggregated bandwidth)

Andrea Righi – LinuxDay 2007 (Oct 27) Conclusion

Andrea Righi – LinuxDay 2007 (Oct 27) Advantages Quicker deployment of images Safer deployment (better error handling) Less load on the image server ✔ no need to buy a powerful machine

Andrea Righi – LinuxDay 2007 (Oct 27) Disadvantages More disk space consumption! ✔ images + tarballs of images Images and tarballs must be kept in-sync ✔ Re-generate tarball and.torrent at each image change

Andrea Righi – LinuxDay 2007 (Oct 27) Future work Improve security (encryption of BT tarballs) Exploit the p2p approach to create distributed and redundant repositories of custom image Virtual cluster deployment (re-imaging using the same physical resource pool) Use BT transport also for updates (pushing changes/differences of images)

Andrea Righi – LinuxDay 2007 (Oct 27) References Web: ✔ Mailing list: ✔ ✔ IRC: ✔ #sisuite (irc.freenode.net)

Andrea Righi – LinuxDay 2007 (Oct 27) Tank you for attending