1 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok A PlanetLab (Europe) Short Tutorial Serge Fdida Université Pierre & Marie Curie, LIP6 Paris, France An Open Federated Laboratory Supporting Network Research for the Future Internet

2 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok PlanetLab (-Europe) An open, shared platform for developing, deploying, and accessing planetary scale applications

3 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok PlanetLab short Tutorial ● Introduction Starting from the Customer end PlanetLab - what is it? First steps PlenatLab architecture ● Operation Operation of the testbed? Monitoring ● Beyonf PlanetLab Federation

4 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok In this tutorial Introduction to PlanetLab ● Tutorial Step 1: Register with the system ● Tutorial Step 2: Generate an SSH key PlanetLab Architecture ● Tutorial Step 3: Adding a Node ● Tutorial Step 4: Creating a Slices ● Tutorial Step 5: Assign users to a slice ● Tutorial Step 6: Adding nodes to a slice PlanetLab Tools ● Tutorial Step 7: Log in to a sliver ● Other things to do with on a sliver

5 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Overview ● Global distributed system infrastructure platform for long running services testbed for network experiments ● Launched in March 2002

6 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Why Use PL? ● Example: Content delivery network. ● Deploys content and allows access ● How do I build a content deliver network? ● Proof: Does it work? ● Start with a few nodes, for content delivery system. Build up. Test to see if the system works as the system builds and grows, and is a large scale testbed.

7 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok About PlanetLab nodes around the world – 41 countries – 487sites (universities, research labs) – 5030 registered users, 630 active slices – more than 1000 researchers A collection of machines distributed over the globe – Most of the machines are hosted by research institutions – All of the machines are connected to the Internet Software – All PlanetLab machines are administered by a system called MyPLC The software is based on Fedora Core 8

8 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Content Distribution Network ~330 (open) caching proxy servers Open to all users (see URL) Highly available (after lots of work!) Spawned many subprojects / services: – CoBlitz, scalable distribution of large files. – CoDeploy, efficient synchronization for slices. – CoDNS, fast and reliable name lookup. – CoMon, node monitoring for PlanetLab – CoTest, login debugging tool for nodes – PlanetSeer, distributed network anomaly tracing Illustrates how deployment of a real service spurs research – Example: CoDeeN (Princeton)

9 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok PlanetLab is not… ● A distributed supercomputer ● A simulation platform ● An Internet emulator ● An arena for repeatable experiments ● Completely representative of the current Internet ● Grid

10 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Organizational Challenges ● PlanetLab Consortium Licensing Agreement ● Organization is responsible for the activities of their researchers ● Users may not use PlanetLab to disseminate copyright material ● PlanetLab tracks resources usage and maintains a list of published intentions ● Otherwise, PlanetLab is a free for all

11 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok The PlanetLab Europe Consortium ● Membership ● The Consortium includes eight membership levels Charter (300k € annual dues) Full (75k € annual dues) Associate (25k € annual dues) Sponsor (10k € annual dues) SME (1k € annual dues) Director (no annual dues) Academic (no annual dues) FP7 (no annual dues)

12 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Using PlanetLab ● Central Website that manages All accounts All nodes All resources ● Registering with PLC (your PlanetLab Central) ● 3 PLC PL USA (planet-lab.org) PL Europe (planet-lab.eu) PL Japan (planet-lab.jp)

13 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Architecture (people) ● 3 Roles PI (Principal investigator) ■ Manage users / slices of his site Tech ■ Manage nodes User ■ Use slices and nodes to launch experiments ● Special role for administration / Management

14 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok PI’s Roles and Responsibilities Oversight. PIs are responsible for overseeing all slices that they create on behalf of the users at their site. Account management. PIs can: – Enable, disable, and delete user accounts. – Create slices. – Delete slices. – Assign users to slices. – Allocate resources to slices. Node management. PIs are responsible for the physical maintenance of the nodes at their site

15 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok PlanetLab Architecture

16 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Terminology ● Site A: site is a physical location where PlanetLab nodes are located (e.g. Fraunhofer Institute or UCL). ● Node A: node is a dedicated server that runs components of PlanetLab services. ● Slice A: slice is a set of allocated resources distributed across PlanetLab. Slices are assigned to some number of PlanetLab nodes. Slices have a finite lifetime and must be periodically renewed to remain valid. ● Sliver A: sliver is a slice running on a specific node. You can use ssh to login to a sliver on a specific node.

17 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Architecture (Hardware) OVH UPMC INRIA Private MyPLC Princeton (PLC) SVN MyPLC Backup MyPLC Tutorial Private MyPLC Backup MyPLC Mail DNS Mailing list Monitoring Backup

18 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Architecture: general view Users Node SSH DB server (postgresql) MyPLC Boot server (apache) Web server (apache) API server (apache, Mod_python) HTTP(S) XML-RPC HTTPS XML-RPC HTTPS XML-RPC HTTPS XML-RPC HTTPS

19 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Virtualization ● Why virtualization Security ■ A hacked account is « locked » into a virtual machine Isolation ■ Users cannot see each other ■ Fair share of ressources Give root access / own environment ■ yum install,... Ease of install / shutdown

20 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Virtualization Node Context0 (root context) site_admin Slice1 site_adminSlice3site_adminSlice2 For PL's admins Full access For local tech Restricted access For users

21 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Nodes

22 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Slices – hujiple_isis

23 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Slices – upmcple_paristr

24 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Slices

25 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Security (user) User Private SSH key PLE (MyPLC) Public SSH key Node Key Node Key Node Key Public SSH key

26 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Trust Relationships 1) PLC expresses trust in a user by issuing it credentials to access a slice 2) Users trust to the PLC to create slices on their behalf and inspect credentials 3) Owner trusts PLC to set users and map network activity to right user 4) PLC trusts owner to keep nodes physically secure Service Developer (User)T PLC Node Owner

27 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Security Architecture ● Node Operating System – isolates slivers – audits behavior ● PlanetLab Central (PLC) – remotely manages nodes – bootstrap services to instantiate and control slices – monitor sliver/node health

28 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Node Architecture Virtual Machine Monitor (VMM) Hardware Local Admin Slice VM 1 VM 2 VM n & Node Mgr

29 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok VMM Linux – significant mindshare Vserver – scales to hundreds of VMs per node (12MB each) Scheduling – CPU fair share per sliver (guarantees possible) – link bandwidth fair share per sliver average rate limit: 1.5Mbps (24 hour bucket size) peak rate limit: set by each site (100Mbps default) – disk 5GB quota per sliver (limit runaway log files) – memory no limit pl_mom resets biggest user at 90% utilization

30 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Auditing & Monitoring PlanetFlow – logs every outbound IP flow on every node retrieves packet headers, timestamps, context ids (batched) – used to audit traffic – aggregated and archived at PLC SliceStat – has access to kernellevel/ systemwide information – used by global monitoring services – used to performance debug services

31 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok PlanetLab Tutorial A step by step guide

32 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok First steps to using PlanetLab 1. Register with the system 2. Create an ssh key 3. Create a slice 4. Add nodes to the slice 5. Describe the slice 6. Log into a sliver

33 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Step 1: Register with the system ● Go to

34 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok ● Click on Create an account Step 1: Register with the system

35 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok ● Fill in personal information first name, last name, title telephone number Step 1: Register with the system

36 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok ● Choose a password and valid as login Step 1: Register with the system

37 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok ● Select your site Université Pierre et Marie Curie Step 1: Register with the system

38 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Step 1: Register with the system ● Click Register

39 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok User Registration User registers on website User receives to verify registration User verifies address PI receives request to enable user PI enables account Approv e? User receives confirmation Yes Rejected No

40 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok PI Enables Account ● Log in as user ● Choose Enable under CHOOSE ACTION menu

41 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok PI Enables Account

42 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Account approval (as PI) Principal Investigator Account

43 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Account approval (as PI) ● Choose Enable User name Principal Investigator Account

44 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok VIDEO – Register with PLE

45 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Step 2: Generate an ssh key ● Access to resources is secured through public key encryption ● You must upload your public ssh key to use PlanetLab

46 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Step 2: Generate an ssh key ● PlanetLab uses 1024bit RSA keys for authentication on nodes ● Generate an ssh key using OpenSSH: sshkeygen –t rsa f /.ssh/id_planetlab ● Please use a secure passphrase

47 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok VIDEO – ● Generating and uploading an ssh key

48 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Step 3: Adding nodes ● Only a Tech can add a node to a site ● Nodes MUST Have a visible, public DNS name Be outside of a NAT or FIREWALL Recommended to not have an IDS or Proxy upstream of the node

49 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Recommended Configurations ● Recommended Configuration More than 1 GB RAM At least 100 GB hard disk At least 1 Mb/sec connection to the Internet P4 or better CPU External PCU optional 49

50 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok How to add a node ● Tech registers the node with the PLC ● Download iso from the PLC ● Burn to CD or copy to USB ● Insert media on node ● Boot from media ● Answer Yes to installation request ● Wait 60 minutes or more while the node is formatted and installed (depending on network connection) 50

51 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Adding nodes ● Fill in form, Click Add

52 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok VIDEO – Adding a node

53 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Step 4: Creating a slice ● Actually, ask your PI to create a slice… Principal Investigator Account

54 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Step 4: Creating a slice ● Enter slice name _

55 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Step 4: Creating a slice ● Enter project url

56 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Step 4: Creating a slice ● Enter description

57 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Step 4: Creating a slice ● Assign users to slice

58 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Step 4: Creating a slice ● Click Add Slice

59 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Your Slice is ready to serve

60 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok VIDEO – Cretaing a slice

61 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok By now… ● You have a user account ● Your PI has approved it ● Your PI has created a slice ● Your PI has assigned you as a user of the slice ● Next step: add nodes to the slice

62 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Adding nodes to your slice ● Page for adding/removing nodes – Select site to see nodes to add – Shown are all nodes currently in slice – Nodes can be removed ● For your first time, it is recommended: – Add a couple of nodes, one nearby ● Note: Changes will take ~5 minutes to propagate on the tutorial system. It may take up to two hours on production PlanetLab system.

63 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Step 6: Adding Nodes to a slice User

64 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Step 6: Adding nodes to a slice 64

65 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Step 6: Adding nodes to a slice ● Search and select nodes to add easly 65

66 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Step 6: Adding nodes to a slice ● Nodes can be removed 66

67 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok VIDEO – Adding nodes to a slice

68 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Step 7: Log in to a sliver ● Your user name is your slice name ● E.g., with OpenSSH:

69 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Populating your sliver sudo yum install gcc ● sudo allows you to be root in your sliver ● yum is the redhat package manager ● You can use any RPM packages you want in your sliver ● Caveat Emptor: yum may take up to one hour to initialize and begin installation of your packages

70 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Deploying to a sliver scp -l huji_dimes \ -i ~/.ssh/id_rsa -r \ localdir evgsics7.sics.se: Copy localdir to the remote machine rsync -a -e “ssh -l huji_dimes” \ localdir evgsics7.sics.se: Update the remote directory. Copies only those files that are different

71 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Deploying Software to Your Nodes ● Copy binaries to the node to run – scp or rsync works for small numbers of nodes – Ensure library dependencies are satisfied

72 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok References ● PlanetLab official Web site: ● L. Peterson, S. Muir, Timothy Roscoe, and Aaron ● Klingaman PlanetLab Architecture: An Overview. Technical Report, PlanetLab, May 2006 ● L. Peterson and T. Roscoe. The Design Principles of PlanetLab. ● Operating Systems Review (OSR), 40(1):11.16, Jan

73 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Backup

74 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Operations PlanetLab (PLC) & PlanetLab Europe (PLE) An Open Federated Laboratory Supporting Network Research for the Future Internet

75 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Using PlanetLab ● Central Website that manages All accounts All nodes All resources ● Registering with PLx (your PlanetLab Central) ● 3 PLC PL USA (planet-lab.org) = PLC PL Europe (planet-lab.eu) = PLE PL Japan (planet-lab.jp) = PLJ

76 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok PlanetLab Europe overview Main operations centreEverLab opsPLC ops Nodes

77 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Using PlanetLab Europe Main operations centreEverLab opsPLC ops Nodes Researcher

78 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Using PlanetLab Europe Main operations centreEverLab opsPLC ops Nodes Researcher Slice request

79 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Using PlanetLab Europe Main operations centreEverLab opsPLC ops Nodes Request propagation Researcher

80 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Using PlanetLab Europe Main operations centreEverLab opsPLC ops Nodes Slice allocation Researcher

81 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Using PlanetLab Europe Main operations centreEverLab opsPLC ops Nodes Slice allocation Researcher

82 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Using PlanetLab Europe Main operations centreEverLab opsPLC ops Nodes Slice allocated Researcher

83 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Using PlanetLab Europe Main operations centreEverLab opsPLC ops Nodes Slice login Researcher

84 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Using PlanetLab Europe Main operations centreEverLab opsPLC ops Nodes Run experiment Researcher

85 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Using PlanetLab Europe Main operations centreEverLab opsPLC ops Nodes External users Researcher

86 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok PlanetLab Europe overview Main operations centreEverLab opsPLC ops Nodes

87 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok What we manage Main operations centreEverLab opsPLC ops Nodes

88 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Growth of PlanetLab Europe Main operations centreEverLab opsPLC ops Nodes

89 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Growth of PlanetLab Europe Main operations centreEverLab opsPLC ops Nodes Site

90 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Growth of PlanetLab Europe Main operations centreEverLab opsPLC ops Nodes PLC site managed by PLE

91 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Growth of PlanetLab Europe Main operations centreEverLab opsPLC ops Nodes

92 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Growth of PlanetLab Europe Main operations centreEverLab opsPLC ops Nodes Addition of wireless (new)

93 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Growth of PlanetLab Europe Main operations centreEverLab opsPLC ops Nodes Addition of emulation (new)

94 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Growth of PlanetLab Europe Main operations centreEverLab opsPLC ops Nodes

95 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Monitoring

96 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Monitoring ● Who needs monitoring? Operations Users Scientific research purposes European community ● Operations monitoring: Health / availability New features' deployments Load of the system Tools: ■ PlanetFlow ■ Monitor ■ EverStat ■ PLE Graphs

97 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Auditing & Monitoring ● PlanetFlow – logs every outbound IP flow on every node retrieves packet headers, timestamps, context ids (batched) – used to audit traffic – aggregated and archived at PLC ● SliceStat – has access to kernellevel/ systemwide information – used by global monitoring services – used to performance debug services

98 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Monitoring examples

99 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Monitoring examples

100 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Federation

101 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok What is federation? Site 1 API Site 2 API Ressources

102 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok What is federation? Site 1 API Site 2 API Ressources API

103 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Node Classical PLE PLE SM RAM CM ● Registry Users, slices, components, authorities Exports registry interface ● Slice Manager Create & control slices Export research interface ● Aggregate Manager Responsible for a set of components Exports slices & management interface ● Component Manager Controls a component Exports slices & management interface Node CM Node CM

104 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Classical PLE PLE SM RAM CM ● Slice interface Create & control slices/silvers ● Registry interface Bind & resolve naming info ● Managment interface Query & reboot components ● Uber research interface Slice interface,...

105 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Classical PLE PLE RAM CM SM USER

106 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Alternative SM PLE SM RAM CM EMULAB SM USER PLC SM RAM CM EMULAB SM USER

107 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Emulab (users and slices in PLC) PLC SM RAM CM EMULAB SM USER CM AM

108 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Vini PLC (single SM) PLC SM RAM CM VINI USER CM AM

109 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Full federation PLC SM RAM CM USER PLE SM RAM CM USER

110 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Concluding remarks ● Building a facility is a major challenge Complex process, High risk, non technical issues (IPR, Legal,) ● FIRE / Onelab is about: Supporting two complementary dimensions (NS & Exp) Enabling different federations – not one size fits all Basing on an existing ecosystem with an international community ● Onelab is already: Up and running! Independent and Federated Highly visible worldwide, seen as a peer wrt other testbeds Cooperation with « Pilot » projects (PSIRP, ANA, Haggle, 4Ward, FEDERICA) – looking for new partnerships. Aggregate tools of disperse communities

111 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Coffee Break

112 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Tutorial Site The tutorial (pdf slides) are available at: The three tutorials are: PLE Basics, lab.eu/files/PlanetLab_Basics.pdfhttps:// lab.eu/files/PlanetLab_Basics.pdf Technical Overview, lab.eu/files/PlanetLab__Tech_Overview.pdfhttps:// lab.eu/files/PlanetLab__Tech_Overview.pdf Adding a Node, lab.eu/files/PlanetLab_Adding_a_Node.pdfhttps:// lab.eu/files/PlanetLab_Adding_a_Node.pdf The video tutorials are available at: 112

113 - OneLab - AsiaFi– November 17, 2009 – AIT Bangkok Questions