1. WING: A Consistent Computing Platform Yiwei Ci 22/02/2012 ciyiwei@hotmail.com

2. Goals Give processes suitable places for computing. Take processes to the place suitable. Transparency – OS abstraction (SSI). – Hardware abstraction supports heterogeneous system Seamless – Dynamically switch workplace.

3. Challenges Security – Prevent malicious software? Availability – The availability of each computer is high today. How about the whole distributed system? Scalability – Provide just-in-time scalability?

4. Application scenarios Build your data center. – Exchange between data migration and computation migration. Build your virtual cluster. – PC + PC – PC + Mobi – Mobi + Mobi Build your test bed. – Easy deployment

5. Related works XtreemOS – Grid operating system supporting virtual organizations. – Latest version: 3.0 (2012/02/10) OpenSSI – Provide single process space, single I/O space, single IPC space, and single root for the cluster. – Latest version: 1.9.6 (2010/02/18) OpenMosix – A Linux kernel extension for single-system image clustering – Latest version: 2.4.26 (closed)

6. Related works OpenSSIOpenMosixKerrighed Single Administrative DomainYesNoYes Cluster Membership GuaranteesYesNo High-Availability (HA) ClusteringYesNo Single Process Management SpaceYesNoYes Process MigrationFullPartial Process Load BalancingYes Process Migration is HAYesNo Migrate Processes Using SemaphoresYes Migrate Processes Using Shared MemoryYesNoYes Single Thread MigrationOnly thread groupNo Process Checkpointing3rd party Yes Single IPC NamespaceYesNoYes http://wiki.openssi.org/go/Features

7. Related works OpenSSIOpenMosixKerrighed Distributed Shared MemoryMigratesNoYes Single PTY NamespaceYesNo Single-Site File NamingYesNo ? Coherent Cache / File AccessYes ? ? Migrate Active FilesystemExperimentalNo HA Cluster Filesystem (CFS)YesNo HA Single Cluster Name/AddressYesNo HA Network Load Balancer (LB)YesNo LB Auto Detects TCP/UDP IP ServicesYes (UDP 1.9.1)N/A Transparent Socket MigrationNot yetNoYes Automatic Service FailoverYesNo Diskless Nodes via Network BootYes (PXE/EtherBoot)NoYes http://wiki.openssi.org/go/Features

8. Outline Design Implementation Results Conclusions

9. Design Virtualization (1)

10. Design Virtualization (2)

11. Design Virtualization (3)

12. Design Virtualization (4)

13. Design Virtualization (5)

14. Design Basic Architecture

15. Design Metaphor Object Operation System callPathFile operation …/class_key_entry op id flags Metaphor: everything is a file Subject object subject

16. Design Resource – Computation resource Process – Data resource File Memory – Common resource IPC: msg, sem, shm …

17. Design Area Basically, an operating system has two areas for different processes. – Local area: local computing. – Global area: distributed computing. /mnt/repo The resources in local area are invisible to the processes in global area. The resources in global area are consistent.

18. Design Isolation – Process Isolation Processes in WING are “invisible” to the world outside. – Data Isolation Data in WING is “invisible” to the world outside. The private data of one user is “invisible” to other users. – Private files. – The private data inside a file.

19. Implementation Components WTRANS: kernel translator WLN: linker WRES: common resource WCR: checkpoint kernel module CFS: checkpoint file system WMAN: process manager LDFS: lightweight distributed file system IFS: image file system

20. Implementation Checkpoint Record the status of a process (transparent to the programmer) – Registers – Opened files – Signals – Credentials – Memory – IPC

21. Implementation Checkpoint – Incremental saving – Dynamic recovery – Dedicated file system

22. Implementation Distributed IPC message 、 semaphore 、 shared memory – System V IPC does not support distributed communication. – It is complex to dump and to restore the IPC status in kernel.

23. Implementation Distributed IPC – Coexist of conventional System V IPC and distributed IPC (the same interface). – Ensure the consistency of IPC resources in the distributed environment. – Stateless in kernel (for process migration) Re-implement IPC in user-space. provide a pseudo file system to store the status of the IPC resource (RAM based). – High availability

24. Implementation Distributed IPC – Event Driven Event-Flow

25. Implementation Msg: Requirements: 1) msgtyp = 0 ： the first message on the queue is received. 2) msgtyp > 0 ： the first message of type msgtyp is received. 3) msgtyp < 0 ： the first message of the lowest type that is less than or equal to the absolute value of msgtyp is received. Sem Requirements: 1) RPC has a time-out mechanism (for semtimedop). 2) RPC has an undo mechanism (for exit_sem).

26. Implementation Shared memory Problems: – Find (key, value) pair – Frequently update of (key, value) Consistency model: – Sequential consistency Features: – Multi-owner – Versioning – Write invalidation

27. Implementation Shared memory Handle shm_fault

28. Implementation Shared memory Handle shm_fault

29. Implementation Image File System (IFS) Data can be shared, but the data for each user needs to be protected. – Each user can have a different view of a file. – The processes of the same user have the same view of a file.

30. Implementation Image File System

31. Results Environment: – VM (x2): 512MB RAM, 2 processors, NAT – OS: Based on kernel 2.6.29.6 – Host: 2048MB RAM, Intel Core 2 Duo CPU T6570 Experiments: – Msg – Sem

32. Results Msg Leader: 1.Use msg( key 0 ) to collect start requests from members; 2.If all start requests are received then 3. Use msg( key i ) (i > 0) to send start i ; 4. Use msg( key N+1 ) to collect stop requests; 5. if all stop requests are received then 6. return success; 7.return fail; Member i: 1.Use msg( key 0 ) to send start request to leader; 2.If start i is successfully received by msg( key i ) then 3. create_process(msg_snd); 4. create_process(msg_rcv); 5. if msg_snd and msg_rcv are finished then 6. Use msg( key N+1 ) to send stop request to leader; Process msg_snd 1.for n = 1 to ROUNDS do 2. Use msg( key 0 ) to receive req; 3. Build mtext (|mtext| = req.msgsz); 4. i := req.src; 5. Use msg( key i ) to send mtext; Process msg_rcv at member i 1.for n = 1 to ROUNDS do 2. req.msgsz := rand() % MSG_SIZE + 1; 3. req.src := i; 4. Use msg( key 0 ) to send req; 5. Use msg( key i ) to receive mtext; 1 leader, 1 member, ROUNDS = 1000, MSG_SIZE = 128, time = 65.36134 sec

33. Results sem 1 leader, 1 member, ROUNDS = 1000, time = 8.810316 sec Leader: 1.Create sem( key 0 ) (sem( key 0 ) has N items); 2.Assign each item of sem( key 0 ) to 0; 3.If all items of sem( key 0 ) are not 0 then 4. for i = 1 to N do 5. remove sem( key i ); Member i: 1.Create sem( key i ); 2.If all sem( key j ) (j ≠ i) are created then 3. for n = 1 to ROUNDS do 4. k := rand() % N + 1; 5. down(sem( key k )); 6. up(sem( key k )); 7. endfor 8. sem( key 0 ).item i := 1;

34. Conclusions By operating system virtualization, WING provides processes in different nodes with consistent views of the distributed resources. There are no additional libraries required. Conventional multi-task applications can be used for distributed computing.

35. Future work Components – WMAN: process manager – LDFS: lightweight distributed file system Tools: – Profiler – Test suits Stability Security

36. Thanks

37. Q&A