Processes: code migration

Process Concept An operating system executes a variety of programs: Batch system – jobs Time-shared systems – user programs or tasks Process – a program in execution; process execution must progress in sequential fashion A process includes: program counter stack data section

Esempio: kernel linux 2.6.20.4 struct task_struct { volatile long state; /* -1 unrunnable, 0 runnable, >0 stopped */ … #ifdef CONFIG_SMP #ifdef __ARCH_WANT_UNLOCKED_CTXSW int oncpu; #endif int prio, static_prio, normal_prio; cpumask_t cpus_allowed; unsigned int time_slice, first_time_slice; /* task state */ struct linux_binfmt *binfmt; long exit_state; int exit_code, exit_signal; pid_t pid; pid_t tgid;

Esempio: kernel linux 2.6.20.4 #ifdef CONFIG_CC_STACKPROTECTOR /* Canary value for the -fstack-protector gcc feature */ unsigned long stack_canary; #endif … /* process credentials */ uid_t uid,euid,suid,fsuid; gid_t gid,egid,sgid,fsgid; struct group_info *group_info; kernel_cap_t cap_effective, cap_inheritable, cap_permitted; /* CPU-specific state of this task */ struct thread_struct thread; /* filesystem information */ struct fs_struct *fs; /* open file information */ struct files_struct *files; /* namespaces */ struct nsproxy *nsproxy; /* signal handlers */ struct signal_struct *signal; struct sighand_struct *sighand; };

Diagram of Process State

CPU Switch From Process to Process

Single and Multithreaded Processes

User level threads Cheap to create and destroy threads Thread administration is in user-space Create: allocating memory for a thread stack Destroy: freeing memory of the stack Create and Destroy are cheap Switching between threads is fast Store the CPU registers of the current thread Load the ones for the next thread

User level threads: Problem? Invocation of a blocking call immediately block the entire process including all the threads of that process eg: blocking on I/O A thread blocking on I/O should never prevent other parts from being executed

Kernel Level threads Implement threads in the OS’s kernel Create, delete, synchronization, … all take place via a system call Switching threads is as expensive as switching processes So, lose the benefit of using a thread instead of a process

Windows XP Threads Implements the one-to-one mapping Each thread contains A thread id Register set Separate user and kernel stacks Private data storage area The register set, stacks, and private storage area are known as the context of the threads The primary data structures of a thread include: ETHREAD (executive thread block) KTHREAD (kernel thread block) TEB (thread environment block)

Linux Threads Linux refers to them as tasks rather than threads Thread creation is done through clone() system call clone() allows a child task to share the address space of the parent task (process)

Thread Usage in Nondistributed Systems Context switching as the result of IPC

Thread Implementation Combining kernel-level lightweight processes and user-level threads.

Multithreaded Servers (1) A multithreaded server organized in a dispatcher/worker model.

Multithreaded Servers (2) Model Characteristics Threads Parallelism, blocking system calls Single-threaded process No parallelism, blocking system calls Finite-state machine Parallelism, nonblocking system calls Three ways to construct a server.

The X-Window System Motif - Open Software Foundation (OSF) Provides GUI interface to UNIX based systems A protocol, not a product Operating system independent X win implementation: Motif - Open Software Foundation (OSF) Open Look - Sun/AT&T The Client/Server terminology is REVERSED for X Windows Client: the application which requests the GUI display and provides the service. Located on the server node Server: Program that provides the GUI display

The X-Window System The basic organization of the X Window System

Servers: General Design Issues Client-to-server binding using a daemon as in DCE Client-to-server binding using a superserver as in UNIX 3.7

Reasons for Migrating Code The principle of dynamically configuring a client to communicate to a server. The client first fetches the necessary software, and then invokes the server.

Degrees of Mobility Data Control Code State Execution Navigational Autonomy Transfer Direction Message Passing Move In/Out RPC Out Remote Execution Code on Demand In Process Migration Mobile Agents (weak) Own Mobile Agents (strong)

System Examples Types Systems Message Passing Socket, PVM, MPI RPC Xerox Courier, SunRPC, RMI Remote Execution Servlets, Remote evaluation, Tacoma Code on Demand Applets, VB/Jscripts Process Migration Condor, Sprite, Olden Mobile Agents (Weak Migration) IBM Aglets, Voyager, Mole Mobile Agents (Strong Migration) Telescript, D’Agent, Ara

Remote Execution Procedure code is sent together with arguments. Server behaves like a general cycle server. Server can evolve itself. Client Server Control Main Program Function Object f( ) Function/object transfer Dispatcher Function Object Argument transfer Arguments Remote execution Return value

Code on Demand Server behaves like a general remote object server. A remote function/object is sent back as a return value. Client executes the function/object locally. Client execution control stays in local while suspended upon a request to a server. Client Server Control Main Program func( ) Request a remote function/object Dispatcher Locally executed Function Object Function/object itself is returned. Remote Function Object

Process Migration Source Site Destination Site Selecting a process to be migrated Selecting the destination node Suspending the process Capturing the process state Sending the state to the destination Resuming the process Forwarding future messages to the destination Process P1 : Execution suspended Source Site Destination Site Resumed Transfer of control Time Freezing time

Process Migration Benefits Better response time and execution speed-up Dynamic load balancing among multiple nodes Using a faster CPU Higher throughput and Effective resource utilization Migrating I/O and CPU-bound processes to file and cycle servers. Reducing network traffic Migrating processes closer to the resources they are using most heavily. Improving system reliability Migrating processes from a site in failure to more reliable sites Replicating and migrating critical processes to a remote.

Process Migration State Capturing CPU registers Captured upon a freeze Address space Difficult to restore pointers I/O state: Fast I/O Operations Completed before a process migration Durable I/O Operations like files and user interactions Difficult to carry files in use and to freeze/restore system calls. Necessity to maintain a connection with I/O established at the source node. Some popular files available at the destination node

Code Migration Issues Not always possible to migrate resources A TCP/IP connection, for example Transferring a reference is not always a problem Eg: A URL to a file 3 Types of process-to-resource bindings Binding by Identifier Binding by Value Binding by Type

Code Migration Issues Binding by Identifier (Strongest form) process precisely requires the identified resource Eg: refer to an FTP server by its IP address Binding by Value only the value of the resource is needed another resource can provide the same value (C libraries) Binding by Type (Weakest form) just need a resource of specific type Eg: a local printer, monitor, …

Code Migration Issues Unattached Resources can be easily moved between various machines Eg: data files associated with the program Fastened Resources Very expensive to move these Eg: Local databases, complete Web sites Fixed Resources intimately bound to a specific machine Eg: local devices

Initiation of Migration Operating system when goal is load balancing Process when goal is to reach a particular resource

Migration Must destroy the process on the source system Process control block and any links must be moved

What is Migrated? Eager (all):Transfer entire address space no trace of process is left behind if address space is large and if the process does not need most of it, then this approach my be unnecessarily expensive

Precopy Precopy: Process continues to execute on the source node while the address space is copied pages modified on the source during precopy operation have to be copied a second time reduces the time that a process is frozen and cannot execute during migration

What is Migrated? Eager (dirty): Transfer only that portion of the address space that is in main memory any additional blocks of the virtual address space are transferred on demand the source machine is involved throughout the life of the process good if process is temporarily going to another machine good for a thread since the threads left behind need the same address space

Copy on reference Copy-on-reference: Pages are only brought over on reference variation of eager (dirty) has lowest initial cost of process migration

What is Migrated? Flushing: Pages are cleared from main memory by flushing dirty pages to disk later use copy-on-reference strategy relieves the source of holding any pages of the migrated process in main memory

Process Migration Address Transfer Mechanisms Total Freezing Pretransferring Transfer-on-reference Source node Destination node Source node Destination node Source node Destination node Suspended Migration decision Migration decision Migration decision Suspended Freezing time Transfer of address space Transfer of address space On-demand transfer Freezing time resumed Suspended Freezing time resumed resumed Merits: easy implementation Demerits: long delay time Merits: freezing time reduce Demerits: total time extended Merits: quick migration Demerits: large memory latency

Process Migration Message Forwarding Mechanisms Resending messages Ask origin site Sender Origin Dest 1 Dest 2 Receiver Migrate Migrate again Resend Resend again Origin Sender Receiver Send Send Migrate Forward Dest 1 Migrate again Dest 2

Process Migration Message Forwarding Mechanisms (Cont’d) Link traversal Link Update Origin Origin Sender Receiver Sender Receiver Send Send New location Forward Send Link Migrate Migrate Send Dest 1 Dest 1 New location Forward Link Migrate again Send Migrate again Current location Dest 2 Dest 2

Process Migration Heterogeneous Systems Using external data representation Floating-point data External data representation must have at least as much space as the longest floating-point data representation Process migration is restricted to only the machines that can avoid the over/underflow and the loss of precision. Architectural-dependent data representation Signed-infinity and signed-zero In general, process migration over heterogeneous systems are too expensive Conversion work Architectural-dependent representation handling Always interrupting external data representation Java

Process migration examples Early work XOS, Worm, Butler, DEMOS/MP Transparent migration in Unix-like systems Locus, OSF/1 AD, MOSIX, Sprite OS with Message-passing interface Charlotte, Accent, V Kernel Microkernels RHODOS, Arcade, Chorus, Amoeba, Birlix, Mach User-space migrations Condor, Migratory PVM, LSF, … Application-specific migrations Freeman, Skordos, Bharat & Cardelli (Migratory Applications) Mobile objects Emerald, SOS, COOL Mobile agents: derived from 2 fields AI Distributed systems Telescript, Agent Tcl, TACOMA, Mole, ..

Distributed Global States Operating system cannot know the current state of all process in the distributed system A process can only know the current state of all processes on the local system Remote processes only know state information that is received by messages these messages represent the state in the past

Thread Migration Only thread stack and registers copied All shared resources of the process remain on the source machine Assumptions: A process for the same program has been started in the destination machine The code is in the same virtual memory area on both machines Problems: Pointers to stack Pointers to heap Heap accessed by multiple threads …

Thread Migration Problems Heap Pointers Disallow the use of the heap Virtual common heap by a DSM Stack Pointers Pointer manipulation Preventive stack reservation

Thread migration goals Exploitation of resource locality Accessing more processing power Resource sharing Fault resilience Load balancing

Thread Migration Examples Emerald Ariadne Amber Millipede UPVM Active Threads …

Process Migration Mechanism Condor (Univ. of Wisconsin) Harvesting Idle Machines from Cluster of Workstations Checkpoint-based Migration Homogeneous system Checkpointing/Migration No source code change Link with Condor Checkpointing Library New signal handler System call wrapper Migration Invocation Asynchronously using signal: sig_checkpoint

Process Migration in Condor(1) Process Address Space Text, data and stack Stack Heap Uninitialized data initialized data Text Data

Process Migration in Condor(2) Open Files State: file descriptor number, mode, offset, dup info Use system call augmentation Wrapper open(), dup(), lseek(), close(), etc. Catch info and store at process’ data space, Then, call original system call Signals Signal handling attr block/ignore, default/custom_handler Store using sigprocmask(), sigaction() Pending signals Use sigispending()

Process Migration in Condor(3) CPU state Registers: integer/floating point, special purpose H/W status: floating point hardware, instruction queues Do we need to manually save all these state info? No! Note that checkpointing is handling of a custom signal Unix signal handling does this job for us. File Access Require system to have something like NFS? No. it is too much! Solution: Remote File Access Shadow process Augment all system calls using file descriptors Use RPC in wrapper

Agent Technology Based on human agents travel agent insurance agent real-estate agent personal assistant (a.k.a. secretary) Result  agents have specialized knowledge represent our interests find / filter / customize information

Mobile Agents Low network traffic and latency Agents-server communication takes place locally. Encapsulation All code and data are carried with an agent. Autonomous and asynchronous navigation Agent disconnect communication with their client and visits servers as their own. Run-time adaptability Agents can dynamically load new objects as they migrate over network. Robustness Agents are active to get out of faulty nodes.

Software Agents in Distributed Systems Property Common to all agents? Description Autonomous Yes Can act on its own Reactive Responds timely to changes in its environment Proactive Initiates actions that affects its environment Communicative Can exchange information with users and other agents Continuous No Has a relatively long lifespan Mobile Can migrate from one site to another Adaptive Capable of learning Some important properties by which different types of agents can be distinguished.

Slides from: http://www.cs.vu.nl/~ast/books/ds1 http://codex.cs.yale.edu/avi/os-book/os7