1. Processes
Chapter 3

2. Processes Topics discussed in this chapter:
Threads improve performance.
In distributed systems threads allow clients and servers to be constructed such that communication and local processing can overlap.
Clients: various GUI systems.
Servers: object servers
Code migration can help achieve scalability and dynamically configure clients and servers.
Software agents are a type of process which attempt to reach a common goal.

3. Introduction to Threads
Basic idea: An operating system creates virtual processors in software on top of physical processors.
A processor provides the capability of executing a series of instructions.
A process is a program in execution.
A software processor in whose context one or more threads may be executed. Executing a thread means executing a series of instructions in the context of that thread.
A thread is the execution of a part of a program.
A thread maintains the minimal information to allow a CPU to be shared by several threads.
A thread context consists of the CPU context and the information for the thread.
Saving a thread context implies stopping the current execution and saving all the data needed to continue the execution at a later stage.

4. Context Switching
Processor context: The minimal collection of values stored in the registers of a processor used for the execution of a series of instructions (e.g., stack pointer, addressing registers, program counter).
Process context: The minimal collection of values stored in registers and memory, used for the execution of a series of threads (i.e., thread context, but now also at least MMU register values).
Thread context: The minimal collection of values stored in registers and memory, used for the execution of a series of instructions (i.e., processor context, state).

5. Thread Usage in Nondistributed Systems
Context switching as the result of IPC

6. Context Switching
Observation 1: Threads share the same address space. Thread context switching can be done entirely independent of the operating system.
Observation 2: Process switching is generally more expensive as it involves getting the OS in the loop, i.e., trapping to the kernel.
Observation 3: Creating and destroying threads is much cheaper than doing so for processes.

7. Threads and Operating Systems
Main issue: Should an OS kernel provide threads, or should they be implemented as user-level packages?
User-space solution:
We'll have nothing to do with the kernel, so all operations can be completely handled within a single process. Implementations can be extremely efficient.
All services provided by the kernel are done on behalf of the process in which a thread resides.  If the kernel decides to block a thread, the entire process will be blocked. Requires messy solutions.
In practice we want to use threads when there are lots of external events: threads block on a per-event basis  If the kernel can't distinguish threads, how can it support signaling events to them.

8. Threads and Operating Systems
Kernel solution: The whole idea is to have the kernel contain the implementation of a thread package. This does mean that all operations return as system calls.
Operations that block a thread are no longer a problem: the kernel schedules another available thread within the same process.
Handling external events is simple: the kernel (which catches all events) schedules the thread associated with the event.
The big problem is the loss of efficiency due to the fact that each thread operation requires a trap to the kernel.
Conclusion: Try to mix user-level and kernel-level threads into a single concept.

9. Solaris Threads
Basic idea: Introduce a two-level threading approach: user-level processes and light-weight processes (LWP) that can execute user-level threads in the kernel.
When a user-level thread does a system call, the LWP that is executing that thread blocks. The thread remains bound to the LWP.
The kernel can simply schedule another LWP having a runnable thread bound to it. Note that this thread can switch to any other runnable thread currently in user space.
When a thread calls a blocking user-level operation, we can simply do a context switch to a runnable thread, which is then bound to the same LWP.
When there are no threads to schedule, a LWP may remain idle, and may even be removed (destroyed) by the kernel.

10. Thread Implementation
Combining kernel-level lightweight processes and user-level threads.

11. Threads and Distributed Systems
Multithreaded clients: To establish a high degree of distribution transparency, distributed systems need hiding network latencies (so that the user is not knowing there is a interprocess communication across a wide-area network).
Example: Multithreaded Web Client
A Web browser scans an incoming HTML page, and finds that more files need to be fetched.
Each file is fetched by a separate thread, each doing a (blocking) HTTP request.
As files come in, the browser displays them.
Example: Multiple RPCs
A client does several RPCs at the same time, each one by a different thread.
It then waits until all results have been returned.
Note: If RPCs are to different servers, we may have a linear speed-up compared to doing RPCs one after the other.

12. Threads and Distributed Systems
Multithreaded servers: Main issues are improving performance and better programming structure.
Improve performance
Starting a thread to handle an incoming request is much cheaper than starting a new process.
Having a single-threaded server prohibits simply scaling the server to a multiprocessor system.
Servers hide network latency by reacting to next request while previous one is being replied.
Better programming structure
Most servers have high I/O demands. Using simple, well-understood blocking calls simplifies the overall structure.
Multithreaded programs tend to be smaller and easier to understand due to simplified flow of control.

13. Multithreaded Servers
A multithreaded server organized in a dispatcher/worker model.

14. Multithreaded Servers
Model
Characteristics
Threads
Parallelism, blocking system calls
Single-threaded process
No parallelism, blocking system calls
Finite-state machine
Parallelism, nonblocking system calls
Three ways to construct a server.

15. Clients - User Interfaces
Essence:
A major task of most clients is to interact with a human user and a remote server.
A major part of client-side software is focused on (graphical) user interfaces.
The X Window System is used to control bit-mapped terminals.

16. The X Window System
The heart of the system is formed by the X kernel. It contains all the terminal-specific device drivers.
The X kernel interface for controlling the screen is made available to applications in the Xlib library.
There are two types of X applications: ordinary applications and window managers.
A window manager is an application that is given special permission to manipulate the entire screen.
The X protocol is a network-oriented communication protocol by which an instance of Xlib can exchange data and events with the X kernel.
The client which runs only the X kernel is called X terminals.

17. The basic organization of the X Window System

18. User Interfaces – Compound Documents
Modern user interfaces allow applications to share a single graphical window and use that window to exchange data through user actions.
Compound documents are a collection of documents which are seamlessly integrated at the user-interface level.: Make the user interface application-aware to allow inter-application communication:
drag-and-drop: move objects to other positions on the screen, possibly invoking interaction with other applications. Drag a file to drop in a trash can.
in-place editing: integrate several applications at user-interface level (word processing + drawing facilities). Edit an image in a text document.

19. Client-Side Software
In many cases, a part of processing is executed on the client side.
Essence: Client software comprises components for achieving distribution transparency.
Access transparency is handled through client-side stubs for RPCs and RMIs.
Location/migration transparency can be handled by letting client-side software keep track of actual location.
Replication transparency can be achieved by forwarding an invocation request to multiple replica.
Failure transparency can often be placed only at client (we're trying to mask server and communication failures).
Concurrency transparency can be handled through special intermediate servers (transaction monitors).

20. Client-Side Software for Distribution Transparency
A possible approach to transparent replication of a remote object using a client-side solution.

21. Servers
Basic model:
A server is a process implementing a specific service on behalf of a collection of clients.
A server is a process that waits for incoming service requests at a specific transport address.
In practice, there is a one-to-one mapping between a port (endpoint) and a service.
How do clients know the endpoint of a service?
Endpoints for well-known services are globally assigned by the Internet Assigned Numbers Authority (IANA).
DCE uses a special daemon which listens to a known port and keeps track of the current endpoint of each service. A client contact the daemon, request the endpoint, and then contact the specific server.

22. Servers - General Organization
Iterative vs. concurrent servers: Iterative servers itself handles the request itself and can handle only one client at a time. A concurrent server does not handle the request itself but pass it to a separate thread or another process.
Superservers: Servers that listen to several ports, i.e., provide several independent services. In practice, when a service request comes in, they start a subprocess to handle the request (UNIX inetd)

23. Servers: General Design Issues
3.7
Client-to-server binding using a daemon as in DCE
Client-to-server binding using a superserver as in UNIX

24. Out-of-Band Communication
Issue: Is it possible to interrupt a server once it has accepted (or is in the process of accepting) a service request?
Non-Solution: Exit the client application and restart it.
Solution 1:Use a separate port for urgent data (possibly per service request):
The Server has a separate thread (or process) waiting for incoming urgent messages.
When the urgent message comes in, the other request is put on hold.
Note: The OS is required to support high-priority scheduling of specific threads or processes.
Solution 2:Use out-of-band communication facilities of the transport layer.
Example: TCP allows to send urgent messages in the same connection.
Urgent messages can be caught using OS signaling techniques.

25. Servers and State
Stateless servers never keep information on the state of a client.
Don't record whether a file has been opened (simply close it again after access).
Don't promise to invalidate a client's cache.
Don't keep track of the clients.
Example: Web servers.
Consequences
Clients and servers are completely independent.
State inconsistencies due to client or server crashes are reduced.
Possible loss of performance because, e.g., a server cannot anticipate the client behavior (think of prefetching file blocks).
Question: Does connection-oriented communication fit into a stateless design?
Stateful connections do not violate the statelessness of servers.

26. Servers and State
Stateful servers keep track of the state of their clients.
Record a file that has been opened, so that prefetching can be done.
Know which data a client has cached, and allows clients to keep local copies of shared data.
Observation: The performance of stateful servers can be extremely high, provided clients are allowed to keep local copies.
Consequence: If the server crashes, a stateful server needs to recover its entire state before its crash. Enabling recovery introduces complexity.
A stateless server can use a cookie. It is a small piece of data containing the client information which is stored on the client.
Example: Web browser.

27. Object Servers
An object server is a server tailored to support distributed objects.
An object server by itself does not really provide a specific service.
Specific services are implemented by the objects that reside in the server.
It is easy to change services by simply adding and removing objects.
An object server acts as a place where objects live.
An object consists of two parts: data representing its state and code forming the implementation of its methods.

28. Object Servers Invoking objects
Use a uniform way to invoke objects such as DCE. It is inflexible and constrains developers.
Support different policies:
Object invocation:
Invoke a (transient) object one at a time (consuming time to invoke objects).
Invoke all objects at a time (consuming resource).
Data sharing:
Objects share neither data nor code. (e.g. separate security policies)
share code (e.g. database object that access a database).
Threading: A thread per object (adv.: concurrent control) or a thread per method.

29. Object Adapter
Decisions on how to invoke an object are referred to as activation policies.
A mechanism to group objects per policy is called an object adapter, or object wrapper.
An object adapter can best be thought of as software implementing a specific activation policy.
An object adapter has one or more objects under its control. Several object adapters (each adapter represent a different activation policy) may reside in the same server.

30. Object adapter Object adapter: The ‘manager’ of a set of objects:
Inspects (as first) incoming requests.
Ensures referenced object is activated (requires identification of servant).
Passes request to appropriate skeleton, following specific activation policy.
Responsible for generating object references.
Observation: Object servers determine how their objects are constructed.

31. Object Servers Skeleton: Server-side stub for handling network I/O:
Generated from interface specifications
Unmarshall incoming requests, and call the appropriate servant code.
Marshall results and send the reply message.
Servant: The actual implementation of an object, sometimes containing only method implementations:
Collection of C, Fortran, or COBOL functions, that act on structs, records, database tables, etc.
Java or C++ classes

32. Object Adapter
Organization of an object server supporting different activation policies.

33. Object Adapter Implementation
Consider an object adapter that manages a number of objects. The adapter implements the policy that it has a single thread of control fro each of its objects.
Object adapter implementation
The header file of the adapter (e.g. header.h)
The thread library (e.g. thread.h)
The implementation code of a thread-per-object
The request demultiplexer can be replaced by an object adapter. CORBA uses this approach.

34. Object Adapter
/* Definitions needed by caller of adapter and adapter */ #define TRUE #define MAX_DATA 65536
/* Definition of general message format */ struct message { long source /* senders identity */ long object_id; /* identifier for the requested object */ long method_id; /* identifier for the requested method */ unsigned size; /* total bytes in list of parameters */ char **data; /* parameters as sequence of bytes */ };
/* General definition of operation to be called at skeleton of object */ typedef void (*METHOD_CALL)(unsigned, char* unsigned*, char**);
long register_object (METHOD_CALL call); /* register an object */ void unrigester_object (long object)id); /* unrigester an object */ void invoke_adapter (message *request); /* call the adapter */
The header.h file used by the adapter and any program that calls an adapter.

35. The thread.h file used by the adapter for using threads.
Object Adapter
typedef struct thread THREAD; /* hidden definition of a thread */
thread *CREATE_THREAD (void (*body)(long tid), long thread_id); /* Create a thread by giving a pointer to a function that defines the actual */ /* behavior of the thread, along with a thread identifier */
void get_msg (unsigned *size, char **data); void put_msg(THREAD *receiver, unsigned size, char **data); /* Calling get_msg blocks the thread until of a message has been put into its */ /* associated buffer. Putting a message in a thread's buffer is a nonblocking */ /* operation. */
The thread.h file used by the adapter for using threads.

36. Object Adapter
The main part of an adapter that implements a thread-per-object policy.

37. Code Migration Reasons for migrating code
Overall system performance can be improved if processes are moved from heavily-loaded to lightly-loaded machines.
Code migration makes sense to process data close to where those data reside.
Code migration exploits parallelism.
Flexibility – dynamically configuring distributed systems.

38. Code Migration Dynamically configuring distributed systems:
The code is available to the client at the time the client application is being developed.
The code is downloaded when required.
Advantages:
Clients need not have all the software preinstalled.
The client-server protocol and its implementation can be changed as requested.
Disadvantages: It is not secure to blindly trust the downloaded code.

39. Reasons for Migrating Code
The principle of dynamically configuring a client to communicate to a server. The client first fetches the necessary software, and then invokes the server.

40. Models for Code Migration
Object components [Fugetta et al., 1998]
Code segment contains the set of instructions.
Data segment contains the data references such as files, printers, devices, other processes, and more.
Execution segment contains current execution state of a process, consisting of private data, the stack, and the programming counter.
Weak mobility: Move only code and data segment and start execution from the beginning after migration:
Relatively simple, especially if code is portable
Example: Java applets.

41. Models for Code Migration
Strong mobility: The execution segment can be transferred as well.
Migration: move the entire object from one machine to the other.
Cloning: simply start a clone, and set it in the same execution state.
Example: D’Agents
In sender-initiated migration, the migration is initiated at the machine where the code currently resides or is being executed. Examples:
Uploading code to the sever: The clients need to be registered and authenticated at the server.
Sending a search program across the Internet to a Web database server to perform the queries at that server.

42. Models for Code Migration
In receiver-initiated migration, the initiative for code migration is taken by the target machine. Examples:
Java applets
Downloading code can often be done anonymously.
The migrated code can be executed:
At the target process
For example, Java applets are downloaded and executed in the browser’s address space.
The advantage is no separate process is required. The drawback is the target process needs to protect against malicious code.
In a new separate process
It has the resource-access problems.

43. Models for Code Migration
Alternatives for code migration.

44. Managing Local Resources
Problem: The resource segment cannot be simply transferred without change. An object uses local resources that may or may not be available at the target site. Examples:
Port: The migrated code gives up the port and requests a new one at the destination.
URL remains valid irrespective of the code migration.
Object-to-resource binding [Fuggetta et al.]
By identifier: The object requires a specific instance of a resource (e.g. a URL, FTP server’s Internet address).
By value: The object requires the value of a resource (e.g. standard libraries).
By type: The object requires that only a type of resource is available (e.g. a local device such a color monitor, a printer).
Resource types
Unattached: The resource can easily be moved along with the object (e.g. a data file).
Fastened: The resource can, in principle, be migrated but only at high cost (e.g. local databases and Web site).
Fixed: The resource cannot be migrated, such as local devices or communication endpoint.

45. Migration and Local Resources
Resource-to machine binding
Unattached
Fastened
Fixed
By identifier
By value
By type
MV (or GR)
CP ( or MV, GR)
RB (or GR, CP)
GR (or MV)
GR (or CP) GR
RB (or GR)
Process-to-resource binding
Actions to be taken with respect to the references to local resources when migrating code to another machine.
MV: Move the resource
GR: establish a Global system-wide Reference
CP: Copy the value of the resource
RB: Rebind process to locally available resource

46. Managing Local Resources
Scenario of binding by identifier with unattached resource:
When a process is bound to a resource by identifier and the resource is unattached, it is best to move it along with the migrating code.
If the resource is shared by other processes, a global reference can be established. But the cost of the global reference might be high (Consider the situation if the resource is a video stream).

47. Managing Local Resources
Scenario of binding by identifier with fixed resource:
When migrating a process that is making use of a local communication endpoint.
Two possible solution:
Set up a connection to the source machine after it has migrated and has a proxy process at the target machine. What happens if the source machine malfunctions?
Change their global reference, and send messages to the new communication endpoint at the target machine.

48. Managing Local Resources
Scenario of binding by value:
Unattached resource: copying the resource to the new destination is better than a global reference.
Fasten resource: Establishing a global reference is better than copying runtime libraries.
Fixed resource: A process assumes that memory can be shared between processes. Establishing a global reference needs to implement distributed shared memory.
Scenario of binding by type:
The obvious solution is to rebind the process to a locally available resource of the same type.

49. Migration in Heterogeneous Systems
Main problem:
The target machine may not be suitable to execute the migrated code.
The definition of processor/process/thread context is highly dependent on local hardware, operating system, and runtime system (e.g. platform-specific data).
For weak mobility it suffices to recompile the source. For strong mobility the execution segment is also migrated. The recompilation is not enough.

50. Migration in Heterogeneous Systems
Solution (migration stack):
The migration stack is the copy of the program stack maintained on the runtime system.
Existing languages allow migration at specific ‘transferable’ points, such as just before a function call.
Solution (virtual machine): Make use of an abstract machine that is implemented on different platforms.
Interpreted languages running on a virtual machine (Java/JVM; scripting languages)

51. Migration in Heterogeneous Systems
3-15
The principle of maintaining a migration stack to support migration of an execution segment in a heterogeneous environment

52. Example: D'Agents
D’Agents formerly called Agent Tcl (Tool Command Language), is a system that is built around the concept of an agent. Refer to
An agent in D’Agents is a program that can migrate between different machines. Programs can be written in an interpretable language such as Tcl, Java, or Scheme.

53. D'Agents D'Agents provides support for
Sender-initiated weak mobility (agent_submit command)
Strong mobility by process migration (agent_jump command)
Strong mobility by process cloning (agent_fork command)

54. Overview of Code Migration in D'Agents
proc factorial n { if ($n  1) { return 1; } # fac(1) = 1 expr $n * [ factorial [expr $n – 1] ] # fac(n) = n * fac(n – 1) }
set number … # tells which factorial to compute
set machine … # identify the target machine
agent_submit $machine –procs factorial –vars number –script {factorial $number }
agent_receive … # receive the results (left unspecified for simplicity)
A simple example (send-initiated weak mobility) of a Tcl agent in D'Agents submitting a script to a remote machine (adapted from [gray.r95])
agent_receive is blocked until the result is received.

55. Overview of Code Migration in D'Agents
all_users $machines
proc all_users machines { set list "" # Create an initially empty list foreach m $machines { # Consider all hosts in the set of given machines agent_jump $m # Jump to each host set users [exec who] # Execute the who command append list $users # Append the results to the list } return $list # Return the complete list when done }
set machines … # Initialize the set of machines to jump to set this_machine # Set to the host that starts the agent
# Create a migrating agent by submitting the script to this machine, from where # it will jump to all the others in $machines.
agent_submit $this_machine –procs all_users -vars machines -script { all_users $machines }
agent_receive … #receive the results (left unspecified for simplicity)
An example (send-initiated strong mobility) of a Tcl agent in D'Agents migrating to different machines where it executes the UNIX who command (adapted from [gray.r95])
agent_jump allows the process to move to other machine.

56. D'Agents Organization: Each machine is built as a five-layered system:
TCP/IP and layer implements a common interface to the communication facilities of the underlying network.
Server layer is responsible for agent management, authentication, and communication management between agents.
Agent Run Time System (Core) layer consists of a language-independent core that supports the basic model of agents such as starting and ending an agent.
Language interpreter layer consists of the language interpreter, a security module, an interface to the core layer, a module to capture the state of a running agent.
Agent layer: Each agent is executed by a separate process.

57. Implementation Issues
The architecture of the D'Agents system.

58. D'Agents
The more difficult part in the implementation of D’Agents, is capturing the state of a running agent and shipping that state to another machine.
There are four tables containing global definitions of variables and scripts, and two stacks for keeping track of the execution status.
When an agent calls agent_jump, by which the agent migrates to another machine.
The complete state of the agent as just described is marshaled into a series of bytes.
The D’Agents server on the target machine subsequently creates a new process to handle the marshaled data, which is then unmarshals into the state when the agent called agent_jump.

59. Implementation Issues
Status
Description
Global interpreter variables
Variables needed by the interpreter of an agent
Global system variables
Return codes, error codes, error strings, etc.
Global program variables
User-defined global variables in a program
Procedure definitions
Definitions of scripts to be executed by an agent
Stack of commands
Stack of commands currently being executed
Stack of call frames
Stack of activation records, one for each running command
The parts comprising the state of an agent in D'Agents.

60. What is a Software Agent?
Definition: An autonomous process capable of reacting to and initiating changes in its environment, and of performing a task in collaboration with users and other (remote) agents.
The feature that makes an agent more than just a process is its capability to
act on its own and take initiative where appropriate
cooperate with other agents

61. Taxonomy of Agents
It seems hard for researchers to reach agreement on a single taxonomy:
collaborative agent: collaborate with others in a multiagent system.
A multiagent system is a system in which agents seek to achieve some common goal through collaboration.
Example: Agents collaborate in setting up a meeting
mobile agent: can move between machines
Example: Agents which retrieve information distributed on the Internet
interface agent: assist users in the use of one or more applications (usually with learning ability).
Example: agents which seek to bring buyers an sellers together.
information agent: manage information from physically different sources
Example: agent, mobile information agent.

62. Software Agents in Distributed Systems
Property
Common to all agents?
Description
Autonomous
Yes
Can act on its own
Reactive
Responds timely to changes in its environment
Proactive
Initiates actions that affects its environment
Communicative
Can exchange information with users and other agents
Continuous No
Has a relatively long lifespan
Mobile
Can migrate from one site to another
Adaptive
Capable of learning
Some important properties by which different types of agents can be distinguished.

63. Agent Technology
The Foundation for Intelligent Physical Agent (FIPA) is developing a general model for software agents [FIPA, 1998b] (
In this model, agents are registered at, and operate under the management of an agent platform:
Management: Keeps track of where the agents on this platform are.
creating and deleting agents.
mapping globally unique agent ID to a local communication endpoint (port)
Directory: Mapping of agent names and attributes to agent IDs
ACC: Agent Communication Channel, used to communicate with other platforms
Communication between ACCs on different platforms follows Internet Inter-ORB Protocol (IIOP).
Example: server in D'Agents

64. The general model of an agent platform (adapted from [fipa98-mgt]).
Agent Technology
The general model of an agent platform (adapted from [fipa98-mgt]).

65. Agent Language
Communication between agents takes place by means of an application-level communication protocol, which is referred to as an agent communication language (ACL).
ACL makes distinction between purpose and content of a message.

66. Agent Language An ACL message can have a limited number of purposes.
A message consists of:
a header
Identifier of the purpose of message,
Fields for identifying the sender and receiver,
A field to identify the language or encoding scheme,
A field to identify a standardized mapping of symbols to their meaning. Such a mapping is called ontology.
the actual content.

67. Agent Communication Languages
Message purpose
Description
Message Content
INFORM
Inform that a given proposition is true
Proposition
QUERY-IF
Query whether a given proposition is true
QUERY-REF
Query for a give object
Expression
CFP
Ask for a proposal
Proposal specifics
PROPOSE
Provide a proposal
Proposal
ACCEPT-PROPOSAL
Tell that a given proposal is accepted
Proposal ID
REJECT-PROPOSAL
Tell that a given proposal is rejected
REQUEST
Request that an action be performed
Action specification
SUBSCRIBE
Subscribe to an information source
Reference to source
Examples of different message types in the FIPA ACL [fipa98-acl], giving the purpose of a message, along with the description of the actual message content.

68. Agent Communication Languages
Field
Value
Purpose
INFORM
Sender
Receiver
Language
Prolog
Ontology
genealogy
Content
female(beatrix),parent(beatrix,juliana,bernhard)
A simple example of a FIPA ACL message sent between two agents using Prolog to express genealogy information.