Intelligent Agents -Agent Mobility and Cloning Katia Sycara The Robotics Institute email: katia@cs.cmu.edu www.cs.cmu.edu/~softagents

Information server Host Agent1 Network Agent2 Idle Host Host Agent1 Network Agent2

Mobile Agents Overview In some types of applications, it is useful to have mobile agents: software agents that can migrate between machines to better utilize resources agents that reside on a mobile platform, and need to adjust to changes in the machine location Mobile agents can improve task performance and efficiency of using bandwidth and CPU Mobile agents require special services to allow for their application

Mobile Agents Programs typically written in a script language that may be dispatched from a client computer to execute remotely on a server sending a program without state to a remote CPU to execute there; program possibly communicates by RPC with other CPUs and then terminates (e.g., Java programs) sending a program that carries its execution state to a remote CPU; program executes and then moves to a third CPU or returns. This is called a mobile agent

Mobile Agents (cont) The mobile agent execution environment is called the Agent Meeting Place. It allows passing parameters between the application program and the modules of the agent binding of user interface libraries of the client’s operating system binding to other operating system functions such as the memory manager, the file system, etc, in particular to the message transport service (sending and receiving agents via the communication infrastructure)

Mobility Process Suspend current process or create daughter process Collect new or suspended process including execution state into a message expressed in machine independent form Send message to a particular known a priori machine a post office that performs address resolution intermediate designation that routes the agent based on message content (semantic routing)

Mobility Process (cont) Upon receipt of the message, the destination message subsystem delivers it to the server’s agent execution environment In server execution environment, message is reconstituted into the execution thread(s) and is dispatched Execution continues at the next instruction following the instruction that initiated mobility

Mobility Process (cont) During execution at the server, the agent passes information from the client application and receives information from it, or Interacts with other agents on the server The agent may: terminate its execution suspend its execution at the server waiting for some event from a server application or some other agent repeat the migration process, returning to its origin or going somewhere else, depending on the situation

What to Move? When agents collect and process information on the web, they consume bandwidth Attempting to reduce bandwidth consumption, a mobile agent can move to the information site instead of bringing the information to it But this requires bandwidth for code transfer, so if the agent is “heavy weight”, or it needs a little information - migration is not beneficial In practice, only when information transfer is significantly greater than code transfer migration is beneficial

Infrastructure Requirements A mobile agent that arrives at a foreign machine needs: to be accepted by that machine: be provided with access to machine resources to be written in a transportable language, able to run on any platform not to violate privacy or harm host machine be able to maintain state, to run itself, load code dynamically, stop itself protect itself from malicious host

What is the Price? Locally, a mobile agent needs to reason about itself, its needs, the network, and decide how to proceed Altogether, agent mobility imposes overheads: in computation for decision making in infrastructure for receiving agents in bandwidth for migration

And What is the Promise? In the right settings, agent migration can improve performance significantly In some cases, mobility is unavoidable E.g., mobile platforms (laptops, palms): when a machine is about to disconnect from the network it is beneficial for an agent to: get off the machine with essential information, state and processing continue living on the network until the machine is re-connected, possibly somewhere else move back to the machine, update it with changes, state

Promise (cont) Reduction of overall communication traffic over low bandwidth, high latency, high cost access networks employed by mobile computers Ability of the agent to engage in high bandwidth communication to search large free text databases on the server Ability of lightweight mobile computers to interact with heavy applications without prior detailed knowledge of the server’s abilities

Promise (cont) Ability of the agent to integrate knowledge from the client and server while performing interfacing at the server Ability of the user to personalize services by customizing agents that are resident at a server

Current State Only a hand full of mobile agent systems support strong mobility, ie the ability to capture and transfer the full execution state of the agent. Since strong mobility requires capture of thread execution state, and since this is not currently possible in current Java VM, modifying the Java VM has been the strategy (e.g. NOMADS, ARA, Sumatra, WASP)

Current State (cont) In current Java Security Manager, there is no way to control: -disc storage -number of I/O operations -number fo simultaneous print jobs -thread and process priorities -number of allowable open windows

Additional Mobile Agent Info Mobile agents list: http://www.informatic.uni-stuttgart.de/ipvr/projecte/mole/mal/ lists over 60 mobile agent systems voyager, oddyssey, aglets, D’agents are some mobile agent frameworks mobile agent languages: safe-tcl, telescript, Java, Milog

Agent Cloning/Merging Cloning supersedes agent mobility Cloning could be done with or without the agent’s knowledge Cloning goals: --To mitigate against agent overload --To achieve load balancing in a distributed system --To achieve higher throughput

Agent5 c5 Agent1 c1 Agent6 c6 T4 Agent7 c7 T5 T7 T9 T8 T T2 T T1 T T3 Idle Host Agent9 c9 T6 T10 Agent10 c10

Overloads in MAS Agents receive a stream of tasks T Tasks require capabilities and resources Agent’s capacity is limited. Therefore: an agent’s resources for task execution may be insufficient. As a result: agents may be overloaded at the same time others, on other machines are under-loaded the system utilizes resources inefficiently

Cloning Self-aware agents reason and anticipate overloads (predictive cloning) Self-aware agents detect overload after it has occurred (reactive cloning) Agents explore computational environment to find other agents, machines, resources Agents plan for task delegation, migration and cloning to resolve overloads Cloning is the creation and activation of other agents (on local or remote machines)

Create a clone Agent5 c5 Agent1 c1 Agent6 c6 T8 T8 T7 T9 Agent7 c7 T8 Agent2 (clone) T T2 T1 T1 T T3 Create a clone T T Idle Host Task delegation Agent9 c9 T6 T9 Agent8 (clone) Agent10 c10

The Cloning Algorithm(1) (performed by each agent) Self awareness activity (do periodically, forever (in a separate thread of control)): retrieve self available resources R(t0) (cpu, memory, disk space, comm. band width, etc) retrieve self current loads L(t0) anticipate current and expected Tl - capabilities and resources required, deadlines. Accordingly: compute future (expected) loads L(t), resources R(t) compare R(t), L(t). Locate periods of overload OV(t) (what type of overload, what tasks cause it).

Algorithm(2) - Reasoning If OV(t) not empty, reason for cloning: locate other agents and hosts for each located entity E: retrieve current loads LE (t0) anticipate future tasks and loads LE (t) solve optimization problem to allocate tasks G in Tl that may cause OV(t) to entities. Refer to: the remote loads LE (t) task deadlines and required capabilities, resources if E not agent - include creation/activation overhead

Algorithm(3) - Task Delegation The optimization process provides: S, set of agents that can execute tasks in G for each ai c S, a unique, timed task list Tai (for each task in Tai, est and lst) H, set of hosts with available resources a timed task list TH for remote hosts For each ai c S, delegate tasks in Tai at appropriate time

Algorithm(4) - Create, Activate, Reason After While TH is not empty select a preferred host h in H create and activate a clone ah on h delegate tasks from TH to ah Reason, learn after cloning (subsequent slide) Reason about underloads. Consider: merging with under-loaded agents self-extinction

Reasoning Guidelines Given the current task list and expected task distribution (provided or learned): If required resources at any time exceed agent maximum capacity- move tasks of the overloaded time to split-list if no agents can perform these tasks: locate a host, create a clone, pass split-list

Reasoning for Cloning

Cloned-Agent Activity Introduces itself to MAS (via middle agent) Receives self task list Tl from creator’s TH Performs tasks (independently from creator) When Tl completed, if no new tasks arrived (reasons periodically w.r.t. underload): self-extincts, or merges with creator (or with another agent)

Reasoning After Cloning Collect info. about success, loads and availability of agents and hosts w.r.t. time Compute averages, variances and predictors of this information Use these results to update probabilities (of success of using agents/hosts) to the optimization problem Future cloning decisions will be affected

Cloning Simulation Setting: Results: 10 to 20 agents, up to 10 additional clones up to 1000 dynamically arriving tasks an agent can perform 20 tasks simultaneously Results: up to 100 tasks - close to 100% of tasks are executed w/wo cloning. 100 to 350 tasks: w/o cloning, MAS overall task execution poor with cloning approx. 85% of tasks met deadlines

System Throughput 44

Additional Utility of Cloning Can be used for agent migration and mobility by: creating a clone passing all tasks to the clone original agent performs self extinction

Agent Merging When an agent or the underlying infrastructure detects underutilization of an agent: kills the agent, if its capability is replicated, notifies its current customers, or selects agent with replicated capability and passes it the tasks of the underutilized agent, notifying its customers