1. Concurrent Programming for DAI Gul A. Agha & Needem Jamali

2. Concurrent Programming2 Concurrent programming introduces many challenges: A key challenge is the difficulty of programming parallel and distributed architectures: Some models are very low-level (shared variables model). One approach would be to use concurrent objects in a reflective architecture. Using agent programming is such an architecture.

3. Concurrent Programming3 Patterns of Concurrent Problem Solving. Three common Patterns: Pipelined Concurrency Enumeration of potential solutions and the concurrent testing of these solutions. Divide and Conquer Concurrent elaboration of different sub problems and joining the solutions. No interaction between the procedures. Cooperative problem solving. Processes dynamically interact in order to solve the problem.

4. Concurrent Programming4 Agent Programming Paradigms A few agent programming languages: Agent Oriented Programming (AOP). Concurrent MetateM. Telescript. Actors. I will present all four, with a focus on Actors.

5. AOP5 Agent Programming Paradigms A few agent programming languages: Agent Oriented Programming (AOP). Introduction to Multiagent Systems (Wooldridge). An Overview of Agent Oriented Programming (Shoham). Concurrent MetateM. Telescript. Actors.

6. AOP6 Agent Oriented Programming (AOP) Due to Yoav Shoham (Stanford). Based on cognitive and societal view of computation. An agent is an entity that functions continuously and autonomously in an environment in which other agents exist. An agent’s state is viewed as consisting of mental components such as: beliefs, capabilities, choices, commitments and desires. “Agenthood is in the mind of the programmer.” What makes a system - an agent, is the fact that one has chosen to analyze and control it in these mental terms.

7. AOP7 Agent Oriented Programming – AGENT0 Shoham has developed a basic programming language to support AOP: AGENT0. An agent is specified in terms of four sets: Capabilities (Things the agent can do). [Initial] beliefs (Things the agent knows/believes). [Initial] Commitments. Commitment rules.

8. AOP8 Agent Oriented Programming – AGENT0 - 2 The component which determines how the agent acts is the commitment-rules set. A commitment rule consists of three components: A message condition. A mental condition. An action. The rule fires if the agent received a message that matches the message condition, and the agent’s beliefs match the mental condition. The agent than becomes committed to the action.

9. AOP9 Agent Oriented Programming – AGENT0 - 3 An action may be private (sends no message), or communicative(sends messages). Messages are constrained to three type: Request. Unrequest. Inform.

10. AOP10 Agent Oriented Programming – AGENT0 – Example A commitment rule example: COMMIT( ( agent, REQUEST, DO(time, action) ), ;;; msg condition ( B, [now, Friend agent] AND CAN(self, action) AND NOT [time, CMT(self, anyaction)] ), ;;; mental condition self, DO(time, action) )

11. AOP11 Agent Oriented Programming – AGENT0 – Main Loop 1. Read all current messages, update beliefs, and hence commitments – when necessary. 2. Execute all commitments for the current cycle where the capability condition of the associated action is satisfied. 3. Goto (1).

12. AOP12 AOP - Properties of the Agent’s Components The programmer may assume several properties of the agent’s components: Internal consistency. Good faith. Introspection. Persistence of mental state.

13. 13 Agent Programming Paradigms A few agent programming languages: Agent Oriented Programming (AOP). Concurrent MetateM. Introduction to Multiagent Systems (Wooldridge). Telescript. Actors.

14. Concurrent MetateM14 Concurrent MetateM Due to Michael Fisher. Based on the direct execution of logical formulae. An agent is programmed by giving it a temporal logic specification of the behavior it should exhibit.

15. Concurrent MetateM15 Concurrent MetateM - Execution Agent’s execution: Iteratively build a logical model for the temporal agent’s specification. It is possible to prove that the procedure used to execute an agent specification is correct. If it is possible to satisfy the specification, then the agent will do so.

16. Concurrent MetateM16 Concurrent MetateM – Agent Structure Communication through broadcast message passing. An agent has two main components: An interface Computational engine

17. Concurrent MetateM17 Concurrent MetateM – Agent Structure - Interface The interface consists of three components: Unique Agent ID. Environment propositions. Component propositions. Example: Stack(pop, push)[popped, full]. Agent ID Component propositions Environment propositions

18. Concurrent MetateM18 Concurrent MetateM – Agent Structure - Engine An agent specification is given as a set of program rules, which are temporal logic of the formulae of the form: antecedent about the past  consequent about present and future. The name of the paradigm: ‘Declarative past and imperative future’.

19. Concurrent MetateM19 Concurrent MetateM – Syntax PML – Propositional MetateM Logic.

20. Concurrent MetateM20 Concurrent MetateM – Syntax Example important(agents) It is now and will always be true the agents are important.  important(Janine) Sometime in the future, Janine will be important. (¬friends(us)) U apologize(you) We are not friends until you apologize. Oapologize(you) Tomorrow(in the next state), you apologize.

21. Concurrent MetateM21 Concurrent MetateM – Execution 1. Update the history of the agent by receiving messages (environment propositions) from other agents an adding them to its history. 2. Check which rules fire, by comparing past time antecedents of each rule against the current history to see which is satisfied. 3. Jointly execute the fired rules together with any commitments carried over from previous cycles. 4. Goto 1.

22. Concurrent MetateM22 Concurrent MetateM – Execution Example

23. Telescript23 Agent Programming Paradigms A few agent programming languages: Agent Oriented Programming (AOP). Concurrent MetateM. Telescript. Mobile Agents (James E. White). Actors.

24. Telescript24 Telescript Telescript implements the concept of mobile agents in a commercial setting. One approach for sending tasks over a network is by using RPC (Remote Procedure calling). One computer is able to call procedures on another computer. Ongoing interaction requires ongoing communication.

25. Telescript25 Telescript A better approach would be to use RP (Remote Programming). One computer is able to supply procedures to be executed on another computer. Better performance (no ongoing communication). Telescript follows the RP approach, and extends it to the mobile agents concept.

26. Telescript26 Telescript Main concepts implemented by Telescript Places: A network of computer is a collection of places. A place offers a service to the mobile agents that enter it. Example: A mainframe computer may function as a shopping center which houses several stores. Each store is a place. Agents: A communication application is an agent. Each agent occupies a particular place. An agent may move from one place to another. A place is permanently occupied by one distinguished agent which represents the place, and provides its services.

27. Telescript27 Telescript Main Telescript concepts (Contd.) Travel: Travel lets an agent obtain a service offered remotely, and then return to its starting place. Example: A user’s agent might travel from home to a ticketing place to obtain tickets for a show, and then return home to report the user about the tickets it obtained. Implemented in Telescript by the go instruction. Meetings: Two agents in the same place may meet. Example: The agent in pursuit for tickets, may meet the ticket agent, and purchases the tickets Implemented in Telescript by the meet instruction.

28. Telescript28 Telescript Main Telescript concepts (Contd.) Connections: Two agents in different places may make a connection between them. Example: The agent that travels in search of tickets, might send a diagram to the home agent, representing the seats available. Implemented in Telescript by the connect instruction.

29. Telescript29 Telescript Main Telescript concepts (Contd.) Authorities: The authority of an agent is the individual or organization in the physical world that it represents. One agent may discern the authority of another. Telescript technology verifies the authority of an agent whenever it travels from one place to another. A region is a collection of places provided by computers that are all operated by the same authority. Unless a source region can prove the authority of the agent to the destination region, the agent is denied entry. To determine an agent’s authority, the name instruction used. The result of the instruction is a telename.. Identities distinguish agents of the same authority.

30. Telescript30 Telescript Main Telescript concepts (Contd.) Permits: A permit is data that grants capabilities. An agent can discern its capabilities, but cannot increase them. Example: An agent’s permit may give it the right to create other agents. To determine an agent’s permit, the permit instruction is used. Permits help guard against malicious agents, and against unbridled consumption of resources by ill programmed or ill intentioned agents.

31. Actors31 Agent Programming Paradigms A few agent programming languages: Agent Oriented Programming (AOP). Concurrent MetateM. Telescript. Actors.

32. Actors32 Actors - References Concurrent Programming for DAI (Gul A. Agha and Neede, Jamali) Concurrent Object Oriented programming (Gul Agha)

33. Actors33 Why Actors? Object Oriented Programming seems suitable for agent programming. Concurrent Objects must also specify ways of interactions and other concepts relevant for concurrent programming. The Actor paradigm provides this abstraction.

34. Actors34 What Are Actors? An actor is basically a reasoning agent (introduced by Carl Hewit in the early `70s) This term was refine over the years into a model of concurrency. An actor encapsulates behavior (data and procedure) as well as a process.

35. Actors35 Actors - Overview Actors carry out their action asynchronously and communicate by sending messages. The basic mechanism is asynchronous and buffered. However, other forms may be defined in the context of the model. Actors may be dynamically created and reconfigured. Provides flexibility in organizing concurrent activity. The internal behavior is encapsulated. Thus any agent may be defined using any programming language.

36. Actors36 Actors – Basic Primitives Three basic primitives: newactor/create(e) Creates a new actor which is evaluating expression e returns its address Send (a, v) Sends receiver a, the message v. Ready/become(b) Alters the behavior of the executing actor to b. Frees the actor to accept another message.

37. Actors37 Actors – More About Basic Primitives The newactor primitive extends the dynamic creation capability in sequential programming languages by allowing creation of processes. The send primitive is the asynchronous analog of function application. It causes a message to be put in an actor’s mailbox (queue) The become primitive give actors a history- sensitive behavior necessary for shared mutable data objects by delineating a group of actions as atomic. This is in contrast to pure functional programming.

38. Actors38 Actors –About State Changes A state change is specified using replacement behaviors. When an actor processes a communication, it also computes its behavior in response to the next communications. For a fully functional actor, the replacement behavior is identical to the original behavior. A behavior change may represent A change of state variables (balance of bank account) A change of in the operations performed in response to a message.

39. Actors39 Actors – More About State Changes One of the advantages of replacement behavior is this: The analysis of the system is easier. The granularity of the system is more flexible. The programmer may aggregate changes avoiding control flow dependencies. Allows an easy determination of when a process execution is finished.

40. Actors40 Actors – Event Diagrams Concurrent computations can be visualized by event diagrams. Lifeline Events

41. Actors41 Actors – Event Diagram For Factorial Computation

42. Actors42 History sensitive behavior Bank account example Given the above definition, one can create a new back-account actor with an initial balance of 1000: (define my-account (create BankAccount 1000))

43. Actors43 Join Continuations One of the three main concurrent programming paradigms is: Divide and conquer. The way to implement this paradigm using actors is by creating Join –Continuation (jc) actors. A jc actor, collect (and/or merges) the results other actors have produced

44. Actors44 Join Continuations - Example A filtered search: Given a multi-ary tree, we would like to search it for values by filtering them. The algorithm: After checking for the base case, the behavior FILTEREDSEARCH, creates a jc actor with behavior COLLECT. Then it creates a FILTEREDSEARCH agent for each of the sub- trees.

45. Actors45 Join Continuations – Example contd. A filtered search (defActor FILTERSEARCH() (let ((filter (lambda (list) …))) (method (cust tree) (if (= (num-children tree) 0) (send cust (content tree)) (let ((jc (newActor COLLECT (cust (num-children tree) (list (content tree) filter)))) (map (lambda (x) (let ((f (newActor FILTERSEARCH()))) (send f jc x))) (children tree)) (ready FILTERSEARCH()))))))

46. Actors46 Join Continuations – Example – The JC Actor. (defActor COLLECT (cust n results filter) (method (res) (cond ((> n 1) (ready COLLECT (cust (- n 1) (append res results)))) ((= n 1) (send cust (filter (append res results))) (ready SINK())))))

47. Actors47 Semantics of Actors The λ calculus will be extended for the actor semantics. An instantaneous snapshot of the actor system is called a configuration. The notion of open systems is captured by explicitly representing a set of receptionists which may receive messages from actors outside a configuration, and a set of actors external to the configuration which may receive messages from the actors within.

48. Actors48 Semantics of Actors – Configuration Maps a finite set of addresses to their behaviors Finite sets of actor addresses Finite multiset of (pending) messages Receptionists (actors that may receive messages from external actors) External actors

49. Actors49 Semantics of Actors – Basic Actor

50. Actors50 Fairness of Actor Systems The define actor semantics is fair. Which means every enabled transition eventually fires. Every busy actor eventually makes progress. Every actor that is ready to receive a message will eventually receive a message, provided there is a message pending for it. If an actor does not become “stuck” it will eventually process every message sent to it.

51. Actors51 Equivalence of Actor Systems When reasoning as to a concurrent system, one may wish to define an equivalence of two systems. Since the actor system is fair, one may reason about eventuality properties. Thus, equivalence may be defined. The notion of equivalence is defined by adding an observable distinguished event to the set of transitions.

52. Actors52 Equivalence of Actor Systems - 2 Two actor expressions may be plugged into the same context to see if the event occurs in one or the other case. Two expressions are considered equivalent if they have the same observations over all possible contexts.

53. Actors53 Equivalence of Actor Systems - 3 Since actor computation is nondeterministic, three notions of observation over a computation tree are raised: There are many computational paths in the tree The event may occur on some, but not all paths. May equivalence - “always occurs” is as good as “sometimes occurs” Must equivalence - “never occurs” is as good as “only sometimes occurs” Convex equivalence – the intersection of the two equivalences.

54. Actors54 Extension To The Basic Actors Model Mobility of actors Resource bounded agents.

55. Actors55 Mobility of Actors To take advantage of the agents’ ability to autonomously decide whether, when and where they want to migrate, the Actor model must be extended. True migration must allow an actor to migrate to a different node while it is in the middle of its execution. In order for an agent to migrate, we first need a suitable syntax: …(let (( f (newActor FILTERSEARCH()) @ (host-of x)

56. Actors56 Mobility of Actors - Implementation We need to identify an actor’s current location. Instead of a, we now have h.a The name a at any node corresponds to the same actor. Migration may be represented in two ways: Provide a migration primitive, or Provide an agent with a way to grab its own state and send it over (inside a message) to a remote host. The original agent can then become a forwarder (of messages). We will use the latter.

57. Actors57 Mobility of Actors – Migration Syntax A new primitive is added: ccf which grabs the agent’s current state. Using this new primitive, higher level operations may be represented as macros. We will assume that each host also has a manager actor: h.m that acts on behalf of the host and manages its resources.

58. Actors58 Mobility of Actors – Migration Syntax Example for a macro used for migration: migrate@h. migrate@h This macro is used for migration of an actor to a remote host h.

59. Actors59 Mobility of Actors – Migration Macro The function given to ccf first sends a move request to the remote host’s manager – h.m using the reduction context enclosed in y as its behavior. (The host than creates an actor with the same name as the requesting actor). The actor then changes its behavior to WAIT-ACK. (let ((k (getkey))) (ccf (lambda (y) (seq (send h.m move self y k (personal-name self)) (ready WAIT-ACK (h m k ((personal-name self))))))

60. Actors60 Mobility of Actors – Migration Macro (defActor WAIT-ACK (lambda (ret-h ret-m ret-k) (if (and (= h ret-h) (= m ret-m) (=k ret-k)) (ready FORWARDER (h.a))

61. Actors61 Mobility of Actors – Semantics The transition semantics should be modified to address support for migration. A superscript is now added to the actor state to represent the host. We assume that all creation is local and that only message co-located on the same host as an actor are consumed. Other messages are forwarded. A superscript is added to each message to identify the host of the intended recipient. A new transition is added to represent message forwarding.

62. Actors62 Mobility of Actors – Semantics

63. Actors63 Resource Bounded Agents An economic model may be used to protect against resource consumptive behavior of agents in a multiagent system. I will use the problem of Filtered search to demonstrate the resource bound model. The Actor with the behavior FILTERSEARCH will now receive a new parameter - res – to represent the resources that it can consume in the search.

64. Actors64 Resource Bounded Agents (defActor FILTERSEARCH() (let ((filter (lambda (list) …))) (method (cust tree) (if (= (num-children tree) 0) (send cust (content tree)) (let ((jc (newActor TELLCOLLECT (count 0)) (map (lambda (x) (if (> (my-resources) delta) (let ((f (newActor FILTERSEARCH()) @(host-of x))) (send f jc x (part (my-resources))) (setf count (+ count 1))))) (children tree)( (send jc cust count (content tree) (part (my-resources)) filter) (ready FILTERSEARCH()))))))))

65. Actors65 Resource Bounded Agents The application will stop when only delta resources remains. We need delta in order to send the results to the client. The jc does not know in advance, how many messages it should expect. Thus we now have the TELLCOLLECT behavior which waits for this number, then becomes COLLECT. Note that this example does not account for the resources actors consume while they are inactive.

66. Actors66 Resource Bounded Agents – Economic Model We need a notion of a universal currency, in order to implement an economic model. Resource allocation will be measure in GCUs (Global Currency Units) The analog of renting resource should be in mind (rather than purchasing them). Waiting actors should also pay for their hosts.

67. Actors67 Resource Bounded Agents – Semantics A value r is added to the agent’s state, to represent the GCUs available to the agent. Β is now added to represent the system map: Host agents and their connecting network. [s] h Means host agent h has state s. T st is a function that takes the current state of a host and the transition being applied, to give the next state. T res(a, h) is a function representing a contract between an agent a and host h hosting it, and determines the cost of transition t when the host is in state s.

68. Actors68 Resource Bounded Agents – Semantics

69. Actors69 Resource Bounded Agents – Semantics

70. Actors70 Reasoning About Agent Systems When building a multiagent/actor system, one would like to reason about its behavior and eventuality properties. Correctness (out of the scope of this presentation). Fairness – We saw the Actor model is fair. Actor systems equivalence. Concurrency index. Debugging (out of the scope of this presentation).

71. Actors71 Concurrency Index The concurrency index of a system, is the number of processors currently active in the system. A concurrency index of 1 means sequential execution. A high concurrency index may indicate bad programming.

72. Actors72 Concurrency Index - Mergesort

73. Actors73 Concurrency Index – Mergesort With Fewer Dependencies