cloud resourcing a middleware example Simon Lynch

features a network of "willing" nodes dynamic resource allocation (similar-ish to Contract Net Protocol) nodes can initiate or perfom computations nodes join/leave explicitly (& leave implicitly on failure) NB: unlike CNP, nodes register with subcontractors indicating willingness to perform computations - no further negotiation

structure dynamic, 3 tier, meta-agent subsystem… Systems Cluster deals with nodes joining and leaving loaded (auto/manual) on all candidate nodes Management Cluster deals with task allocation dynamic – build when node becomes a resource Applications Cluster performs allocated tasks dynamic – build when performing computation(s)

structure

node N joins the pool... 1.N sends message to a pool management agent (PMA) to register availability 2.PMA accepts N 3.N's systems cluster builds Management Cluster (M) 4.M is provided with identity of a subcontractor (subcontractor manages requests for processing) 5.M sets up new comms link with subcontractor 6.N provides details to the subcontractor… purpose (resource or initiator) capabilities (nature of support, processing capacity…) time period available (if known)

other notes... M handles negotiations necessary for N to accept & run tasks (on behalf of other nodes) M also deals with negotiations to enlist help of other nodes when N's computations can be run in parallel subcontractors maintain white/yellow page information (details of nodes & capabilities) can use networks of subcontractors… each manages subset contacts contacts are nodes & other subcontractors

various parrallel forms... parallel( C 0.. C n )results are required from all computations C 0.. C n p-any( C 0.. C n )one result required from any computation C 0.. C n p-and( C 0.. C n )true iff all computations C 0.. C n are true p-or( C 0.. C n )true iff any computations C 0.. C n are true p-backtrack(C 0.. C n )results from C 0.. C n are followed as they arrive - backtrack on failure

an AI example... an artificial life/evolutionary learning system possible parallel executions… divides population into subpopulations allowing them to evolve independently before re-dividing each sub-pop is tested in shared environment (isolated from other sub-pops) runs 5-6 independent tests with each sub-pop

notation... AiNiAiNi agent A i active on node N i CiCi a computation C i (a task or series of statements) C i | A i N i agent A i performing computation C i on node N i { x; y }tasks x and y execute concurrently A i  A j [m]agent A i sends agent A j message m A i  A j [ m ] / R i agent A i sends agent A j message m requesting that replies are posted to the receiver R i A j  A i / R i [ m 2 ]A j sends message m2 to the receiver R i of agent A i M 0 : CREATE( A 0 )M 0 creates a new agent A 0 A i SLEEPSA i pauses, usually until reactivated by an incoming message A i WAKESA i reactivates A i TERMINATESA i completes all activities and ceases to exist C x COMPLETEScomputation C x finishes

C 0 ; parallel(C 1 ; C 2 ); C 3 part-1 A 0 N 0 processes statements C 0 C 0 | A 0 N 0 COMPLETES A 0 N 0 reaches parallel statement & informs management agent M 0 N 0 A 0 N 0  M 0 N 0 [ C 1, C 2 ] / R a0 A 0 N 0 SLEEPS M 0 N 0 elects to process C 1 so creates A 1 N 0 and gives it C 1 M 0 N 0 : CREATE( A 1 N 0 ) M 0 N 0  A 1 N 0 [C 1 ] / R b0(c1) M 0 N 0 requests support for C 2 from its subcontractor S M 0 N 0  S [ requestNode, C 2 ] / R b0(c2) S forwards the request to a registered and available node M 0 N 1 S  M 0 N 1 [ C 2 ] / R b0(c2) new node M 0 N 1 creates application layer agent &initiates its computation M 0 N 1 CREATE( A 0 N 1 ) M 0 N 1  A 0 N 1 [C 2 ] / R b1(c2)

M 0 N 1 creates an application layer agent and initiates its computation M 0 N 1 CREATE( A 0 N 1 ) M 0 N 1  A 0 N 1 [C 2 ] / R b1(c2) A 0 N 1 completes C 2 with result β, β is rippled back as appropriate, A 0 N 1 terminates, M 0 N 1 informs S it is no longer busy C 2 | A 0 N 1 COMPLETES A 0 N 1  M 0 N 1 / R b1(c2) [ β ] M 0 N 1  M 0 N 0 / R b0(c2) [ β ] A 0 N 1 TERMINATES M 0 N 1  RA [ node-idle ] A 1 N 0 completes C 1 with result α, passes a to M 0 N 0 then terminates C 1 | A 1 N 0 COMPLETES A 1 N 0  M 0 N 0 / R b0(c1) [α ] A 1 N 0 TERMINATES M 0 N 0 receives results for both C 1 & C 2 (R b0(c1) & R b0(c2) have triggered) A 0 N 0 gets results & is re-activated M 0 N 0  A 0 N 0 / R a0 [α, β ] A 0 N 0 WAKES A 0 N 0 completes parallel( C 1 ; C 2 ){ C 1 ; C 1 } | A 0 N 0 COMPLETES A 0 N 0 processes statements C 3 C 3 | A 0 N 0 C 0 ; parallel(C 1 ; C 2 ); C 3 part-2