Event Infrastructure Alasdair Allan University of Exeter Alasdair Allan University of Exeter Robert R. White Los Alamos National Laboratories The eSTAR/TALONS Interconnect

VOEvent Workshop II, Tucson 1 The eSTAR concept It’s an ad-hoc peer-to-peer network of heterogeneous resources utilising a collaborative agent paradigm for decision making Treat telescopes and databases in a similar manner, both being made available on an observational grid Disguise complexity, users hate complexity

VOEvent Workshop II, Tucson 2 eSTAR in a nutshell

VOEvent Workshop II, Tucson 3 What is an agent anyway? An agent is “just software” not magic Loosely, an agent is a computational entity which: Acts on behalf of another entity in an autonomous fashion Performs its actions with some level of proactivity and/or responsiveness Exhibits some level of the key attributes of learning, co-operation and mobility

VOEvent Workshop II, Tucson 4 Multi-agent systems A multi-agent system is one that consists of a number of agents, which interact with one-another. In the most general case, agents will be acting on behalf of users with different goals and motivations. To successfully interact, they will require the ability to cooperate, coordinate, and negotiate with each other, much as people do…

VOEvent Workshop II, Tucson 5 Multi-agent systems for eSTAR We’ve built the first agent based astronomical system, and it was clearly the correct choice of architecture. What we’ve built in eSTAR is a collaborative agent system which schedules telescopes. Complicated telescope schedules are built by humans using a multi-pass approach. Traveling salesman problem…

VOEvent Workshop II, Tucson 6 Peer-to-Peer Agents operate in a peer-to-peer manner and can make use of these interconnections between people and data. Carrying out intelligent resource discovery could mean that your agent looks to your collaborators agent for data and expertise before it looks to “central” sources.

VOEvent Workshop II, Tucson 7 The world is flat… The world is small and flat, but it is none the less still very complex. Architectures which take account of this are intuitive, and will map well into the real world. © Terry Pratchett

VOEvent Workshop II, Tucson 8 The eSTAR network © Nik Szymanek User Agents Embedded Agent The VO Embedded Agents GRB rapid follow-up programme Search for exo-planets Testbed for Robonet-1.0 open time Gateway Service

VOEvent Workshop II, Tucson 9 The eSTAR network © Nik Szymanek User Agents Embedded Agent The VO Gateway Service Embedded Agents GCN Alert Agent Broker

VOEvent Workshop II, Tucson 10 What we’ve done so far? Convenience layer and parser (Perl & C++) –building common cases (including GCN) –different parser methodologies Prototype event network live and on sky Limited (hard-wired logic) brokering Archiving and persistent storage RSS test feed(s) –programmatic, or human readable?

VOEvent Workshop II, Tucson 11 Where are we going? Event brokering (aggregators) –time critical –non-time critical (data mining?) Event archiving and persistent storage –REST interface –persistence of data products Search service –REST interface Event Feeds –pull, or push?

VOEvent Workshop II, Tucson 12 Event feeds There are two ways to feed events to consumers, Pull model (feeds) –polling –e.g. RSS feed Push model (forwarding) –via REST, SOAP or vanilla TCP

VOEvent Workshop II, Tucson 13 Pull model Advantage –under client control Disadvantage –as slow as the polling interval A pull model, e.g. RSS, is never going to work for rapid response cases like GRB or transient follow-up.

VOEvent Workshop II, Tucson 14 Push model Advantage –theoretically faster, depending on architecture Disadvantage –administrative nightmare for publisher A push model, e.g. GCN, is the only way to do rapid response work. However it isn’t really optimal, so we’ll probably be stuck using both approaches. Shouldn’t optimise our standard for distribution via RSS model.

VOEvent Workshop II, Tucson 15 Aggregators Why should we aggregate event feeds? Consolidation –Do we need to support multiple publishers? Removal of duplicate events Added semantic content Trust issues

VOEvent Workshop II, Tucson 16 Storage Why should we permanently store event messages? The data itself will be replicated elsewhere in the VO. Why do we care about the original message? If we want to search, we have to store. However, to me, the best use case for persistent storage is actually event feeds…

VOEvent Workshop II, Tucson 17 Atom feed eSTAR/TALONS GCN Feed eSTAR Project ivo://gcn.gsfc/hete/396943a. ]]> GCN

VOEvent Workshop II, Tucson 18 RSS feed VOEvent GCN Notices en RSS feed for GCN notices MILAGRO_Source trigger Possible GRB The event time is T16:24:15 UT. Location RA Dec (J2000) Fri, 02 Dec :09:52 PST Scott Barthelmy GCN

VOEvent Workshop II, Tucson 19 Its an optional sub-element of the RSS tag that allows the feed publisher to include a link to a file. It has three required attributes. The url=“ ” attribute says where the enclosure is located, length=“ ” says how big it is in bytes, and type=“ ” says what its type is, a standard MIME type. e.g, This is how podcasting works..

VOEvent Workshop II, Tucson 20 RSS feed VOEvent GCN Notices en RSS feed for GCN notices MILAGRO_Source trigger Possible GRB The event time is T16:24:15 UT. Location RA Dec (J2000) Fri, 02 Dec :09:52 PST Scott Barthelmy GCN

VOEvent Workshop II, Tucson 21 RSS feed VOEvent GCN Notices en RSS feed for GCN notices MILAGRO_Source trigger Possible GRB The event time is T16:24:15 UT. Location RA Dec (J2000) Fri, 02 Dec :09:52 PST Scott Barthelmy GCN

VOEvent Workshop II, Tucson 22 Unanswered questions We have our document standard, so at this stage there should be only two issues outstanding, Message passing protocol(s) –How the documents are passed Transport standard(s?) –How the documents are carried

VOEvent Workshop II, Tucson 23 Protocol work The recent work on interoperability between eSTAR and TALONS has show the importance of, ACK messages IAMALIVE messages Gateway Service eSTAR Broker

VOEvent Workshop II, Tucson 24 Should we mandate transport? Why? Nobody thought about RFC 1149 when IP datagram packets were standardised… See

VOEvent Workshop II, Tucson 25 Authentication Transport layer issue? Format issue?

VOEvent Workshop II, Tucson 26 Authorisation Transport layer issue?

VOEvent Workshop II, Tucson 27 Things to think about… Standards never survive widespread contact with users intact. At least no good standard does… If people twist and pull the standard to do something differently than we designed it to do, that’s our fault. The users are always right. No, seriously…