The rise of the robots… Alasdair Allan School of Physics, University of Exeter Alasdair Allan School of Physics, University of Exeter

VOTech DSRP, Sorrento, 6 - 9th March 1 eSTAR in a nutshell

VOTech DSRP, Sorrento, 6 - 9th March 2 The eSTAR network © Nik Szymanek User Agents Embedded Agent The VO Gateway Service Embedded Agents OGLE III Alert Agent GCN Alert Agent Broker

VOTech DSRP, Sorrento, 6 - 9th March 3 What does VOEvent have to do with the VO? –event messages from DBs –event messages from workflow –data mining for added semantic content –testbed for agent architectures in the VO Why should I be interested? –major drivers for the adoption of the VO VOEvent and VOTech?

VOTech DSRP, Sorrento, 6 - 9th March 4 What we’ve done so far? Convenience layer and parser (Perl & C++) –building common cases (including GCN) –different parser methodologies – Prototype event network live and on sky Limited (hard-wired logic) brokering Archiving and persistent storage RSS test feed(s) –programmatic, or human readable?

VOTech DSRP, Sorrento, 6 - 9th March 5 Where are we going? Event brokering (aggregators) –time critical –non-time critical (data mining?) Event archiving and persistent storage –REST interface –persistence of data products Search service –REST interface Event Feeds –pull, or push?

VOTech DSRP, Sorrento, 6 - 9th March 6 An architecture diagram showing; publishers, subscribers, relays and repositories, along with the more complicated aggregators and brokers. “Atomic” services Publisher Subscriber Relay Repository Author

VOTech DSRP, Sorrento, 6 - 9th March 7 UML diagram CREDIT: Philip Warner, NOAO

VOTech DSRP, Sorrento, 6 - 9th March 8 Event feeds There are two ways to feed events to consumers, Pull model (feeds) –polling –e.g. RSS feed Push model (forwarding) –via REST, SOAP or vanilla TCP

VOTech DSRP, Sorrento, 6 - 9th March 9 Pull model Advantage –under client control Disadvantage –as slow as the polling interval A pull model, e.g. RSS, is never going to work for rapid response cases like GRB or transient follow-up.

VOTech DSRP, Sorrento, 6 - 9th March 10 Push model Advantage –theoretically faster, depending on architecture Disadvantage –administrative nightmare for publisher A push model, e.g. GCN, is the only way to do rapid response work. However it isn’t really optimal, so we’ll probably be stuck using both approaches. Shouldn’t optimise our standard for distribution via RSS model.

VOTech DSRP, Sorrento, 6 - 9th March 11 Aggregators Why should we aggregate event feeds? Consolidation –Do we need to support multiple publishers? Removal of duplicate events Added semantic content Trust issues

VOTech DSRP, Sorrento, 6 - 9th March 12 Storage Why should we permanently store event messages? The data itself will be replicated elsewhere in the VO. Why do we care about the original message? If we want to search, we have to store. However, to me, the best use case for persistent storage is actually event feeds…

VOTech DSRP, Sorrento, 6 - 9th March 13 Its an optional sub-element of the RSS tag that allows the feed publisher to include a link to a file. It has three required attributes. The url=“ ” attribute says where the enclosure is located, length=“ ” says how big it is in bytes, and type=“ ” says what its type is, a standard MIME type. e.g, This is how podcasting works..

VOTech DSRP, Sorrento, 6 - 9th March 14 RSS feed VOEvent GCN Notices en RSS feed for GCN notices MILAGRO_Source trigger Possible GRB The event time is T16:24:15 UT. Location RA Dec (J2000) Fri, 02 Dec :09:52 PST Scott Barthelmy GCN

VOTech DSRP, Sorrento, 6 - 9th March 15 RSS feed VOEvent GCN Notices en RSS feed for GCN notices MILAGRO_Source trigger Possible GRB The event time is T16:24:15 UT. Location RA Dec (J2000) Fri, 02 Dec :09:52 PST Scott Barthelmy GCN

VOTech DSRP, Sorrento, 6 - 9th March 16 Unanswered questions We have our VOEvent document standard, so there are only two issues outstanding, Protocol standard(s?) –How the documents are passed… Transport standard(s?) –How the documents are carried…

VOTech DSRP, Sorrento, 6 - 9th March 17 Protocol work The recent work on interoperability between eSTAR and RAPTOR/TALONS has show the importance of, “ack” messages “iamalive” messages Gateway Service eSTAR Broker

VOTech DSRP, Sorrento, 6 - 9th March 18 Mini-interoperability meeting Tentative agreement on a protocol standard –“ack” and “iamalive” message types Tentative agreement on transport standards –pull via RSS feeds –push via vanilla TCP/IP TALONSCaltecheSTAR

VOTech DSRP, Sorrento, 6 - 9th March 19 Should we mandate transport? Why? Nobody thought about RFC 1149 when IP datagram packets were standardised… See

VOTech DSRP, Sorrento, 6 - 9th March 20 Infrastructure changes for VOEvent Main change needed is to the registry, need to be able to register the “atomic” VOEvent services –author, publisher, relay, repository, subscriber Also need to handle services which do more than one atomic operation, –aggregator (relay + repository) –broker (relay + repository + publisher)

VOTech DSRP, Sorrento, 6 - 9th March 21 Infrastructure changes for agents Again the main (only?) change needed is to the registry, need to be able to register the agents. New types of services, –agents at telescopes –agents doing science –brokers for data mining

VOTech DSRP, Sorrento, 6 - 9th March 22 Things I’m going to promise you… Deliverables –agents deliverables –VOEvent standards & prototypes –integration into the VO

VOTech DSRP, Sorrento, 6 - 9th March 23 VOEvent deliverables Prototype “first cut” infrastructure VOEvent parsers (Perl & Java) Library to co-ordinate with Registry –convenience layer to hide registries Standards work, input into standards documents VOEvent prototype network –message handling & forwarding –storage & archiving –search services

VOTech DSRP, Sorrento, 6 - 9th March 24 Agent deliverables Event Broker (Agent) –aggregation of messages –data-mining for added semantic content –publishing its own event feed Query Broker (Agent) –marshalling asynchronous queries –encapsulation queries, hides low level interfaces Neural networks –more intelligent, intelligent agents –enhanced decision making for the VO

VOTech DSRP, Sorrento, 6 - 9th March 25 Integration with the VO? Emitting VOEvent messages from Workflow Wrapping ACR and Workflow inside agents Distributed storage of event messages in VOSpace

VOTech DSRP, Sorrento, 6 - 9th March 26 Conclusions Two different groups of people working on VOEvent Conflict between these groups now (mostly) resolved Standards moving forward, a lot of prototypes now being written at Exeter, Caltech and Los Alamos A good testbed for the intelligent agent work