Upload
casandra-galler
View
215
Download
0
Tags:
Embed Size (px)
Citation preview
The rise of the robots…The rise of the robots…Alasdair AllanSchool of Physics, University of Exeter
Alasdair AllanSchool of Physics, University of Exeter
VOTech DSRP, Sorrento, 6 - 9th March
3
The eSTAR network
© Nik Szymanek
User Agents
Embedded Agent The VO
GatewayService
Embedded Agents
OGLE IIIAlertAgent
GCNAlertAgent
Broker
VOTech DSRP, Sorrento, 6 - 9th March
4
• What does VOEvent have to do with the VO?– event messages from DBs– event messages from workflow– data mining for added semantic content– testbed for agent architectures in the VO
• Why should I be interested?– major drivers for the adoption of the VO
VOEvent and VOTech?
VOTech DSRP, Sorrento, 6 - 9th March
5
What we’ve done so far?
• Convenience layer and parser (Perl & C++)– building common cases (including GCN)– different parser methodologies– http://voevent.sourceforge.net/
• Prototype event network live and on sky• Limited (hard-wired logic) brokering• Archiving and persistent storage• RSS test feed(s)
– programmatic, or human readable?
VOTech DSRP, Sorrento, 6 - 9th March
6
Where are we going?
• Event brokering (aggregators)– time critical– non-time critical (data mining?)
• Event archiving and persistent storage– REST interface– persistence of data products
• Search service– REST interface
• Event Feeds– pull, or push?
VOTech DSRP, Sorrento, 6 - 9th March
7
Unanswered questions
• Protocol standard(s?)– How the documents are passed…
• Transport standard(s?)– How the documents are carried…
VOTech DSRP, Sorrento, 6 - 9th March
8
Protocol work
The recent work on interoperability between eSTAR and
RAPTOR/TALONS has show the importance of,
• “ack” messages
• “iamalive” messages
GatewayService
eSTARBroker
VOTech DSRP, Sorrento, 6 - 9th March
9
Mini-interoperability meeting
• Tentative agreement on a protocol standard– “ack” and “iamalive” message types
• Tentative agreement on transport standards– pull via RSS feeds– push via vanilla TCP/IP
TALONSCaltech
eSTAR
VOTech DSRP, Sorrento, 6 - 9th March
10
Should we mandate transport?
• Why? Nobody thought about RFC 1149 when IP datagram packets were standardised…
• See http://www.blug.linux.no/rfc1149/
VOTech DSRP, Sorrento, 6 - 9th March
11
An architecture diagram showing; publishers, subscribers, relays and repositories, along with the more complicated aggregators and brokers.
“Atomic” services
• Publisher• Subscriber• Relay• Repository
• Author• Author
VOTech DSRP, Sorrento, 6 - 9th March
13
Event feeds
There are two ways to feed events to consumers,
• Pull model (feeds)– polling– e.g. RSS feed
• Push model (forwarding)– via REST, SOAP or vanilla TCP
VOTech DSRP, Sorrento, 6 - 9th March
14
Pull model
• Advantage– under client control
• Disadvantage– as slow as the polling interval
A pull model, e.g. RSS, is never going to work for rapidresponse cases like GRB or transient follow-up.
VOTech DSRP, Sorrento, 6 - 9th March
15
Push model
• Advantage– theoretically faster, depending on architecture
• Disadvantage– administrative nightmare for publisher
A push model, e.g. GCN, is the only way to do rapidresponse work. However it isn’t really optimal, so we’llprobably be stuck using both approaches. Shouldn’toptimise our standard for distribution via RSS model.
VOTech DSRP, Sorrento, 6 - 9th March
16
Aggregators
Why should we aggregate event feeds?
• Consolidation– Do we need to support multiple publishers?
• Removal of duplicate events
• Added semantic content
• Trust issues
VOTech DSRP, Sorrento, 6 - 9th March
17
Storage
Why should we permanently store event messages?
• The data itself will be replicated elsewhere in the VO.
• Why do we care about the original message?
• If we want to search, we have to store.
• However, to me, the best use case for persistent storage is actually event feeds…
VOTech DSRP, Sorrento, 6 - 9th March
18
<enclosure>
• Its an optional sub-element of the RSS <item> tag that allows the feed publisher to include a link to a file.
• It has three required attributes. The url=“ ” attribute says where the enclosure is located, length=“ ” says how big it is in bytes, and type=“ ” says what its type is, a standard MIME type. e.g,
<enclosure url=“http://estar.org.uk/event.pl?id=886-1” length=“2440” type=“application/xml+voevent” />
• This is how podcasting works..
VOTech DSRP, Sorrento, 6 - 9th March
19
RSS feed
<?xml version="1.0" ?>
<rss version="2.0">
<channel>
<title>VOEvent GCN Notices</title>
<language>en</language>
<description>RSS feed for GCN notices</description>
<link>http://voeventnet.caltech.edu/GCN.html</link>
<item>
<title>MILAGRO_Source trigger 886-1</title>
<description>
<p>Possible GRB The event time is 2005-12-01T16:24:15 UT. Location RA 232.1248 Dec 62.9797 (J2000)</p>
</description>
<pubDate>Fri, 02 Dec 2005 16:09:52 PST</pubDate>
<link>http://voeventnet.caltech.edu/Notices/ivo_GCN_MILAGRO_58_886-1_2005-12-01T16:24:15.xml</link>
<author>Scott Barthelmy GCN [email protected]</author>
<comments>http://gcn.gsfc.nasa.gov/gcn3_archive.html</comments>
</item>
.
.
.
</channel>
</rss>
VOTech DSRP, Sorrento, 6 - 9th March
20
RSS feed
<?xml version="1.0" ?>
<rss version="2.0">
<channel>
<title>VOEvent GCN Notices</title>
<language>en</language>
<description>RSS feed for GCN notices</description>
<link>http://voeventnet.caltech.edu/GCN.html</link>
<item>
<title>MILAGRO_Source trigger 886-1</title>
<description>
<p>Possible GRB The event time is 2005-12-01T16:24:15 UT. Location RA 232.1248 Dec 62.9797 (J2000)</p>
</description>
<pubDate>Fri, 02 Dec 2005 16:09:52 PST</pubDate>
<link>http://voeventnet.caltech.edu/Notices/ivo_GCN_MILAGRO_58_886-1_2005-12-01T16:24:15.xml</link>
<author>Scott Barthelmy GCN [email protected]</author>
<comments>http://gcn.gsfc.nasa.gov/gcn3_archive.html</comments>
<enclosure url=“http://estar.org.uk/event.pl?id=886-1” length=“2440” type=“application/xml+voevent” />
</item>
.
.
.
</channel>
</rss>
VOTech DSRP, Sorrento, 6 - 9th March
21
Infrastructure changes for VOEvent
• Main change needed is to the registry, need to be able to register the “atomic” VOEvent services– author, publisher, relay, repository, subscriber
• Also need to handle services which do more than one atomic operation,– aggregator (relay + repository)– broker (relay + repository + publisher)
VOTech DSRP, Sorrento, 6 - 9th March
22
Infrastructure changes for agents
• Again the main (only?) change needed is to the registry, need to be able to register the agents.
• New types of services, – agents at telescopes– agents doing science– brokers for data mining
VOTech DSRP, Sorrento, 6 - 9th March
23
Things I’m going to promise you…
• Deliverables– agents deliverables– VOEvent standards & prototypes– integration into the VO
VOTech DSRP, Sorrento, 6 - 9th March
24
VOEvent deliverables
• Prototype “first cut” infrastructure• VOEvent parsers (Perl & Java)• Library to co-ordinate with Registry
– convenience layer to hide registries• Standards work, input into standards documents• VOEvent prototype network
– message handling & forwarding– storage & archiving– search services
VOTech DSRP, Sorrento, 6 - 9th March
25
Agent deliverables
• Event Broker (Agent)– aggregation of messages– data-mining for added semantic content– publishing its own event feed
• Query Broker (Agent)– marshalling asynchronous queries– encapsulation queries, hides low level interfaces
• Neural networks– more intelligent, intelligent agents– enhanced decision making for the VO
VOTech DSRP, Sorrento, 6 - 9th March
26
Integration with the VO?
• Emitting VOEvent messages from Workflow
• Wrapping ACR and Workflow inside agents
• Distributed storage of event messages in VOSpace
VOTech DSRP, Sorrento, 6 - 9th March
27
Conclusions
• Two different groups of people working on VOEvent
• Conflict between these groups now (mostly) resolved
• Standards moving forward, a lot of prototypes now being written at Exeter, Caltech and Los Alamos
• A good testbed for the intelligent agent work