Upload
zenia
View
41
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Skyalert: Rapid Transients for you and your Robot. Roy Williams LIGO Laboratory, Caltech Andrew Drake, Ashish Mahabal Astronomy, Caltech. Supported by US National Science Foundation OCI-0915473. http://lib.skyalert.org/skyalert/Guide2Skyalert-1.3.pdf. Skyalert is …. - PowerPoint PPT Presentation
Citation preview
1
Skyalert: Rapid Transients for you and your Robot
Roy WilliamsLIGO Laboratory, Caltech
Andrew Drake, Ashish MahabalAstronomy, Caltech
Supported by US National Science Foundation OCI-0915473
2
Skyalert is ….A clearinghouse and repository of information about astronomical transients, each described by a collection of VOEvent packets that may be multiply authored. The components of Skyalert are:• A web-based event broker, allowing subscription so that information about transients can
be delivered to users and their telescopes immediately upon receipt.• A web-based authoring system, so that authenticated users can inject events direct from
automated discovery pipelines, or fill in web forms, that may be delivered rapidly to others.
• An event repository, storing all events that come through the broker, and allowing bulk queries and drill-down.
• A ‘click or code’ paradigm that allows people web based access and machines web service access.
• A way to browse recent and past transients, as tables, multi-layered web pages, or with popular astronomical software.
• A development platform for building real-time decision rules about transients, and for mining the repository.
• Open-source software to allow local implementations as well as the web-based application.
http://lib.skyalert.org/skyalert/Guide2Skyalert-1.3.pdf
3
Installations
• Production: Skyalert.org at Caltech• Test: Betelgeuse.ligo.caltech.edu:8000• IUCAA, India• LSST, Tucson, Arizona• AMON, Penn State, Pennsylvania• GAIA, Cambridge UK
http://lib.skyalert.org/skyalert/
4
MicrolensingOptical transients
Radio transientsX-ray transients
Gamma transients
Follow-up Scheduler
TelescopeTelescope
Telescope
Authors Subscribers
annotation from archives
Events and annotation disseminated to subscribers
in real time with intelligence
skyalert.org
AstronomersAmateursStudents
Skyalerthttp://skyalert.org
GCN Broker
5
VOEvent is the common language
http://ivoa.net/Documents/VOEvent/
http://hotwireduniverse.org
http://www.skyalert.org/s/sampleEvent.html
6
<voe:VOEvent xsi:schemaLocation="http://www.ivoa.net/xml/VOEvent/v2.0 /Users/roywilliams/Documents/VOEventLib/schema/VOEvent-v2.0.xsd" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:voe="http://www.ivoa.net/xml/VOEvent/v2.0" version="2.0" ivorn="ivo://ldas-pcdev1.ligo.caltech.edu/VOEvent#a8" role="test">
<Description>Candidate galaxies for LIGO trigger event</Description> <Reference uri="http://ldas-pcdev1.ligo.caltech.edju/VOEventRepository/1"/>
<Citations> <EventIVORN cite="followup">ivo://ldas-pcdev1.ligo.caltech.edu/VOEvent#a7</EventIVORN> </Citations>
<Who> <Author> <title>LSCVirgo</title> <contactName>Ed Daw</contactName> <contactEmail>[email protected]</contactEmail> <contactPhone>+44 114 268 6895</contactPhone> </Author> </Who> <What> <Param name="EVETut_ID" value="012345678" ucd="meta.id"> <Description>use the gps second as an event ID </Description> </Param> <Param name="SNR" value="2.4" ucd="stat.snr"> <Description>Significance of the trigger</Description> </Param> <Param name="UTC" value="2005-04-15T23:59:59" ucd="time.epoch"> <Description>Time of the trigger</Description> </Param> <Param name="NO_OF_GALAXIES" value="3" ucd="meta.number"> <Description>Number of candidate source galaxies in this VOEvent</Description> </Param> <Table> <Description>Candidate sources for the LSC/Virgo trigger</Description> <Field name="Source rank" ucd="meta.code"> <Description>this integer is 1 for the most likely source, 2 for 2nd, etc</Description> </Field> <Field name="GWGC_CATALOGUE_NUMBER" ucd="meta.number"> <Description>this is the integer ID of the galaxy in Darren's catalogue</Description>
VOEvent
• Who – author’s provenance• What – author’s data packet• WhereWhen – targeting in spacetime• Why – scientific characterization• Citation – related events• IVORN – unique identifier
http://ivoa.net/Documents/VOEvent/
http://hotwireduniverse.org
http://www.skyalert.org/s/sampleEvent.html
7
VOEvent validation
Choose herefor validation
http://www.skyalert.org/submit/
8
VOEventLib
• Python Library for VOEvent– Read, modify, create,
write VOEvents– lib.skyalert.org/
VOEventLib– Code built
automatically from XML schema• GenerateDS package• Thanks Dave Kuhlman!
from VOEventLib.VOEvent import *from VOEventLib.Vutil import *import sys# get filename with event to be modified
# parse the event from the file namev = parse(filename)
# set the ivorn to something elsev.set_ivorn('ivo://silly/billy#iuw6e7F72ufh')
# set the authorv.get_Who().get_Author().set_contactName(['Mickey Mouse'])
# look for a specific param in the event to changeparam = findParam(v, '', 'magnitude')if param: val = paramValue(param) param.set_value("16.45")
xml = stringVOEvent(v)print xml
http://lib.skyalert.org/VOEventLib/
9
Who uses VOEvent
• IAU/CBAT, AAVSO• GAIA• CRTS, CSS asteroids, MOA, OGLE• SWIFT, Fermi, Integral, Agile, etc
(GCN translation)
• LIGO, LOFAR• LSST?• amateurs (DC3.com)
10
Event timeline
click event for drilldown
http://www.skyalert.org/
11
Worldwide Telescopeview of all recent events
CLICK
12
Skyalert Key Concepts• Identifier
• IVORN• Stream (class) IVORN inside event (instance) IVORN
• Stream• Event semantics defined by author
– What does Burst_SOD mean in the Fermi stream?• Like a data dictionary, a schema, common metadata
• Portfolio• Event + associated data
– As collection of VOEvents with the observation first
• Alert/Select• Define what is ‘interesting’ as Python expression on Params
– eg CRTS[“magnitude”] < 17– Get message, run annotator, run telescope
• Actually … everything is an Alert
?
FermiCRTS3PIoftheSKY
13
Concept: Identifier== IVORN
• Can identify a stream:• ivo://moa.massey/transient_alerts
• Can identify an event from that stream:• ivo://moa.massey/transient_alerts#gb10-R-8-16933_1
The event IVORN contains the stream IVORN
http://www.ivoa.net/Documents/latest/IDs.html
14
Concept: Stream as Event Template
Event author defines parameters and meaning in advance.
stream event
made byperson in daytime
made byrobot system at night
15
Stream DefinitionName, IVORN, ownerDescription
Overview Template
See events, delete tests
Parameters that can be used in the overview
Sample event
Owner view Public view
http://www.skyalert.org/streams/54/
16
Skyalert Stream Registry... will be VO registry
http://www.skyalert.org/streams/
17
Concept: Portfolio
==“data holdings about a transient”Multiple authors
Primary event plus secondary data: follow-up, archives, data-mining results etc
In VOEvent there are <Citation> tags
18
Portfolio skins
OverviewParamsXML
http://www.skyalert.org/events/46556/
19
CRTS portfolio
CRTS observation Later CRTSobservation
Is it in the archivesyes: NED and NVSS
Telescopic followup
Classification asblazar outburst
http://www.skyalert.org/events/33547/
20
SWIFT portfolio
Notices from the satellite Circulars from the humans
http://www.skyalert.org/events/49299/
21
What is an Alert?
• Selects “interesting” events– For rapid messaging– For response to query– Default action is to send email, some others:
• addconstellation• alert_email• coincidence• csannotator• post2skyalert• sdssannotator• senddakota
?
22
Annotators
• When triggered, annotator builds event, appends to portfolio
• Annotator can be local or remote
• Example• For each event, find
nearest galaxy in SDSS
http://www.skyalert.org/s/annotator.html
24
Skyalert Overview
GCNEmailJabberDakota etc
MessagingEmailJabberDakota etc
Authoring
= http POST
Annotation
Replication
BrowsingJSON servicesFeeds
Stre
ams
Events
Portfolio
?
??
Alert
25
Users, Groups, Streams Auth and Auth
Users registerGet email & respond
Each stream has User who is owner
Users can join Groups
Groups can have permission to view streams
26
Tutorials
• Tut1: Stream, Sample event, Authoring• Tut2: Authoring with Code• Tut3: Alert, Annotation• Tut4: Mining
Please use test machine https://betelgeuse.ligo.caltech.edu:8000/
27
Tut1: StreamLook at “Test” stream
Anonymous and Logged in
Play with template
Look at bottom for “Sample Event”
https://betelgeuse.ligo.caltech.edu:8000/streams/54/
28
Tut1: Sample
Copy sample and Edit
Change IVORNivorn="ivo://testing/test#5551"Change RA, DecChange ISOtimeChange Params
Go to Authoring page
https://betelgeuse.ligo.caltech.edu:8000/streams/sampleEvent/54/
Keep this on your desktop
29
Tut1: Validation and Authoring
Select “Validate XML”Click to Go
Now authoring …Enter stream name “Test”Enter username/passClick to Go
https://betelgeuse.ligo.caltech.edu:8000/submit/
30
Tut1: Authoring
Event successfully saved
Portfolio visible
31
Tut2: Authoring by code
• Very simple code– http gets through firewalls– Standard Python/Perl etc
• Can also author with – Jabber, Dakota, Email
http://www.skyalert.org/s/authorcode.html
import urllibdict = {}
# the server that will handle the submit requesturl = "https://betelgeuse.ligo.caltech.edu:8000/submit/"
# choose 'dryrun' for validation and 'author' for authoringdict['checker'] = 'dryrun'
# for command line, we want plain text output, not HTMLdict['plainResponse'] = 'on'
# uncomment these for authoring#dict['checker'] = 'author'
# Skyalert username and password#dict['username'] = username#dict['password'] = password
# This is the short name for the stream,# must match credentials and event!#dict['streamName'] = 'myEventStream'
# Should alerts be run once the event is ingested?#dict['doRules'] = 'on'
# open a file for the XMLdict['xmlText'] = open('sample.xml').read()
# Now send it off and print the resultparams = urllib.urlencode(dict)f = urllib.urlopen(url, params)result = f.read()print result
Change serverPut in username/passwordMake XML file from desktop
must change IVORN
32
Log in to build your own alerts and feeds
Select the “Test” stream
Tut3: Building an Alerthttps://betelgeuse.ligo.caltech.edu:8000/rules/newrule/
?
33
Tut3: Building an Alert
Select the “Test” streamGive it a nameChoose action as csannotator
Make trigger expressionCan use red dots
belowOr type Python
Click Save
https://betelgeuse.ligo.caltech.edu:8000/rules/337/
?
34
Tut3: Authoring with Consequences
Run with Alerts
Have changed RA and Dec of event to194.95 and 27.98
35
Tut3: Annotator Ran
Alerts run on Test events
Annotator fetches data from VO archives
36
Tut3: Discover Coma Cluster
Annotator (VO archives) reveals Coma Cluster from NED
https://betelgeuse.ligo.caltech.edu:8000/events/49663/
37
Tut4: Mining the Events
Alert is a query
Use JSON to get query result-- page/resumption
Then JSON to get portfolio detail
38
Tut4: Select• List the portfolios that satisfy an
alert with JSON return: /jtable/{ "rule": "285", "query": "trigger expression", "next": resumption URL, "list": [ [ "http://....", "ivo://....", "2011-09-19 02:12:23" ],
from django.utils import simplejsonimport urllib
sk = ‘http://skyalert.org’
# CRTS and CRTSCircular["First"]["eventClass"] == "Supernova"query = "285"
jtableurl = "%s/events/jtable/%s/" % (sk, query)
while 1: jsontable = urllib.urlopen(jtableurl).read() pytable = simplejson.loads(jsontable) list = pytable["list"] print "Found %d portfolios" % len(list) for triple in list: ivorn = triple[1] print ivorn
http://www.skyalert.org/events/jtable/285/
39
Tut4: Resolve• Resolve an IVORN to the
portfolio: /resolve/ { "url": "http://skyalert.org/events/50425", "portfolio": { "ivo://.....": { "Group1": { "Param1": "...", "Param2": "….",
# get magnitude from a CRTS eventsk = "https://betelgeuse.ligo.caltech.edu:8000"dict = {‘ivorn’:ivorn, ‘JSON’:’on’}params = urllib.urlencode(dict)resolveurl = "%s/resolve/" % skjsondetail = urllib.urlopen(resolveurl, params).read()pydetail = simplejson.loads(jsondetail)primaryevent = pydetail["portfolio"][ivorn]mag = primaryevent["FirstDetectionparams"]["magnitude"]
http://www.skyalert.org/resolve/
40
Tut4: Mining the Eventsfrom django.utils import simplejsonimport urllib
#sk = "http://skyalert.org"sk = “https://betelgeuse.ligo.caltech.edu:8000”
# CRTS and CRTSCircular["First"]["eventClass"] == "Supernova"query = "285"
jtableurl = "%s/events/jtable/%s/" % (sk, query)
while 1: jsontable = urllib.urlopen(jtableurl).read() pytable = simplejson.loads(jsontable) list = pytable["list"] print "Found %d portfolios" % len(list) for pf in list: ivorn = pf[1]
dict = {} dict['ivorn'] = ivorn dict['JSON'] = 'on' params = urllib.urlencode(dict) resolveurl = "%s/resolve/" % sk jsondetail = urllib.urlopen(resolveurl, params).read() pydetail = simplejson.loads(jsondetail) # ivorn of the primary event of the portfolio should be here crtsevent = pydetail["portfolio"][ivorn] mag = crtsevent["FirstDetectionparams"]["magnitude"]
tok = ivorn.split('#') localivorn = tok[1] date = crtsevent[""]["ISOtime"] print localivorn, date, mag
jtableurl = jsonresult["next"] if jtableurl == "finished": break
1107030071174105491 2011-07-03T08:36:01 18.9354001107030091124153498 2011-07-03T08:01:30 18.7563991107031180794128821 2011-07-03T04:39:47 20.1698001107031070764142675 2011-07-03T04:07:39 18.1661001107021180094118167 2011-07-02T10:46:06 18.6014001107021041264109182 2011-07-02T09:31:06 15.8796001107011150104107901 2011-07-01T10:47:36 18.2796991107011120104119909 2011-07-01T10:46:15 17.8748001107010040834139996 2011-07-01T06:17:32 18.4836011107011040674116917 2011-07-01T04:13:08 19.630899
Magnitudes of CRTS supernovae
41
Skyalert Statistics
303 registered
users
61 gmail.com 24 aol.com 15 yahoo.com 7 hotmail.com 6 mail15.com 6 astro.caltech.edu 5 camerabuy.info 5 cacr.caltech.edu 4 caltech.edu 3 lsst.org 2 yahoo.com.br 2 yahoo.co.uk 2 ya.ru
http://webstats.skyalert.org
42
Skyalert Statistics2724 emails sent May 18 thru June 18
new full
Which streams generated them?910 SWIFT785 CRTS340 CBAT204 CRTSCircular192 MOA156 CRTS2111 Fermi20 AAVSO6 HST_MCT
Who got them?324 [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected]
Emails per day
43
Questions?
Please register at skyalert.org
44
Slide 2: http://lib.skyalert.org/skyalert/Guide2Skyalert-1.3.pdfSlide 3: http://lib.skyalert.org/skyalert/Slide 4: http://www.skyalert.orgSlide 5 and 6: http://ivoa.net/Documents/VOEvent/Slide 7: http://www.skyalert.org/submit/Slide 8: http://lib.skyalert.org/VOEventLib/Slide 10: http://www.skyalert.orgSlide 13: http://www.ivoa.net/Documents/latest/IDs.htmlSlide 15: http://www.skyalert.org/streams/54/Slide 16: http://www.skyalert.org/streams/Slide 18: http://www.skyalert.org/events/46556/Slide 19: http://www.skyalert.org/events/33547/Slide 20: http://www.skyalert.org/events/49299/Slide 22: http://www.skyalert.org/s/annotator.htmlSlide 23: http://pipes.yahoo.com/pipes/Slide 27: https://betelgeuse.ligo.caltech.edu:8000/streams/54/Slide 28: https://betelgeuse.ligo.caltech.edu:8000/streams/sampleEvent/54/Slide 29: https://betelgeuse.ligo.caltech.edu:8000/submit/Slide 31: http://www.skyalert.org/s/authorcode.htmlSlide 32: https://betelgeuse.ligo.caltech.edu:8000/rules/newrule/Slide 33: https://betelgeuse.ligo.caltech.edu:8000/rules/337/Slide 36: https://betelgeuse.ligo.caltech.edu:8000/events/49663/Slide 38: http://www.skyalert.org/events/jtable/285/Slide 39: http://www.skyalert.org/resolve/Slide 41: http://webstats.skyalert.org