1 HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY 12-15, 201 Name of Meeting Location...
Preview:
Citation preview
- Slide 1
- 1 HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY
12-15, 201 Name of Meeting Location Date - Change in Slide Master
LSST Alert Production Pipelines Simon Krughoff for the DM team LSST
Data Management at UW Hot-Wiring the Transient Universe May 12-15,
Santa Barbara
- Slide 2
- HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY
12-15, 201 Basic idea behind LSST: a uniform sky survey 90% of time
will be spent on a uniform survey: every 3-4 nights, the whole
observable sky will be scanned twice per night after 10 years, half
of the sky will be imaged about 1000 times (in 6 bandpasses,
ugrizy): a digital color movie of the sky ~100 PB of data: about a
billion 16 Mpix images, enabling measurements for 37 billion
objects Left: a 10-year simulation of LSST survey: the number of
visits in the r band (Aitoff projection of eq. coordinates) LSST in
one sentence: An optical/near-IR survey of half the sky in ugrizy
bands to r~27.5 (36 nJy) based on 1000 visits over a 10-year
period: deep wide fast.
- Slide 3
- HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY
12-15, 201 Required system characteristics Large primary mirror:
(at least 6m) to go faint and to enable short exposures (30 s)
Agile telescope: 5 sec for slew and settle Large field of view:
enable fast surveying Impeccable image quality: weak lensing Fast
Readout: ~2 sec for 3 GPix camera Sophisticated software: 20
TB/night, 37 billion objects, 30 trillion measurements
- Slide 4
- HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY
12-15, 201 LSST Telescope 8.4m, 6.7m effective 5 sec slew &
settle
- Slide 5
- HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY
12-15, 201 The field-of-view comparison: Gemini vs. LSST
- Slide 6
- HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY
12-15, 201 LSST camera
- Slide 7
- HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY
12-15, 201 LSST camera Modular design: 3200 Megapix = 189 x16
Megapix CCD 9 CCDs share electronics: raft (=camera) Problematic
rafts can be replaced relatively easily
- Slide 8
- 8 HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY
12-15, 201 Petascale Computing, Gbps Networks The computing cluster
at the LSST Archive at NCSA will run the processing pipelines.
Single-user, single-application data center Commodity computing
clusters. Distributed file system for scaling and hierarchical
storage Local-attached, shared-nothing storage when high bandwidth
needed Long Haul Networks to transport data from Chile to the U.S.
2x100 Gbps from Summit to La Serena (new fiber) 2x40 Gbps for La
Serena to Champaign, IL (path diverse, existing fiber) Archive Site
and U.S. Data Access Center NCSA, Champaign, IL Archive Site and
U.S. Data Access Center NCSA, Champaign, IL Base Site and Chilean
Data Access Center La Serena, Chile Base Site and Chilean Data
Access Center La Serena, Chile
- Slide 9
- 9 HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY
12-15, 201 LSST From the Users Perspective A stream of ~1-10
million time-domain events per night, detected and transmitted to
event distribution networks within 60 seconds of observation. A
catalog of orbits for ~6 million bodies in the Solar System. A
catalog of ~37 billion objects (20B galaxies, 17B stars), ~7
trillion observations (sources), and ~30 trillion measurements
(forced sources), produced annually, accessible through online
databases. Deep co-added images. Services and computing resources
at the Data Access Centers to enable user-specified custom
processing and analysis. Software and APIs enabling development of
analysis codes. Level 3 Level 1 Level 2
- Slide 10
- 10 HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY
12-15, 201 Level 1 Overview Primary purpose: Satisfy science cases
requiring rapid identification and follow-up (transients,
fast-moving NEOs, etc.) Transient science Nova, supernova, GRBs
Source characterization Instantaneous discovery Nearby Solar System
Objects NEOs, PHAs
- Slide 11
- 11 HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY
12-15, 201 Level 1 Overview Primary purpose: Satisfy science cases
requiring rapid identification and follow-up (transients,
fast-moving NEOs, etc.) Transient science Nova, supernova, GRBs
Source characterization Instantaneous discovery Nearby Solar System
Objects NEOs, PHAs
- Slide 12
- 12 HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY
12-15, 201 Alert Production: Pipeline overview ( , , flux, )
DIASource Raw visit DiffimDetectMeasure ( , , flux, ) DIASource
DIAObject SSObject Association Update DIAObject Alert Packet
DIAObject record DIASource record DIASource records Construct Alert
Transmit to event brokers LSST Simple broker Community Brokers
- Slide 13
- 13 HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY
12-15, 201 Introducing Some Jargon In normal survey mode, LSST will
operate by capturing two back-to-back, 15- second exposures for
each pointing. The two exposures are referred to as snaps They are
combined to a visit, which is the basic input image product for
Level 1 data products The primary purpose of the snaps is to
enhance cosmic ray rejection Level 1 processing will populate the
Level 1 database tables, including: DIASource : table of sources
detected on difference images DIAObject : objects and their
characteristics, inferred from DIASource detection SSObject : the
catalog of orbits of objects in the Solar System
- Slide 14
- 14 HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY
12-15, 201 Level 1 Pipeline: Source Detection A visit is acquired
and reduced (bias subtraction, flat fielding, cosmic ray rejection,
combining of snaps etc.). The visit image is then differenced
against the appropriate template DIASources are detected DIASource
position, flux, and shape are measured with multiple algorithms The
list is stored to a relational database for Level 1 data products
(into the DIASource table) ( , , flux, ) DIASource Calibrated visit
DiffimDetectMeasure
- Slide 15
- 15 HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY
12-15, 201 Level 1 Pipeline: Source Association The Level 1
database is searched for a match with an existing DIAObject or an
SSObject. If no match is found a new DIAObject is created If the
DIASource has been associated with an SSObject, we issue an alert
and stop. All further SSObject processing (eg., orbit
recomputation) occurs in daytime. Otherwise, the associated
DIAObject measurements are updated with new data (centroids,
light-curves, etc.). Forced photometry is performed for all
DIAObjects overlapping the field of view to which a DIASource from
this visit has not been associated. ( , , flux, ) DIASource
DIAObject SSObject Association Update DIAObject
- Slide 16
- 16 HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY
12-15, 201 Level 1 Pipeline: Issuing Alerts An alert packet is
constructed that includes all previous DIASource measurements, all
information contained in the DIAObject record, image cut-outs, and
all metadata necessary to query the Level 1 database for a copy of
this alert. We plan to use VOEvent as the format for alert
transmission The packet will be transmitted to VOEvent Brokers We
expect the community to provide sophisticated brokers with
classification engines, cross-match capabilities to other catalogs,
etc. LSST will provide a default, limited, broker Alert Packet
DIAObject record DIASource record DIASource records Construct Alert
Transmit to the internal or External Brokers
- Slide 17
- 17 HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY
12-15, 201 Level 1 Processing: System Architecture
- Slide 18
- 18 HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY
12-15, 201 Level 1 Processing: System Architecture
- Slide 19
- 19 HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY
12-15, 201 Level 1 Data Products Processing to enable rapid
detection and follow-up of time-domain events Real-time image
differencing as observing unfolds each night Measurement of
position, brightness and shape for each detection Alerts to
detected changes transmitted within 60 seconds of observing,
enabling rapid follow-up Transient Detection with Image
Differencing (CANDELS;
http://www.spacetelescope.org/images/heic1306d/)
- Slide 20
- 20 HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY
12-15, 201 Level 1: Transients Alerts LSST computing is sized for
10M alerts/night (average), 10k/visit (average), 40k/visit (peak)
Dedicated networking for moving data from Chile to the US Dedicated
image processing clusters New image differencing pipelines with
improved algorithms Will measure and transmit with each alert:
position flux, size, and shape light curves in all bands (up to a
~year; stretch: all) variability characterization (eg., low-order
light-curve moments, probability the object is variable) cut-outs
centered on the object (template, difference image)
- Slide 21
- 21 HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY
12-15, 201 A note on the LSST broker # Keep only never-before-seen
events within two # effective radii of a galaxy. This is for
illustration # only; the exact methods/members/APIs may change. def
filter(alert): if len(alert.sources) > 1: return False nn =
alert.diaobject.nearest_neighbors[0] if not nn.flags.GALAXY: return
False return nn.dist < 2. * nn.Re No cross-match or
classification Intended for simple filtering User defined filtering
See DPDD: https://ls.st/dpdd
- Slide 22
- 22 HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY
12-15, 201 Level 1 Catalogs: DIASources Includes measures of:
Position Shape (adaptive Gaussian moments; Bernstein & Jarvis
2002) Model fits: -Point source model Measure of flux and position
assuming the object is a stationary point source -Trailed source
model Measure of flux, position, and direction of motion, assuming
object moves sufficiently fast to trail in the image. Designed for
Solar System objects. -Dipole model fit Fit the source with a
dipole model, a positive next to a negative point source Some
DIASources will be false positives. We will flag suspect
DIASources
- Slide 23
- 23 HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY
12-15, 201 Level 1 Catalogs: DIAObjects Characterization of the
underlying astrophysical objects detected in difference images
Computed from associated DIASource records (recomputed as needed)
Primary goal: Enable object classification Include: Fits of
position, parallax, and proper motion Mean point source flux
(difference image and direct image) Variability characterization
(e.g., Richards et al. 2011 parameters) Pointers to nearby objects
in the Level 2 catalog
- Slide 24
- 24 HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY
12-15, 201 Level 1 Catalogs: SSObjects Catalog of Solar System
objects Asteroids, comets, KBOs, etc. Includes: Orbital elements
MOID Estimates of mean absolute magnitude (H, G) -Estimates
performed in LSST bands Convenience functions to compute the phase
angle, reduced, H( ), and absolute, H, asteroid magnitudes will be
provided by the Level 1 database.
- Slide 25
- 25 HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY
12-15, 201 Challenges Difference imaging algorithms Template
generation Flux dependent PSF Many more 0.6 FWHM Antilogus et al.
2014 Becker et al. Winter 2014 Report
- Slide 26
- 26 HOT-WIRING THE TRANSIENT UNIVERSE | SANTA BARBARA, CA | MAY
12-15, 201 Thanks!