28
ALeRCE broker: current status and demo

ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

ALeRCE broker: current status and demo

Page 2: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

Outline

● Introduction● Goals● Team / Methodology● Infrastructure● Machine Learning

● Demo○ Web Interfaces○ API○ Jupyter notebooks

Page 3: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

Future time domain astronomy ecosystem

Survey telescopes

Alert brokers/TOMs

Follow up telescopes

+ other brokers/TOMs

Page 4: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

Astronomical infrastructure in Chile

TAOE-ELT

LSST

ALMA

SOAR

La Silla

CTIO

VLT

CTAGMT

CCAT

Magellan

VISTA

Gemini

Page 5: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure
Page 6: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

ALeRCE is a Chilean-led initiative to build a community broker for LSST and other large etendue

survey telescopes

Page 7: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

Goals

To facilitate the study of non-moving, variable and transients objects:

● Fast classification of transients, variable stars and active galactic nuclei

● Flexibility to adapt to different science cases (taxonomy, data products)

● Connect survey and follow up resources in Chile and abroad

Page 8: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

Scientific Questions

Page 9: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

ALeRCE Team

Page 10: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

Agile Methodology

● Daily meetings + 2 weeks sprints● Product owner● Kanban / Backlog● Weekly ML/training set meetings

Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio

Reyes, Camilo Valenzuela

Page 11: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

ALeRCE infrastructure

Distributed Storage

Container orchestrator

Distributed messaging

Distributed database

Services

Page 12: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

ALeRCE infrastructure

Distributed Storage

Container orchestrator

Distributed messaging

Distributed database

Services

Page 13: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

ALeRCE pipeline

Page 14: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

ZTF Data Model

Page 15: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

Classification

● Current Classifiers:

○ Convolutional Neural Network (stamps)

○ Random Forest (light curve)

● Future Classifiers:

○ Domain Adaptation○ Data Augmentation○ Outlier Detection○ Forecasting

Page 16: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

Late ClassifierEarly ClassifierConvolutional Neural Network

(using first stamps)Random Forest

(using light curve, at least 5 observations)

Classes: AGN, SN, VS, asteroid, bogusClasses:

- Stochastic: AGN-I, Blazar, CV/Nova- Transients: SNe Ia, SNe Ibc,SNe II,SNe IIn, SLSNe- Periodic stars: EBSD/D, EBC, DSCT, RRL, Ceph, LPV, Periodic Other

Page 17: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

δ

Page 18: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

18

Detection Threshold

Convolutional recurrent neural network

Carrasco-Davis+18PASP

Pclass i (t0)

Page 19: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

19

Detection Threshold

Convolutional recurrent neural network

Carrasco-Davis+18PASP

Pclass i (t0)

Page 20: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

Training vs domain overfitting

Carrasco-Davis+18PASP

Page 21: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

Summary● Goals:

○ Fast classification: transients, variable stars and AGN○ Flexibility to adapt to different science cases○ Connect survey and follow up

● Interdisciplinary team:○ Machine Learning○ Astronomy ○ Computer Science

● Infrastructure:○ Modular / Distributed / Robust

● Machine Learning:○ Classification: CNN / Random Forest○ Outlier Detection○ Domain Adaptation / Data Augmentation○ Forecasting

Page 22: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

DEMOhttp://alerce.science

Page 23: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

Web InterfacesJupyter

Notebooks

TOMs

API

Page 24: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

Web Interfaces

ZTF Explorerhttp://alerce.online

SN Hunterhttp://snhunter.alerce.online

Page 25: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

Mobile Phones

Page 26: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

SNe reported by ALeRCE

Page 27: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

API

● ZTF Database: http://ztf.alerce.online

● Avro/Stamps: http://avro.alerce.online

● catsHTM Cone Search & Xmatch: http://catshtm.alerce.online *

* Soumagnac & Ofek (2018), (Ofek 2014; ascl.soft 07005)

Page 28: ALeRCE broker: current status and demo · Javier Arredondo, Ernesto Castillo, Francisco Forster, Diego Rodríguez, Daniela Ruz, Ignacio Reyes, Camilo Valenzuela. ALeRCE infrastructure

Jupyter Notebooks

● API

● Transients

● Variable Stars

● Active Galactic Nuclei

https://github.com/alercebroker/usecases/tree/master/notebooks