38
2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 1 Gaia Solving equations with a billion unknowns Lennart Lindegren Lund Observatory Lund University, Sweden

Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

Embed Size (px)

Citation preview

Page 1: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 1

Gaia

Solving equations with a billionunknowns

Lennart Lindegren

Lund ObservatoryLund University, Sweden

Page 2: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 2

Outline of talk

• Gaia in a nutshell

• Observation principle

• Hardware

• Data processing challenge

• The solution

Page 3: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 3

Gaia in a nutshell

• All-sky astrometric survey carried out 2012 – 2017

‒ final results around 2020‒ positions, parallaxes (distances), proper motions (transverse vel.)‒ flux measurements at low spectral resolution

• All point objects in the detection range (magnitude 6 to 20)

‒ stars, asteroids, quasars, extragalactic supernovae, etc‒ about 109 objects

• Using Hipparcos principle (scanning, two fields of view)

‒ positional accuracy from 6 μas (bright stars) to 200 μas (faint)

‒ tied to a non-rotating frame via ~500,000 quasars

• Spectroscopic radial velocities (V < 16, few km/s)

Page 4: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 4

How small is 6 micro-arcseconds?

distance 0.1 km 6 km 360 km 360,000 km 60 million km

angle 1 degree 1 arcmin 1 arcsec 1 mas 6 μas (3600") (60") (1") (0.001") (0.000006")

Hip

parc

hos,

130

BC

Tych

o, 1

590

Röm

er, 1

704

1900

Hip

parc

os, 1

989

Gai

a, 2

012

Page 5: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 5

Measurement of parallax (π)

π

π

J1

J2

Sun

picture taken whenthe Earth is at J1

picture taken whenthe Earth is at J2

distance = A / (sin π)

A

Page 6: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 6

Principle of parallax measurement - from the ground

Friedrich Wilhelm Bessel's measurementof the parallax of 61 Cyg (1838) - relativeto two background stars - gave � = 0.294”(error was about 0.01”)

Page 7: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 7

1%

2%

5%

10%

20%

“Pinwheel Galaxy” (M101)Credit: ESA/NASA/CFHT/NOAO

You are here

30,0

00 li

ghty

ears

Parallax horizon for K5III stars (no extinction)

Hipparcos limit

Page 8: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 8

Gaia - schedule

ProposalConcept & Technology Study

Mission Selection

Re-Assessment Study

Phase B1

Scientific operation

Launch 2012-Aug

Final

Studies

Mission Data Processing

Implementation

Data Processing

Definition

Operation

Mission ProductsIntermediate

now

Selection of Prime Contractor (EADS Astrium)

Phase B2Phase C/D

Software Development

1995

2000

2005

2010

2015

2020

1994

1993

1997

1998

1999

2019

2018

2017

2016

2014

2013

2012

2011

2009

2008

2007

2006

2004

2003

2002

2001

1996

2021

Page 9: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 9

Soyuz/Fregat launch from Kourou (French Guyana)

~1 month transfer orbit to L2

Lissajous orbit around L2

~1 orbit correction per month

5 - 6 years of (almost)continuous observation

Gaia launch and orbit(Credit: EADS Astrium)

Earth-Moon barycentre, a = 1 AU

L2, a = 1.01 AU

Page 10: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 10

10 m diam.deployablesunshield

solar array

optical instrument(under thermal cover)in permanent shadow

Page 11: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 11

2 off-axis telescopes1.45 x 0.5 m2 aperture35 m focal length

common focal plane, 106 CCDs(1 Gigapixel)0.93 x 0.42 m2

basic angle = 106.5°

Page 12: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 12

Gaia focal plane (106 CCDs)

detectionand FOV

discrimination

astrometricmeasurements

photometry(dispersed

images)

radial velocity(dispersedimages)

BAM = basic angle monitor, WFS = wavefront sensor

0.42

m

0.93 m

SM

1

SM

2

AF

1

AF

2

AF

3

AF

4

AF

5

AF

6

AF

7

AF

8

AF

9

BP

RP

RV

S1

RV

S2

RV

S3

WF

S2

WF

S1

BA

M2

BA

M1

Page 13: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 13

CCD data collection in the astrometric field

on-chip binning of pixels across-scan

window of 6-18 samples transmitted to ground

"Time of observation" for image centre relative to CCDdetermined to ~200 μas precision (magnitude 15)

Some 700 such measurements per object in 5 years

scan

Page 14: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 14

path of viewing directions over 4 days

path of spin axisover 4 days

Gaia scanning: Motion of viewing directions over 4 days

Page 15: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 15

sun

spinaxis

path of spin axis over 4 months

4 rev/day

path of sun over 4 months

45°

Gaia scanning: Motion of the spin axis over 4 months

Page 16: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 16

Equatorial projection

sky average = 80

Number of FoV crossings per star (5 years)

Page 17: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 17

SiC torus (optical bench) finished (Jan 2010)

Credit: Boostec Industries

Page 18: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 18

Data collection and reduction: ESA and DPAC responsibilities

positionsproper motionsparallaxesradial velocitiesmagnitudesvariabilityorbits, massesTeff , log g ……catalogue access

Final resultsData acquisition

Mission operations

centre

Telemetry data

Data processing

Page 19: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 19

Gaia Data Processing and Analysis Consortium (DPAC)

• Individual persons organized in an ad hoc structure

• About 430 members(January 2010)

• 24 funding agencies

• 6 data processing centres

Credit: F. Mignard

Page 20: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 20

How does the Gaia data processing differ from other (ground-based) astrometric projects?

Ole Rømer's meridian circle (ca 1704)

Page 21: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 21

The meridian circle at Lund Observatory(ca 1920)

Page 22: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 22

Linking stars by focal plane measurements

PrecedingField of View

FollowingField of View

Focal planewith CCDs

Page 23: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 23

The observations depend on many thingsbesides the star positions...

projection onto the sky of one of the CCDs throughthe preceding FoV

projection onto the sky of the same CCD through the following FoV

x

y

z

x y z = ScanningReferenceSystem (SRS)

CCD

fiducial observation line

The geometric instrumentcalibration specifies thefiducial line in the SRS

The attitude is the orientation of the SRS in the celestial frame

Page 24: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 24

Principle of self-calibration:An example from computer vision

• Extrinstic camera parameters: (X, Y, Z) → (u, v) [rotation, translation]

• Intrinsic camera parameters: (u, v) → pixels [origin, scales, shear]

u

vc

cameraX

Z

worldcoordinates

pixels

Page 25: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 25

Camera calibration in computer vision (1/2)

• Classical (standard) calibration method:

u

vc

standard pattern camera

Page 26: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 26

Camera calibration in computer vision (2/2)

• Self-calibration method using a moving camera

Maybank & Faugeras (1992) showed that with≥ 7 point correspondencesand ≥ 3 camera positions,the extrinsic and intrinsiccamera parameters canbe recovered up to a scale factor.

Assumptions:● intrinsic param constant● object constant in (X,Y,Z)

Page 27: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 27

Gaia is a self-calibrating instrument

• Pre-launch laboratory calibration is not feasible to required accuracy

• Gaia cannot not rely on any previous astrometric measurements for its calibration

• No special "calibration observations" are made

• Most of the observations contribute to the calibration

• Instrument stability on short time scales is essential

• The data reduction becomes very complicated because the observations are all coupled together in a big system of equations

Page 28: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 28

The global astrometric solution

• The inertial motion of a single star on the celestial sphere is described by 5 astrometric parameters (unknowns):

‒ two for the position at a fixed reference epoch: α0, δ0

‒ two for the angular rates: μα, μδ

‒ one for the parallax: π

• About 200 million stars can be used as ”primary sources” for the global astrometric solution

‒ this gives ~ 1000 million astrometric unknowns‒ they are linked through simultaneous measurements on the CCDs

(α0, δ0)

(μα, μδ)

π

Page 29: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 29

Structure of the observation equations

• Every observation tobs depends on:

‒ the astrometric parameters si of exactly one star (i)

‒ the attitude parameters aj of exactly one time interval ( j)

‒ the calibration parameters ck of exactly one calibration unit (k)

Page 30: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 30

Method of Least Squares (C.F. Gauss, 1795)

• The observation equations form an overdetermined system of equations

where A is an extremely sparse matrix of size ~ 1011 ⋅ 109

• The "best" solution minimizes the sum of the squares of the residuals:

• It can be found by solving the normal equations

Page 31: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 31

Structure of the normal equations matrix

calibration (~106)

Filled

Sparse

Zeroes

s1s2s3 ... a1 a2 a3 ... c

stars attitude calibration

source (109)

attitude (4·107)

N =

one 5⋅ 5 matrix per source

one band matrix per

attitudeinterval

one matrix per cal. unit

the non-zeroelements hereconnect the s, a, c parametersand make thesystem hard to solve

Page 32: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 32

Iterative solution method

calibration(~106)

Filled

Sparse

Zeroes

s1s2s3 ... a1 a2 a3 ... c

stars attitude calibration

source (109)

attitude(4·107)

0 0

0N =

Page 33: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 33

Astrometric Global Iterative Solution (AGIS)

set s, a, c to some initial approximation

iterate until convergence

final s, a and c

one star at a time

one attitude interval at a time

one calibration unit at a time

Source Block

Attitude Block

Calibration Block

Page 34: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 34

40 - 100 iterations needed for full convergence

initial errors

doubleprecisonroundingerrors

x 10-9

Page 35: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 35

Number of floating-point operations (flop)

• The astrometric solution is currently run on a cluster ~1 Tflop/s

• Can handle about 10 million stars (1/20 of final size)

• A solution takes 10 days for convergence

• Final system (2012) ~10 Tflop/s

• Full solution ~20 days

• About 10 such solutions needed (~2 ⋅ 1020 flop)

• Total data reduction effort estimated to 1021 flop

We are helped by Moore's law....

Page 36: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 36

Page 37: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum 37

Number of floating-point operations (flop)

• The astrometric solution is currently run on a cluster ~1 Tflop/s

• Can handle about 10 million stars (1/20 of final size)

• A solution takes 10 days for convergence

• Final system (2012) ~10 Tflop/s

• Full solution ~20 days

• About 10 such solutions needed (~2 ⋅ 1020 flop)

• Total data reduction effort estimated to 1021 flop

This is one of the largest computational challenges in observational astronomy, but the rewards will be immense:

A 3D map of our Galaxy!

Page 38: Gaia - datamuseum.dkdatamuseum.dk/site_dk/20100506/gaia.pdf · Earth-Moon barycentre, a = 1 AU L2, ... Gaia scanning: Motion of the spin axis over 4 months. 2010 May 6 Dansk Datahistorisk

2010 May 6 Dansk Datahistorisk Forening og Kroppedal Museum38

Thank you!