Data Reduction & Photometric Calibration-

IPM School and Workshop on Weak Lensing andPhoto-z Techniques

Hendrik Hildebrandt, Leiden Observatory

April 22, 2009

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Outline

1 Introduction

2 Data Reduction

3 Photometric Calibration

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Who am I?

Bonn: That’s where I studiedand finished my PhD in August2007.

Leiden: That’s where I’m sinceas a postdoctoral fellow in theDUEL network.

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Overview of this lectureBasics of data reduction

How do raw data look like?Aims of data reductionTHELI pipeline

Photometric calibrationMagnitude systemsFiltersStandard starsHow do I calibrate data?

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Layout of WFI@MPG/ESO2.2m

2046 pix

E

4098

pix

1 2 3 4

8 7 6 5

N

23"

1 pix = 0.238"

Gui

ding

CC

D

14"

127 mm = 34’

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Rawdata from WFI@MPG/ESO2.2m

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Aims of data Reduction

100×

⇒

1×

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Multicolour Mosaic of the CDFS

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Aims of data Reduction

In words...Remove the instrumental signature.Mask defects.Calibrate the image

astrometrically (rel. & abs.)photometrically (rel. & abs.)(others, e.g. wavelengthcalibration)

Produce a stack.

Caution!The term “reduction” is somewhatmisleading. Don’t expect your HDDspace to free up.

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Pipelines

Steps in Data ReductionOverscan/Bias correctionFlatfield correctionSuper-FlatfieldingFringe removalCreation of weight imagesAstrometric calibrationSky subtractionCoaddition

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Pipelines

SoftwarePipelines are software packages thattake you from the raw science- andcalibration-data to the desiredend-product.

irafMIDASESO pipelinesTHELI (GaBoDS):

based on existing software(TERAPIX, imcat, eclipse,...)open sourceinstrument-independenteasily learn- and adaptablefast (multi-CPU capable)

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Overscan/Bias correction

What it is and how to correct for:The bias is a small negativepotential in the CCD.Not necessarily totally constant.Overscan regions arenon-illuminated parts at the chipboundary.Subtracting the bias corrects for theuniform negative level.Subtracting the overscan can helpto remove horizontal patterns.After overscan correction theoverscan regions should be cutaway.

Master-BIAS frameCombination of 0s exp.

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

DARK frames

All modern astronomical CCD camerasare cooled with liquid N2 ⇒ Negligibledark current.

Reasons for still taking DARK frames:

Some CCD defects only appearafter some time.Thus, DARK frames are the bestpossible base for efficient weightimages.DARK frames should be exposedapproximately as long as thescience exposures.Can be done in the day.

Master-SKYFLAT frameCombination of skyflats

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Flatfield correction

Flatfielding - Why and how to correct for:

CCDs are not homogeneouslysensitive.CCDs are not illuminatedhomogeneously.Take an image of a uniformlyilluminated object (twilight sky orilluminated dome)⇒ flatfieldexposure.Divide by a normalised flatfieldexposure to take out sensitivityvariations.

Master-SKYFLAT frameCombination of skyflats

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Super-Flatfielding

If flatfielding doesn’t work (e.g. large-scale gradients):

Estimate the flatfield from your (or someone else’s)science exposures themselves.Only possible with many dithered exposures on emptyfields.Detect all objects.Mask these objects.Stack the masked exposures.Smooth the stack.Divide by this smoothed Super-Flatfield.

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Fringe removal

Reasons for fringes and removal strategy:

Fringes in the red bands are caused by interference ofincident light-rays and reflected ones (thin-layer effect).Fringe pattern is contained in the super-flatfields.Substract a heavily smoothed version of the super-flatfrom the unsmoothed version to extract the fringepattern.Subtract this pattern from your science data afterre-scaling it to the sky-background level of the scienceexposure.

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Fringe removal - before and after

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Cosmics

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Reflections and Satellite Tracks

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Creation of Weight Images

Why weights?

Image defects should be masked before coaddition.Keep track of the noise properties.Create not only a stacked science image in the end butalso a stacked weight.

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Object Detection

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Astrometric Calibration

174.8 174.6 174.4 174.2

-11.8

-11.6

-11.4

Ra

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Astrometric calibration

Astrometric Packages

LDACoriginal THELI packagefast, but unstable for large numbers of chipsonly used for websites now

ASTROMETRIXrobust and accurateslow!

SCAMProbust and accuratevery fast!photometric calibration includedeasy to use

Astrometric Standard Star Catalogues

USNO-A2, USNO-B1, GSC-1/2, UCAC-2, SDSS, 2MASS...Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Sky subtraction

How is it done?Objects are detected in the single exposures.Object pixels are replaced by the mode of the image.An analytic function is fitted on a grid to this objectsubtracted image.This sky-background model is then substracted.

Caution:If the sky-background varies appreciably over a chip (e.g. forMEGAPRIME@CFHT) this approach might fail.

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Coaddition

Transformation

Original Coarse

Pixel GridOutput Fine

Pixel Grid

Geometric

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Magnitudes

Historical accident:

m1 −m2 = −2.5 log(

f1f2

)Traditionally Vega is used as a Zeropoint, i.e. mVega∗ = 0 in allbands:

m = −2.5 log(

f1fVega∗

)In recent years more and more people use AB magnitudes.Zeropoint is a source with fν = const.

mAB,λ = mVega,λ + ABcorr.,λ

andmAB,V = mVega,V

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Filter sets

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Filter setsSDSS

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Photometric Standard star catalogues

Available catalogues

Landolt (1973, 1983, 1992, 2007, 2009)based on photo-electric observationsUBVRI photometry722 equatorial stars

Stetson (2000)based on CCD observationsBVRI photometry>∼ 15 000 equatorial starsfainter than Landolt

SDSS-DR7 (2008)based on CCD observationsugriz photometrymillions of Northern hemisphere stars

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Conversion to Instrumental Magnitudes

If you do not have exactly the same filters as the ones used for thestandard star catalogue, you have to convert your instrumentalmagnitudes to the system of the standard star catalogue:

mStandard −minst. = ZP + K× sec z + CT× colour

ZP: instrumental zeropointK: extinction coefficientsec z: airmass (z is the zenith angle)CT: colour term

ZP, K, and CT are determined from standard star observationsthrough your filters over a range of airmasses and a range ofstandard stars with varying colour.

Caution: The linear relation above might break down for verydifferent filters.

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Conversion from Instrumental Magnitudes into theStandard system

Sometimes it is necessary to transform your observed instrumentalmagnitudes into the standard system. Then you need at leastobservations in two filters. This again assumes a linear relation andmight be highly inaccurate.

For many purposes you can entirely work in the instrumental systemand only use the standard star catalogue to get you ZP right. Afterabsolute calibration of your ZP you can then forget about CT and K.

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Photometric Calibration - Overview

Steps in photometric calibrationSource ExtractionMatching to a standard starcatalogueFit for zeropoint, extinctioncoeff., and colour termQuality control

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Source Extraction

SExtractorTERAPIX software to detect sources andextract their propertiesSources are defined as a number ofcontiguous pixels above a user-specifiedthreshold w.r.t. the sky-level.Astrometry and photometry is performedon these sources.Overlapping sources can be de-blended.SExtractor can use weight images.Dual-image mode for multi-colourphotometry.2510 citations!

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

SExtractor with a filter file

sex image.fits -c config_file

sex image.fits-CATALOG_TYPE FITS_LDAC \-CATALOG_NAME image.cat \-PIXEL_SCALE 0.238 \-FLAG_IMAGE image.flag.fits\-FLAG_TYPE MAX\-WEIGHT_IMAGE image.weight.fits\-WEIGHT_TYPE MAP_WEIGHT

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Star Selection

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Association

Object Overlapof associationsDirected graph

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden

Introduction Data Reduction Photometric Calibration Hendrik Hildebrandt, Leiden