Observational procedures and data reduction Lecture 1: Introduction and observing strategies

Observational procedures and data reduction

Lecture 1: Introduction and

observing strategies

XVII Canary Islands Winter School of Astrophysics: ‘3D Spectroscopy’

Tenerife, Nov-Dec 2005

James E.H. Turner

Gemini Observatory

Introduction & observing strategies

Introduction: James’s lectures

● Four ~1-hr lectures on observing and basic data reduction for IFUs

– Lecture 1 discusses strategies for observing with IFUs

● Optical (visible) and infrared observing techniques

● IFU-specific issues

– Lecture 2 presents some background on image sampling

● How do we reconstruct the spatial information in raw IFU data?

● How do we best preserve the integrity of the data?

– Lectures 3 & 4 cover data reduction and data formats

● Calibrating and formatting data ready for scientific analysis (Pierre);

removing instrumental and atmospheric effects

● Optical vs. infrared and fibres vs. microlenses or image slicers


Introduction: background

● Integral field spectroscopy (IFS) techniques have been in development

for at least a couple of decades (Vanderriest, 1980)

… but it is only during the last few years (<5) that IFUs have become

widely available at major observatories, for everyone to use

(with a few exceptions—eg. the Lyon microlens IFUs at CFHT)

● IFS poses new data reduction and analysis issues

– It introduces 3D datasets to mainstream optical/IR astronomy

– Spatial information is scrambled (and often not on a square grid)

– The software has arguably lagged behind hardware, especially in terms of

general-purpose tools

● Hence the Euro3D effort, started ~2002 (and, eg., recent additions to IRAF)


Introduction: background

● Now we have new instrumentation and software, but not so much

experience with them in the community…

– Astronomers have been using ‘standard’ spectroscopy for centuries!

(and it is technically more straightforward than IFS)

● The current generation of students and postdocs growing up with IFS

will be the ones that spread the expertise within the community


Start with a quick tour of how IFS compares with

other observing modes…


Observing with IFUs vs. other instruments

● Current IFUs are used at optical (visible) or near-infrared wavelengths

– Optical: ~0.4-1m (CCD detectors); NIR: ~1-5m (HgCdTe/InSb arrays)

– In future also mid-IR (JWST MIRI) and far-IR (FIFI-LS on Sofia)

sketch based on

plot from NASA



● IFUs can be dedicated instruments

…or insertable modules inside multi-purpose spectrographs

GNIRS image slicing IFU in the slit slide(Gemini)

SAURON on the WHT(ING newsletter)



● We can also have multiple deployable IFU fields within the telescope

field of view (like MOS fibres)—more instruments like this in future

GIRAFFE multi-IFU design for VLT(Observatoire de Paris?)



● Fields of view are typically small

– A few arcseconds, compared with arcminutes for typical imagers

– Target acquisition is not quite point-and-shoot—but it’s easier than

aligning an object with a narrow slit

● IFUs have a wider aperture and can provide 2D images for alignment

● Some IFUs have larger fields but coarse spatial sampling

and/or short spectra

– eg. SAURON has a mode with a ~0.5’ field and ~1” spatial pixels

● Greater field of view for nearby targets

● Better sensitivity for regions of low surface brightness



● Compared with a slit, IFUs introduce nonuniformities over the field

– Have to flat field both the detector and IFU

– Together with small field sizes, this makes dithering and mosaicing

important—to average over variations and cover a larger area

● High-res IFUs are a good way to use adaptive optics

– Capture more light with an IFU than a very narrow (AO-scale) slit

– Acquisition is easier with an IFU than a very narrow slit

– Projects that benefit from AO often benefit from 2D spatial coverage


More details on optical/NIR

observing with IFUs…


Observing process—typical procedure: optical

Acquire target

onto the IFU

Observe

flat/arc

Observe science

target N

Observe

flat/arc

Offsets onsource?Nod to sky?

Science target

If flexure isimportant(usually is)


Observing process—typical procedure: near-IR

Acquire target

onto the IFUObserve flats

Observe science

target

Observe blank

sky

Observe flats

Observe telluric

std at position A

Acquire telluric

std onto the IFU

Observe telluric

std at position B

N

N ~1-4

Observe flats?

Science target

(with flexure)

Standard – before or after

Offsets onsource?


Observing process—separate calibrations

Twilight flats

Biases

(optical)

Darks

Flux standard

(mainly optical)Daytime

flats/arcs?

Day time or twilight

(dark dome)Twilight Night time

Special

calibrations?


Observing process—acquisition (single IFU)

● Want to centre the science target at a suitable place in the IFU field

– Usually the middle!

– Blind telescope pointing rarely gets the target right in the centre of a

small IFU field without tweaking

– Take a short exposure, measure the target position and move the

telescope (or IFU) to adjust it

● Two approaches:

● Use a normal imaging camera that is fixed with respect to the IFU

(maybe another mode of the same instrument)

● Centre the target at co-ordinates known to correspond to the IFU centre

● Assumes the position of the IFU centre on the camera is repeatable (not too

much flexure etc.) if this is the only acquisition step


Observing process—acquisition

Direct imaging acquisition onto known ‘hot spot’



● Reconstruct a 2D image of what the IFU is looking at

● Where possible, taking undispersed images through the IFU (or using a

single emission line) gives the best sensitivity

– Rearrange a 1D slit or set of micropupil spots to a 2D image

● Otherwise (maybe to avoid saturation), sum in wavelength or take a spatial

cross-section and then rearrange to 2D

Reconstructed acquisition image

through the IFU



● Alternatively, create a 3D datacube and collapse it in wavelength to make an

acquisition image (if it doesn’t take too long!)

● …or use a direct image to get the target onto the field and then fine-

tune the pointing using the IFU

● For large-pixel IFUs, it may be important that repeated acquisitions

(eg. on different nights) are done identically, to allow combining the

data optimally

● Otherwise, there is less of a need for periodic re-acquisition than for a

long slit, where the target can drift slowly out of the aperture


Observing process—object and sky spectra

● Once the object is in the right place, we want to keep taking spectra

until we get enough signal-to-noise

● Normally need to observe blank sky as a reference for subtracting out

telluric (sky) emission lines and any other background counts

– For IFUs it is more likely that we have to nod away from the target

● Optical wavelengths

– Can sample the sky at the same time as the target using several methods

● Use blank sky from the edges of the IFU field, if the target is small enough

● Use an IFU with a separate sky field or sky fibres, placed far enough away

from the science field (eg. a few arcminutes)

● Nod up and down the field (and later subtract pairs of frames), if the target

is compact enough—or dither around to remove objects with rejection



Object

fibres

Sky

fibres


– Otherwise, if the target is too extended, we have to nod off to blank sky

from time to time and spend <100% of the time observing the target

● Infrared wavelengths

– Standard practice is to nod to sky, usually every other exposure, so we

can remove both telluric/thermal emission and detector dark current by

subtracting pairs of raw exposures

– Have to nod more frequently than in the optical, since sky lines are

stronger and vary on timescales of a few minutes

– For point-like targets, nod within the IFU field to get 2 the flux

– At non-thermal wavelengths (~1m) and high enough spectral resolution,

an alternative for some projects is to spend 100% of the time on source

and interpolate over sky lines after dark subtraction







Observing process—integration times

● Exposure times are mainly determined by the same factors as for other

spectroscopic modes

● Minimum exposure

– For faint targets, need to integrate long enough for the background noise

to overcome the detector read-out noise

– It often takes longer to get the same counts per pixel as with a slit:

● High spatial/spectral resolution IFUs have smaller apertures than a typical

slit (since they can have without losing light overall)

● IFUs introduce extra optics (=losses) in the telescope beam

● Sometimes there is extra magnification (=more pixels) involved

– Frequently the main constraint



● Maximum exposure

– In the infrared, we have to start a new exposure often enough to sample

variations in sky lines, for accurate subtraction

● For bright targets, take short exposures (eg. 0.5-2 minutes) to sample fast

emission-line variations

● For faint targets, take longer exposures (10-30 minutes) to average over the

fast sky variations

– Must avoid saturating the detector capacity with too many photons, for

bright targets (or perhaps bright sky lines)



– We may want to divide a fixed observing time into multiple exposures

for various reasons, such as:

● To allow for changes in flexure between the slit and detector or the

telescope image and the IFU

● Using repeated samples to help remove cosmic rays

● Dithering on the sky

● Avoiding too much time loss if something goes wrong with an integration

● Typical exposures are from a few minutes up to 1 hour (optical) or

~20 minutes (infrared)


Observing process—dithering & mosaicing

● Three reasons for moving where an IFU is pointing on the sky

between exposures (other than sky subtraction):

● ‘Dithering’: because IFUs use reflective or transmissive optics, rather

than just a clear slit, they tend to introduce artificial spatial structure

● Flat-field variations, including dead elements such as broken optical fibres

● Possible variations in spectral line profiles between IFU elements

● Although flat-fielding removes systematic throughput differences, the

resulting noise variations and ‘holes’ due to dead elements remain

– Dithering the IFU position with respect to the target object helps to

produce a homogeneous dataset and ‘fill in’ any missing spectra

– Use small offsets, eg. 1-2 IFU elements (fibres, slices or lenslets)

– For short exposures, using integer-element offsets may make the data a

bit easier to combine (just co-add corresponding fibres/slices/lenses)



● ‘Mosaicing’: since IFU fields are often just a few arcseconds in size,

sometimes we need to observe multiple pointings in order to cover a

large enough area of the target

● Use offsets comparable to the size of the IFU field, for small overlaps

● ‘Subsampling’: IFUs with larger fields tend to have coarse spatial pixels

that can’t capture all the detail in telescope images

● Try offseting by a fraction of a spatial pixel (fibre, lens, pixel) between

frames to get better sampling (like for HST WFPC+Drizzle)

– eg. steps of 1/2 or 1/3 (smaller increments don’t necessarily gain much)

● Unlike HST, ground-based observatories have variable seeing and cloud

– This may limit the ability to combine data accurately enough

– Subsampling is not yet well tested for IFUs, but I’m told it has been

used successfully for ground-based imaging



● Observing strategy

– Change the telescope pointing slightly between frames, or offset the IFU

within the telescope field (if it is movable)

– In all 3 cases, we need a way to register the relative positions accurately,

so we can combine the data with the right shifts

● For dithering and small mosaics, keeping the centre of the target (eg. galaxy

nucleus) inside the field of view at every pointing gives a reliable reference

– Allows up to 4x the field of view

● Without at least one reference peak in the field at every position, we need to

have well-known pointing offsets, ie. accurate guider (etc.) movements

● For subsampling, telescope offsets must be accurate to a small fraction of a

spatial pixel …unless there are enough peaks in the field to measure a

statistically accurate offset from several approximate centroids



– In the infrared, dithering & mosaicing may allow spending a larger

fraction of time on source than for a single object pointing

● Typical single-pointing sequence: sky-object-object-sky … (N)

– For faint targets, we achieve best S/N by spending equal time on sky

and object so that the background noise is equal in both cases

● ‘Short cut’ for dithering: sky-object-object … (N)

– Because we have to shift and add the pointings, we can subtract the

same sky from 2 object frames but still have 2 independent sky

measurements at any given position

– In practice, the most conservative schemes give the most accurate sky

subtraction (sky-object-object-sky … or sky-object…)


Observing process—flat fielding

● Need to measure instrumental efficiency (flat-field) variations, so we

can separate them out from real features in the data

– Across the detector: pixel-to-pixel variations & other features

– Across the IFU field: differences in transmission between different

fibres, lenslets or image slices (and possibly along image slices)

● Eg. due to fibre stresses/FRD, alignment variations, optical bonding, slicer

reflectivity differences, diffraction losses etc.

● Detector flat exposures

– Need a dispersed illumination source that is spectrally smooth

● Dispersed to allow for variations in detector response with wavelength etc.

● Smooth so we can fit and remove the spectral profile of the lamp, leaving

just the intrinsic pixel variations



– Detector flat may also include fringing

● IFU flat exposures

– Need an illumination source that is spatially flat

● The flattest reference is the twilight sky—but there are only a few minutes

twice a day to observe this at the right brightness level

– For IFUs with small spatial pixels and/or high spectral dispersion, it is

sometimes necessary to take sky flats when the sun is up!

● ‘Dome flats’, taken by illuminating a blank spot inside the dome with

appropriate lamps, can be relatively flat

● Given the small sizes of many IFU fields, the calibration source used for

detector flats may be flat enough

– Matching the spatial slit-detector flexure of science exposures is more

important than for a long slit, since the apertures are much shorter




– In the absence of flexure, or if flats are taken frequently enough, we may

choose to use a combined detector+IFU flat

● Can be taken before or after night-time observations if there is no flexure

– Where there is flexure (more common), we probably want to take flats at

the same telescope pointing as the science data in order to:

● illuminate the same detector pixels in the same way

● help determine the locations of IFU elements on the detector

– If some optical element (eg. the disperser tilt) moves non-repeatably. we

may need to take flats/arcs before changing instrument configuration

(eg. to a different wavelength setting)



Lamp flat

Twilight flat



Fibre flat field variations


Observing process—wavelength calibration

● Want to know the wavelength accurately at each detector pixel

– Measure (and interpolate between) the positions of well-known spectral

lines in a reference spectrum

● Wavelength references

– Arc lamp spectrum (eg. CuAr, ThAr, Ar, Xe, Kr)

– Sky emission lines, in the redinfrared

– Sky absorption lines, primarily in the infrared


– Normally get detailed wavelength variation (including nonlinear terms)

from an arc lamp exposure



– Can correct small zero-point shifts due to flexure using sky lines

● Observe an arc during the day (or twilight) and shift the zero-point to match

each science exposure

– If there is flexure and no sky lines are available (eg. at high dispersion in

the blue), we have to observe arc spectra in between science exposures

● Frequently enough that the telescope pointing doesn’t change much

● Before changing the instrument configuration (eg. grating tilt)

– Need to calibrate wavelength as a function of pixel index separately for

each 1D fibre or point in a 2D spectrum



Optical fibre arc


Observing process—telluric calibration

● In the infrared (and far red), telluric absorption lines are important

– The I/z/J/H/K/L/M bandpasses are defined in spectral regions with

reasonable atmospheric transmission, but there are still many minor

absorption features within the bands, eg. due to water vapour

● Occasionally we may even want to work in between clean bands, eg. to

measure a strong emission line that is redshifted from the visible

– The amount of absorption scales with airmass and varies with time

– For most purposes, telluric absorption in the science data is bad news

● Confuse telluric lines with stellar features—especially when using an

automatic algorithm to measure velocities, for example

● Telluric lines can overlap real spectral features, changing their profiles so

we measure spurious line widths, centres, strengths etc.



G star, with telluric features




– To calibrate telluric features, observe a star of known spectral type, with

little or no intrinsic absorption at wavelengths of interest (eg. A type)

● Immediately before or after the corresponding science observation

● At an RA & Dec chosen to match the airmass (ie. elevation / zenith

distance) of the science target

– Match the average airmass during the science observation, where the

range of variation is relatively small (eg. <0.3 airmasses).

– For longer observations, bracket the range of airmass of the science

observation with telluric standards before and after

– If the instrument’s spectral profile varies over the IFU field, we might

dither the star around the IFU to get light through different elements

● For slices, profiles could also vary between point-like and diffuse sources


Observing process—flux calibration

● In order to compare fluxes meaningfully, we have to account for:

● Instrumental efficiency variation as a function of wavelength

● Spectral equivalent of flat-fielding the IFU spatially

● Eg. if we want to measure the true continuum slope of the target or take line

ratios from different ends of the spectrum

● The total throughput / sensitivity of the instrument + telescope + sky

● If we want to determine the absolute brightness of the source or a particular

spectral feature (and the observing conditions permit this)


– Derive an instrumental sensitivity spectrum by observing a standard star

with well-known intrinsic brightness as a function of wavelength





– In the visible, there are numerous spectrophotometric standards with

brightness already tabulated as a function of wavelength (eg. Oke, 1990)

– In the IR, we often observe a star with just a well-known broad-band

magnitude and spectral type

● Model the intrinsic continuum using a black-body curve for the appropriate

temperature and magnitude and compare with the real data

– Standards can be observed occasionally during a given observing run

– If we’re only interested in correcting the relative throughput variation

with wavelength then it’s OK to observe through cloud (which is grey)

– Line strengths (equivalent widths) can still be measured relative to the

continuum without performing absolute calibration



● IFU vs. long slit

– For slit spectroscopy, if we need absolute flux calibration we have to use

a special wide slit in order to capture all the light from the standard star

● Wider slit = lower spectral resolution than for the science data

● IFUs can capture all the light without affecting the spectral resolution

– Can possibly use a single standard observation for both telluric

calibration and absolute flux calibration

– For narrow slits, atmospheric dispersion causes colour-dependent

throughput losses unless observing at the parallactic angle

● Relative flux calibration is also easier and more accurate with an IFU


Observing process—detector bias

● Need to determine the zero-point readout level of each detector pixel,

so we can measure the accumulated counts above that level

– For CCD detectors, take a few very short exposures in the dark

● Gives the value in each pixel when no electrons are stored

● Average together several such exposures to overcome read-out noise

– Infrared arrays are normally read out by subtracting the difference in

counts between the start and end of each exposure

● The bias level is removed automatically, so there is no need to measure it

separately

– Why the difference?

● IR arrays can be read out quickly without affecting the stored charge,

whereas reading out a CCD involves shuffling the charge off the detector


Observing process—detector bias

● Observing strategy (CCDs)

– The bias level is typically stable enough to take occasional reference

exposures during the daytime

– Sometimes an overscan region is created for each exposure by continuing

to read out the detector after shuffling out all the accumulated charge

● Allows an overall zero-point correction to be made if necessary, on top of

the pixel-to-pixel differences from the bias exposures

Biases are the dullest thing

you will get to observe!


Observing process—dark current

● During an exposure, detector pixels accumulate some electrons due to

thermal excitation & array defects, as well as from incident photons

– The detector is cooled to minimize thermal current, but too much cooling

would cause the quantum efficiency to drop

● For modern CCDs, the dark current may be low enough not to matter

(eg. 1e- / hr)

● For infrared arrays, the dark current is higher and tends to vary

strongly between pixels

– Hot pixels obscure features in the raw images

– Need a reference to separate dark current from counts due to photons

from the target source



High dark-current pixels

in a raw NIR spectrum



● Observing strategy: for science data

– For a given exposure time, the dark current can be determined by

exposing the detector in complete darkness for the same length of time

● Average several dark exposures to account for read noise and cosmic rays

– When taking separate sky exposures, the same dark current is present in

both object and sky frames (assuming the exposures are equal)

● In the infrared, subtracting object-sky pairs removes dark current

automatically, without the need for special calibrations

● Usually many of the hot pixels subtract out well, leaving just a statistical

increase in noise (the remainder have to be masked out during reduction)

– Separate darks are needed if simple pixel-for-pixel sky subtraction is not

used (eg. if scaling sky frames or not nodding to sky)



● Observing strategy: for calibrations

– For flat-field observations, equal dark (or ‘lamps off’) exposures must be

taken if the flats are long enough to have significant dark current

– For arc lamp exposures, darks are needed if uncorrected hot pixels appear

as spikes that could be confused with real emission lines

● In short:

– In the IR, one typically takes darks for flat fields; for on-sky exposures

they are optional, depending on the reduction method

● It’s time-consuming to take darks for science exposures of ≥10 min!

– In the optical darks aren’t always needed with modern detectors

● …but for a CCD with high dark current, it is essential to take darks

matching the science exposures (unless nodding off to blank sky)


Observing process—disclaimer

● Small details vary a lot between different instruments

– Especially when it comes to configuring the instrument, repeatibility etc.

– Get advice from your observatory contact scientist!!

– If your projects are queue scheduled, visit the telescope for a month

anyway and help out… you’ll understand your data better!


Observing process—disclaimer

● Small details vary a lot between different instruments

– Especially when it comes to configuring the instrument, repeatibility etc.

– Get advice from your observatory contact scientist!!

– If your projects are queue scheduled, visit the telescope for a month

anyway and help out… you’ll understand your data better!

● (Observatories like cheap labour)


Summary

● On the whole, observing with a single IFU isn’t too different from

standard slit spectroscopy

– As usual, we have different techniques in the optical and near-IR

● A number of details are different from other spectroscopic modes

(some are more like imaging, since IFUs do that too):

– Target acquisition

● 2D image reconstruction through the IFU

– Data inspection

● Have to learn how to read the scrambled images and/or use special software

– Flat fielding requirements

● Spatial structure due to the IFU as well as the detector

● More important to match the flexure of science data


Summary

– Sky subtraction strategies

● Optical: eg. use separate sky fibres instead of the ends of a slit

● NIR: eg. nod off to sky instead of up and down a slit

– Spatial dithering and mosaicing

● Large and small offsets in both dimensions

– Sensitivity

● Difficult to go faint with a high-res IFU (want a 30m/100m telescope!)

● Can do better with large-pixel IFUs

● Next lecture: Sampling images


Summary

– Sky subtraction strategies

● Optical: eg. use separate sky fibres instead of the ends of a slit

● NIR: eg. nod off to sky instead of up and down a slit

– Spatial dithering and mosaicing

● Large and small offsets in both dimensions

– Sensitivity

● Difficult to go faint with a high-res IFU (want a 30m/100m telescope!)

● Can do better with large-pixel IFUs

● Next lecture: Sampling images

Complete with Dirac

delta functions!

Documents

Observational procedures and data reduction Lecture 1: Introduction and observing strategies