Beating Noise Observational Techniques ASTR 3010 Lecture 11 Textbook

Beating NoiseObservational Techniques

ASTR 3010

Lecture 11

Textbook

From previous classes

• We learned that in the astronomical data pre-processing,

bias subtraction, dark subtraction, flat-fielding, bad pixel correction, cosmic ray hits removal

are important.

• There are several observation techniques that can take care of the above correction nearly automatically.

Subtraction of two images taken with a small offset

• image1 – image2

• bad pixels, bias, and dark are removed!• But, cosmic rays remain. • Need to apply flat-fielding

- =

?Dither Observation?• Dither Observation

Series of images taken with telescope offsets.

box5

box4

Dithering Observation

dead pixels

hot pixels

cosmic rays

Observation No 1

Detector Array

Dithering Observation

dead pixels

hot pixels

cosmic rays

Observation No 2

Dithering Observation

dead pixels

hot pixels

cosmic rays

Observation No 3

Dithering Observation

dead pixels

hot pixels

cosmic rays

Observation No 4

Dithering Observation

dead pixels

hot pixels

cosmic rays

Observation No 4

Dithering Observation

• Five observations

obs1 … obs5

flux1

flux2

flux3

flux4

flux5

True Flux

Dithering Observation

• Five observations

This will take care of bias, dark, cosmic ray hits, bad pixels!

obs1 … obs5

flux4 < flux5 < flux2 < flux1 < flux3

Ordered list

median value

Dozens of Dithered images with small offsets

• SuperResolution

With the P&A telescope

Not Yet!

P&A Dithering by Hand

Real example : Keck NIRC2 AO observation of HIP88945

Shift and Add

look at those hot pixels!!

# demonstrating the Shift and Add concept.## import librariesfrom ds9 import *from numpy import *import pyfits

im1=pyfits.getdata('n0002.fits',ignore_missing_end=True)im2=pyfits.getdata('n0003.fits',ignore_missing_end=True)im3=pyfits.getdata('n0004.fits',ignore_missing_end=True)im4=pyfits.getdata('n0005.fits',ignore_missing_end=True)im5=pyfits.getdata('n0006.fits',ignore_missing_end=True)

# create a big enough array to cover any shifted images...BIG = zeros((1201,1201))# Then, we will find the positions of the star in each image.# Shift the image in the BIG array such that the star appears# at the center of the array (601,601)posx_im1=276posy_im1=279BIG[601-posx_im1:601-posx_im1+512,601-posy_im1:601-posy_im1+512] += im1

posx_im1=397posy_im1=162BIG[601-posx_im1:601-posx_im1+512,601-posy_im1:601-posy_im1+512] += im2

posx_im1=155posy_im1=403BIG[601-posx_im1:601-posx_im1+512,601-posy_im1:601-posy_im1+512] += im3

posx_im1=397posy_im1=401BIG[601-posx_im1:601-posx_im1+512,601-posy_im1:601-posy_im1+512] += im4

posx_im1=154posy_im1=161BIG[601-posx_im1:601-posx_im1+512,601-posy_im1:601-posy_im1+512] += im5

d=ds9()d.set_np2arr( transpose(BIG) )

Shift and Median is better!!

Python HW2:

Create a Python script doing this!

Median sky subtraction

• When your object is extended… or you need to make a pretty picture…

• in near-IR: useful for 10-20 min depends on the quality of the night.

In Astronomical observations…

• Three major noise sources…o Poisson noise (aka “shot noise”)o Sky noiseo Electronic noise

• Sky noise limited observation• Dark current limited observation• Readout noise limited observation(?)

Typical stellar sources in the Solar Neighborhood

Noise versus signal at various wavelengths

Optical near IR mid IR

101

109

Sky noise

Detector noise

Sign

al S

tren

gth

Extremely bright objects

High red shift galaxies

105

Coadds

• Because of the increased sky background, frames will get saturated in short exposures.

variable sky is the limiting factor sky limited observations.

• Take as many unsaturated frames as necessary and combined them later.too many files for a single object (several hundreds)on-chip combine of multiple exposures = “coadd”

Near-IR observation Tips

• Need to sample the skyo through ditheringo or from a dedicated sky observation in the nearby field needs to be dithered

also. why?• Choose a right number of coadds. o Choose the maximum frame exposure time which will stay in the linear regime.o Then, from the required total exposure time, calculate the necessary number of

coadds.• Sky level is varying faster at longer wavelengths!o at K-band (2.2 μm) : up to 20 minuteso at L-band (3.6 μm) : max is about a minuteo at mid-IR (10 μm) : fraction of a second different strategy

• Dither• Repeat the dither sequence as many times as necessary.

Observation Planning : near-IR

• You will be observing at the K-band (2.2 micron), and you expect that your source would have a count rate of about 1,000 counts per second where this signal will be spread over ~10 pixels due to the PSF structure. Some other relevant data areo Detector linear regime: count < 10,000 countso Sky brightness at K = 100 counts per second per pixelo Readout noise = 10 counts per readouto Dark current = 25 counts per second per pixelWhat would be the best observing strategy for this object if you want your final S/N~100?

Observation Planning : near-IR

• Source ~ 2,000 count per second over 4 pixels. Goal S/N >= 100o Detector linear regime: count < 8,000 countso Sky brightness at K = 1,000 counts per second per pixelo Readout noise = 10 counts per readouto Dark current = 100 counts per second per pixel

Per pixel per second signal = 500 source count + 1,000 sky count + 100 dark current = 1,600 counts/pxl/sec In 5 sec exposure, the source will get into a non-linear regime max exposure time should be less than 5sec or we should “coadd”

In a single 5 second exposure : S/N = 500 / sqrt(1600)~12 if we coadd 16 frames, then a single exposure will be 40 second long and S/N=12*sqrt(16)=48

For a box5 dither observation pattern, we will get five images total readout noise = 10 * sqrt(5) = 22.4 counts. This is negligible to shot noises from source+sky+detector.

If we take one image (40 sec long with 8 coadds) using a box5 pattern final S/N=48 * sqrt(5) = 107

The total duration of the observation is 5 x 40sec + total overhead (~1 min) = 4 min. At K-band, the sky will be invariable for up to ~20 min. So, this will be OK.

Nod & Shuffle

• "Nod and Shuffle" is a technique used to obtain very faint spectra• the natural glow of the nighttime sky would overwhelm the extremely faint

objects (e.g., far, far away galaxies)• N&S allows astronomers to effectively subtract away the bright spectral

emission lines and fainter continuum of our atmosphere's nighttime glow while retaining the faint spectra of dim, red galaxies. This also significantly reduces readout noise.

1. obtaining a spectrum of an object2. moving the electrical charge built up by the image of the spectrum on the

CCD to another location "buffer storage" on the CCD (Shuffling)3. shifting the position of the telescope slightly (Nodding) so that the spectrum

of the target object shifts to a different part of the CCD4. obtaining another spectrum in the nodded position5. shuffling the charges moved in step 1 back to the original position (while

moving the charges from step 4 to the buffer storage)6. nodding the telescope back to the position in step 17. repeating until enough light is collected.

Sky cancellation: ‘nod and shuffle’Storage of ‘sky’ image next to object image via ‘charge shuffling’Zero extra noise introduced, rapid switching (60s)

A

B

A-B

Typically A=60s/15 cy: 1800s exposure10-3 subtraction

Another example

Weird image artifacts

Latent images : badly saturated pixels do not come back to normal right away!

Weird image artifacts

ghost images : multiple reflections within an optical element (e.g., filter, beam splitter, entrance pupil) seen near a very bright star

Handling various problems at IRNon-uniform QE Flat-field

High background level• Point sources• Extended sources

Dithering and coadds• Small dither offset• Off-chip dither

Non-linearity of detectors Stay within linear range

Memory effects (latent) Do not saturate

Bad pixels Mask or median out from dithers

Cosmic rays Shift and add

Dark current Subtracts out with sky

Hot rows, hot pixels, amplifier glow

Subtracts out well

In summary…

Important Concepts• Various noise sources at different

wavelengths and their relation to the observing techniques.

• Median combining of images

Important Terms• Dither observation• Shift and add• Shift and median• Coadd• Nod-and-shuffle

Chapter/sections covered in this lecture : 9