Multi-Beaming And Calibration - ASTRON · O. Smirnov - AvA 2013 3 Dr. Sidelobes, Or How I Learned To Stop Worrying & Love Simulations O. Smirnov (Rhodes University & SKA SA)

Multi-BeamingAnd

Calibration

O. Smirnov - AvA 2013 2

Multi-BeamingAnd

Calibration


Dr. Sidelobes, Or How I Learned To Stop Worrying & Love Simulations

O. Smirnov (Rhodes University & SKA SA)


Introduction

“A high quality radio map is a lot like a sausage, you might be curious about how it was made, but trust me you really don't want to know.”

– Jack Hickish, Oxford

Interferometry is hard and unintuitive, but we've carved out some comfort zones

New technologies, new observatories and new surveys will be [are] boldly going where no man has gone before

A lot of this thanks to Arnold Result: we live in interesting times


Radio Interferometer...

What lay people think I do What funding agenciesthink I do

What cosmologists & astrophysicists think I do What my engineers think I do What I actually do

(In celebration of the passing of an extremely lame but blissfully short-lived internet meme)


The Four Horsemenof interferometer limitations

Thermal noise lucky if we can

reach it Classical confusion

(i.e. depth limited by resolution) Residual calibration and deconvolution

artefacts (& calibration “noise”) Dr. Sidelobes

thistalk

Wel

l-un

ders

too

d (?

)


Dr Sidelobes: Missing Information

Response to a point source: Point Spread Function (PSF)

PSF = FT(uv-coverage)

Observed “dirty image” is convolved with the PSF

Structure in the PSF = uncertainty in the flux distribution (corresponding to missing data in the uv-plane)

24

PSF of WSRT. The regular rings are due to the regular spacing of its antennas in theEast-West direction.

PSF of MeerKAT

PSF sidelobesextend over the

entire sky


Why Is This Bad?

A whole continuum of skies fits the dirty image (pick any value for the missing uv-components)

Deconvolution picks one = interpolates the missing info from extra assumptions (e.g.: “sources are point-like”).

Real-life WSRT dirty image

Dirty image dominated by PSF sidelobes from the stronger sources

Deconvolution required to get at the faint stuff underneath.


Good News First

3C147 @21cm12h WSRT synthesis160 MHz bandwidth

22 Jy peak (3C147)13.5 μJy noise1,600,000:1 DRthermal noise-limited

Regular calibration does not reach the noise, leaves off-axis artefacts due to direction-dependenteffects (left inset)

Addressed via differential gains (right inset)

3C147 22Jy

30 mJy


JVLA Can Do It Too

Recent result from 3GC3 workshop

3C147

JVLA-D @1.4 GHz

Best image afterregular selfcal


JVLA Can Do It Too II

Recent result from 3GC3 workshop

3C147

JVLA-D @1.4 GHz

Best image afterregular selfcal

...and direction-dependent (DD)calibration on a fewsources


When Primary Beams Go Bad...(Courtesy of Ian Heywood)

EVLA 8 GHz: Looking for sub-mm galaxies and QSOs in the William Herschel Deep Field.

Dominant effect: bright calibrator source rotating through first sidelobe of the primary beam.

(This also has a horrible PSF, being an equatorial field.)This is your

phase calibrator

This is your science(good luck!)

Brightness scale 0~50μJy


Keep Your Friends Close,and your calibrators as far away as you can...

An approximation of the primary beam response, overlaid on top of the image.

As the sky rotates, the sidelobes of the PB sweep over the source, thus making it effectively time-variable.

This is your phase calibrator

This is your science(good luck!)

(Brightness scale 0~50μJy)


Deconvolution Doesn't Help...

Residual image, after deconvolution.

The contaminating source cannot be deconvolved away properly, due to its instrumental time-variability.

...5 years ago this would observation would probably be a complete write-off.

(Brightness scale 0~50μJy)


Same Problem Here

The artefacts in this image have the same underlying cause.

But here, the dominant source is at the centre (where PB variation is minimal) and the “offending” sources are relatively faint.

But we did address them via differential gains...


Differential Gains To The Rescue

Residual image after applying differential gain solutions to the contaminating source



Multi-Band Image

Multi-band residual image:noise-limited, no trace of contaminating source.


Phase calibrator used to be here


Flush With Success

3C147, WHDF, LOFAR maps: some real successes

Thermal noise-limited maps are being produced

Though nowhere near routinely... T&Cs apply: extended

sources are still notoriouslyhard to deconvolve

….though new algorithms are emerging

Is this the light at the end of thetunnel?


2004: The Ghosts Of Cyg AWSRT 92cm observation of J1819+3845 by Ger de Bruyn

String of ghosts connecting brightest source to Cyg A(20° away!)

“Skimming pebbles in a pond”

Positions correspond to rational fractions(1/2, 1/3, 2/3, 2/5, etc...)

Wasn't clear if they were a one-off correlator error, a calibration artefact, etc.

(...and if you did low-frequency in 2004, you had it coming anyway.)


2010: Ghosts Return

WSRT 21cm observation

...with intentionally strong instrumental errors

String of ghosts extending through dominant sources A (220 mJy) and B (160 mJy)

Second, fainter, string from source A towards NNE

Qualitatively similar to Cyg A ghosts


Don't Mess With My L-Band!

Eventually nailed via simulations


Ghosts In The (Selfcal) Machine

Ghosts arise due to missing flux in the calibration sky model

Mechanism: selfcal solutions try to compensate for this by moving flux around

Not enough DoFs to do this perfectly ...so end up dropping flux all over the map ...with a lot of help from the good Dr Sidelobes

Regular structure in this case due to WSRT's redundant layout = regular sidelobes

JVLA, MeerKAT: “random” (but not Gaussian!)


JVLA Ghost Sim


Ghastly Questions

Does calibration always introduce ghosts?

YES. But most of the time they're buried in the noise. ...unless you have a complete sky model (i.e. if all your

science targets are known in advance) Why don't we always see them?

Not enough sensitivity Will they average out?

NO. Push the sensitivity, they pop out. What will they do to my statistical detections (hello EoR)?

Dunno. Simulations needed. What else is that redistributed flux doing?


Ghosts, The Flip Side

WSRT “Field From Hell” (Abell 773 @300 MHz),residual map


Getting There, Right?

After diligent (direction-dependent) calibration


Noise-limited Is Not Always Good

Suppression of non-model sources

Our target


The Dangers Of Direction-Dependent Solutions

Suppression is less with more conservative calibration

Our target


Ghosts & Source Suppression

Both ghosts and suppression operate via the same mechanism, and Dr Sidelobes pulls the strings

Ghosts are usually buried in the noise Suppression always present, but more severe with

DD calibration (more DoFs...) A noise-limited map is not necessarily a good

science map!

“What if we were to somehow break the thermal noise barrier, butall we'd find beneath would be the bones of Arnold's enemies?”

– Anon., 3GC-II Workshop

(names and places changed to protect the guilty)


HI Stacking:Picking over the bones of Arnold's enemies

Individual HI lines hidden in the noise So apply priors from optical surveys... ...by shifting each spectrum to its known redshift ...and probe the total signal by co-adding spectra

HI lines add up, noise suppressed by ~N -2

For bonus points, use optical positions to stack sources that are sub-noise in radio

(Delhaize & Staveley-Smith)


HI Stacking: The Ian “Bad News” Heywood Simulation I

Simulated HI emission (S3-SAX)

Pure sky model

No noise

No continuum

No interferometerresponse

So, what happens when Dr. Sidelobes shows up?


“Stacking experiment for high-z HI on the celestial equator? LOL!” – Anon.

No noise No continuum No calibration or

deconvolution residuals

SKA-Mid sim Equatorial field:

high N-S sidelobes


True vs. Observed

How many of these galaxies will stack up coherently?


LADUMA Stacking Sim

No noise No continuum No calibration or

deconvolution residuals

MeerKAT sim (... with some

YouTube artefacts...)


Dr Sidelobes at Work

TOP: A selection of individual galaxy spectra BOTTOM: A selection of

“Dr. Sidelobes's spectra”


HI + Dr. Sidelobes

TOP: original HI signal BOTTOM: HI + Dr. Sidelobes

...most of which originate with undetected sources

Ordinal number of galaxy(in order of HI mass)

75% of the top 500...


Pixel Values Histogram

no negativevalues

Most HI signal lives here

dominantsources

negative sidelobes of dominant

sources

positive sidelobesof dominant

sources


How Radio Interferometry Really Works

Bright sources(DD solutions, uv-subtraction, deconvolution)

Faint sources(DDE interpolation, deconvolution)

ther

mal

noi

se 1-2σ sources(stacking)

<1 σ sources(a stiff drink?)


Stacking In WHDF

Should have been possible: ~1700 optical positions available in this field.

Failed miserably.

Prime suspect: the bones of Arnold's enemies left behind by the phase calibrator.

Can simulate the whole thing...


And Things Can Only Get Worse...

Radio interferometers of the past: simple machines built of beautiful, massively over-engineered components

“Simple”experiments


The Future

Complex machines built out of cheap junk “We'll just solve it in software!”

Increasingly subtle andelaborateexperiments


Conclusion

It's a brave new world that Arnold's had a hand in building

And it's also full of surprises But we have the tools to discover them, and

hopefully avoid the bad ones Another Golden Age in instrumental radio

astronomy?

Documents

Multi-Beaming And Calibration - ASTRON · O. Smirnov - AvA 2013 3 Dr. Sidelobes, Or How I Learned To Stop Worrying & Love Simulations O. Smirnov (Rhodes University & SKA SA)