RFI Mitigation Techniques for RadioAstronomy

RFI Mitigation Techniques for RadioAstronomy

Michael KestevenAustralia Telescope National FacilityGroningen, 28 March 2010

RFI mitigation. Groningen, 2010

• Introduction• The issues• RFI-free environments• Blanking• RFI cancellation• The low RFI levels problem• The large dataset problem• Prospects


Introduction

In the past decade a number of RFI mitigation techniques have been trialled and shown to work.

Yet few observatories have on-line RFI mitigation installed.

Has its time now arrived, perhaps?


There is no universal solution

• Different sources of RFI• TV/Communications• Satellites• Observatory-based

• Different types of Telescopes• Single dish• arrays

• Different observing regimes• Low frequency; high frequency• VLBI• Pulsars• Spectral line• Continuum


The ITU RA-769 argument

A.R.Thompson provided a useful framework to describe the impact of RFI. Of interest here is the link between the observation mode and the RFI levels.

Recognise that RFI entering the main beam of a telescope (LOFAR apart) is generally a lost cause. Pitch the debate at RFI in the far sidelobes - at the level corresponding to a 0 dBi gain.


Single Dish Operation

Natural defences are fewantenna sidelobes (0 dBi gain)

Mitigation techniques work welladaptive filtersblanking

Datasets are modest (relatively speaking)detailed probes over the entire dataset are realistic

Vulnerable to low level interference


Arrays

Natural defences are better:antenna sidelobesphase tracking decorrelationdelay decorrelation (continuum observations)spatial resolution

Mitigation techniques less well developed

Datasets could become hugeSome advanced techniques may not be realistic in the near term


Current response to RFI

- Flag/blank post-detection data.

- Retune the receiver to an adjacent frequency

- Tolerate it

- Reschedule the observations


The Challenge

We need machinery to reduce the impact of RFI which is damaging the astronomer’s data.

- It should be automatic, reliable and robust.

- It should not introduce artefacts which mimic real results.

- The cost of applying the machinery should be predictable.

- The cost should be less than the cost of doing nothing.

(cost : $, science, time)


Mitigation Options

Pro-active mitigation : AvoidanceRemove the RFI at source.

Re-active mitigation : Remove the RFI from the dataBlank those parts of the astronomical data space which

contain RFI ---- excision.

Identify and remove the RFI while leaving the astronomy untouched --- cancellation.


Avoidance no RFI to mitigate

- Remote Locations

- Regulation- Spectrum Management- Radio quiet zones

- Good observatory practice- Discipline- Good design- Maintenance- Constant monitoring



Blanking, Flagging

This is the current mitigation strategy of choice.

It is attractive to observers because it is simple and its consequences are predictable:

• The loss in sensitivity is related to the amount of data discarded.• The effect on the image quality can be estimated.

• It is straightforward in its implementation, and can easily be automated.


Excision

Radiometer integration period (~msec)Time

Pulsed interferer (~sec)

Requirements:

- The RFI events occupy a small fraction of the data space.- Each RFI event has to be detectable – needs INR > 1 in small number of samples.


Excision – Real Time blanking

- Given the mean and rms of good data, define RFI to be those samples above a threshold (= *rms)

- Need to buffer some small number of data samples in order to be able to distinguish good from bad.

- The buffer allows you to apply some intelligence: determine the local mean and rms in order to identify the outliers.

- The buffer allows further options – one could blank a known pulse shape, for example.


Implementations

A number of observatories have built hardware, on-line blanking devices.

- Arecibo, for example, addresses the serious RFI from neighbouring radar. The known timing details of the pulsing assists the blanking trigger.

- WSRT have demonstrated an impressive unit built around digital processing boards which, amongst in many capabilities, can provide on-line blanking.


Excision Post-correlation Blanking.

Flag the data – instruct the downstream imaging/processing machinery to ignore the corrupt samples.

This is the RFI-mitigation strategy of last resort.

Tedious when done manually; automated scripts now available.

When this is applied to the correlator output data, the minimum quantum of rejected data is the size of the correlator dump cycle.


ATCA – Middleberg Automated flagging


Frequency Blanking

Discarding data in frequency space is a variant of this approach: modern high speed processing allows fine on-line spectral analysis, so that corrupted channels can be identified and excised.

This is an option if the discarded fraction of frequency space is modest compared to the overall bandwidth.

LOFAR includes this in its armoury.


Excision Issues

The technique relies on the ability to detect RFI from a small number of samples (or from a priori information). It generally requires good INR. Long integrations with low INR will be compromised

INR > 10 is a rough guide. There may be little to be gained by integration if the RFI is pulsed, as the INR is essentially based on a relatively small number of samples. (Periodic RFI is a separate case).

Downstream processing should not be compromised. Care needed in defining the replacement sample.

Discarding data in synthesis arrays will affect the (u,v) plane population and may therefore compromise the imaging quality.


Excision – Bottom line

It can be a viable technique if the cost to science is modest.

It depends on some prior definition of “badness”, and it depends on a low duty cycle.


Cancellation

This is the more ambitious approach – identify and characterise the RFI; then remove just the RFI.

This is a two-step process:

1. Characterise the RFI. 2. Subtract the RFI from the data – to give the astronomer an

RFI-free dataset.


Cancellation -- How is the RFI identified?

- Extract the RFI details from the data itself.

- Point a reference antenna towards the RFI -- use adaptive filter.

- Predict the RFI from published data (eg, GLONASS) – use software adaptive filter.


Questions:

Mitigate on the data on-the-fly ? (each correlator dump)

Or

Mitigate on the entire observation.

SNR is the issue with the first; Data volume the problem with the second.


Filter Variants

• Image plane filtering

• Spatial filtering

• Null Steering

• Cyclo-stationary filters

• Adaptive filters


Clean/self-calibration filter (Cornwell-NRAO)

Identify the RFI in the imaging stage. Apply self-calibration to the RFI. Remove the RFI.

Stationary RFI will map to the pole.

The self-calibration operates simultaneously in two areas :- The astronomical target;- The RFI which is stationary with respect to the observatory. The self-calibration accounts for the phasing and amplitude variations.

It requires the data to be sampled much faster than the astronomical target would require.


Cornwell – 327 MHz.

No Filtering Filter active


Spatial Filtering

• Each object within the field of view of the array will add a specific signature to the full set of correlation products between the antennas.

• An eigenvalue decomposition of the product matrix will isolate the strongest sources.

• A projection operation can then remove the RFI sources.

• This scheme has long history, most recently successfully demonstrated in the LOFAR trials.


The LOFAR snapshot variant

1. Within each widefield (whole sky) snapshot identify and remove the RFI point sources (as found by spatial filtering).

This cleans the snapshot down to sky noise.

2. Stacking the sky-aligned snapshots builds the SNR on the astronomical objects while dissolving the remaining RFI.


This scheme is best suited to low frequency arrays (LOFAR)

There are problems at higher frequencies, where very short correlator cycle times are required.

The computing load for a detailed spatial filtering operation may be a limiting factor.


Cyclostationary Filters

• The concept here is to identify the RFI by its temporal signature, cyclostationarity. This attribute is specific to RFI.

• The classical spatial filtering matrix is replaced by a variant which is matched to a cyclic frequency.

• The projection operation then proceeds as before, to remove the RFI.

• This scheme has had some initial (promising) trials on LOFAR.


Null Steering

The ATA is an array of 42 antennas that includes a beamformer mode of operation, each beam directed to a potential target. This opens the possibility of adjusting the beamformer weights to position nulls in the direction of known RFI sources – fixed or mobile.

Wide-band nulls may be required (and have been demonstrated).

The process works well, but has serious implications for the bandwidth of the phase tracking machinery.


Adaptive Filters

These have been applied to :

• Single Dish• Arrays

The starting point is to obtain a copy of the RFI.

We manipulate this copy to match the RFI in the data – the function of the adaptive filter.

We then subtract the modified copy from the data.


Cancellation Real-time adaptive filter


Cancellation. Issues with the real-time filter

It requires modest INR. Averaging at the correlation step helps.

It can cope with multi-pathing, but not with multiple transmitters on the same frequency channel.

With no RFI there is no added noise. Gain drops to zero.

It adapts automatically to changes in the relative transmission path details.


Filter OFF Filter ON


Real-time adaptive filter

Best suited to continuum single dish observations.

Works well for pulsar and VLBI observations.

It may not be suitable for spectral line observations, as the cancellation is not complete, and the residuals will mimic the original RFI spectrum.

It could be difficult to implement in an array.


Post-Correlation adaptive filter

• We combine three cross-products to get a good estimate of the interference in the astronomical channel.

• No total power products in the cross-products, thus no bias.

• Noise*RFI products are also removed.

• The signal/noise is set by the ratio of Correlated RFI to noise products -

cBINR~ SNR


Reference antenna

Parkes 64m

Single Dish – post-correlation


Adaptive filters

• Both real-time and post-correlation filters use the INR as a control factor – the filter switches off when INR <~ 1This makes the filter robust.

• Both filters subtract the correction term from the raw astronomy signal – they do not modify the astronomy.

• The real-time filter provides attenuation, leaving some residual RFI power;

• The post-correlation provides cancellation, with some added

residual zero-mean noise.


Postcorrelation filter applied to an array

• The Post-correlation adaptive filter has been successfully applied to an array.

• A reference antenna provided the RFI copy.

• This copy followed the same conversion chain and correlation path as all the antennas of the array.

• Each baseline was corrected for RFI after computing a baseline-specific correction term.


Synthesis Array Filtering(ATCA, 1503 MHZ, 4 MHZ BW)


Before and after images


Post-correlation Adaptive filter - Issues

- Still effective with low INR (to ~ 0.1)

- Will require additional correlator capacity

- Works well with single dish.

- Works well with an array, but may require short correlator dump times


The Low INR problem

The mitigation schemes generally work on short sections of data,but the astronomer works with the entire dataset.

Low level RFI which may only show up in the final product is of concern.

Future arrays may have too much data to allow RFI mitigation predicated on the entire dataset. The ASKAP, for example, needs to complete the processing on-the-fly, when in high-resolution spectral mode.


Conclusions

The prospects look good at the low-frequency (LOFAR) end of the spectrum.

The issue is less clear at SKA frequencies and above.

A number of niche areas, such as VLBI and pulsars, look tractable.

Contact UsPhone: 1300 363 400 or +61 3 9545 2176

Email: [email protected] Web: www.csiro.au

Thank you

Australia Telescope National FacilityMichael Kesteven

Phone: 61 2 9372 4544Email: [email protected]: www.csiro.au/group


Array Adaptive Filter

termcorrection the- - - R2R1

R2BR1A

RFI astronomy the--- BA

We introduce two additional “antennas” into the correlator system, R1 and R2, from the reference antenna.


Real-time & Post-Correlation Filters

(f)C - (f)C (f)C :filter n correlatio-Post

(t)V - (t)V (t)V :filter adaptive time-Real

corrinout

corrinout


Preliminaries types of RFI

- Continuous (TV)likely to be an issue for long integrations, single dish.

handled well by adaptive filters.

- Impulsive (radar)amenable to blanking if strong.diluted and not important if low level (minor increase in Tsys)

- Short term, strong (satellites)predictable, so precautionary measures possible.


Adaptive Filter Performance (1)

Adopting the RA-769 criteria we can estimate the limitations of the adaptive filter.

1. The ratio of the RFI powers in the main and reference antennas is :

r = (Area of 0 dBi antenna) / (Area of the reference antenna)

r = (R)2 [= 0.0016 at 600 MHz, 4m ref antenna]

2. The residual power (due to the RFI) in the filtered output :

resid = r * Tsys * INR / (1 + INR)


3. resid tends to r*Tsys for large INR.

4. The filter starts to fade out at INR ~ 1, with resid ~ r*Tsys/2

5. Thereafter resid falls gracefully to 0 as the INR decreases.


General Issues

What does the RFI look like in the final data product?

- RA769 assumed white noise (Tsys argument)

- But fringe decorrelation has poor cancellation on small baselines, and for v=0 baselines. Leads to (1/f) –type rubbish.


General issues

When does the RFI mitigation machinery detect the RFI ?

- On a sample-by-sample basis?- In modest sample packets?- In the entire dataset?

Dilemma: low INR RFI may only be visible after a long integration.

- Need to boost the INR in the short sample mitigation strategy. antenna with gain >


Post-Correlation adaptive Filter in an ARRAY


ATCA - 1503 MHz; 4 MHz BW

Mean amp

filter OFF filter ON

Mean phase

freqfreq


MRO Protection

• Observatory grounds (120 km2)• Full/self-control

• Boolardy Pastoral Station (3467 km2/856,835 acres)• CSIRO held and operated

• Mineral Management Area (80 km radius) - State• Controls for non-licensed radiators

• Section 19 - State• Embargo on new mines in the region

• ACMA RALI September 2007 - Commonwealth• (Aus Communication & Media Authority Radiocommunications

Assignment and Licensing Instruction)• “FCC RQZ” protection (various radii)

• Additional State/Commonwealth Legislation being pursued


MRO Protection


Excision – The COST

Data is lost.

samplesofNumberSNR __

In time-frequency blanking, some frequency channels may be abandoned.


Boolardy, 2008 median


Scaling the RFI copy – adaptive filter

• Use an adaptive filter (real-time, or post-correlation). - Note the importance of the ref antenna receiver noise –

ensures the distinction between zero correlation (= filter optimally adjusted) and zero correlation (no RFI).

- Optimally adjusted filter -> the (reference antenna – main antenna pair) now has a null positioned on the RFI.


MRO Protection

• Observatory grounds (120 km2)• Full/self-control

• EMC control region (30 km radius)• No inhabitants• Control, but negotiation with pastoral activities

• Mineral Management Area (80 km)• Controls for non-licensed radiators

• Section 19• Large region limiting mining activity (no new mines in the region)

within blue


RALI


MRO RQZ Protection

Frequency Range (MHz)

Restricted Zone Radius (km)

Coordination Zone Radius (km)

Threshold (dBm/Hz)

100-230 150 260 -214

230-400 100 180 -222

400-520 100 165 -224

520-820 100 190 -224

820-1000 100 145 -228

1000-2300 100 140 -230

2300-6000 100 120 -232

6000-10000 100 - -232

10000-25250 100 - -236


Cancelation

We have attenuation of the RFI:- 2)1(1INR

att

We have additional receiver noise added to the output, sinceg is non-zero.

)1()(

INRFastronomyIPP RFI

Added noise power : -


Cancellation Broadband adaptive filter


Excision – Identify the RFI

The challenge is to find a characteristic which distinguishes the RFI from the “RFI-free” condition.

In time : test for RFI in S(t); S above some threshold.

In time/frequency : test for RFI in S(f,t)

Prior information : Abandon polluted frequency channels Spatial blanking – Null steering if the location of RFI sources

is known or can be determined.


Post-Correlation adaptive filter


Cancellation Real-time adaptive filter


ATCA – Kesteven & Manchester real-time single IF adaptive filter


Types of RFI (2)

- Observatory-generated computers

high-speed digital electronicsfaulty equipment


The Copy of the RFI

Some options:

• Use a reference antenna pointed directly towards the source of the RFI.

• RFI is distributed over the focal plane, unlike the astronomical field. Could use a multiple feed package to extract the RFI.

• GLONASS – using the published data.

Documents

RFI Mitigation Techniques for RadioAstronomy