RFI mitigation for the Parkes Galactic All Sky Survey (GASS)

Peter M.W. Kalberla Argelander Institut für Astronomie Bonn

Galactic All Sky Survey (GASS) N. M. McClure-Griffiths, D. J. Pisano, M. R. Calabretta, H. Alyson Ford, Felix J. Lockman, L. Staveley-Smith, P. M. W. Kalberla, J. Bailin, L. Dedes, S. Janowiecki, B. K. Gibson, T. Murphy, H. Nakanishi, K. Newton-McGee, J. Kerp, B. Winkel McClure-Griffiths et al. (2009) Kalberla et al. (2010)

2

GASS data final version (-0.12 to 50 K, log scale)

3

• • • • • •

GASS: survey parameters

13 beam receiver 21-cm line survey of the Galactic HI emission – – – – Declinations δ < 1 deg (-500) < -468 < v LSR Δv = 1 km/s < +468 < (+500) km/s In band frequency switching, Δv = 660 km/s Beam FWHM 14.4 arcmin OTF mapping in RA and DEC, two coverages 2.8·10 7 spectra, 5 sec dumps, noise ~0.4 K 10 observing sessions between 2005 and 2006 • FITS maps: noise at full resolution (15.6 arcmin): 60 mK 4

Every Thing You Always Wanted to Know About..

5

Problems

• RFI at fixed frequency without significant variation in time – Causing in many cases negative signals (ghosts) • Broad lines (Δv ~ 15 km/s) in March 2006 • Bandpass ghosts from HVC gas due to folding • Footprints: strong RFI signals for short time intervals • Ringing (Gibbs phenomenon) from correlator 6

First step: Use livedata flags

• LAB data are used for fitting the instrumental baseline • At that stage it is easy to replace channels flagged by

livedata

during first stage of reduction with LAB data • Alternatively flagged data can be interpolated from neighboring channels of Parkes data • The replacement using LAB is far better!!

• 0.1% of all data affected 7

Remaining RFI: „footprints“

8

Clean

9

Median filter (at any observed position)

•

Determine median, mean and rms fluctuations

radius of 6 arcmin (consistent with HIPASS) within a •

Find channels

– – that have High rms scatter (> 3 σ rms )

and

Large differences between median and mean (>σ m ) •

Replace data

– Do not that deviate > 2 σ m from median by filter for T > 0.5 K (T > 2 K at b > 10 deg)

median

– Do not filter at positions with continuum > 200 mJy • 0.07% of all data affected 10

Clean data

11

Observed, flagged RFI replaced

12

•

Peirce criterion (1852) AJ 2, 161 Criterion for the rejection of doubtful observations

•

Cutoff limit for exclusion of outliers depends on number of available data points

• For 40 profiles (typically) a 2 σ rms 10% of the data are suspect limit is adequate if about • A 1.6 σ rms limit would be adequate if about 20% of the data are suspect • We use a fixed 2 σ rms limit with deviations from the

median

13

•

Extra treatment:

Eliminate spectra with

high noise

(>3 times average) and with

more than 30 flagged channels (0.3% affected)

• Bandpass

ghosts

can be minimized by median filtering • RFI in March 2006 (broad Gaussian lines) – Fit parameters – – Flag data accordingly Median filtering as usual RFI • Emission lines > 2 K – – No automatic filtering Inspect data and filter only those regions that are affected 14

Reorganize database for computational reasons

•

300 GB sdfits

files with 2.8·10 7 spectra are hard to handle • Generate compressed

random access database

–

135 GB

in single file, pointer information – fast access of individual profiles • Benefit of new data format: – Allows fast filtering – Very fast on-the-fly processing of FITS cubes 15

Stray radiation (the reason for the second data release)

16

• • • • • • • • • •

21cm line work and Darwinism

Correction for stray radiation suffers from detailed observations of the antenna diagram Antenna parameters: Model parameters need to be self-consistent Does the solution ~60 different runs survive?

Baseline correction: Code and parameters need to survive ~50 different versions necessary

RFI mitigation Comparison of all profiles at any position within 6 arcmin (10 9 cases) >2 CPU years in total

Hornet magazine, 1871 17

Summary

• RFI post-processing needs

redundancy

– Typically no more that 25% of the data are bad – Limit: 50% • Fast data access necessary for filtering – New data format needed (random access) – Advantages: generation of FITS cubes very fast • Replace bad data by LAB data or by median – Surprisingly simple recipe to use other data 18

This all was about…

RFI in the protected band

But <0.5% of data affected

19

Dirty stuff you don’t want to see….

20