Planning for MSSS - Lorentz Center fileLOFAR Surveys Workshop March 9, 2010 Michael Wise* Planning for MSSS *On behalf of the LOFAR collaboration

LOFAR Surveys WorkshopMarch 9, 2010

Michael Wise*

Planning for MSSS

*On behalf of the LOFAR collaboration

Planning for MSSS

● System and pipeline functionality● Data management and operations● Specifications and scheduling

Outline

ASTRON LOFAR Surveys Meeting, 09 March 20103

Why do an MSSS ?

LOFAR needs a Global Sky Model (GSM) for the northern sky which — has a proper flux scale

— has validated (initial) source parameters (spectrum, structure, ..)

— is astrometrically correct to better than 0.5”— interfaces efficiently to calibration & imaging pipeline (through LSM)

Moreover, carrying out MS3 will

— create a joint focus for activities— integrates scheduling, monitoring, processing, calibration & imaging

— test all KSP-pipelines

— provides a field-test for storage and processing resource needs— provide the conditions for a rehearsal of full LOFAR operations

*Registered trademark, Ger de Bruyn (2007)


Initial Proposal for MSSSObservations: — 20 NL stations (13+7) => multiple snapshots for decent uv coverage

— limit to two (broad?) frequency ranges: 60 MHz & 150 MHz

— complete in < 3 months (30% efficiency) & ‘real-time’ processing — 4 beams of ~10 MHz (+ CasA beam, ~1 MHz)

Products:

— 1 million sources, of which ~ 100,000 will be high S/N (i.e. ~ 5 / deg2 )— spectral indices for the ~ 100,000 sources seen in both bands

— structural information: ~20 - 60” PSF (~VLSS/WENSS/NVSS)

— fully tested pipelines

— arcsec images of ~ 4,000 European-LOFAR calibrator sources— lists of polarized calibration sources for ionospheric RM-monitoring


Multiple cuts of 5m + Very broadband (30 MHz)

LBA Band uv Coverages


Tasks StatusPreparation

MSSS Observation proposal draftMSSS scheduling neededFlux scale in progressFinalized station beam models implemented

Stations20 validated stations readyHBA ring splitter readymulti beam observations readyHBA station taper neededStation Calibration LBA implementedStation Calibration HBA in progress

SAS / MACsingle beam observations implementedmulti beam observations implemented

OLAPwriting to new storage facilities readymulti beam observations implementedqueue scheduling in progressrapid switching in progressparallel incoherent beam mode implemented

System Functionality


Pipeline Functionality


Place, date- 1 -Title

Standard Imaging Pipeline

• General– Integration of Pipeline with MAC under way. Expected to be ready for

MSSS dress rehearsal (in 4 weeks)– Assessment and improvement of pipeline performance has high priority– UVW coordinate flip is fixed. Other differences are being looked into.– Major Cycle closed for CASApy source components. Source Finding

components remain to be done.• DPPP

– overhaul: 40% performance gain. Needs final testing. ParSet file changes: Cookbook needs updating.

– full resolution flags will be kept in compressed data– Andre Offringa improved his Flagger. It remains outside DPPP.

• BBS– HBA station beams ready for testing. Who?– Caching in Predict is implemented.– Groups of sources can have same instrumental gains.– Gaussian source models operational

Imaging Pipeline Status


Imaging Pipeline Status

Place, date- 2 -Title

• BBS Ionosphere– Clock – ionosphere phase separation works, but suffers from 2 pi

ambiguities.– SPAM processing with distributed files is implemented– Next step: direction dependent calibration– First working version could be available for MSSS dress rehearsal

• Imager– Initial implementation for Direction Dependent Corrections available for 2

months now! Who will test this?– Comparing performance of CImager with CASA Imager under way

• Source Finding– Hanno Spreeuw documented the Source Finding module

• Some remarks– The 5th busy week provided great feedback. A lot of issues are being

followed up on.– Warning: using non-LOFAR software may be the way of least

resistance, but we do need feedback on DPPP / BBS / CIMager!– MSSS dress rehearsal is being planned for


Operations and Archiving


Estimated Data Volumes

Current archive resources● Currently using ~50 TBytes● 300 TBytes available @ SARA in Amsterdam● 60 TBytes @ Astrowise server in Groningen● 500 TBytes (to 1 PBytes) via TARGET in Groningen● 1 PBytes tape storage @ Jülich● Additional 2-3 PBytes to be added @ BiGGRID in Amsterdam

(R. Nijboer, March 2009)

MSSS Dry Runs

16 pointing centersCircumpolar fieldsAll-day observing

4 beams of 12 MHz6 hrs per field45 & 150 MHzTotal of 4 days

(courtesy D. Rafferty, N. Mohan)

⇒ March 15-19 April 12-16


Scheduling

Busy WeeksEngineering timeMSSS

WorkshopsDry runs


The End

Documents

Planning for MSSS - Lorentz Center fileLOFAR Surveys Workshop March 9, 2010 Michael Wise* Planning for MSSS *On behalf of the LOFAR collaboration