Introduction to CASA and Basic Data Reduction Overview Steven T. Myers CASA Project Scientist (NRAO)

Introduction to CASA and Basic Data

Reduction OverviewSteven T. Myers

CASA Project Scientist (NRAO)

CASA Tutorial -- 7 October 20082

Beta Release StatusCASA Status:

in Beta Release since October 2007roughly quarterly “patches”

Current Release: Beta Release Patch 3

target release date October 15general release

register at my.nrao.edu , no password for download


Beta DisclaimerThis really is a Beta Release!

still missing functionalitythere are known failure modeswill find new problems Improvements in upcoming patchesmove towards handling ALMA/EVLA “use cases”better handling of calibration


Curriculumschedule:

~1 hour orientation lecture + demo (this talk)hands-on data reduction with datasets:

VLA Continuum Polarization (3C129)

VLA Spectral Line (NGC2403)

BIMA 3mm line mosaic (NGC4826)

intended to be interactiveif you need help - ask!


Starting CASAFrom your workstation (running RHEL):

type ‘casapy-test’ at promptthis is the pre-release test version of Beta Patch 3 at AOC

quit CASA using ‘exit’brings up IPython shell plus casalogger GUI

Other releases available for downloadnot yet built for Patch 3...


Getting User SupportFirst stop: Read the Manual!

CASA Home: http://casa.nrao.eduCookbook, and “help” within CASA

CASA Helpdesk & Support Cadres“Helpdesk” under my.nrao.edu

submit Issues

User Support Specialists (Gustaaf at AOC)will handle your Helpdesk ticket and interface with developers

http://casa.nrao.edu/


Tutorial ContentsI - InterfaceII - Tasks & ToolsIII - Data in CASAIV - Data Processing WorkflowV - Example Use Cases


I - InterfaceDocumentationIPython & PythonCrashes & RecoveryCASA helpCASA task interfaceFunctionality


CASA DocumentationCASA Analysis cookbook (draft):

http://casa.nrao.edu/Doc/Draft/Cookbook/casa_cookbook.pdf

CASA User Reference Manual: http://casa.nrao.edu/docs/casaref/CasaRef.html

Python: http://python.org/doc (e.g., see Tutorial for novices)

IPython: http://ipython.scipy.org/moin/Documentation

matplotlib: http://matplotlib.sourceforge.net/

http://www.aoc.nrao.edu/~smyers/naug/doc/cookbook/casa_cookbook.pdf

http://casa.nrao.edu/docs/casaref/CasaRef.html

http://python.org/

http://ipython.scipy.org/moin/Documentation

http://matplotlib.sourceforge.net/


CASA InterfaceIPython

shell accessautoparenthesis (autocall)command historysession logging (ipython.log, casapy.log)numbered input/outputhistory/searching

Pythonsee standard manuals, Cookbook Appendix D


Python Pointersto run a .py script:execfile(‘<scriptname>’)

example: execfile(‘ngc5921_demo.py’)

indentation matters!be careful when doing cut-and-paste to Pythoncut a few (4-6) lines at a time

variables are global!tasknames are objects (not variables)


Crashes & RecoveryBeta Release still has crash modes

plotting in particular (plotxy and plotcal) can get a segmentation fault

If in doubt, exit and restart casapyget rid of orphaned loggersyou may have to use ps to find rogue casapy processes and kill them


Part II - Tasks & Toolsoverview of tasks and tools in CASAintroduction to the task interfacehow to get help on tasks and toolshow tools underly the taskshow to run tasks asynchronously


Tasks and tools in CASA

Tasks - high-level functionalityfunction call or parameter handling interfacethese are what you should use in tutorial

Tools - complete functionalitytool.method calls, used by taskssometimes shown in tutorial scripts


Key TasksTask list:

may be different in test and release versionsasap_init to load single-dish tasks and tools

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.


Task Executiontwo ways to invoke:

call from Python as functions with arguments

taskname( arg1=val1, arg2=val2, ... )

unspecified parameters will be defaulted (globals not used)

use standard tasking interfaceuse global variables for task parameters

see Chapter 1.3 in Cookbook


Task Interfacestandard tasking interface

use parameters set as global Python variables

set <param> = <value> (e.g. vis = ‘ngc5921.demo.ms’ )

parameter manipulation commandsusing inp , default , saveinputs , tget

execute<taskname> or go ( e.g. clean() )

return valuessome tasks return Python dictionaries, e.g. myval=imval()


Task Interfaceexamine task parameters with inp :




Expandable Parameters

boldface parameter are expandable






Parameter Checkingsanity checks of parameters in inp :

erroneousvalues in red


Help on TasksIn-line help <taskname> command:




Tools in CASACASA Toolkit underneath tasks

core AIPS++ code (mostly in C++)

tools are functions call from casapy as <tool>.<method>()default tool objects are pre-constructed

e.g. imager (im) , calibrater (cb), ms (ms) , etc. (see toolhelp)


CASA Tool Listlist of default tools from toolhelp :

tools described in the CASA User Reference Manual:






Asynchronous Tasksrun tasks in background

set parameter async=Trueruns as separate process

use taskmanager tool (tm) to monitor (Cookbook Chapter 1.3.2)

messages will appear in logger in the asynchronous order

useful to avoid known problems (e.g. exportfits / importfits)

cannot retrieve return values from async tasks


III - Data in CASAdata in CASA are stored in tables

interferometry data in Measurement Sets (MS)also tables for images and calibration tablesformats and structures defined in CASA memos

accessed through table and ms toolsrow and column structure is logical not physical


The Measurement Setthe MS is a directory on disk

the MAIN table in table.* filesalso contains sub-tables

e.g. FIELD, SOURCE, ANTENNA, etc.

sub-tables are sub-directoriesto copy must cp -rf to get contents

WARNING: moving a MS can break cal-table dependencies


Example MSExample: ls ngc5921.usecase.ms

ls ngc5921.usecase.ms/FIELD


MAIN Table ContentsExample using task browsetable:




More MS Detailsoften will refer to “rows” & “columns”

rows = integrations in spectral windowscolumns = time, u, v, w, DATA, weights, etc.

there are “scratch columns”created when needed - can bloat the MS

but (usually) less space than scratch copy of MS!

CORRECTED_DATA , MODEL_DATA, IMAGING_WEIGHTS


Data Selectionstandard MS selection syntax

see Chapter 2.5 of CASA Cookbooksyntax detailed in CASA Memo

http://casa.nrao.edu/Memos/msselection/index.html


Selection Examplestandard selection parameters

e.g. for task gaincal:

field and spw common standard selections

expandable selectdata with other selections as sub-parameters


Selection Syntaxsee Chapter 2.5 of Cookbook

field - string with source name or field IDcan use ‘*’ as wildcard, first checks for name, then ID

example: field = ‘1331+305’ ; field = ‘3C*’ ; field = ‘0,1,4~5’

spw - string with specwindow ID plus channels

use ‘:’ as separator of spw from optional channelization

use ‘^’ as separator of channels from step/width

example: spw = ‘0~2’ ; spw = ‘1:10~30’ ; spw = ‘2~5:5~54^5’


Selection Syntaxsee Chapter 2.5 of Cookbook

antenna - string with antenna name or IDfirst check for name, then ID (beware VLA name 1-27, ID 0-26)

example: antenna = ‘1~5,11’ ; antenna = ‘VA*’

timerange - string with date/time rangespecify ‘T0~T1’ , missing parts of T1 default to T0, can give ‘T0+dT’

example: timerange = ‘2007/10/16/01:00:00~06:30:00’


IV - Reduction Philosophy

data is imported into MSkeep original DATA inviolate

calibration is carried in cal-tableswill discuss cal-table philosophy later

imaging creates images from MSfeed back model images for self-calibration


Data PhilosophyOriginal data kept intact by default

DATA (and WEIGHT) columnsdo not change in original MS

user can use toolkit to mess with these also

MS “scratch columns” are usedused for adjusted data and model visibilities

BETA: user must keep track ; Post-Beta: hide table and MS state


CASA Workflow Chartworkflow:

importexamine / flagcalibrateimageanalyze


Workflow Stepsfollow rough workflow path:

import data (from UVFITS or archive formats)examine data and edit out bad visibilitiescalibrate data (through cal-tables)image data (various deconvolution methods)image and uv analysis (e.g. moments)data and image visualization (whenever needed)


Import Datasee Chapter 2 of Cookbook

task importuvfits UVFITS data (e.g. from AIPS)

task importvla VLA “export” format (e.g. from archive)

task importasdm ALMA data format (also EVLA eventually)


Data Examinationsee Chapter 3 of Cookbook

task listobslist summary of MS

task plotxyx-y line plots

task browsetable (casabrowser outside casapy)

Qt browser for tables (e.g. MS and images)

casaviewer - view raster images of MS


Flaggingsee Chapter 3 of Cookbook

task plotxydraw boxes around points and flag

casaviewer ( task viewer )draw boxes on raster and flag (modes similar to AIPS TVFLG)

task flagdata - manual flagging, clipping, etc.task flagmanager - backup & restore flags


Task plotxyuse plotxy for simple data plots & flagging

before (left) and after (right) editing and calibration


Task plotxyinteractive plotting and flagging

averagingtimebin, width (time & channel averaging)

iteration iteration = ‘antenna’ , ‘field’ , ‘time’ , ‘spw’ (or combinations)

GUI controlsbuttons for MarkRegion , Flag , Unflag , Locate , Next , Quit

plot optionsuse subplot = yxn to set multi-panels (e.g. subplot=311)controls for fontsize , markersize , other options


Example inputsset up to plot amp vs. time




Editing Jupiterisolate blocks of bad data and remove

single bad scan on Jupiter both spw , all antennas


MS viewingthe casaviewer can handle MS (e.g. TVFLG)




Calibrationsee Chapter 4 of Cookbook

prior calibrationsolvingcalibration table manipulationcalibration table examinationapplication to data


Calibrationworkflow:

input datasetsolving => tablesincrementalaccumulationapplication


Calibration Tablescalibration tables are exchanged between the calibration tasks:


Calibration Tasksprior calibration: setjysolvers: gaincal, bandpass, polcal, blcal, fringecalmanipulation: plotcal, accum, smoothcalapplication: applycal, splitother: uvcontsub, uvmodelfit


Parameterizationsprior calibration:

gaincurve: VLA-only for nowopacity: single tau value (for now)

gaincal: gaintype=’G’ (solution per solint time slot)gaintype=’GSPLINE’

bandpass: bandtype=’B’ (solution per channel)bandtype=’BPOLY’

No time-dependence yet


Calibration Processingsetjy (or ft)

specify a model image (for selfcal or resolved cals)

smoothcalMean, median smoothing of cal tables

accumcalibration re-sampling

Incremental/cumulative calibration


Imagingsee Chapter 5 of Cookbook

task cleansingle-field cleaning, variety of algorithms

mosaicing using uv-gridder (uv-plane mosaicing on single image)

interactive and non-interactive operation

task widefield - widefield imaging prototypew-projection and faceting

task feather - combine single-dish and uvMS


Interactive CLEANclean has interactive mode

interactive=True

uses the viewercan change cycles during cleaningdraw boxes and polygon regions for masking




Cube ChannelizationWhen using mode=‘channel’

select spectral windows of the data, e.g. spw=‘0,1’

the width is then set in units of first data spw channel width




Imaging spectral cubes

Other cube considerations... Averaging of data channel in imaging is always ‘mfs’ style…i.e gridded with different (u,v)If a mode=’velocity’ cube is chosen, then data will be gridded into LSRK binsIf simple data averaging is needed split is recommended


Mosaicing in cleanuse imagermode=’mosaic’

deconvolution on “Sault” (constant noise) image

User can set make PB corrected image at end

primary beam coverage x Sky in image planeConvolution with aperture cross-correlation function in uv-domain

Advantages of doing mosaicing in uvFasterapply correct beam transform for each baseline (e.g. different dish size, change of pb with frequency, beam errors etc.)

Parameter “ftmachine” sets thisPossibilities are 'ft' or 'mosaic'


More cleanMosaicing parameters continued:mosweight: important/different for uniform and Brigg's weighting scheme. Use per field weighting if mosweight=True

Cotton-Scwab major cyclesimagermode=’csclean’ or ‘mosaic’returns to residual visibilities at major cyclescyclefactor : controls major cycle depth

larger => more major cycles (clean will take longer)

to a lesser extent cyclespeedup (for a non-converging clean)


Image DisplayThe viewer

Cookbook Chapter 7






Image Analysissee Chapter 6 of Cookbook

task imhead - get and change image header information

task immoments - computes moment images of spectral cube

task immath - combine images and do image mathusing Lattice Expression Language (LEL)also: Spectral Index, Linear Polarized Intensity and Angle

task imstat - return statistics on regions of image

task imval - return values for pixel or region of image

task imfit - fit a 2D Gaussian to the image


Task immomentsuse viewer to look at NGC5921 moment images

Overlay contours of moment 1 (velocity) on moment 0(intensity)


V - Scripts & DataTutorial Home Page

http://casa.nrao.edu/Tutorial/20081007/

CASA Scripts & Data Pagethese are still Patch 2 versionshttp://casa.nrao.edu/casatraining.shtml

http://www.aoc.nrao.edu/~smyers



Tutorial Homepagehttp://casa.nrao.edu/Tutorial/20081007/



SummerSchool08 Data

The VLA tutorials:3C129 continuum data

B and C configuration export data

NGC2403 HI line dataexport data files AS649_* (VLA Archive files)

Millimeter tutorial:NGC4826 BIMA line mosaic

BIMA uvfits files


Example Use CasesNGC5921 VLA 1.4GHz spectral line

simple regression data set and scriptngc5921_demo.py

Jupiter 6cm continuuminteractive use case script

jupiter6cm_demo.py

also broken up by operations (flagging, calibration, imaging, pol)

Documents

Introduction to CASA and Basic Data Reduction Overview Steven T. Myers CASA Project Scientist (NRAO)