Upload
bob
View
39
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Resources on forecast evaluation/verification. Barbara Brown. Joint Numerical Testbed. NCAR, Boulder, CO. July 2011. WMO Verification web page. Developed and maintained by. working group on. forecast verification research. Includes FAQs,. references and links to - PowerPoint PPT Presentation
Citation preview
Resources on forecastevaluation/verification
Barbara BrownJoint Numerical Testbed
NCAR, Boulder, CO
July 2011
WMO Verification web pageDeveloped andmaintained byworking group onforecast verificationresearch
Includes FAQs,references and links totutorial presentations
http://www.cawcr.gov.au/projects/verification/
Verification discussiongroup:[email protected]
EUMETCAL Online Learning Module
http://www.eumetcal.org/resources/ukmeteocal/verification/www/english/courses/msgcrs/index.htm
Free on-linetutorial withexercises
Includes modulesonCategoricalContinuousProbabilistic
Verificationmethods
Warner (2010):Numerical
BooksJolliffe and
Stephenson:Forecast
Verification - APractitioner’s
Guide inAtmospheric
SciencesPublished by Wiley
(new version in2011)
Weather andClimate
PredictionPublished by
CambridgeUniverstity PressIncludes chapter
Wilks (2011): StatisticalMethods in the AtmosphericSciencesPublished by ElsevierIncludes extensive chapter onverification methods
Forecast Evaluation Tools
R Statistics Package:Includes many statistical tools,including the R VerificationLibrary
MET is freely available andsupported to the community
Main focus: ModelverificationIncludes tools for point andgridded data, ensembleforecasts, spatial methodshttp://www.dtcenter.org/met/users/
Introduction to R
Tara JensenNCAR/RAL/JNT
What is R?
• A statistical programming language• In part, developed from the S ProgrammingLanguage from Bell Labs (John Chambers)
• Created to allow rapid development ofmethods for use in different types of data.
• Create new graphics. Many defaultparameters are chosen, but users retaincomplete control.
Why R?
• R has become the dominant language in the statisticalresearch community.
• R is Open Source and free.• Runs on all operatingsystems• Nearly 2,400 packages contributed.• Packagesand applicationsin nearly every field ofscience, business and economics.
• See R Notes, R Journal and Journal of StatisticalSoftware.www.jstatsoft.org
• More than 100 books with accompanying code• Very large, active user base.
Why not R?
• NCL, IDL, Matlab, SAS, … are all viablealternatives to R. If you are a part of an activecommunity of researchers using anotherlanguage, do likewise.
• If we were biostatisticians we would be usingSAS. Book Title: “Analyzing Receiver Operating Characteristic Curves with SAS”
• Consider building verification functions andutilities as part of code development .Verification need not be an external process toforecasting.
The R Community• Developers
– R Core Group (17 members), only 2 have left since1997
– Major update in April/October (freeze dates, betaversions, bug tracking, ...)
• Mailing lists– Help list ~ 150 messages/day, archived,searchable.
• 5 International Conferences, 2 US, 1 China
• Source code• Binary compilations (Windows, Mac OS, Linux• Documentation ( Main documents, plus numerouscontributed. Some in foreign languages.)
• Newsletter (replaced by R Journal.)• Mailing list (Several search engines)• Packages on every topic imaginable• Wiki with examples• Reference list of books using R. ( more than 100)• Task Manager
Everything about R is at www.r-project.org
Use R with scripts• In Linux - Emacs Speaks Statistics
– Providessyntax-based– Object name completion– Key strokeshort cuts– Commandhistory– Alt-x R to invoke R with Xemacs.
• In Windows, use editor– Added GUI features– <control>R sends a line or highlighted section into R.– Install package with GUIs– Save graphics by point and click.
• Mac OS– Similarto Windows with advantagesof system calls.
Coding principles
• Make verification code transparent and easyto read
• Comment and document liberally• Archive your code• Share your code• Label and save your data• Share your data
Packages in R
• Contributed by people world wide.• Allow scientists or statisticians to push theirideas.
• Apply and extend R capabilities to meet theneeds of specific communities.
• Accompany many statistical textbooks• Accompany applied articles (Adrian Raftery,Doug Nychka, Tilman Gneiting, Barbara Casati,Matt Briggs)
A sample of useful packages
•••••
verificationfields (spatial stats)radiosondesextRemesBMA(BayesianModelAveraging)
• BMAensemble• circular• Rsqlite
• Rgis, spatstat (GIS)• ncdf ( support fornetcdf files )
• Rcolorbrewer• randomForests
Packages
• Packages must be installedto call.• Packages must be called to use.• Base packages are installed by default.
10 most useful function in R
• aggregate - applies a function to groups ofdata subset by categories.
• apply - incredibly efficient in avoiding loops.Applies functions across dimensions of arrays.
• layout - creatively divide a print region.• xyplot (in the lattice package) slightly advancegraphic techniques
• %in% returns logical showing which elementsin A are in B. (e.g A%in%B)
More top 10
•
•
•
•
•
table – create contingency tabel counts.boot – apply bootstrap function correctlyread.fwf – read fixed width format datapar – control everything in a graphsystem( ) – allows you to call systemcommand from R
• pairs – the most under utilized plot – plots amatrix of 4 columns in a 4x4 plot layout
R Exercises
•
•
•
•
•
Choose groups of 3-4 – find a computerLog onto machinesBring up at least 2 xterms>cp /wrfhelp/MET/R-packages/example/* .>vi intro2R.2011comet.R
or>kwrite intro2R.2011comet.R &