Financial Data Access with SQL, Excel & VBA

Computational Finance and Risk Management

Financial Data Accesswith SQL, Excel & VBA

Guy YollinInstructor, Applied Mathematics

University of Washington

Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 1 / 59

http://www.r-programming.org

Outline

1 What is RExcel?

2 Installation and configuration

3 Getting help with RExcel

4 Getting started with RExcel

5 The RCommander

6 Basic data transfer

7 Scratchpad mode

8 Scratchpad example



Lecture references

T. Baier, E. NeuwirthUsing R from within Excel.2007.

T. Baier, E. NeuwirthCreating and Deploying an Application with (R)Excel and R.The R Journal Vol. 3/2, December 2011.




Let’s not kid ourselves: The most widely usedpiece of software for statistics is Excel.

Brian D. Ripley



Outline

1 What is RExcel?




5 The RCommander


7 Scratchpad mode




RExcel - an interface between R and Excel

RExcel is an interface program that uses R as an add-in to Excel

Transfers data between R and Excel, using Excel as a data editor fordata to be analyzed with R

Uses Excel as a container and simple code editor for R commands,and running R commands from Excel

Runs R commands from Excel VBA macros allowing users to createExcel based applications and Excel add-ins using R

Uses R functions in Excel worksheets functions, integrating Rfunctionality into the automatic recalculation engine of Excel



RExcel modes of operation

RExcel supports three modes of operation:

Scratchpad and data transfer modeMenus control data transfer from R to Excel and back; commandscan be executed immediately, either from Excel cells or from Rcommand line

Macro modeMacros, invisible to the user, control data transfer and R commandexecution

Spreadsheet modeFormulas in Excel cells control data transfer and command execution,automatic recalculation is controlled by Excel



RExcel components

statconnDCOM A DCOM† server which allows to seamlessly integrate Ror Scilab into other applications.

rcom An R package allowing to have R and another programrunning at the same time and let them communicate.

rscproxy An R package which is required for rcom and for thestatconn (D)COM Server.

RExcel An Excel addin using statconn (D)COM or rcom toallow Excel to call R from within Excel.

RAndFriends An installation wrapper containing installation programsfor R (including many packages), statconnDCOM,RExcel, and the example files for the book R throughExcel.

†http://en.wikipedia.org/wiki/Component_Object_ModelGuy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 8 / 59

http://en.wikipedia.org/wiki/Component_Object_Model


Outline

1 What is RExcel?




5 The RCommander


7 Scratchpad mode




RAndFriends

Download the latest version ofRAndFriends from the statconnwebsite. Download includes:

Latest version of R (2.15.1)

Package rscproxy 2.0-5

Package rcom 2.2-5

To install:

Run the executable and acceptdefault installation options

http://rcom.univie.ac.at/download.html#RAndFriends



Excel add-ins and developer tabs

Add-Ins Tab

Contains RExcel add-in menu

Developer Tab

Contains access to macrocreation and VBA development

If the Add-Ins and/or Developer tabis not visible:

Go to File → Options →Customize Ribbon. Activatethe Add-Ins and Developercheckbox under "Main Tabs".



Establishing a reference to RExcelVBAlib

To use RExcel VBA procedures, a reference must be established to theRExcelVBAlib object library

From the VBE in Excel:Select the "Tools" tab

Select "References"

Find the object library labeledRExcelVBAlib and activate thecheckbox

Click "OK" and exit the VBAeditor



Outline

1 What is RExcel?




5 The RCommander


7 Scratchpad mode




Getting help

To access the RExcel help click theRExcel add-in menu and select"RExcel Help"

This help option opens "Using Rfrom within Excel" in a webbrowser

Contents

"Using R from within Excel" is aquick reference guide

For additional information, see"R Through Excel"



RExcel tutorial video

"Combining R and Excel" tutorialvideo

T. Baier, E. Neuwirth

Provides RExcel walkthroughcommentary

Walkthrough material isavailable through the RExceladd-in menu

http://rcom.univie.ac.at/RExcelDemo/

It is highly recommended that students view this material before getting started with RExcel.Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 15 / 59


Walkthroughs

RExcel comes with walkthrough demo worksheets.To start working on the demo material click the RExcel add-in andselect "Demo Worksheets"

Demo worksheets include:

Data transfer

Writing macros

Worksheet functions

Graphics with sliders

Interactive graphics



Outline

1 What is RExcel?




5 The RCommander


7 Scratchpad mode




Starting RExcel

To start RExcel:

Navigate to the add-ins tab

Click on the RExcel add-in tobring up the RExcel add-inmenu

Click "Start R"

The RExcel splash screen shouldappear briefly



The RExcel add-in menu

Once the R Server is started, the RExcel add-in menu will appear asfollows†:

†If foreground server is started, option will read Disconnect R rather then Close RGuy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 19 / 59


RExcel add-in menu commands

RExcel add-in commands:

Start R Starting RExcel will begin a new R session.

Close R Closing RExcel will end your R session.

Run Code Executes valid R code from a cell or a range of cells.

Get R Value Puts the value of an R expression into the active cell orrange of cells.

Put R Var Puts a valid selection of cells into an array or dataframeobject.

Get R output Retrieves R output from the most recent command.



RExcel add-in menu commands cont.

Set R working dir Allows the user to set a new working directory.

Load R file Loads an .R file into the current R session from theworking directory.

Copy Code Copies selected R code and inputs the selection into aVBA procedure.

Debug R Displays the debug window in which all calls to R aredisplayed before execution.

Error Log Displays an additional window with error messages.

Options Displays an options menu.

Set R server Allows the user to switch between a foreground serveror background server.



RExcel add-in menu commands cont.

RExcel Help Opens "Using R from within Excel" in abrowser window.

R Help Opens the R help documentation.

RCommander Opens the RCommander with Excel orseparate menu options.

Demo Worksheets Opens the Demo walkthroughs in a newExcel spreadsheet.

RthroughExcel Worksheets Opens RExcel projects with advance graphicsand analysis.

Mark Calc cells Marks cells containing calculated results.

About RExcel Opens the "About RExcel" menu.



The RExcel RMB menu

Once the R Server is started, there will be RExcel specificright-mouse-button options:



The RExcel RMB menu cont.

The RExcel RMB menu adds the following options:

Insert Current R Plot Inserts the current R Plot into a given range.

Name Range Create a named range for easier selection.

Prettyformat Numbers Aligns decimal points in a selected range.



Setting the R Server type

Setting the R Server

Before starting RExcel, the "SetR server" option is available.

There are two methods by whichto run RExcel:

Background server

Foreground server

Starting RExcel initiates a number of different processes at startup. These processes are explained at greater length in theStartup section of "Using R from within Excel".



Setting the R Server type

Background ServerThe background server is totally hidden from the user, all interaction withR has to be done in Excel.

Foreground ServerThe foreground server allows direct access to the R GUI command linewhile working in Excel.

See Using R from with Excel, T. Baier and E. NeuwirthGuy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 26 / 59


Background server

To initiate the background server:

Open the RExcel add-in menu

Select Set R server

Under server type, click"Background"

Note

No visible R GUI

All R commands must be sentfrom Excel



Foreground server

To initiate the foreground server:


Select Set R server

Under server type, click"Foreground"

Note

R GUI is visible

R commands can be typeddirectly into the command lineor sent from Excel



Background vs. foregroundBackground advantages

R operates in the background, invisible to front-end users

Background disadvantagesR operates in the background, invisible to front-end users

Foreground advantagesFor those familiar with R, a foreground server provides a moreconvenient method by which to pass R commandsA foreground server is simply supplemental, no RExcel functionality islost by running a foreground server

Foreground disadvantagesDo not close the R GUI before disconnecting from the DCOM server.This may cause system instability.If R is upgraded but RExcel is not reinstalled, RExcel will not workwith the foreground server



Outline

1 What is RExcel?




5 The RCommander


7 Scratchpad mode




RCommander

R Commander is a GUI for the R†

Developed by John Fox atMcMaster University

Licensed under GNU GPL

Along with plug-ins, arguablythe most fully featured GUI forR along the lines of S-PLUS

†http://socserv.mcmaster.ca/jfox/Misc/Rcmdr/Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 31 / 59

http://socserv.mcmaster.ca/jfox/Misc/Rcmdr/


The RCommander Window

The RCommander windows can be thought of as a re-skinned version ofthe R GUI.

RCommander contains three distinctsub-windows:

Script Window Type or open Rscripts

Output Window Displays outputfrom submitted Rcommands

Messages Displays errormessages andgeneralinformation



The RCommander functions and methods

RCommander functionality:

File Save and/or open scripts, set a new working directoryEdit Copy, paste, cut, undo, redo, etc.Data Import data from various sources and manage datasetsStatistics Get statistics regarding the active datasetGraph Plot the active datasetModels Create statistical models based upon the active datasetDistributions Plot distributions and compute relevant statistical valuesTools Load packages/plug-ins and access RCommander optionsHelp Access R and RCommander Help

Depending upon the nature of the active dataset, some options may not be available.



Starting the RCommmander

RExcel provides menu options, viathe RCommander, for some datatransfer, analysis and visualization.To start the RCommander:

Start RExcel


Click "RCommander"

Select either "with Excel menus"or "with separate menus"



Activating a dataset

In order create models anddistributions using the RCommander,a dataset must be defined:

To activate a dataset click"<No active dataset>" locatedbelow the tab bar

Notice that the user must defineat least one dataset in thecurrent R session to activate adataset (to be explained in thefollowing sections)

Select a dataset from thedropdown menu and click "OK"



Outline

1 What is RExcel?




5 The RCommander


7 Scratchpad mode




Arrays and dataframes

RExcel can handle two classes of data:

ArrayRExcel uses the term array to mean R arrays, matrices, and vectors. BasicR data types where all elements are of the same class.

DataframeA dataframe is a two-dimensional matrix-like structure where each columncan be of different classes.



Using RExcel to create a data.frame in R

There are various methods by which RExcel can create a data.frame in R.

RExcel add-in menu options

RMB menu options

RCommander menu options

R commands typed directly intothe R GUI or the RCommanderscript window

RExcel worksheet functions

RExcel VBA procedures



Sending a data.frame from Excel via RExcel add-in menuFor beginning users, the most convenientmethod is to use the RMB menu options.

Start RExcelHighlight a selection of dataRight-click the data selectionChoose "Put R DataFrame"Create a dataframe name or acceptdefault name

NoteRExcel assumes that the data selectioncontains column headersBy default, RExcel names thedataframe after the spreadsheet title.This can be changed in the RExceladd-in menu under "Options".



Importing data via RCommander

A dataframe can also be created through theRCommander menu options:

Start RExcel and open theRCommanderOpen the "Data" tabImport data

NoteThe RCommander accepts various dataforms:

.txt, .dat, .csv, .xls, .xlsx, etc.The RCommander can also load datafrom any attached R package

To attach additional packages go tothe "Tools" tab in the RCommanderand select "Load package(s)"



Importing data via RCommander cont.

RCommander will create R code based upon user input from the dataimport wizard.



Retrieving a data.frame from R

There are a few methods by which toretrieve a dataframe from R.

RExcel add-in menu options

RMB menu options

RExcel VBA procedures



Retrieving a data.frame from R via RExcel RMBRMB menu options

Insure RExcel is runningRight click the cell in which you wantthe dataframe to beginChoose "Get R DataFrame"Click "Get from R"Select a dataframe from the dropdown

Note

If you ended the connection to theDCOM server, previously createddataframes will have been deletedSequential numbering will be includedunless the "with rownames" checkbox isoff



Outline

1 What is RExcel?




5 The RCommander


7 Scratchpad mode




RExcel as code container and editorRExcel can run valid R code from a cell or selection of cells. Sending andreceiving information between R and Excel is accessible through theRExcel add-in menu and the RMB menu.



The "Run code" option

By interfacing R and Excel throughRExcel, R code can be typed directlyinto a cell or a range of cells and besent to R:

There is no need to enclose theR expression in quotations

Code can be run by accessingthe RExcel add-in menu or theRMB menuIf there is a coding issue, anpop-up will relay error messagesdirectly from R



Special RExcel commands

RExcel implements a few special commands that allow the automation ofdata exchanage without VBA. They can be run through the "Run Code"option.

Command Description

#!rput variable range store the value (contents) of a range in an R variable#!rputdataframe variable range store the value of a range in an R data frame#!rputpivottable variable range store the value of a range in an R variable#!rget r-expression range store the value of the R expression in the range#!rgetdataframe r-expression range store the data frame of the R expression in the range#!insertcurrentrplot cell-address insert the active plot into the worksheet



Outline

1 What is RExcel?




5 The RCommander


7 Scratchpad mode




The quantmod package

The quantmod package for R is designed to assist the quantitative traderin the development, testing, and deployment of statistically based tradingmodels.

Key functions:getSymbols load or download price data

Yahoo Finance / Google FinanceFREDOandacsv, RDataMySQL, SQLite

chartSeries charting tool to create standard financial charts

Author:Jeffrey Ryan



The getSymbols function

The getSymbols function loads (downloads) historic price data

R Code: The getSymbols function> library(quantmod)> args(getSymbols)

function (Symbols = NULL, env = .GlobalEnv, reload.Symbols = FALSE,verbose = FALSE, warnings = TRUE, src = "yahoo", symbol.lookup = TRUE,auto.assign = TRUE, ...)

NULL

Main arguments:Symbols symbols to be loadedsrc source of the data (Yahoo, Google, FRED, etc.)

Return value:a time series object



Federal reserve economic data

The function getSymbols can also be used to access data from theFederal Reserve Economic Data (FRED) database

http://research.stlouisfed.org/fred2/Guy Yollin (Copyright © 2012) Data Access with SQL, Excel & VBA RExcel 51 / 59

http://research.stlouisfed.org/fred2/


Sending command to R with Run Code



Ready to Run code



Data and plot retrieved from R



Data and plot retrieved from R



Sending data.frame to R via special command










http://depts.washington.edu/compfin


http://depts.washington.edu/compfin


Education

Financial Data Access with SQL, Excel & VBA