59
Rcpp Making R Applications Go Faster and Further Dirk Eddelbuettel EARL 2015 Keynote Address September 16, 2015 Released under a Creative Commons Attribution-ShareAlike 4.0 International License. 1/59

Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

  • Upload
    others

  • View
    9

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

RcppMaking R Applications Go Faster and Further

Dirk EddelbuettelEARL 2015 Keynote AddressSeptember 16 2015Released under a Creative Commons Attribution-ShareAlike 40 International License

159

Introduction

259

A Very Kind Tweet

359

And Another Tweet

459

And Yet Another Tweet

559

And Why Not Another Tweet

659

And Last But Not Least

759

Extending R

859

Why R Programming with Data

ChambersComputationalMethods for DataAnalysis Wiley1977

Becker Chambersand Wilks TheNew S LanguageChapman amp Hall1988

Chambers andHastie StatisticalModels in SChapman amp Hall1992

ChambersProgramming withData Springer1998

ChambersSoftware for DataAnalysisProgramming withR Springer 2008

Thanks to John Chambers for sending me high-resolution scans of the covers of his books

959

A Simple Example

xx lt- faithful[eruptions]fit lt- density(xx)plot(fit)

1059

A Simple Example

1 2 3 4 5 6

00

01

02

03

04

05

densitydefault(x = xx)

N = 272 Bandwidth = 03348

Den

sity

1159

A Simple Example - Refined

xx lt- faithful[eruptions]fit1 lt- density(xx)fit2 lt- replicate(10000

x lt- sample(xxreplace=TRUE)density(x from=min(fit1$x) to=max(fit1$x))$y

)fit3 lt- apply(fit2 1 quantilec(00250975))plot(fit1 ylim=range(fit3))polygon(c(fit1$xrev(fit1$x)) c(fit3[1]rev(fit3[2]))

col=grey border=F)lines(fit1)

1259

A Simple Example - Refined

1 2 3 4 5 6

00

01

02

03

04

05

densitydefault(x = xx)

N = 272 Bandwidth = 03348

Den

sity

1359

So Why R

R enables us to

middot work interactivelymiddot explore and visualize datamiddot access retrieve andor generate datamiddot summarize and report into pdf html hellip

making it the key language for statistical computing and apreferred environment for many data analysts

1459

So Why R

R has always been extensible via

middot C via a bare-bones interface described in Writing RExtensions

middot Fortran which is also used internally by Rmiddot Java via rJava by Simon Urbanekmiddot C++ but essentially at the bare-bones level of C

So while in theory this always worked ndash it was tedious inpractice

1559

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1659

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1759

Why Extend R

Chambers proceeds with this rough map of the road ahead

middot Againstmiddot Itrsquos more workmiddot Bugs will bitemiddot Potential platform dependencymiddot Less readable software

middot In Favormiddot New and trusted computationsmiddot Speedmiddot Object references

1859

Why Extend R

The Why boils down to

middot speed Often a good enough reason for us hellip and a focusfor us in this workshop

middot new things We can bind to libraries and tools that wouldotherwise be unavailable in R

middot references Chambers quote from 2008 foreshadowed thework on Reference Classes now in R and built upon viaRcpp Modules Rcpp Classes (and also RcppR6)

1959

And Why C++

middot Asking Google leads to about ~ 50 million hitsmiddot Wikipedia C++ is a statically typed free-form

multi-paradigm compiled general-purpose powerfulprogramming language

middot C++ is industrial-strength vendor-independentwidely-used and still evolving

middot In science amp research one of the most frequently-usedlanguages If there is something you want to use connect to it probably has a CC++ API

middot As a widely used language it also has good tool support(debuggers profilers code analysis)

2059

Why C++

Scott Meyers View C++ as a federation of languages

middot C provides a rich inheritance and interoperability as UnixWindows hellip are all build on C

middot Object-Oriented C++ (maybe just to provide endlessdiscussions about exactly what OO is or should be)

middot Templated C++ which is mighty powerful template metaprogramming unequalled in other languages

middot The Standard Template Library (STL) is a specifictemplate library which is powerful but has its ownconventions

middot C++11 (and C++14 and beyond) add enough to becalled a fifth language

NB Meyers original list of four languages appeared years before C++11 2159

Why C++

middot Mature yet currentmiddot Strong performance focus

middot You donrsquot pay for what you donrsquot usemiddot Leave no room for another language between the

machine level and C++

middot Yet also powerfully abstract and high-levelmiddot C++11 is a big deal giving us new language featuresmiddot While there are complexities Rcpp users are mostly

shielded

2259

Interface Vision

2359

Bell Labs May 1976

2459

Interface Vision

R offers us the best of both worlds

middot Compiled code withmiddot Access to proven libraries and algorithms in

CC++Fortranmiddot Extremely high performance (in both serial and parallel

modes)middot Interpreted code with

middot An accessible high-level language made for Programmingwith Data

middot An interactive workflow for data analysismiddot Support for rapid prototyping research and

experimentation

2559

Why Rcpp

middot Easy to learn as it really does not have to be thatcomplicated ndash we will see numerous few examples

middot Easy to use as it avoids build and OS system complexitiesthanks to the R infrastrucure

middot Expressive as it allows for vectorised C++ using RcppSugar

middot Seamless access to all R objects vector matrix listS3S4RefClass Environment Function hellip

middot Speed gains for a variety of tasks Rcpp excels preciselywhere R struggles loops function calls hellip

middot Extensions greatly facilitates access to external librariesusing eg Rcpp modules

2659

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 2: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Introduction

259

A Very Kind Tweet

359

And Another Tweet

459

And Yet Another Tweet

559

And Why Not Another Tweet

659

And Last But Not Least

759

Extending R

859

Why R Programming with Data

ChambersComputationalMethods for DataAnalysis Wiley1977

Becker Chambersand Wilks TheNew S LanguageChapman amp Hall1988

Chambers andHastie StatisticalModels in SChapman amp Hall1992

ChambersProgramming withData Springer1998

ChambersSoftware for DataAnalysisProgramming withR Springer 2008

Thanks to John Chambers for sending me high-resolution scans of the covers of his books

959

A Simple Example

xx lt- faithful[eruptions]fit lt- density(xx)plot(fit)

1059

A Simple Example

1 2 3 4 5 6

00

01

02

03

04

05

densitydefault(x = xx)

N = 272 Bandwidth = 03348

Den

sity

1159

A Simple Example - Refined

xx lt- faithful[eruptions]fit1 lt- density(xx)fit2 lt- replicate(10000

x lt- sample(xxreplace=TRUE)density(x from=min(fit1$x) to=max(fit1$x))$y

)fit3 lt- apply(fit2 1 quantilec(00250975))plot(fit1 ylim=range(fit3))polygon(c(fit1$xrev(fit1$x)) c(fit3[1]rev(fit3[2]))

col=grey border=F)lines(fit1)

1259

A Simple Example - Refined

1 2 3 4 5 6

00

01

02

03

04

05

densitydefault(x = xx)

N = 272 Bandwidth = 03348

Den

sity

1359

So Why R

R enables us to

middot work interactivelymiddot explore and visualize datamiddot access retrieve andor generate datamiddot summarize and report into pdf html hellip

making it the key language for statistical computing and apreferred environment for many data analysts

1459

So Why R

R has always been extensible via

middot C via a bare-bones interface described in Writing RExtensions

middot Fortran which is also used internally by Rmiddot Java via rJava by Simon Urbanekmiddot C++ but essentially at the bare-bones level of C

So while in theory this always worked ndash it was tedious inpractice

1559

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1659

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1759

Why Extend R

Chambers proceeds with this rough map of the road ahead

middot Againstmiddot Itrsquos more workmiddot Bugs will bitemiddot Potential platform dependencymiddot Less readable software

middot In Favormiddot New and trusted computationsmiddot Speedmiddot Object references

1859

Why Extend R

The Why boils down to

middot speed Often a good enough reason for us hellip and a focusfor us in this workshop

middot new things We can bind to libraries and tools that wouldotherwise be unavailable in R

middot references Chambers quote from 2008 foreshadowed thework on Reference Classes now in R and built upon viaRcpp Modules Rcpp Classes (and also RcppR6)

1959

And Why C++

middot Asking Google leads to about ~ 50 million hitsmiddot Wikipedia C++ is a statically typed free-form

multi-paradigm compiled general-purpose powerfulprogramming language

middot C++ is industrial-strength vendor-independentwidely-used and still evolving

middot In science amp research one of the most frequently-usedlanguages If there is something you want to use connect to it probably has a CC++ API

middot As a widely used language it also has good tool support(debuggers profilers code analysis)

2059

Why C++

Scott Meyers View C++ as a federation of languages

middot C provides a rich inheritance and interoperability as UnixWindows hellip are all build on C

middot Object-Oriented C++ (maybe just to provide endlessdiscussions about exactly what OO is or should be)

middot Templated C++ which is mighty powerful template metaprogramming unequalled in other languages

middot The Standard Template Library (STL) is a specifictemplate library which is powerful but has its ownconventions

middot C++11 (and C++14 and beyond) add enough to becalled a fifth language

NB Meyers original list of four languages appeared years before C++11 2159

Why C++

middot Mature yet currentmiddot Strong performance focus

middot You donrsquot pay for what you donrsquot usemiddot Leave no room for another language between the

machine level and C++

middot Yet also powerfully abstract and high-levelmiddot C++11 is a big deal giving us new language featuresmiddot While there are complexities Rcpp users are mostly

shielded

2259

Interface Vision

2359

Bell Labs May 1976

2459

Interface Vision

R offers us the best of both worlds

middot Compiled code withmiddot Access to proven libraries and algorithms in

CC++Fortranmiddot Extremely high performance (in both serial and parallel

modes)middot Interpreted code with

middot An accessible high-level language made for Programmingwith Data

middot An interactive workflow for data analysismiddot Support for rapid prototyping research and

experimentation

2559

Why Rcpp

middot Easy to learn as it really does not have to be thatcomplicated ndash we will see numerous few examples

middot Easy to use as it avoids build and OS system complexitiesthanks to the R infrastrucure

middot Expressive as it allows for vectorised C++ using RcppSugar

middot Seamless access to all R objects vector matrix listS3S4RefClass Environment Function hellip

middot Speed gains for a variety of tasks Rcpp excels preciselywhere R struggles loops function calls hellip

middot Extensions greatly facilitates access to external librariesusing eg Rcpp modules

2659

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 3: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

A Very Kind Tweet

359

And Another Tweet

459

And Yet Another Tweet

559

And Why Not Another Tweet

659

And Last But Not Least

759

Extending R

859

Why R Programming with Data

ChambersComputationalMethods for DataAnalysis Wiley1977

Becker Chambersand Wilks TheNew S LanguageChapman amp Hall1988

Chambers andHastie StatisticalModels in SChapman amp Hall1992

ChambersProgramming withData Springer1998

ChambersSoftware for DataAnalysisProgramming withR Springer 2008

Thanks to John Chambers for sending me high-resolution scans of the covers of his books

959

A Simple Example

xx lt- faithful[eruptions]fit lt- density(xx)plot(fit)

1059

A Simple Example

1 2 3 4 5 6

00

01

02

03

04

05

densitydefault(x = xx)

N = 272 Bandwidth = 03348

Den

sity

1159

A Simple Example - Refined

xx lt- faithful[eruptions]fit1 lt- density(xx)fit2 lt- replicate(10000

x lt- sample(xxreplace=TRUE)density(x from=min(fit1$x) to=max(fit1$x))$y

)fit3 lt- apply(fit2 1 quantilec(00250975))plot(fit1 ylim=range(fit3))polygon(c(fit1$xrev(fit1$x)) c(fit3[1]rev(fit3[2]))

col=grey border=F)lines(fit1)

1259

A Simple Example - Refined

1 2 3 4 5 6

00

01

02

03

04

05

densitydefault(x = xx)

N = 272 Bandwidth = 03348

Den

sity

1359

So Why R

R enables us to

middot work interactivelymiddot explore and visualize datamiddot access retrieve andor generate datamiddot summarize and report into pdf html hellip

making it the key language for statistical computing and apreferred environment for many data analysts

1459

So Why R

R has always been extensible via

middot C via a bare-bones interface described in Writing RExtensions

middot Fortran which is also used internally by Rmiddot Java via rJava by Simon Urbanekmiddot C++ but essentially at the bare-bones level of C

So while in theory this always worked ndash it was tedious inpractice

1559

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1659

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1759

Why Extend R

Chambers proceeds with this rough map of the road ahead

middot Againstmiddot Itrsquos more workmiddot Bugs will bitemiddot Potential platform dependencymiddot Less readable software

middot In Favormiddot New and trusted computationsmiddot Speedmiddot Object references

1859

Why Extend R

The Why boils down to

middot speed Often a good enough reason for us hellip and a focusfor us in this workshop

middot new things We can bind to libraries and tools that wouldotherwise be unavailable in R

middot references Chambers quote from 2008 foreshadowed thework on Reference Classes now in R and built upon viaRcpp Modules Rcpp Classes (and also RcppR6)

1959

And Why C++

middot Asking Google leads to about ~ 50 million hitsmiddot Wikipedia C++ is a statically typed free-form

multi-paradigm compiled general-purpose powerfulprogramming language

middot C++ is industrial-strength vendor-independentwidely-used and still evolving

middot In science amp research one of the most frequently-usedlanguages If there is something you want to use connect to it probably has a CC++ API

middot As a widely used language it also has good tool support(debuggers profilers code analysis)

2059

Why C++

Scott Meyers View C++ as a federation of languages

middot C provides a rich inheritance and interoperability as UnixWindows hellip are all build on C

middot Object-Oriented C++ (maybe just to provide endlessdiscussions about exactly what OO is or should be)

middot Templated C++ which is mighty powerful template metaprogramming unequalled in other languages

middot The Standard Template Library (STL) is a specifictemplate library which is powerful but has its ownconventions

middot C++11 (and C++14 and beyond) add enough to becalled a fifth language

NB Meyers original list of four languages appeared years before C++11 2159

Why C++

middot Mature yet currentmiddot Strong performance focus

middot You donrsquot pay for what you donrsquot usemiddot Leave no room for another language between the

machine level and C++

middot Yet also powerfully abstract and high-levelmiddot C++11 is a big deal giving us new language featuresmiddot While there are complexities Rcpp users are mostly

shielded

2259

Interface Vision

2359

Bell Labs May 1976

2459

Interface Vision

R offers us the best of both worlds

middot Compiled code withmiddot Access to proven libraries and algorithms in

CC++Fortranmiddot Extremely high performance (in both serial and parallel

modes)middot Interpreted code with

middot An accessible high-level language made for Programmingwith Data

middot An interactive workflow for data analysismiddot Support for rapid prototyping research and

experimentation

2559

Why Rcpp

middot Easy to learn as it really does not have to be thatcomplicated ndash we will see numerous few examples

middot Easy to use as it avoids build and OS system complexitiesthanks to the R infrastrucure

middot Expressive as it allows for vectorised C++ using RcppSugar

middot Seamless access to all R objects vector matrix listS3S4RefClass Environment Function hellip

middot Speed gains for a variety of tasks Rcpp excels preciselywhere R struggles loops function calls hellip

middot Extensions greatly facilitates access to external librariesusing eg Rcpp modules

2659

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 4: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

And Another Tweet

459

And Yet Another Tweet

559

And Why Not Another Tweet

659

And Last But Not Least

759

Extending R

859

Why R Programming with Data

ChambersComputationalMethods for DataAnalysis Wiley1977

Becker Chambersand Wilks TheNew S LanguageChapman amp Hall1988

Chambers andHastie StatisticalModels in SChapman amp Hall1992

ChambersProgramming withData Springer1998

ChambersSoftware for DataAnalysisProgramming withR Springer 2008

Thanks to John Chambers for sending me high-resolution scans of the covers of his books

959

A Simple Example

xx lt- faithful[eruptions]fit lt- density(xx)plot(fit)

1059

A Simple Example

1 2 3 4 5 6

00

01

02

03

04

05

densitydefault(x = xx)

N = 272 Bandwidth = 03348

Den

sity

1159

A Simple Example - Refined

xx lt- faithful[eruptions]fit1 lt- density(xx)fit2 lt- replicate(10000

x lt- sample(xxreplace=TRUE)density(x from=min(fit1$x) to=max(fit1$x))$y

)fit3 lt- apply(fit2 1 quantilec(00250975))plot(fit1 ylim=range(fit3))polygon(c(fit1$xrev(fit1$x)) c(fit3[1]rev(fit3[2]))

col=grey border=F)lines(fit1)

1259

A Simple Example - Refined

1 2 3 4 5 6

00

01

02

03

04

05

densitydefault(x = xx)

N = 272 Bandwidth = 03348

Den

sity

1359

So Why R

R enables us to

middot work interactivelymiddot explore and visualize datamiddot access retrieve andor generate datamiddot summarize and report into pdf html hellip

making it the key language for statistical computing and apreferred environment for many data analysts

1459

So Why R

R has always been extensible via

middot C via a bare-bones interface described in Writing RExtensions

middot Fortran which is also used internally by Rmiddot Java via rJava by Simon Urbanekmiddot C++ but essentially at the bare-bones level of C

So while in theory this always worked ndash it was tedious inpractice

1559

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1659

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1759

Why Extend R

Chambers proceeds with this rough map of the road ahead

middot Againstmiddot Itrsquos more workmiddot Bugs will bitemiddot Potential platform dependencymiddot Less readable software

middot In Favormiddot New and trusted computationsmiddot Speedmiddot Object references

1859

Why Extend R

The Why boils down to

middot speed Often a good enough reason for us hellip and a focusfor us in this workshop

middot new things We can bind to libraries and tools that wouldotherwise be unavailable in R

middot references Chambers quote from 2008 foreshadowed thework on Reference Classes now in R and built upon viaRcpp Modules Rcpp Classes (and also RcppR6)

1959

And Why C++

middot Asking Google leads to about ~ 50 million hitsmiddot Wikipedia C++ is a statically typed free-form

multi-paradigm compiled general-purpose powerfulprogramming language

middot C++ is industrial-strength vendor-independentwidely-used and still evolving

middot In science amp research one of the most frequently-usedlanguages If there is something you want to use connect to it probably has a CC++ API

middot As a widely used language it also has good tool support(debuggers profilers code analysis)

2059

Why C++

Scott Meyers View C++ as a federation of languages

middot C provides a rich inheritance and interoperability as UnixWindows hellip are all build on C

middot Object-Oriented C++ (maybe just to provide endlessdiscussions about exactly what OO is or should be)

middot Templated C++ which is mighty powerful template metaprogramming unequalled in other languages

middot The Standard Template Library (STL) is a specifictemplate library which is powerful but has its ownconventions

middot C++11 (and C++14 and beyond) add enough to becalled a fifth language

NB Meyers original list of four languages appeared years before C++11 2159

Why C++

middot Mature yet currentmiddot Strong performance focus

middot You donrsquot pay for what you donrsquot usemiddot Leave no room for another language between the

machine level and C++

middot Yet also powerfully abstract and high-levelmiddot C++11 is a big deal giving us new language featuresmiddot While there are complexities Rcpp users are mostly

shielded

2259

Interface Vision

2359

Bell Labs May 1976

2459

Interface Vision

R offers us the best of both worlds

middot Compiled code withmiddot Access to proven libraries and algorithms in

CC++Fortranmiddot Extremely high performance (in both serial and parallel

modes)middot Interpreted code with

middot An accessible high-level language made for Programmingwith Data

middot An interactive workflow for data analysismiddot Support for rapid prototyping research and

experimentation

2559

Why Rcpp

middot Easy to learn as it really does not have to be thatcomplicated ndash we will see numerous few examples

middot Easy to use as it avoids build and OS system complexitiesthanks to the R infrastrucure

middot Expressive as it allows for vectorised C++ using RcppSugar

middot Seamless access to all R objects vector matrix listS3S4RefClass Environment Function hellip

middot Speed gains for a variety of tasks Rcpp excels preciselywhere R struggles loops function calls hellip

middot Extensions greatly facilitates access to external librariesusing eg Rcpp modules

2659

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 5: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

And Yet Another Tweet

559

And Why Not Another Tweet

659

And Last But Not Least

759

Extending R

859

Why R Programming with Data

ChambersComputationalMethods for DataAnalysis Wiley1977

Becker Chambersand Wilks TheNew S LanguageChapman amp Hall1988

Chambers andHastie StatisticalModels in SChapman amp Hall1992

ChambersProgramming withData Springer1998

ChambersSoftware for DataAnalysisProgramming withR Springer 2008

Thanks to John Chambers for sending me high-resolution scans of the covers of his books

959

A Simple Example

xx lt- faithful[eruptions]fit lt- density(xx)plot(fit)

1059

A Simple Example

1 2 3 4 5 6

00

01

02

03

04

05

densitydefault(x = xx)

N = 272 Bandwidth = 03348

Den

sity

1159

A Simple Example - Refined

xx lt- faithful[eruptions]fit1 lt- density(xx)fit2 lt- replicate(10000

x lt- sample(xxreplace=TRUE)density(x from=min(fit1$x) to=max(fit1$x))$y

)fit3 lt- apply(fit2 1 quantilec(00250975))plot(fit1 ylim=range(fit3))polygon(c(fit1$xrev(fit1$x)) c(fit3[1]rev(fit3[2]))

col=grey border=F)lines(fit1)

1259

A Simple Example - Refined

1 2 3 4 5 6

00

01

02

03

04

05

densitydefault(x = xx)

N = 272 Bandwidth = 03348

Den

sity

1359

So Why R

R enables us to

middot work interactivelymiddot explore and visualize datamiddot access retrieve andor generate datamiddot summarize and report into pdf html hellip

making it the key language for statistical computing and apreferred environment for many data analysts

1459

So Why R

R has always been extensible via

middot C via a bare-bones interface described in Writing RExtensions

middot Fortran which is also used internally by Rmiddot Java via rJava by Simon Urbanekmiddot C++ but essentially at the bare-bones level of C

So while in theory this always worked ndash it was tedious inpractice

1559

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1659

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1759

Why Extend R

Chambers proceeds with this rough map of the road ahead

middot Againstmiddot Itrsquos more workmiddot Bugs will bitemiddot Potential platform dependencymiddot Less readable software

middot In Favormiddot New and trusted computationsmiddot Speedmiddot Object references

1859

Why Extend R

The Why boils down to

middot speed Often a good enough reason for us hellip and a focusfor us in this workshop

middot new things We can bind to libraries and tools that wouldotherwise be unavailable in R

middot references Chambers quote from 2008 foreshadowed thework on Reference Classes now in R and built upon viaRcpp Modules Rcpp Classes (and also RcppR6)

1959

And Why C++

middot Asking Google leads to about ~ 50 million hitsmiddot Wikipedia C++ is a statically typed free-form

multi-paradigm compiled general-purpose powerfulprogramming language

middot C++ is industrial-strength vendor-independentwidely-used and still evolving

middot In science amp research one of the most frequently-usedlanguages If there is something you want to use connect to it probably has a CC++ API

middot As a widely used language it also has good tool support(debuggers profilers code analysis)

2059

Why C++

Scott Meyers View C++ as a federation of languages

middot C provides a rich inheritance and interoperability as UnixWindows hellip are all build on C

middot Object-Oriented C++ (maybe just to provide endlessdiscussions about exactly what OO is or should be)

middot Templated C++ which is mighty powerful template metaprogramming unequalled in other languages

middot The Standard Template Library (STL) is a specifictemplate library which is powerful but has its ownconventions

middot C++11 (and C++14 and beyond) add enough to becalled a fifth language

NB Meyers original list of four languages appeared years before C++11 2159

Why C++

middot Mature yet currentmiddot Strong performance focus

middot You donrsquot pay for what you donrsquot usemiddot Leave no room for another language between the

machine level and C++

middot Yet also powerfully abstract and high-levelmiddot C++11 is a big deal giving us new language featuresmiddot While there are complexities Rcpp users are mostly

shielded

2259

Interface Vision

2359

Bell Labs May 1976

2459

Interface Vision

R offers us the best of both worlds

middot Compiled code withmiddot Access to proven libraries and algorithms in

CC++Fortranmiddot Extremely high performance (in both serial and parallel

modes)middot Interpreted code with

middot An accessible high-level language made for Programmingwith Data

middot An interactive workflow for data analysismiddot Support for rapid prototyping research and

experimentation

2559

Why Rcpp

middot Easy to learn as it really does not have to be thatcomplicated ndash we will see numerous few examples

middot Easy to use as it avoids build and OS system complexitiesthanks to the R infrastrucure

middot Expressive as it allows for vectorised C++ using RcppSugar

middot Seamless access to all R objects vector matrix listS3S4RefClass Environment Function hellip

middot Speed gains for a variety of tasks Rcpp excels preciselywhere R struggles loops function calls hellip

middot Extensions greatly facilitates access to external librariesusing eg Rcpp modules

2659

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 6: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

And Why Not Another Tweet

659

And Last But Not Least

759

Extending R

859

Why R Programming with Data

ChambersComputationalMethods for DataAnalysis Wiley1977

Becker Chambersand Wilks TheNew S LanguageChapman amp Hall1988

Chambers andHastie StatisticalModels in SChapman amp Hall1992

ChambersProgramming withData Springer1998

ChambersSoftware for DataAnalysisProgramming withR Springer 2008

Thanks to John Chambers for sending me high-resolution scans of the covers of his books

959

A Simple Example

xx lt- faithful[eruptions]fit lt- density(xx)plot(fit)

1059

A Simple Example

1 2 3 4 5 6

00

01

02

03

04

05

densitydefault(x = xx)

N = 272 Bandwidth = 03348

Den

sity

1159

A Simple Example - Refined

xx lt- faithful[eruptions]fit1 lt- density(xx)fit2 lt- replicate(10000

x lt- sample(xxreplace=TRUE)density(x from=min(fit1$x) to=max(fit1$x))$y

)fit3 lt- apply(fit2 1 quantilec(00250975))plot(fit1 ylim=range(fit3))polygon(c(fit1$xrev(fit1$x)) c(fit3[1]rev(fit3[2]))

col=grey border=F)lines(fit1)

1259

A Simple Example - Refined

1 2 3 4 5 6

00

01

02

03

04

05

densitydefault(x = xx)

N = 272 Bandwidth = 03348

Den

sity

1359

So Why R

R enables us to

middot work interactivelymiddot explore and visualize datamiddot access retrieve andor generate datamiddot summarize and report into pdf html hellip

making it the key language for statistical computing and apreferred environment for many data analysts

1459

So Why R

R has always been extensible via

middot C via a bare-bones interface described in Writing RExtensions

middot Fortran which is also used internally by Rmiddot Java via rJava by Simon Urbanekmiddot C++ but essentially at the bare-bones level of C

So while in theory this always worked ndash it was tedious inpractice

1559

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1659

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1759

Why Extend R

Chambers proceeds with this rough map of the road ahead

middot Againstmiddot Itrsquos more workmiddot Bugs will bitemiddot Potential platform dependencymiddot Less readable software

middot In Favormiddot New and trusted computationsmiddot Speedmiddot Object references

1859

Why Extend R

The Why boils down to

middot speed Often a good enough reason for us hellip and a focusfor us in this workshop

middot new things We can bind to libraries and tools that wouldotherwise be unavailable in R

middot references Chambers quote from 2008 foreshadowed thework on Reference Classes now in R and built upon viaRcpp Modules Rcpp Classes (and also RcppR6)

1959

And Why C++

middot Asking Google leads to about ~ 50 million hitsmiddot Wikipedia C++ is a statically typed free-form

multi-paradigm compiled general-purpose powerfulprogramming language

middot C++ is industrial-strength vendor-independentwidely-used and still evolving

middot In science amp research one of the most frequently-usedlanguages If there is something you want to use connect to it probably has a CC++ API

middot As a widely used language it also has good tool support(debuggers profilers code analysis)

2059

Why C++

Scott Meyers View C++ as a federation of languages

middot C provides a rich inheritance and interoperability as UnixWindows hellip are all build on C

middot Object-Oriented C++ (maybe just to provide endlessdiscussions about exactly what OO is or should be)

middot Templated C++ which is mighty powerful template metaprogramming unequalled in other languages

middot The Standard Template Library (STL) is a specifictemplate library which is powerful but has its ownconventions

middot C++11 (and C++14 and beyond) add enough to becalled a fifth language

NB Meyers original list of four languages appeared years before C++11 2159

Why C++

middot Mature yet currentmiddot Strong performance focus

middot You donrsquot pay for what you donrsquot usemiddot Leave no room for another language between the

machine level and C++

middot Yet also powerfully abstract and high-levelmiddot C++11 is a big deal giving us new language featuresmiddot While there are complexities Rcpp users are mostly

shielded

2259

Interface Vision

2359

Bell Labs May 1976

2459

Interface Vision

R offers us the best of both worlds

middot Compiled code withmiddot Access to proven libraries and algorithms in

CC++Fortranmiddot Extremely high performance (in both serial and parallel

modes)middot Interpreted code with

middot An accessible high-level language made for Programmingwith Data

middot An interactive workflow for data analysismiddot Support for rapid prototyping research and

experimentation

2559

Why Rcpp

middot Easy to learn as it really does not have to be thatcomplicated ndash we will see numerous few examples

middot Easy to use as it avoids build and OS system complexitiesthanks to the R infrastrucure

middot Expressive as it allows for vectorised C++ using RcppSugar

middot Seamless access to all R objects vector matrix listS3S4RefClass Environment Function hellip

middot Speed gains for a variety of tasks Rcpp excels preciselywhere R struggles loops function calls hellip

middot Extensions greatly facilitates access to external librariesusing eg Rcpp modules

2659

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 7: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

And Last But Not Least

759

Extending R

859

Why R Programming with Data

ChambersComputationalMethods for DataAnalysis Wiley1977

Becker Chambersand Wilks TheNew S LanguageChapman amp Hall1988

Chambers andHastie StatisticalModels in SChapman amp Hall1992

ChambersProgramming withData Springer1998

ChambersSoftware for DataAnalysisProgramming withR Springer 2008

Thanks to John Chambers for sending me high-resolution scans of the covers of his books

959

A Simple Example

xx lt- faithful[eruptions]fit lt- density(xx)plot(fit)

1059

A Simple Example

1 2 3 4 5 6

00

01

02

03

04

05

densitydefault(x = xx)

N = 272 Bandwidth = 03348

Den

sity

1159

A Simple Example - Refined

xx lt- faithful[eruptions]fit1 lt- density(xx)fit2 lt- replicate(10000

x lt- sample(xxreplace=TRUE)density(x from=min(fit1$x) to=max(fit1$x))$y

)fit3 lt- apply(fit2 1 quantilec(00250975))plot(fit1 ylim=range(fit3))polygon(c(fit1$xrev(fit1$x)) c(fit3[1]rev(fit3[2]))

col=grey border=F)lines(fit1)

1259

A Simple Example - Refined

1 2 3 4 5 6

00

01

02

03

04

05

densitydefault(x = xx)

N = 272 Bandwidth = 03348

Den

sity

1359

So Why R

R enables us to

middot work interactivelymiddot explore and visualize datamiddot access retrieve andor generate datamiddot summarize and report into pdf html hellip

making it the key language for statistical computing and apreferred environment for many data analysts

1459

So Why R

R has always been extensible via

middot C via a bare-bones interface described in Writing RExtensions

middot Fortran which is also used internally by Rmiddot Java via rJava by Simon Urbanekmiddot C++ but essentially at the bare-bones level of C

So while in theory this always worked ndash it was tedious inpractice

1559

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1659

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1759

Why Extend R

Chambers proceeds with this rough map of the road ahead

middot Againstmiddot Itrsquos more workmiddot Bugs will bitemiddot Potential platform dependencymiddot Less readable software

middot In Favormiddot New and trusted computationsmiddot Speedmiddot Object references

1859

Why Extend R

The Why boils down to

middot speed Often a good enough reason for us hellip and a focusfor us in this workshop

middot new things We can bind to libraries and tools that wouldotherwise be unavailable in R

middot references Chambers quote from 2008 foreshadowed thework on Reference Classes now in R and built upon viaRcpp Modules Rcpp Classes (and also RcppR6)

1959

And Why C++

middot Asking Google leads to about ~ 50 million hitsmiddot Wikipedia C++ is a statically typed free-form

multi-paradigm compiled general-purpose powerfulprogramming language

middot C++ is industrial-strength vendor-independentwidely-used and still evolving

middot In science amp research one of the most frequently-usedlanguages If there is something you want to use connect to it probably has a CC++ API

middot As a widely used language it also has good tool support(debuggers profilers code analysis)

2059

Why C++

Scott Meyers View C++ as a federation of languages

middot C provides a rich inheritance and interoperability as UnixWindows hellip are all build on C

middot Object-Oriented C++ (maybe just to provide endlessdiscussions about exactly what OO is or should be)

middot Templated C++ which is mighty powerful template metaprogramming unequalled in other languages

middot The Standard Template Library (STL) is a specifictemplate library which is powerful but has its ownconventions

middot C++11 (and C++14 and beyond) add enough to becalled a fifth language

NB Meyers original list of four languages appeared years before C++11 2159

Why C++

middot Mature yet currentmiddot Strong performance focus

middot You donrsquot pay for what you donrsquot usemiddot Leave no room for another language between the

machine level and C++

middot Yet also powerfully abstract and high-levelmiddot C++11 is a big deal giving us new language featuresmiddot While there are complexities Rcpp users are mostly

shielded

2259

Interface Vision

2359

Bell Labs May 1976

2459

Interface Vision

R offers us the best of both worlds

middot Compiled code withmiddot Access to proven libraries and algorithms in

CC++Fortranmiddot Extremely high performance (in both serial and parallel

modes)middot Interpreted code with

middot An accessible high-level language made for Programmingwith Data

middot An interactive workflow for data analysismiddot Support for rapid prototyping research and

experimentation

2559

Why Rcpp

middot Easy to learn as it really does not have to be thatcomplicated ndash we will see numerous few examples

middot Easy to use as it avoids build and OS system complexitiesthanks to the R infrastrucure

middot Expressive as it allows for vectorised C++ using RcppSugar

middot Seamless access to all R objects vector matrix listS3S4RefClass Environment Function hellip

middot Speed gains for a variety of tasks Rcpp excels preciselywhere R struggles loops function calls hellip

middot Extensions greatly facilitates access to external librariesusing eg Rcpp modules

2659

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 8: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Extending R

859

Why R Programming with Data

ChambersComputationalMethods for DataAnalysis Wiley1977

Becker Chambersand Wilks TheNew S LanguageChapman amp Hall1988

Chambers andHastie StatisticalModels in SChapman amp Hall1992

ChambersProgramming withData Springer1998

ChambersSoftware for DataAnalysisProgramming withR Springer 2008

Thanks to John Chambers for sending me high-resolution scans of the covers of his books

959

A Simple Example

xx lt- faithful[eruptions]fit lt- density(xx)plot(fit)

1059

A Simple Example

1 2 3 4 5 6

00

01

02

03

04

05

densitydefault(x = xx)

N = 272 Bandwidth = 03348

Den

sity

1159

A Simple Example - Refined

xx lt- faithful[eruptions]fit1 lt- density(xx)fit2 lt- replicate(10000

x lt- sample(xxreplace=TRUE)density(x from=min(fit1$x) to=max(fit1$x))$y

)fit3 lt- apply(fit2 1 quantilec(00250975))plot(fit1 ylim=range(fit3))polygon(c(fit1$xrev(fit1$x)) c(fit3[1]rev(fit3[2]))

col=grey border=F)lines(fit1)

1259

A Simple Example - Refined

1 2 3 4 5 6

00

01

02

03

04

05

densitydefault(x = xx)

N = 272 Bandwidth = 03348

Den

sity

1359

So Why R

R enables us to

middot work interactivelymiddot explore and visualize datamiddot access retrieve andor generate datamiddot summarize and report into pdf html hellip

making it the key language for statistical computing and apreferred environment for many data analysts

1459

So Why R

R has always been extensible via

middot C via a bare-bones interface described in Writing RExtensions

middot Fortran which is also used internally by Rmiddot Java via rJava by Simon Urbanekmiddot C++ but essentially at the bare-bones level of C

So while in theory this always worked ndash it was tedious inpractice

1559

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1659

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1759

Why Extend R

Chambers proceeds with this rough map of the road ahead

middot Againstmiddot Itrsquos more workmiddot Bugs will bitemiddot Potential platform dependencymiddot Less readable software

middot In Favormiddot New and trusted computationsmiddot Speedmiddot Object references

1859

Why Extend R

The Why boils down to

middot speed Often a good enough reason for us hellip and a focusfor us in this workshop

middot new things We can bind to libraries and tools that wouldotherwise be unavailable in R

middot references Chambers quote from 2008 foreshadowed thework on Reference Classes now in R and built upon viaRcpp Modules Rcpp Classes (and also RcppR6)

1959

And Why C++

middot Asking Google leads to about ~ 50 million hitsmiddot Wikipedia C++ is a statically typed free-form

multi-paradigm compiled general-purpose powerfulprogramming language

middot C++ is industrial-strength vendor-independentwidely-used and still evolving

middot In science amp research one of the most frequently-usedlanguages If there is something you want to use connect to it probably has a CC++ API

middot As a widely used language it also has good tool support(debuggers profilers code analysis)

2059

Why C++

Scott Meyers View C++ as a federation of languages

middot C provides a rich inheritance and interoperability as UnixWindows hellip are all build on C

middot Object-Oriented C++ (maybe just to provide endlessdiscussions about exactly what OO is or should be)

middot Templated C++ which is mighty powerful template metaprogramming unequalled in other languages

middot The Standard Template Library (STL) is a specifictemplate library which is powerful but has its ownconventions

middot C++11 (and C++14 and beyond) add enough to becalled a fifth language

NB Meyers original list of four languages appeared years before C++11 2159

Why C++

middot Mature yet currentmiddot Strong performance focus

middot You donrsquot pay for what you donrsquot usemiddot Leave no room for another language between the

machine level and C++

middot Yet also powerfully abstract and high-levelmiddot C++11 is a big deal giving us new language featuresmiddot While there are complexities Rcpp users are mostly

shielded

2259

Interface Vision

2359

Bell Labs May 1976

2459

Interface Vision

R offers us the best of both worlds

middot Compiled code withmiddot Access to proven libraries and algorithms in

CC++Fortranmiddot Extremely high performance (in both serial and parallel

modes)middot Interpreted code with

middot An accessible high-level language made for Programmingwith Data

middot An interactive workflow for data analysismiddot Support for rapid prototyping research and

experimentation

2559

Why Rcpp

middot Easy to learn as it really does not have to be thatcomplicated ndash we will see numerous few examples

middot Easy to use as it avoids build and OS system complexitiesthanks to the R infrastrucure

middot Expressive as it allows for vectorised C++ using RcppSugar

middot Seamless access to all R objects vector matrix listS3S4RefClass Environment Function hellip

middot Speed gains for a variety of tasks Rcpp excels preciselywhere R struggles loops function calls hellip

middot Extensions greatly facilitates access to external librariesusing eg Rcpp modules

2659

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 9: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Why R Programming with Data

ChambersComputationalMethods for DataAnalysis Wiley1977

Becker Chambersand Wilks TheNew S LanguageChapman amp Hall1988

Chambers andHastie StatisticalModels in SChapman amp Hall1992

ChambersProgramming withData Springer1998

ChambersSoftware for DataAnalysisProgramming withR Springer 2008

Thanks to John Chambers for sending me high-resolution scans of the covers of his books

959

A Simple Example

xx lt- faithful[eruptions]fit lt- density(xx)plot(fit)

1059

A Simple Example

1 2 3 4 5 6

00

01

02

03

04

05

densitydefault(x = xx)

N = 272 Bandwidth = 03348

Den

sity

1159

A Simple Example - Refined

xx lt- faithful[eruptions]fit1 lt- density(xx)fit2 lt- replicate(10000

x lt- sample(xxreplace=TRUE)density(x from=min(fit1$x) to=max(fit1$x))$y

)fit3 lt- apply(fit2 1 quantilec(00250975))plot(fit1 ylim=range(fit3))polygon(c(fit1$xrev(fit1$x)) c(fit3[1]rev(fit3[2]))

col=grey border=F)lines(fit1)

1259

A Simple Example - Refined

1 2 3 4 5 6

00

01

02

03

04

05

densitydefault(x = xx)

N = 272 Bandwidth = 03348

Den

sity

1359

So Why R

R enables us to

middot work interactivelymiddot explore and visualize datamiddot access retrieve andor generate datamiddot summarize and report into pdf html hellip

making it the key language for statistical computing and apreferred environment for many data analysts

1459

So Why R

R has always been extensible via

middot C via a bare-bones interface described in Writing RExtensions

middot Fortran which is also used internally by Rmiddot Java via rJava by Simon Urbanekmiddot C++ but essentially at the bare-bones level of C

So while in theory this always worked ndash it was tedious inpractice

1559

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1659

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1759

Why Extend R

Chambers proceeds with this rough map of the road ahead

middot Againstmiddot Itrsquos more workmiddot Bugs will bitemiddot Potential platform dependencymiddot Less readable software

middot In Favormiddot New and trusted computationsmiddot Speedmiddot Object references

1859

Why Extend R

The Why boils down to

middot speed Often a good enough reason for us hellip and a focusfor us in this workshop

middot new things We can bind to libraries and tools that wouldotherwise be unavailable in R

middot references Chambers quote from 2008 foreshadowed thework on Reference Classes now in R and built upon viaRcpp Modules Rcpp Classes (and also RcppR6)

1959

And Why C++

middot Asking Google leads to about ~ 50 million hitsmiddot Wikipedia C++ is a statically typed free-form

multi-paradigm compiled general-purpose powerfulprogramming language

middot C++ is industrial-strength vendor-independentwidely-used and still evolving

middot In science amp research one of the most frequently-usedlanguages If there is something you want to use connect to it probably has a CC++ API

middot As a widely used language it also has good tool support(debuggers profilers code analysis)

2059

Why C++

Scott Meyers View C++ as a federation of languages

middot C provides a rich inheritance and interoperability as UnixWindows hellip are all build on C

middot Object-Oriented C++ (maybe just to provide endlessdiscussions about exactly what OO is or should be)

middot Templated C++ which is mighty powerful template metaprogramming unequalled in other languages

middot The Standard Template Library (STL) is a specifictemplate library which is powerful but has its ownconventions

middot C++11 (and C++14 and beyond) add enough to becalled a fifth language

NB Meyers original list of four languages appeared years before C++11 2159

Why C++

middot Mature yet currentmiddot Strong performance focus

middot You donrsquot pay for what you donrsquot usemiddot Leave no room for another language between the

machine level and C++

middot Yet also powerfully abstract and high-levelmiddot C++11 is a big deal giving us new language featuresmiddot While there are complexities Rcpp users are mostly

shielded

2259

Interface Vision

2359

Bell Labs May 1976

2459

Interface Vision

R offers us the best of both worlds

middot Compiled code withmiddot Access to proven libraries and algorithms in

CC++Fortranmiddot Extremely high performance (in both serial and parallel

modes)middot Interpreted code with

middot An accessible high-level language made for Programmingwith Data

middot An interactive workflow for data analysismiddot Support for rapid prototyping research and

experimentation

2559

Why Rcpp

middot Easy to learn as it really does not have to be thatcomplicated ndash we will see numerous few examples

middot Easy to use as it avoids build and OS system complexitiesthanks to the R infrastrucure

middot Expressive as it allows for vectorised C++ using RcppSugar

middot Seamless access to all R objects vector matrix listS3S4RefClass Environment Function hellip

middot Speed gains for a variety of tasks Rcpp excels preciselywhere R struggles loops function calls hellip

middot Extensions greatly facilitates access to external librariesusing eg Rcpp modules

2659

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 10: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

A Simple Example

xx lt- faithful[eruptions]fit lt- density(xx)plot(fit)

1059

A Simple Example

1 2 3 4 5 6

00

01

02

03

04

05

densitydefault(x = xx)

N = 272 Bandwidth = 03348

Den

sity

1159

A Simple Example - Refined

xx lt- faithful[eruptions]fit1 lt- density(xx)fit2 lt- replicate(10000

x lt- sample(xxreplace=TRUE)density(x from=min(fit1$x) to=max(fit1$x))$y

)fit3 lt- apply(fit2 1 quantilec(00250975))plot(fit1 ylim=range(fit3))polygon(c(fit1$xrev(fit1$x)) c(fit3[1]rev(fit3[2]))

col=grey border=F)lines(fit1)

1259

A Simple Example - Refined

1 2 3 4 5 6

00

01

02

03

04

05

densitydefault(x = xx)

N = 272 Bandwidth = 03348

Den

sity

1359

So Why R

R enables us to

middot work interactivelymiddot explore and visualize datamiddot access retrieve andor generate datamiddot summarize and report into pdf html hellip

making it the key language for statistical computing and apreferred environment for many data analysts

1459

So Why R

R has always been extensible via

middot C via a bare-bones interface described in Writing RExtensions

middot Fortran which is also used internally by Rmiddot Java via rJava by Simon Urbanekmiddot C++ but essentially at the bare-bones level of C

So while in theory this always worked ndash it was tedious inpractice

1559

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1659

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1759

Why Extend R

Chambers proceeds with this rough map of the road ahead

middot Againstmiddot Itrsquos more workmiddot Bugs will bitemiddot Potential platform dependencymiddot Less readable software

middot In Favormiddot New and trusted computationsmiddot Speedmiddot Object references

1859

Why Extend R

The Why boils down to

middot speed Often a good enough reason for us hellip and a focusfor us in this workshop

middot new things We can bind to libraries and tools that wouldotherwise be unavailable in R

middot references Chambers quote from 2008 foreshadowed thework on Reference Classes now in R and built upon viaRcpp Modules Rcpp Classes (and also RcppR6)

1959

And Why C++

middot Asking Google leads to about ~ 50 million hitsmiddot Wikipedia C++ is a statically typed free-form

multi-paradigm compiled general-purpose powerfulprogramming language

middot C++ is industrial-strength vendor-independentwidely-used and still evolving

middot In science amp research one of the most frequently-usedlanguages If there is something you want to use connect to it probably has a CC++ API

middot As a widely used language it also has good tool support(debuggers profilers code analysis)

2059

Why C++

Scott Meyers View C++ as a federation of languages

middot C provides a rich inheritance and interoperability as UnixWindows hellip are all build on C

middot Object-Oriented C++ (maybe just to provide endlessdiscussions about exactly what OO is or should be)

middot Templated C++ which is mighty powerful template metaprogramming unequalled in other languages

middot The Standard Template Library (STL) is a specifictemplate library which is powerful but has its ownconventions

middot C++11 (and C++14 and beyond) add enough to becalled a fifth language

NB Meyers original list of four languages appeared years before C++11 2159

Why C++

middot Mature yet currentmiddot Strong performance focus

middot You donrsquot pay for what you donrsquot usemiddot Leave no room for another language between the

machine level and C++

middot Yet also powerfully abstract and high-levelmiddot C++11 is a big deal giving us new language featuresmiddot While there are complexities Rcpp users are mostly

shielded

2259

Interface Vision

2359

Bell Labs May 1976

2459

Interface Vision

R offers us the best of both worlds

middot Compiled code withmiddot Access to proven libraries and algorithms in

CC++Fortranmiddot Extremely high performance (in both serial and parallel

modes)middot Interpreted code with

middot An accessible high-level language made for Programmingwith Data

middot An interactive workflow for data analysismiddot Support for rapid prototyping research and

experimentation

2559

Why Rcpp

middot Easy to learn as it really does not have to be thatcomplicated ndash we will see numerous few examples

middot Easy to use as it avoids build and OS system complexitiesthanks to the R infrastrucure

middot Expressive as it allows for vectorised C++ using RcppSugar

middot Seamless access to all R objects vector matrix listS3S4RefClass Environment Function hellip

middot Speed gains for a variety of tasks Rcpp excels preciselywhere R struggles loops function calls hellip

middot Extensions greatly facilitates access to external librariesusing eg Rcpp modules

2659

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 11: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

A Simple Example

1 2 3 4 5 6

00

01

02

03

04

05

densitydefault(x = xx)

N = 272 Bandwidth = 03348

Den

sity

1159

A Simple Example - Refined

xx lt- faithful[eruptions]fit1 lt- density(xx)fit2 lt- replicate(10000

x lt- sample(xxreplace=TRUE)density(x from=min(fit1$x) to=max(fit1$x))$y

)fit3 lt- apply(fit2 1 quantilec(00250975))plot(fit1 ylim=range(fit3))polygon(c(fit1$xrev(fit1$x)) c(fit3[1]rev(fit3[2]))

col=grey border=F)lines(fit1)

1259

A Simple Example - Refined

1 2 3 4 5 6

00

01

02

03

04

05

densitydefault(x = xx)

N = 272 Bandwidth = 03348

Den

sity

1359

So Why R

R enables us to

middot work interactivelymiddot explore and visualize datamiddot access retrieve andor generate datamiddot summarize and report into pdf html hellip

making it the key language for statistical computing and apreferred environment for many data analysts

1459

So Why R

R has always been extensible via

middot C via a bare-bones interface described in Writing RExtensions

middot Fortran which is also used internally by Rmiddot Java via rJava by Simon Urbanekmiddot C++ but essentially at the bare-bones level of C

So while in theory this always worked ndash it was tedious inpractice

1559

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1659

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1759

Why Extend R

Chambers proceeds with this rough map of the road ahead

middot Againstmiddot Itrsquos more workmiddot Bugs will bitemiddot Potential platform dependencymiddot Less readable software

middot In Favormiddot New and trusted computationsmiddot Speedmiddot Object references

1859

Why Extend R

The Why boils down to

middot speed Often a good enough reason for us hellip and a focusfor us in this workshop

middot new things We can bind to libraries and tools that wouldotherwise be unavailable in R

middot references Chambers quote from 2008 foreshadowed thework on Reference Classes now in R and built upon viaRcpp Modules Rcpp Classes (and also RcppR6)

1959

And Why C++

middot Asking Google leads to about ~ 50 million hitsmiddot Wikipedia C++ is a statically typed free-form

multi-paradigm compiled general-purpose powerfulprogramming language

middot C++ is industrial-strength vendor-independentwidely-used and still evolving

middot In science amp research one of the most frequently-usedlanguages If there is something you want to use connect to it probably has a CC++ API

middot As a widely used language it also has good tool support(debuggers profilers code analysis)

2059

Why C++

Scott Meyers View C++ as a federation of languages

middot C provides a rich inheritance and interoperability as UnixWindows hellip are all build on C

middot Object-Oriented C++ (maybe just to provide endlessdiscussions about exactly what OO is or should be)

middot Templated C++ which is mighty powerful template metaprogramming unequalled in other languages

middot The Standard Template Library (STL) is a specifictemplate library which is powerful but has its ownconventions

middot C++11 (and C++14 and beyond) add enough to becalled a fifth language

NB Meyers original list of four languages appeared years before C++11 2159

Why C++

middot Mature yet currentmiddot Strong performance focus

middot You donrsquot pay for what you donrsquot usemiddot Leave no room for another language between the

machine level and C++

middot Yet also powerfully abstract and high-levelmiddot C++11 is a big deal giving us new language featuresmiddot While there are complexities Rcpp users are mostly

shielded

2259

Interface Vision

2359

Bell Labs May 1976

2459

Interface Vision

R offers us the best of both worlds

middot Compiled code withmiddot Access to proven libraries and algorithms in

CC++Fortranmiddot Extremely high performance (in both serial and parallel

modes)middot Interpreted code with

middot An accessible high-level language made for Programmingwith Data

middot An interactive workflow for data analysismiddot Support for rapid prototyping research and

experimentation

2559

Why Rcpp

middot Easy to learn as it really does not have to be thatcomplicated ndash we will see numerous few examples

middot Easy to use as it avoids build and OS system complexitiesthanks to the R infrastrucure

middot Expressive as it allows for vectorised C++ using RcppSugar

middot Seamless access to all R objects vector matrix listS3S4RefClass Environment Function hellip

middot Speed gains for a variety of tasks Rcpp excels preciselywhere R struggles loops function calls hellip

middot Extensions greatly facilitates access to external librariesusing eg Rcpp modules

2659

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 12: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

A Simple Example - Refined

xx lt- faithful[eruptions]fit1 lt- density(xx)fit2 lt- replicate(10000

x lt- sample(xxreplace=TRUE)density(x from=min(fit1$x) to=max(fit1$x))$y

)fit3 lt- apply(fit2 1 quantilec(00250975))plot(fit1 ylim=range(fit3))polygon(c(fit1$xrev(fit1$x)) c(fit3[1]rev(fit3[2]))

col=grey border=F)lines(fit1)

1259

A Simple Example - Refined

1 2 3 4 5 6

00

01

02

03

04

05

densitydefault(x = xx)

N = 272 Bandwidth = 03348

Den

sity

1359

So Why R

R enables us to

middot work interactivelymiddot explore and visualize datamiddot access retrieve andor generate datamiddot summarize and report into pdf html hellip

making it the key language for statistical computing and apreferred environment for many data analysts

1459

So Why R

R has always been extensible via

middot C via a bare-bones interface described in Writing RExtensions

middot Fortran which is also used internally by Rmiddot Java via rJava by Simon Urbanekmiddot C++ but essentially at the bare-bones level of C

So while in theory this always worked ndash it was tedious inpractice

1559

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1659

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1759

Why Extend R

Chambers proceeds with this rough map of the road ahead

middot Againstmiddot Itrsquos more workmiddot Bugs will bitemiddot Potential platform dependencymiddot Less readable software

middot In Favormiddot New and trusted computationsmiddot Speedmiddot Object references

1859

Why Extend R

The Why boils down to

middot speed Often a good enough reason for us hellip and a focusfor us in this workshop

middot new things We can bind to libraries and tools that wouldotherwise be unavailable in R

middot references Chambers quote from 2008 foreshadowed thework on Reference Classes now in R and built upon viaRcpp Modules Rcpp Classes (and also RcppR6)

1959

And Why C++

middot Asking Google leads to about ~ 50 million hitsmiddot Wikipedia C++ is a statically typed free-form

multi-paradigm compiled general-purpose powerfulprogramming language

middot C++ is industrial-strength vendor-independentwidely-used and still evolving

middot In science amp research one of the most frequently-usedlanguages If there is something you want to use connect to it probably has a CC++ API

middot As a widely used language it also has good tool support(debuggers profilers code analysis)

2059

Why C++

Scott Meyers View C++ as a federation of languages

middot C provides a rich inheritance and interoperability as UnixWindows hellip are all build on C

middot Object-Oriented C++ (maybe just to provide endlessdiscussions about exactly what OO is or should be)

middot Templated C++ which is mighty powerful template metaprogramming unequalled in other languages

middot The Standard Template Library (STL) is a specifictemplate library which is powerful but has its ownconventions

middot C++11 (and C++14 and beyond) add enough to becalled a fifth language

NB Meyers original list of four languages appeared years before C++11 2159

Why C++

middot Mature yet currentmiddot Strong performance focus

middot You donrsquot pay for what you donrsquot usemiddot Leave no room for another language between the

machine level and C++

middot Yet also powerfully abstract and high-levelmiddot C++11 is a big deal giving us new language featuresmiddot While there are complexities Rcpp users are mostly

shielded

2259

Interface Vision

2359

Bell Labs May 1976

2459

Interface Vision

R offers us the best of both worlds

middot Compiled code withmiddot Access to proven libraries and algorithms in

CC++Fortranmiddot Extremely high performance (in both serial and parallel

modes)middot Interpreted code with

middot An accessible high-level language made for Programmingwith Data

middot An interactive workflow for data analysismiddot Support for rapid prototyping research and

experimentation

2559

Why Rcpp

middot Easy to learn as it really does not have to be thatcomplicated ndash we will see numerous few examples

middot Easy to use as it avoids build and OS system complexitiesthanks to the R infrastrucure

middot Expressive as it allows for vectorised C++ using RcppSugar

middot Seamless access to all R objects vector matrix listS3S4RefClass Environment Function hellip

middot Speed gains for a variety of tasks Rcpp excels preciselywhere R struggles loops function calls hellip

middot Extensions greatly facilitates access to external librariesusing eg Rcpp modules

2659

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 13: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

A Simple Example - Refined

1 2 3 4 5 6

00

01

02

03

04

05

densitydefault(x = xx)

N = 272 Bandwidth = 03348

Den

sity

1359

So Why R

R enables us to

middot work interactivelymiddot explore and visualize datamiddot access retrieve andor generate datamiddot summarize and report into pdf html hellip

making it the key language for statistical computing and apreferred environment for many data analysts

1459

So Why R

R has always been extensible via

middot C via a bare-bones interface described in Writing RExtensions

middot Fortran which is also used internally by Rmiddot Java via rJava by Simon Urbanekmiddot C++ but essentially at the bare-bones level of C

So while in theory this always worked ndash it was tedious inpractice

1559

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1659

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1759

Why Extend R

Chambers proceeds with this rough map of the road ahead

middot Againstmiddot Itrsquos more workmiddot Bugs will bitemiddot Potential platform dependencymiddot Less readable software

middot In Favormiddot New and trusted computationsmiddot Speedmiddot Object references

1859

Why Extend R

The Why boils down to

middot speed Often a good enough reason for us hellip and a focusfor us in this workshop

middot new things We can bind to libraries and tools that wouldotherwise be unavailable in R

middot references Chambers quote from 2008 foreshadowed thework on Reference Classes now in R and built upon viaRcpp Modules Rcpp Classes (and also RcppR6)

1959

And Why C++

middot Asking Google leads to about ~ 50 million hitsmiddot Wikipedia C++ is a statically typed free-form

multi-paradigm compiled general-purpose powerfulprogramming language

middot C++ is industrial-strength vendor-independentwidely-used and still evolving

middot In science amp research one of the most frequently-usedlanguages If there is something you want to use connect to it probably has a CC++ API

middot As a widely used language it also has good tool support(debuggers profilers code analysis)

2059

Why C++

Scott Meyers View C++ as a federation of languages

middot C provides a rich inheritance and interoperability as UnixWindows hellip are all build on C

middot Object-Oriented C++ (maybe just to provide endlessdiscussions about exactly what OO is or should be)

middot Templated C++ which is mighty powerful template metaprogramming unequalled in other languages

middot The Standard Template Library (STL) is a specifictemplate library which is powerful but has its ownconventions

middot C++11 (and C++14 and beyond) add enough to becalled a fifth language

NB Meyers original list of four languages appeared years before C++11 2159

Why C++

middot Mature yet currentmiddot Strong performance focus

middot You donrsquot pay for what you donrsquot usemiddot Leave no room for another language between the

machine level and C++

middot Yet also powerfully abstract and high-levelmiddot C++11 is a big deal giving us new language featuresmiddot While there are complexities Rcpp users are mostly

shielded

2259

Interface Vision

2359

Bell Labs May 1976

2459

Interface Vision

R offers us the best of both worlds

middot Compiled code withmiddot Access to proven libraries and algorithms in

CC++Fortranmiddot Extremely high performance (in both serial and parallel

modes)middot Interpreted code with

middot An accessible high-level language made for Programmingwith Data

middot An interactive workflow for data analysismiddot Support for rapid prototyping research and

experimentation

2559

Why Rcpp

middot Easy to learn as it really does not have to be thatcomplicated ndash we will see numerous few examples

middot Easy to use as it avoids build and OS system complexitiesthanks to the R infrastrucure

middot Expressive as it allows for vectorised C++ using RcppSugar

middot Seamless access to all R objects vector matrix listS3S4RefClass Environment Function hellip

middot Speed gains for a variety of tasks Rcpp excels preciselywhere R struggles loops function calls hellip

middot Extensions greatly facilitates access to external librariesusing eg Rcpp modules

2659

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 14: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

So Why R

R enables us to

middot work interactivelymiddot explore and visualize datamiddot access retrieve andor generate datamiddot summarize and report into pdf html hellip

making it the key language for statistical computing and apreferred environment for many data analysts

1459

So Why R

R has always been extensible via

middot C via a bare-bones interface described in Writing RExtensions

middot Fortran which is also used internally by Rmiddot Java via rJava by Simon Urbanekmiddot C++ but essentially at the bare-bones level of C

So while in theory this always worked ndash it was tedious inpractice

1559

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1659

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1759

Why Extend R

Chambers proceeds with this rough map of the road ahead

middot Againstmiddot Itrsquos more workmiddot Bugs will bitemiddot Potential platform dependencymiddot Less readable software

middot In Favormiddot New and trusted computationsmiddot Speedmiddot Object references

1859

Why Extend R

The Why boils down to

middot speed Often a good enough reason for us hellip and a focusfor us in this workshop

middot new things We can bind to libraries and tools that wouldotherwise be unavailable in R

middot references Chambers quote from 2008 foreshadowed thework on Reference Classes now in R and built upon viaRcpp Modules Rcpp Classes (and also RcppR6)

1959

And Why C++

middot Asking Google leads to about ~ 50 million hitsmiddot Wikipedia C++ is a statically typed free-form

multi-paradigm compiled general-purpose powerfulprogramming language

middot C++ is industrial-strength vendor-independentwidely-used and still evolving

middot In science amp research one of the most frequently-usedlanguages If there is something you want to use connect to it probably has a CC++ API

middot As a widely used language it also has good tool support(debuggers profilers code analysis)

2059

Why C++

Scott Meyers View C++ as a federation of languages

middot C provides a rich inheritance and interoperability as UnixWindows hellip are all build on C

middot Object-Oriented C++ (maybe just to provide endlessdiscussions about exactly what OO is or should be)

middot Templated C++ which is mighty powerful template metaprogramming unequalled in other languages

middot The Standard Template Library (STL) is a specifictemplate library which is powerful but has its ownconventions

middot C++11 (and C++14 and beyond) add enough to becalled a fifth language

NB Meyers original list of four languages appeared years before C++11 2159

Why C++

middot Mature yet currentmiddot Strong performance focus

middot You donrsquot pay for what you donrsquot usemiddot Leave no room for another language between the

machine level and C++

middot Yet also powerfully abstract and high-levelmiddot C++11 is a big deal giving us new language featuresmiddot While there are complexities Rcpp users are mostly

shielded

2259

Interface Vision

2359

Bell Labs May 1976

2459

Interface Vision

R offers us the best of both worlds

middot Compiled code withmiddot Access to proven libraries and algorithms in

CC++Fortranmiddot Extremely high performance (in both serial and parallel

modes)middot Interpreted code with

middot An accessible high-level language made for Programmingwith Data

middot An interactive workflow for data analysismiddot Support for rapid prototyping research and

experimentation

2559

Why Rcpp

middot Easy to learn as it really does not have to be thatcomplicated ndash we will see numerous few examples

middot Easy to use as it avoids build and OS system complexitiesthanks to the R infrastrucure

middot Expressive as it allows for vectorised C++ using RcppSugar

middot Seamless access to all R objects vector matrix listS3S4RefClass Environment Function hellip

middot Speed gains for a variety of tasks Rcpp excels preciselywhere R struggles loops function calls hellip

middot Extensions greatly facilitates access to external librariesusing eg Rcpp modules

2659

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 15: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

So Why R

R has always been extensible via

middot C via a bare-bones interface described in Writing RExtensions

middot Fortran which is also used internally by Rmiddot Java via rJava by Simon Urbanekmiddot C++ but essentially at the bare-bones level of C

So while in theory this always worked ndash it was tedious inpractice

1559

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1659

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1759

Why Extend R

Chambers proceeds with this rough map of the road ahead

middot Againstmiddot Itrsquos more workmiddot Bugs will bitemiddot Potential platform dependencymiddot Less readable software

middot In Favormiddot New and trusted computationsmiddot Speedmiddot Object references

1859

Why Extend R

The Why boils down to

middot speed Often a good enough reason for us hellip and a focusfor us in this workshop

middot new things We can bind to libraries and tools that wouldotherwise be unavailable in R

middot references Chambers quote from 2008 foreshadowed thework on Reference Classes now in R and built upon viaRcpp Modules Rcpp Classes (and also RcppR6)

1959

And Why C++

middot Asking Google leads to about ~ 50 million hitsmiddot Wikipedia C++ is a statically typed free-form

multi-paradigm compiled general-purpose powerfulprogramming language

middot C++ is industrial-strength vendor-independentwidely-used and still evolving

middot In science amp research one of the most frequently-usedlanguages If there is something you want to use connect to it probably has a CC++ API

middot As a widely used language it also has good tool support(debuggers profilers code analysis)

2059

Why C++

Scott Meyers View C++ as a federation of languages

middot C provides a rich inheritance and interoperability as UnixWindows hellip are all build on C

middot Object-Oriented C++ (maybe just to provide endlessdiscussions about exactly what OO is or should be)

middot Templated C++ which is mighty powerful template metaprogramming unequalled in other languages

middot The Standard Template Library (STL) is a specifictemplate library which is powerful but has its ownconventions

middot C++11 (and C++14 and beyond) add enough to becalled a fifth language

NB Meyers original list of four languages appeared years before C++11 2159

Why C++

middot Mature yet currentmiddot Strong performance focus

middot You donrsquot pay for what you donrsquot usemiddot Leave no room for another language between the

machine level and C++

middot Yet also powerfully abstract and high-levelmiddot C++11 is a big deal giving us new language featuresmiddot While there are complexities Rcpp users are mostly

shielded

2259

Interface Vision

2359

Bell Labs May 1976

2459

Interface Vision

R offers us the best of both worlds

middot Compiled code withmiddot Access to proven libraries and algorithms in

CC++Fortranmiddot Extremely high performance (in both serial and parallel

modes)middot Interpreted code with

middot An accessible high-level language made for Programmingwith Data

middot An interactive workflow for data analysismiddot Support for rapid prototyping research and

experimentation

2559

Why Rcpp

middot Easy to learn as it really does not have to be thatcomplicated ndash we will see numerous few examples

middot Easy to use as it avoids build and OS system complexitiesthanks to the R infrastrucure

middot Expressive as it allows for vectorised C++ using RcppSugar

middot Seamless access to all R objects vector matrix listS3S4RefClass Environment Function hellip

middot Speed gains for a variety of tasks Rcpp excels preciselywhere R struggles loops function calls hellip

middot Extensions greatly facilitates access to external librariesusing eg Rcpp modules

2659

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 16: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1659

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1759

Why Extend R

Chambers proceeds with this rough map of the road ahead

middot Againstmiddot Itrsquos more workmiddot Bugs will bitemiddot Potential platform dependencymiddot Less readable software

middot In Favormiddot New and trusted computationsmiddot Speedmiddot Object references

1859

Why Extend R

The Why boils down to

middot speed Often a good enough reason for us hellip and a focusfor us in this workshop

middot new things We can bind to libraries and tools that wouldotherwise be unavailable in R

middot references Chambers quote from 2008 foreshadowed thework on Reference Classes now in R and built upon viaRcpp Modules Rcpp Classes (and also RcppR6)

1959

And Why C++

middot Asking Google leads to about ~ 50 million hitsmiddot Wikipedia C++ is a statically typed free-form

multi-paradigm compiled general-purpose powerfulprogramming language

middot C++ is industrial-strength vendor-independentwidely-used and still evolving

middot In science amp research one of the most frequently-usedlanguages If there is something you want to use connect to it probably has a CC++ API

middot As a widely used language it also has good tool support(debuggers profilers code analysis)

2059

Why C++

Scott Meyers View C++ as a federation of languages

middot C provides a rich inheritance and interoperability as UnixWindows hellip are all build on C

middot Object-Oriented C++ (maybe just to provide endlessdiscussions about exactly what OO is or should be)

middot Templated C++ which is mighty powerful template metaprogramming unequalled in other languages

middot The Standard Template Library (STL) is a specifictemplate library which is powerful but has its ownconventions

middot C++11 (and C++14 and beyond) add enough to becalled a fifth language

NB Meyers original list of four languages appeared years before C++11 2159

Why C++

middot Mature yet currentmiddot Strong performance focus

middot You donrsquot pay for what you donrsquot usemiddot Leave no room for another language between the

machine level and C++

middot Yet also powerfully abstract and high-levelmiddot C++11 is a big deal giving us new language featuresmiddot While there are complexities Rcpp users are mostly

shielded

2259

Interface Vision

2359

Bell Labs May 1976

2459

Interface Vision

R offers us the best of both worlds

middot Compiled code withmiddot Access to proven libraries and algorithms in

CC++Fortranmiddot Extremely high performance (in both serial and parallel

modes)middot Interpreted code with

middot An accessible high-level language made for Programmingwith Data

middot An interactive workflow for data analysismiddot Support for rapid prototyping research and

experimentation

2559

Why Rcpp

middot Easy to learn as it really does not have to be thatcomplicated ndash we will see numerous few examples

middot Easy to use as it avoids build and OS system complexitiesthanks to the R infrastrucure

middot Expressive as it allows for vectorised C++ using RcppSugar

middot Seamless access to all R objects vector matrix listS3S4RefClass Environment Function hellip

middot Speed gains for a variety of tasks Rcpp excels preciselywhere R struggles loops function calls hellip

middot Extensions greatly facilitates access to external librariesusing eg Rcpp modules

2659

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 17: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Why Extend R

Chambers (2008) opens Chapter 11 Interfaces I Using C andFortran

Since the core of R is in fact a program written inthe C language itrsquos not surprising that the mostdirect interface to non-R software is for code writtenin C or directly callable from C All the sameincluding additional C code is a serious step withsome added dangers and often a substantial amountof programming and debugging required You shouldhave a good reason

1759

Why Extend R

Chambers proceeds with this rough map of the road ahead

middot Againstmiddot Itrsquos more workmiddot Bugs will bitemiddot Potential platform dependencymiddot Less readable software

middot In Favormiddot New and trusted computationsmiddot Speedmiddot Object references

1859

Why Extend R

The Why boils down to

middot speed Often a good enough reason for us hellip and a focusfor us in this workshop

middot new things We can bind to libraries and tools that wouldotherwise be unavailable in R

middot references Chambers quote from 2008 foreshadowed thework on Reference Classes now in R and built upon viaRcpp Modules Rcpp Classes (and also RcppR6)

1959

And Why C++

middot Asking Google leads to about ~ 50 million hitsmiddot Wikipedia C++ is a statically typed free-form

multi-paradigm compiled general-purpose powerfulprogramming language

middot C++ is industrial-strength vendor-independentwidely-used and still evolving

middot In science amp research one of the most frequently-usedlanguages If there is something you want to use connect to it probably has a CC++ API

middot As a widely used language it also has good tool support(debuggers profilers code analysis)

2059

Why C++

Scott Meyers View C++ as a federation of languages

middot C provides a rich inheritance and interoperability as UnixWindows hellip are all build on C

middot Object-Oriented C++ (maybe just to provide endlessdiscussions about exactly what OO is or should be)

middot Templated C++ which is mighty powerful template metaprogramming unequalled in other languages

middot The Standard Template Library (STL) is a specifictemplate library which is powerful but has its ownconventions

middot C++11 (and C++14 and beyond) add enough to becalled a fifth language

NB Meyers original list of four languages appeared years before C++11 2159

Why C++

middot Mature yet currentmiddot Strong performance focus

middot You donrsquot pay for what you donrsquot usemiddot Leave no room for another language between the

machine level and C++

middot Yet also powerfully abstract and high-levelmiddot C++11 is a big deal giving us new language featuresmiddot While there are complexities Rcpp users are mostly

shielded

2259

Interface Vision

2359

Bell Labs May 1976

2459

Interface Vision

R offers us the best of both worlds

middot Compiled code withmiddot Access to proven libraries and algorithms in

CC++Fortranmiddot Extremely high performance (in both serial and parallel

modes)middot Interpreted code with

middot An accessible high-level language made for Programmingwith Data

middot An interactive workflow for data analysismiddot Support for rapid prototyping research and

experimentation

2559

Why Rcpp

middot Easy to learn as it really does not have to be thatcomplicated ndash we will see numerous few examples

middot Easy to use as it avoids build and OS system complexitiesthanks to the R infrastrucure

middot Expressive as it allows for vectorised C++ using RcppSugar

middot Seamless access to all R objects vector matrix listS3S4RefClass Environment Function hellip

middot Speed gains for a variety of tasks Rcpp excels preciselywhere R struggles loops function calls hellip

middot Extensions greatly facilitates access to external librariesusing eg Rcpp modules

2659

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 18: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Why Extend R

Chambers proceeds with this rough map of the road ahead

middot Againstmiddot Itrsquos more workmiddot Bugs will bitemiddot Potential platform dependencymiddot Less readable software

middot In Favormiddot New and trusted computationsmiddot Speedmiddot Object references

1859

Why Extend R

The Why boils down to

middot speed Often a good enough reason for us hellip and a focusfor us in this workshop

middot new things We can bind to libraries and tools that wouldotherwise be unavailable in R

middot references Chambers quote from 2008 foreshadowed thework on Reference Classes now in R and built upon viaRcpp Modules Rcpp Classes (and also RcppR6)

1959

And Why C++

middot Asking Google leads to about ~ 50 million hitsmiddot Wikipedia C++ is a statically typed free-form

multi-paradigm compiled general-purpose powerfulprogramming language

middot C++ is industrial-strength vendor-independentwidely-used and still evolving

middot In science amp research one of the most frequently-usedlanguages If there is something you want to use connect to it probably has a CC++ API

middot As a widely used language it also has good tool support(debuggers profilers code analysis)

2059

Why C++

Scott Meyers View C++ as a federation of languages

middot C provides a rich inheritance and interoperability as UnixWindows hellip are all build on C

middot Object-Oriented C++ (maybe just to provide endlessdiscussions about exactly what OO is or should be)

middot Templated C++ which is mighty powerful template metaprogramming unequalled in other languages

middot The Standard Template Library (STL) is a specifictemplate library which is powerful but has its ownconventions

middot C++11 (and C++14 and beyond) add enough to becalled a fifth language

NB Meyers original list of four languages appeared years before C++11 2159

Why C++

middot Mature yet currentmiddot Strong performance focus

middot You donrsquot pay for what you donrsquot usemiddot Leave no room for another language between the

machine level and C++

middot Yet also powerfully abstract and high-levelmiddot C++11 is a big deal giving us new language featuresmiddot While there are complexities Rcpp users are mostly

shielded

2259

Interface Vision

2359

Bell Labs May 1976

2459

Interface Vision

R offers us the best of both worlds

middot Compiled code withmiddot Access to proven libraries and algorithms in

CC++Fortranmiddot Extremely high performance (in both serial and parallel

modes)middot Interpreted code with

middot An accessible high-level language made for Programmingwith Data

middot An interactive workflow for data analysismiddot Support for rapid prototyping research and

experimentation

2559

Why Rcpp

middot Easy to learn as it really does not have to be thatcomplicated ndash we will see numerous few examples

middot Easy to use as it avoids build and OS system complexitiesthanks to the R infrastrucure

middot Expressive as it allows for vectorised C++ using RcppSugar

middot Seamless access to all R objects vector matrix listS3S4RefClass Environment Function hellip

middot Speed gains for a variety of tasks Rcpp excels preciselywhere R struggles loops function calls hellip

middot Extensions greatly facilitates access to external librariesusing eg Rcpp modules

2659

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 19: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Why Extend R

The Why boils down to

middot speed Often a good enough reason for us hellip and a focusfor us in this workshop

middot new things We can bind to libraries and tools that wouldotherwise be unavailable in R

middot references Chambers quote from 2008 foreshadowed thework on Reference Classes now in R and built upon viaRcpp Modules Rcpp Classes (and also RcppR6)

1959

And Why C++

middot Asking Google leads to about ~ 50 million hitsmiddot Wikipedia C++ is a statically typed free-form

multi-paradigm compiled general-purpose powerfulprogramming language

middot C++ is industrial-strength vendor-independentwidely-used and still evolving

middot In science amp research one of the most frequently-usedlanguages If there is something you want to use connect to it probably has a CC++ API

middot As a widely used language it also has good tool support(debuggers profilers code analysis)

2059

Why C++

Scott Meyers View C++ as a federation of languages

middot C provides a rich inheritance and interoperability as UnixWindows hellip are all build on C

middot Object-Oriented C++ (maybe just to provide endlessdiscussions about exactly what OO is or should be)

middot Templated C++ which is mighty powerful template metaprogramming unequalled in other languages

middot The Standard Template Library (STL) is a specifictemplate library which is powerful but has its ownconventions

middot C++11 (and C++14 and beyond) add enough to becalled a fifth language

NB Meyers original list of four languages appeared years before C++11 2159

Why C++

middot Mature yet currentmiddot Strong performance focus

middot You donrsquot pay for what you donrsquot usemiddot Leave no room for another language between the

machine level and C++

middot Yet also powerfully abstract and high-levelmiddot C++11 is a big deal giving us new language featuresmiddot While there are complexities Rcpp users are mostly

shielded

2259

Interface Vision

2359

Bell Labs May 1976

2459

Interface Vision

R offers us the best of both worlds

middot Compiled code withmiddot Access to proven libraries and algorithms in

CC++Fortranmiddot Extremely high performance (in both serial and parallel

modes)middot Interpreted code with

middot An accessible high-level language made for Programmingwith Data

middot An interactive workflow for data analysismiddot Support for rapid prototyping research and

experimentation

2559

Why Rcpp

middot Easy to learn as it really does not have to be thatcomplicated ndash we will see numerous few examples

middot Easy to use as it avoids build and OS system complexitiesthanks to the R infrastrucure

middot Expressive as it allows for vectorised C++ using RcppSugar

middot Seamless access to all R objects vector matrix listS3S4RefClass Environment Function hellip

middot Speed gains for a variety of tasks Rcpp excels preciselywhere R struggles loops function calls hellip

middot Extensions greatly facilitates access to external librariesusing eg Rcpp modules

2659

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 20: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

And Why C++

middot Asking Google leads to about ~ 50 million hitsmiddot Wikipedia C++ is a statically typed free-form

multi-paradigm compiled general-purpose powerfulprogramming language

middot C++ is industrial-strength vendor-independentwidely-used and still evolving

middot In science amp research one of the most frequently-usedlanguages If there is something you want to use connect to it probably has a CC++ API

middot As a widely used language it also has good tool support(debuggers profilers code analysis)

2059

Why C++

Scott Meyers View C++ as a federation of languages

middot C provides a rich inheritance and interoperability as UnixWindows hellip are all build on C

middot Object-Oriented C++ (maybe just to provide endlessdiscussions about exactly what OO is or should be)

middot Templated C++ which is mighty powerful template metaprogramming unequalled in other languages

middot The Standard Template Library (STL) is a specifictemplate library which is powerful but has its ownconventions

middot C++11 (and C++14 and beyond) add enough to becalled a fifth language

NB Meyers original list of four languages appeared years before C++11 2159

Why C++

middot Mature yet currentmiddot Strong performance focus

middot You donrsquot pay for what you donrsquot usemiddot Leave no room for another language between the

machine level and C++

middot Yet also powerfully abstract and high-levelmiddot C++11 is a big deal giving us new language featuresmiddot While there are complexities Rcpp users are mostly

shielded

2259

Interface Vision

2359

Bell Labs May 1976

2459

Interface Vision

R offers us the best of both worlds

middot Compiled code withmiddot Access to proven libraries and algorithms in

CC++Fortranmiddot Extremely high performance (in both serial and parallel

modes)middot Interpreted code with

middot An accessible high-level language made for Programmingwith Data

middot An interactive workflow for data analysismiddot Support for rapid prototyping research and

experimentation

2559

Why Rcpp

middot Easy to learn as it really does not have to be thatcomplicated ndash we will see numerous few examples

middot Easy to use as it avoids build and OS system complexitiesthanks to the R infrastrucure

middot Expressive as it allows for vectorised C++ using RcppSugar

middot Seamless access to all R objects vector matrix listS3S4RefClass Environment Function hellip

middot Speed gains for a variety of tasks Rcpp excels preciselywhere R struggles loops function calls hellip

middot Extensions greatly facilitates access to external librariesusing eg Rcpp modules

2659

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 21: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Why C++

Scott Meyers View C++ as a federation of languages

middot C provides a rich inheritance and interoperability as UnixWindows hellip are all build on C

middot Object-Oriented C++ (maybe just to provide endlessdiscussions about exactly what OO is or should be)

middot Templated C++ which is mighty powerful template metaprogramming unequalled in other languages

middot The Standard Template Library (STL) is a specifictemplate library which is powerful but has its ownconventions

middot C++11 (and C++14 and beyond) add enough to becalled a fifth language

NB Meyers original list of four languages appeared years before C++11 2159

Why C++

middot Mature yet currentmiddot Strong performance focus

middot You donrsquot pay for what you donrsquot usemiddot Leave no room for another language between the

machine level and C++

middot Yet also powerfully abstract and high-levelmiddot C++11 is a big deal giving us new language featuresmiddot While there are complexities Rcpp users are mostly

shielded

2259

Interface Vision

2359

Bell Labs May 1976

2459

Interface Vision

R offers us the best of both worlds

middot Compiled code withmiddot Access to proven libraries and algorithms in

CC++Fortranmiddot Extremely high performance (in both serial and parallel

modes)middot Interpreted code with

middot An accessible high-level language made for Programmingwith Data

middot An interactive workflow for data analysismiddot Support for rapid prototyping research and

experimentation

2559

Why Rcpp

middot Easy to learn as it really does not have to be thatcomplicated ndash we will see numerous few examples

middot Easy to use as it avoids build and OS system complexitiesthanks to the R infrastrucure

middot Expressive as it allows for vectorised C++ using RcppSugar

middot Seamless access to all R objects vector matrix listS3S4RefClass Environment Function hellip

middot Speed gains for a variety of tasks Rcpp excels preciselywhere R struggles loops function calls hellip

middot Extensions greatly facilitates access to external librariesusing eg Rcpp modules

2659

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 22: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Why C++

middot Mature yet currentmiddot Strong performance focus

middot You donrsquot pay for what you donrsquot usemiddot Leave no room for another language between the

machine level and C++

middot Yet also powerfully abstract and high-levelmiddot C++11 is a big deal giving us new language featuresmiddot While there are complexities Rcpp users are mostly

shielded

2259

Interface Vision

2359

Bell Labs May 1976

2459

Interface Vision

R offers us the best of both worlds

middot Compiled code withmiddot Access to proven libraries and algorithms in

CC++Fortranmiddot Extremely high performance (in both serial and parallel

modes)middot Interpreted code with

middot An accessible high-level language made for Programmingwith Data

middot An interactive workflow for data analysismiddot Support for rapid prototyping research and

experimentation

2559

Why Rcpp

middot Easy to learn as it really does not have to be thatcomplicated ndash we will see numerous few examples

middot Easy to use as it avoids build and OS system complexitiesthanks to the R infrastrucure

middot Expressive as it allows for vectorised C++ using RcppSugar

middot Seamless access to all R objects vector matrix listS3S4RefClass Environment Function hellip

middot Speed gains for a variety of tasks Rcpp excels preciselywhere R struggles loops function calls hellip

middot Extensions greatly facilitates access to external librariesusing eg Rcpp modules

2659

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 23: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Interface Vision

2359

Bell Labs May 1976

2459

Interface Vision

R offers us the best of both worlds

middot Compiled code withmiddot Access to proven libraries and algorithms in

CC++Fortranmiddot Extremely high performance (in both serial and parallel

modes)middot Interpreted code with

middot An accessible high-level language made for Programmingwith Data

middot An interactive workflow for data analysismiddot Support for rapid prototyping research and

experimentation

2559

Why Rcpp

middot Easy to learn as it really does not have to be thatcomplicated ndash we will see numerous few examples

middot Easy to use as it avoids build and OS system complexitiesthanks to the R infrastrucure

middot Expressive as it allows for vectorised C++ using RcppSugar

middot Seamless access to all R objects vector matrix listS3S4RefClass Environment Function hellip

middot Speed gains for a variety of tasks Rcpp excels preciselywhere R struggles loops function calls hellip

middot Extensions greatly facilitates access to external librariesusing eg Rcpp modules

2659

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 24: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Bell Labs May 1976

2459

Interface Vision

R offers us the best of both worlds

middot Compiled code withmiddot Access to proven libraries and algorithms in

CC++Fortranmiddot Extremely high performance (in both serial and parallel

modes)middot Interpreted code with

middot An accessible high-level language made for Programmingwith Data

middot An interactive workflow for data analysismiddot Support for rapid prototyping research and

experimentation

2559

Why Rcpp

middot Easy to learn as it really does not have to be thatcomplicated ndash we will see numerous few examples

middot Easy to use as it avoids build and OS system complexitiesthanks to the R infrastrucure

middot Expressive as it allows for vectorised C++ using RcppSugar

middot Seamless access to all R objects vector matrix listS3S4RefClass Environment Function hellip

middot Speed gains for a variety of tasks Rcpp excels preciselywhere R struggles loops function calls hellip

middot Extensions greatly facilitates access to external librariesusing eg Rcpp modules

2659

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 25: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Interface Vision

R offers us the best of both worlds

middot Compiled code withmiddot Access to proven libraries and algorithms in

CC++Fortranmiddot Extremely high performance (in both serial and parallel

modes)middot Interpreted code with

middot An accessible high-level language made for Programmingwith Data

middot An interactive workflow for data analysismiddot Support for rapid prototyping research and

experimentation

2559

Why Rcpp

middot Easy to learn as it really does not have to be thatcomplicated ndash we will see numerous few examples

middot Easy to use as it avoids build and OS system complexitiesthanks to the R infrastrucure

middot Expressive as it allows for vectorised C++ using RcppSugar

middot Seamless access to all R objects vector matrix listS3S4RefClass Environment Function hellip

middot Speed gains for a variety of tasks Rcpp excels preciselywhere R struggles loops function calls hellip

middot Extensions greatly facilitates access to external librariesusing eg Rcpp modules

2659

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 26: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Why Rcpp

middot Easy to learn as it really does not have to be thatcomplicated ndash we will see numerous few examples

middot Easy to use as it avoids build and OS system complexitiesthanks to the R infrastrucure

middot Expressive as it allows for vectorised C++ using RcppSugar

middot Seamless access to all R objects vector matrix listS3S4RefClass Environment Function hellip

middot Speed gains for a variety of tasks Rcpp excels preciselywhere R struggles loops function calls hellip

middot Extensions greatly facilitates access to external librariesusing eg Rcpp modules

2659

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 27: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Speed

2759

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 28: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Speed Example (due to StackOverflow)

Consider a function defined as

f(n) such that n when n lt 2

f(n minus 1) + f(n minus 2) when n ge 2

2859

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 29: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Speed Example in R

R implementation and use

f lt- function(n) if (n lt 2) return(n)return(f(n-1) + f(n-2))

Using it on first 11 argumentssapply(010 f)

[1] 0 1 1 2 3 5 8 13 21 34 55

2959

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 30: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Speed Example Timed

Timing

library(rbenchmark)benchmark(f(10) f(15) f(20))[14]

test replications elapsed relative 1 f(10) 100 0026 1000 2 f(15) 100 0327 12577 3 f(20) 100 3796 146000

3059

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 31: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Speed Example in C C++

A C or C++ solution can be equally simple

int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2))

But how do we call it from R

3159

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 32: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Mattrsquos Example from useR 2015include ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

SEXP fibonacci_c(SEXP n) SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

need to compile link load fibonacci lt- function(n) Call(fibonacci_c n)sapply(010 fibonacci) 3259

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 33: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

One Minor Modification to Mattrsquos Exampleinclude ltRhgtinclude ltRinternalshgt

int fibonacci_c_impl(int n) if (n lt 2) return nreturn fibonacci_c_impl(n - 1) + fibonacci_c_impl(n - 2)

[[Rcppexport]]SEXP fibonacci_c(SEXP n)

SEXP result = PROTECT(allocVector(INTSXP 1))INTEGER(result)[0] = fibonacci_c_impl(asInteger(n))UNPROTECT(1)return result

Rsapply(010 fibonacci_c)

3359

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 34: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Speed Example in C C++

But Rcpp makes this much easier

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

sapply(010 g)

[1] 0 1 1 2 3 5 8 13 21 34 55

3459

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 35: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Speed Example Comparing R and C++

Timing

RcppcppFunction(int g(int n) if (n lt 2) return(n)return(g(n-1) + g(n-2)) )

library(rbenchmark)benchmark(f(25) g(25) order=relative)[14]

test replications elapsed relative 2 g(25) 100 0099 1000 1 f(25) 100 47787 482697

A nice gain of a few orders of magnitude3559

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 36: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Another Angle on Speed

Run-time performance is just one example

Time to code is another metric

We feel quite strongly that helps you code more succinctlyleading to fewer bugs and faster development

A good environment helps RStudio integrates R and C++development quite nicely (eg the compiler error messageparsing is very helpful) and also helps with package building

3659

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 37: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Speed Example Footnote Also Due to Matt

include ltRcpphgt

[[Rcppplugins(cpp11)]]

constexpr int fibonacci_recursive_constexpr(const int n) return n lt 2 n (fibonacci_recursive_constexpr(n - 1) +

fibonacci_recursive_constexpr(n - 2))

[[Rcppexport]]int constexprFib()

const int N = 42constexpr int result = fibonacci_recursive_constexpr(N)return result

3759

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 38: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Popularity

3859

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 39: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Used by 462 CRAN Packages as of last weekend

3959

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 40: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Page Rank One (According to Andrie de Vries)

4059

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 41: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Application Spotlight Rblpapi

4159

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 42: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

History Basic package using the C (Dirk)

4259

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 43: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

History Key package using Java (Ana then John)

4359

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 44: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

History But the vendor API keeps improving

4459

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 45: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Present and Future (Whit Dirk and John)

4559

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 46: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Even Better Present and Future

4659

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 47: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Third Time Lucky The Rblpapi package

The new rewrite is different

middot Lighter ndash no longer uses or requires Javamiddot Simpler ndash leverages Rcppmiddot More flexible ndash easy to add new functionality with C++

4759

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 48: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Status The Rblpapi package

Where we are at now

middot Robust and fastmiddot Implements most widely-used featuresmiddot (Basic) documentation for everythingmiddot Travis CI integrationmiddot On GitHub and in the ghrr repository

NB And now on CRAN too see below

4859

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 49: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Examples

Core Functions known from other API accessors

middot bdp(c(ESA Index SPY US Equity) c(PX_LAST VOLUME))middot bds(GOOG US Equity TOP_20_HOLDERS_PUBLIC_FILINGS)middot bdh(SPY US Equity c(PX_LAST VOLUME)startdate=SysDate()-31)

middot getBars(ESA Index startTime=ISOdatetime(201511000))middot getTicks(ESA Index TRADE Systime()-6060))middot fieldSearch(VWAP)

4959

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 50: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

[Then] Current Status of the Rblpapi package

Things we addressed

middot Fixed-dimension retrieval very easymiddot Now include shared library with rpath-encoded pathmiddot Builds ldquoeverywhererdquo including on Travis CI

5059

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 51: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

[Then] Current Status of the Rblpapi package

Things we [then] need[ed] to address

middot DataFrame class caused trouble need something newmiddot Builds on ldquothat other OSrdquo very difficult while (vendor)

API library built with VC++middot More features subscriptions screens portfolioshellipmiddot Pull requests welcome

5159

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 52: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

[Then] Summary The Rblpapi package

Concluding

middot Bloomberg provides a first-rate API and infrastructuremiddot So the R Community came up with good packagesmiddot LanguageOS choice matter some vendors still

ldquodifferentrdquomiddot We prefer Open Source package may not go onto CRANmiddot But we have alternatives in GitHub-hosted repositories

NB That was then hellip

5259

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 53: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

[Now] Summary and State of the Rblpapi Package

What we learned

middot Powerful APIs are compellingmiddot Providing working code is keymiddot Unexpectedly we got a pull request for Windows supportmiddot With some additional work this got onto CRANmiddot Supporting Linux OS X and Windows ldquoout of the boxrdquo

5359

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 54: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

The End

5459

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 55: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Documentation

middot The Rcpp package comes with nine pdf vignettes andnumerous help pages

middot The introductory vignettes are now published (for Rcppand RcppEigen in J Stat Software for RcppArmadillo inComp Stat amp Data Anlys)

middot The rcpp-devel list is the recommended resourcegenerally very helpful and fairly low volume

middot StackOverflow has almost 900 posts toomiddot And a number of blog posts introducediscuss features

5559

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 56: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Rcpp Gallery

5659

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 57: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

The Rcpp book

5759

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 58: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Very Last Words

Thank Youdirkeddelbuettelcom

httpdirkeddelbuettelcompresentations

5859

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End
Page 59: Rcpp - Making R Applications Go Faster and Further · Rcpp MakingRApplicationsGoFasterandFurther DirkEddelbuettel EARL2015KeynoteAddress September16,2015 Released under a Creative

Collophon

Made using

middot TeXlive 20141024middot Beamer with mthememiddot Pandoc 11242middot R 322middot rmarkdown 07middot Emacs 244middot Ubuntu 1504

5959

  • Introduction
  • Extending R
  • Interface Vision
  • Speed
  • Popularity
  • Application Spotlight Rblpapi
  • The End