Upload
oxbowlakes
View
6.717
Download
4
Tags:
Embed Size (px)
Citation preview
NameChris Marshall @oxbow_lakes
GSA Capital Partners LLPMarch 2012
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority.
Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261
Building Financial Systems in Scala
3
Mathematics Degree
Working in financial software since 1999
• Smalltalk for ~6 months
• Java thereafter
• Scala since Dec 2008
JP Morgan for 6 years
• ~200,000 employees
GSA Capital for 5 ¾ years
• Quant hedge fund
• ~90 employees
Backgroundwho am i
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority. Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261
4
Low-latency links & feeds
• Up to 750,000 trades /day
• 108 market events / day
Historic / Current Market Data
• Listing changes (e.g. SUNW.O becomes JAVA.O becomes ORCL.O)
• News Events
Backtesting / Execution Framework
Everything Else
• This is where I come in
• What are our positions? What is our P&L?
• Are our trades reconciled?
• Reporting (brokers, regulators, administrators)
GSAWhat do we do? Roughly half the company are technologists
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority. Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261
5
Fragmentation / Diversity
• Vertical business silos (Equity, fixed-income etc)
Java
• Widespread adoption in the first decade of this century
Willing to try new technologies
• Even in large organizations
• Potentially driven by M & A
Financial ITDoes financial IT differ from IT elsewhere?
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority. Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261
6
Dependency on Java
• The platform
• The ecosystem
Older, wiser
• 10 years’ accumulated experience
• Java no longer enough
• Frustrations
JVM alternatives
• A plethora of languages
• Dynamically and statically-typed
• Groovy, Clojure, JRuby, Jython, Scala, Kotlin, Ceylon, Gosu
Financial IT nowWhere do we find ourselves in 2012?
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority. Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261
Straw Poll
val mustReport = trades filter (uncoveredShort ∨ exceedsDollarMax)
val european = { val Europe = (_ : Market).exchange.country.region == Region.EU trades filter (_.market ∈: Europe)}
scala> val fibs: Stream[Int] = 0 #:: 1 #:: (fibs zip fibs.tail map { case (x, y) => x + y })fibs: Stream[Int] = Stream(0, ?)
scala> (fibs take 10).toListres12: List[Int] = List(0, 1, 1, 2, 3, 5, 8, 13, 21, 34)
def run = checks.traverse[({type λ[α] = ValidationNEL[String, α]})#λ, Person](_ andThen lfn apply p)
for { c :: _ <- run } yield c.age + 1.5
8
Fully object-oriented
• No primitives or operators at language level
– Compiles down to primitives in the bytecode
• No statics
• traits (interfaces with implementation)
Terse
• Type inference
Functional
• Function types (A ⇒ B) built in to the language, including closures
• Pattern matching and ADTs
• Fully-immutable collections library
• Syntactic support for monads and functors (for-comprehensions)
• Lazy evaluation (optional)
• Everything is an expression
– That goes for if/else, try/catch etc
What is Scala?An “object-functional language”
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority. Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261
9
Unrestrictive
• Symbolic identifiers (including unicode – e.g. , , ) ∃ ∀ ∈• Syntactic sugar
– Unary methods
– Infix notation (methods and types)
– Right-associativity allowed (e.g. 1 :: Nil where :: is a method invocation against Nil)
• Uniform access principle
• Arbitrarily-scoped definitions/values
Rich
• State of the art typesystem
– Self types, higher-kinded types, dependent method types, intersection types, abstract types
– Declaration-site variance annotations (covariance/contravariance)
• Hugely powerful collections library
• Implicits - values, type conversions, definitions
• Multi-threading support (actors, parallelism, STM – via libraries)
What is Scala?Scala is broad where other languages are deep
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority. Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261
10
It compiles down to bytecode
• Two-way Java interop
It runs on the JVM
• Just add scala-library.jar to the classpath
It is active
• Extensive ecosystem (SBT, Lift, Play, Scalate, Akka)
• New languages features
– Macros
– String interpolation
– Reflection
• It has an active community
– Hosted on github (make a pull request!)
– Docsprees
What is Scala?but most importantly…
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority. Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261
11
Binary incompatibility
• Libraries compiled against 2.9.x will not be compatible with 2.10.x
Unwelcoming “academic” community
• FP advocacy
• Do I need to learn Haskell first?
Too Complex!
• Intersection of features
• Unrestrictive syntax options can lead to confusion
• Places too high a burden on a library designer
The flip sideThe problems of Scala
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority. Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261
12
Java interfaces contain no implementation
• Means API designers are miserly about adding methods to them
• Java.util.Collection defines 13 methods
• scala.collection.immutable.Traversable declares >90 methods
Java interfaces are a pain to implement
• If you want to implement Collection, you must implement 13 methods
– OK, you can extend AbstractCollection and implement 2 methods
• If you want to implement TraversableLike, you must implement 1 method
The scala collection library is unbelievably awesome
• Creating new collections from existing ones
• Getting data out of your collection
• Collapsing your collection (folds)
traitsWhy Java’s collections are useless
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority. Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261
scala> val isEven = (_ : Int) % 2 == 0 isEven: Int => Boolean = <function1>
scala> 1 to 10 filter isEven res3: scala.collection.immutable.IndexedSeq[Int] = Vector(2, 4, 6, 8, 10)
scala> 1 to 10 filterNot isEven res4: scala.collection.immutable.IndexedSeq[Int] = Vector(1, 3, 5, 7, 9)
scala> 1 to 10 span (_ < 3) res5: (scala.collection.immutable.Range, scala.collection.immutable.Range) = (Range(1, 2),Range(3, 4, 5, 6, 7, 8, 9, 10))
scala> 1 to 50 by 5 take 3 res6: scala.collection.immutable.Range = Range(1, 6, 11)
scala> Seq("London", "Paris", "New York") map (_.length) res7: Seq[Int] = List(6, 5, 8)
scala> Seq("London", "Paris", "New York") zip res7 res8: Seq[(java.lang.String, Int)] = List((London,6), (Paris,5), (New York,8))
scala> List(1, 2, 3, 4).foldLeft(0)(_ + _) res9: Int = 10
FX rates example
case class FxRate(from: Currency, to: Currency, rate: BigDecimal) {
trait MarketEnv { def rate: Currency ⇒ Currency ⇒ Option[FxRate] }
FX rates example
case class SimpleEnv(rates: Map[(Currency, Currency), FxRate]) extends MarketEnv {
def rate = from ⇒ to ⇒ rates get (from → to)
}
FX rates example
type CcyPair = (Currency, Currency)
case class FxRate(from: Currency, to: Currency, rate: BigDecimal) {
FX rates example
case class SimpleEnv(rates: Map[CcyPair, FxRate]) extends MarketEnv {
def rate = from ⇒ to ⇒ { def attemptDirect(c1: Currency, c2: Currency) = rates get (c1 → c2) attemptDirect(from, to) orElse (attemptDirect(to, from) map (~_)) }
}
FX rates example
type CcyPair = (Currency, Currency)
case class FxRate(from: Currency, to: Currency, rate: BigDecimal) {
def unary_~ = FxRate(to, from, 1 / rate)
}
FX rates example
case class SimpleEnv(rates: Map[CcyPair, FxRate]) extends MarketEnv {
def rate = from ⇒ to ⇒ { def attempt(c1: Currency, c2: Currency) = rates get (c1 → c2) orElse (rates get (c2 → c1) map (~_)) def viaUsd = { val Usd = Currency.getInstance("USD") for { f2u <- attempt(from, Usd) u2t <- attempt(Usd, to) } yield f2u * u2t }
attempt(from, to) orElse viaUsd } }
FX rates example
type CcyPair = (Currency, Currency)
case class FxRate(from: Currency, to: Currency, rate: BigDecimal) { def unary_~ = FxRate(to, from, 1 / rate) def *(that: FxRate) = { require(this.to == that.from) FxRate(this.from, that.to, this.rate * that.rate) } }
scala> val Gbp = Currency.getInstance("GBP") scala> val Usd = Currency.getInstance("USD") scala> val Eur = Currency.getInstance("EUR") scala> val rates = FxRate(Gbp, Usd, 1.57) :: FxRate(Eur, Usd, 1.31) :: Nil scala> val env = SimpleEnv((rates map (r ⇒ r.pair → r)).toMap)
scala> env.rate(Eur)(Gbp) res0 : Option[FxRate] = Some(FxRate(EUR,GBP,0.83439))
scala> val vGbp = env.rate(Gbp)
scala> println(vGbp(Eur)) res1: Option[FxRate] = Some(FxRate(GBP,EUR,1.19847))
scala> println(vGbp(Usd)) res2 : Option[FxRate] = Some(FxRate(GBP,USD,1.57))
scala> val Jpy = Currency.getInstance("JPY")
scala> println(vGbp(Jpy)) res3 : Option[FxRate] = None
FX rates example
type CcyPair = (Currency, Currency)
case class FxRate(from: Currency, to: Currency, rate: BigDecimal) { def pair: CcyPair = from → to def unary_~ = FxRate(to, from, 1 / rate) def *(that: FxRate) = { require(this.to == that.from) FxRate(this.from, that.to, this.rate * that.rate) } }
FX rates example
type CcyPair = (Currency, Currency)
case class FxRate(from: Currency, to: Currency, rate: BigDecimal) { lazy val pair: CcyPair = from → to def unary_~ = FxRate(to, from, 1 / rate) def *(that: FxRate) = { require(this.to == that.from) FxRate(this.from, that.to, this.rate * that.rate) } }
https://gist.github.com/2028575
24
One Extra Level Of Abstraction
• “Shapes”, “Patterns”, “Structures”
• Usually refer to some (embarrassingly simple) mathematical concepts (like monoids)
Typeclasses
• Scala does not have typeclasses
• This is why you should have asked Santa for a proper type system
– Java’s type system is so poor, most developers don’t realize it has one
– Scala’s type system includes higher-kinds
– Scala’s implicit resolution mechanism allows the compiler to solve your problems for you
Functional ProgrammingNot so complex up to now?
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority. Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261
25
Definitions
• A Set with an associative operation and an identity under the operation
• Without the identity, this is a Semigroup
Defined trivially in scala as an interface
With some implementations marked as implicit
MonoidsPreparation ace
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority. Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261
trait Monoid[A] { def identity: A def mplus(a1: A, a2: A): A }
implicit val IntMonoid = new Monoid[Int] {
def identity = 0
def mplus(i1: Int, i2: Int) = i1 + i2
}
26
Retrofit functionality to anything
“Pimp my Library”Adding methods to any type
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority. Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261
trait Id[A] { val value: A
def mplus(b: A)(implicit m: Monoid[A]) = m.mplus(value, b) }
object Syntax { implicit def mkId[A](a: A) = new Id[A] { val value = a } }
scala> import Syntax._._ scala> 1 mplus 2 res0: Int = 3
27
Monoids beget monoids
• If A, B … N all monoids
– the n-tuple (A, B … N) is a monoid
• If A is a monoid
– Then Option[A] is a monoid
• If V is a monoid
– Then Map[K, V] is a monoid
• If B is a monoid
– The function A ⇒ B is a monoid
This is actually really rather powerful
• How do we prove it?
MonoidsSo, huh. You can add numbers
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority. Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261
private def monoid[A: Monoid] = implicitly[Monoid[A]]
implicit def OptionMonoid[A: Monoid] = new Monoid[Option[A]] { def identity = None
def mplus(o1: Option[A], o2: Option[A]) = (o1, o2) match { case (Some(a1), Some(a2)) ⇒ Some(monoid[A].mplus(a1, a2)) case (x @ Some(_), None) ⇒ x case (None, x @ Some(_)) ⇒ x case (None, None) ⇒ None } } implicit def PairMonoid[A: Monoid, B: Monoid] = new Monoid[(A, B)] { def identity = (monoid[A].identity, monoid[B].identity)
def mplus(a1: (A, B), a2: (A, B)) = (monoid[A].mplus(a1._1, a2._1), monoid[B].mplus(a1._2, a2._2)) }
implicit def Function1Monoid[A, B: Monoid] = new Monoid[A ⇒ B] { def identity = a ⇒ monoid[B].identity
def mplus(a1: A ⇒ B, a2: A ⇒ B) = a ⇒ monoid[B].mplus(a1(a), a2(a)) }
scala> some(4) mplus none[Int] res0: Option[Int] = Some(4) scala> some(4) mplus some(5) res1: Option[Int] = Some(9)
scala> (1, “a”, 2.3) mplus (2, “b”, 3.4) res2: (Int, String, Double) = (3, ab, 5.7)
scala> Map(‘a → 1, ‘b → 2) mplus Map(‘b → 3, ‘c → 4) res3: Map[Symbol, Int] = Map(‘a -> 1, ‘b -> 5, ‘c -> 4)
https://gist.github.com/2028579
30
Example 1: P&L
• Inventory P&L
– The amount I have made or lost on the position I started the day with
• Trading P&L
– The amount I have lost or gained on the trades I have done today
• Now, suppose I have a bunch of positions in my trading book, keyed by investment
Academic nonsenseWhat use is this stuff in the real world?
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority. Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261
trait Position { def investment: Investment def tradingPnL: Option[Double] def inventoryPnL: Option[Double] final def totalPnL = inventoryPnL → tradingPnL}
scala> val positions: Map[Investment, Position] = ... scala> val (totalInv, totalTrad) = (positions.values map (_.totalPnL)).msum totalInv: Option[Double] = Some(801.0) totalTrad: Option[Double] = Some(579.0)
scala> totalInv mplus totalTrad res1: Option[Double] = Some(1380.0)
class TraversableW[A](cc: Traversable[A]) { def msum(implicit m: Monoid[A]): A = (m.identity /: cc)(m.mplus) } implicit def Traversable_Is_TraversableW[A](cc: Traversable[A]) = new TraversableW[A](cc)
scala> val petesBook: Map[Investment, Position] = ...
scala> val davesBook: Map[Investment, Position] = ...
scala> petesBook mapValues (_.totalPnL) mplus (davesBook mapValues (_.totalPnL)) res2: Map[Investment, (Option[Double], Option[Double])] = Map( Vod -> (None, Some(123.0)), Ibm -> (None, Some(567.0)), Msft -> (Some(468.0),Some(876.0)) )
scala> List(‘a, ‘b, ‘c).msum <console>:8: error: could not find implicit value for parameter m: Monoid[Symbol] List('a, 'b, 'c).msum ^
scala> "abc".reverse res1: String = cba
scala> Seq('a, 'b, 'c).reverse res2: Seq[Symbol] = List('c, 'b, 'a)
https://gist.github.com/2028584
33
Scala as a better Java
• Tuples, Functions & Closures
• Pattern-matching as a better “switch”
• Symbolic math
• Type Inference
• Amazing collection library
Scala as a functional gateway drug
• Write more modular code
• Lower cyclomatic complexity
• Delay side effects
What I left out
• Too much stuff
ConclusionsWith great power comes great responsibility
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority. Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261
34
This presentation
• http://www.slideshare.net/oxbow_lakes/building-financial-systems-in-scala
• (tinyurl): http://slidesha.re/xGfdww
• Runnable Code: https://github.com/oxbowlakes
References
• Martin Odersky keynote: http://days2011.scala-lang.org/sites/days2011/files/01.%20Martin%20Odersky.pdf
• Scalaz: http://code.google.com/p/scalaz/
• Shapeless: https://github.com/milessabin/shapeless
Examples
• Handling exceptions via types: https://gist.github.com/970717
• Scala collections examples: https://gist.github.com/1869377
Extrasfor the masochists
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority. Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261
35
Contact us
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority. Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261
GSA Capital Partners LLP
[email protected] +44 (0)20 7959 8850
London Office
Stratton House5 Stratton StreetLondon W1J 8LA
T +44 (0)20 7959 8800F +44 (0)20 7959 8845
New York Office
1140 Ave of the Americas9th FloorNew York NY 10036