STATISTICALMETHODS FOR

FINANCIALENGINEERING

BRUNO REMILLARD

CRC PressTaylor & Francis Croup

Boca Raton London New York

CRC Press is an imprint of theTaylor & Francis Croup, an informa business

A CHAPMAN & HALL BOOK

Contents

Preface . ___. xxi

List of Figures xxv

List of Tables xxix

1 Black-Scholes Model 1Summary 11.1 The Black-Scholes Model 11.2 Dynamic Model for an Asset 2

1.2.1 Stock Exchange Data 21.2.2 Continuous Time Models 21.2.3 Joint Distribution of Returns 41.2.4 Simulation of a Geometric Brownian Motion 41.2.5 Joint Law of Prices 5

1.3 Estimation of Parameters 51.4 Estimation Errors 6

1.4.1 Estimation of Parameters for Apple 71.5 Black-Scholes Formula 9

1.5.1 European Call Option 91.5.1.1 Put-Call Parity 101.5.1.2 Early Exercise of an American Call Option . 10

1.5.2 Partial Differential Equation for Option Values . . . . 111.5.3 Option Value as an Expectation 11

1.5.3.1 Equivalent Martingale Measures and Pricingof Options 12

1.5.4 Dividends 131.5.4.1 Continuously Paid Dividends 13

1.6 Greeks 141.6.1 Greeks for a European Call Option 151.6.2 Implied Distribution 161.6.3 Error on the Option Value 161.6.4 Implied Volatility 19

1.6.4.1 Problems with Implied Volatility 201.7 Estimation of Greeks using the Broadie-Glasserman Method-

ologies ' 20

vi Contents

1.7.1 Pathwise Method 211.7.2 Likelihood Ratio Method 231.7.3 Discussion 23

1.8 Suggested Reading 241.9 Exercises 241.10 Assignment Questions 27l.A Justification of the Black-Scholes Equation 27l.B Martingales 28l.C Proof of the Results 29

l.C.l Proof of Proposition 1.3.1 29l.C.2 Proof of Proposition 1.4.1 . . . . ' . ' 301.C.3 Proof of Proposition 1.6.1 30

Bibliography 30

2 Multivariate Black-Scholes Model 33Summary 332.1 Black-Scholes Model for Several Assets 33

2.1.1 Representation of a Multivariate Brownian Motion . . 342.1.2 Simulation of Correlated Geometric Brownian Motions 342.1.3 Volatility Vector 352.1.4 Joint Distribution of the Returns 35

2.2 Estimation of Parameters 36 .2.2.1 Explicit Method 362.2.2 Numerical Method 37

2.3 . Estimation Errors 372.3.1 Parametrization with the Correlation Matrix 382.3.2 Parametrization with the Volatility Vector 382.3.3 Estimation of Parameters for Apple and Microsoft . . 40

2.4 Evaluation of Options on Several Assets 412.4.1 Partial Differential Equation for Option Values . . . . 412.4.2 Option Value as an Expectation 42

2.4.2.1 Vanilla Options 432.4.3 Exchange Option 432.4.4 Quanto Options 44

2.5 Greeks 472.5.1 Error on the Option Value 472.5.2 Extension of the Broadie-Glasserman Methodologies for

Options on Several Assets 482.6 Suggested Reading 502.7 Exercises ' 512.8 Assignment Questions 532.A Auxiliary Result 54

2.A.1 Evaluation of E {eaZN{b + cZ)} 542.B Proofs of the Results 54

2.B.1 Proof of Proposition 2.1.1 54

Contents vii

2.B.2 Proof of Proposition 2.2.1 552.B.3 Proof of Proposition 2.3.1 562.B.4 Proof of Proposition 2.3.2 562.B.5 Proof of Proposition 2.4.1 572.B.6 Proof of Proposition 2.4.2 592.B.7 Proof of Proposition 2.5.1 592.B.8 Proof of Proposition 2.5.3 59

Bibliography 61

Discussion of the Black-Scholes Model 63Summary -.- .- : 633.1 Critiques of the Model 63

3.1.1 Independence 633.1.2 Distribution of Returns and Goodness-of-Fit Tests of

Normality 663.1.3 Volatility Smile 683.1.4 Transaction Costs 68

3.2 Some Extensions of the Black-Scholes Model 693.2.1 Time-Dependent Coefficients 69

3.2.1.1 Extended Black-Scholes Formula 703.2.2 Diffusion Processes 70

3.3 Discrete Time Hedging 723.3.1 Discrete Delta Hedging 73

3.4 Optimal Quadratic Mean Hedging 743.4.1 . Offline Computations 743.4.2 Optimal Solution of the Hedging Problem 753.4.3 Relationship with Martingales 76

3.4.3.1 Market Price vs Theoretical Price 763.4.4 Markovian Models 773.4.5 Application to Geometric Random Walks 77

3.4.5.1 Illustrations 793.4.6 Incomplete Markovian Models 833.4.7 Limiting Behavior 89

3.5 Suggested Reading . \ 893.6 Exercises 903.7 Assignment Questions 923.A Tests of Serial Independence 933.B Goodness-of-Fit Tests 94

3.B.1 Cramer-von Mises Test : 953.B.1.1 Algorithms for Approximating the P-Value . 95

3.B.2 Lilliefors Test 963.C Density Estimation 96

3.C.I Examples of Kernels 973.D Limiting Behavior of the Delta Hedging Strategy 973.E Optimal Hedging for the Binomial Tree 98

viii Contents

3.F A Useful Result 100Bibliography 100

4 Measures of Risk and Performance 103Summary 1034.1 Measures of Risk 103

4.1.1 Portfolio Model 1034.1.2 VaR 1044.1.3 Expected Shortfall 1044.1.4 Coherent Measures of Risk 105

4.1.4.1 Comments . . . . . . ". T ' 1064.1.5 Coherent Measures of Risk with Respect to a Stochastic

Order 1074.1.5.1 Simple Order 1074.1.5.2 Hazard Rate Order 107

4.2 Estimation of Measures of Risk by Monte Carlo Methods . . 1084.2.1 Methodology 1094.2.2 Nonparametric Estimation of the Distribution Function 109

4.2.2.1 Precision of the Estimation of the DistributionFunction 109

4.2.3 Nonparametric Estimation of the VaR I l l4.2.3.1 Uniform Estimation of Quantiles 113

4.2.4 Estimation of Expected Shortfall 1144.2.5 Advantages and Disadvantages of the Monte Carlo

Methodology 1164.3 Measures of Risk and the Delta-Gamma Approximation . . . 116

4.3.1 Delta-Gamma Approximation 1174.3.2 Delta-Gamma-Normal Approximation 1174.3.3 Moment Generating Function and Characteristic Func-

tion of Q . . - 1184.3.4 Partial Monte Carlo Method 119

4.3.4.1 Advantages and Disadvantages of the Method-ology 120

4.3.5 Edgeworth and Cornish-Fisher Expansions 1204.3.5.1 Edgeworth Expansion for the Distribution

Function 1204.3.5.2 Advantages and Disadvantages of the Edge-

worth Expansion . . . . ; 1214.3.5.3 Cornish-Fisher Expansion 1214.3.5.4 Advantages and Disadvantages of the Cornish-

Fisher Expansion 1224.3.6 Saddlepoint Approximation 122

4.3.6.1 Approximation of the Density 1234.3.6.2 Approximation of the Distribution Function 124

Contents ix

4.3.6.3 Advantages and Disadvantages of the Method-ology 124

4.3.7 Inversion of the Characteristic Function 1254.3.7.1 Davies Approximation 1254.3.7.2 Implementation 125

4.4 Performance Measures 1264.4.1 Axiomatic Approach of Cherny-Madan 1264.4.2 The Sharpe Ratio 1274.4.3 The Sortino Ratio 1274.4.4 The Omega Ratio 128

4.4.4.1 Relationship with Expectiles 1284.4.4.2 Gaussian Case and the Sharpe Ratio . . . . 1294.4.4.3 Relationship with Stochastic Dominance . . 1304.4.4.4 Estimation of Omega and G 130

4.5 Suggested Reading 1314.6 Exercises 1314.7 Assignment Questions 1344.A Brownian Bridge 1344.B Quantiles 1354.C Mean Excess Function 135

4.C.I Estimation of the Mean Excess Function 1364.D Bootstrap Methodology 136

4.D.1 Bootstrap Algorithm 1364.E Simulation of <Q>F,a,b 1374.F Saddlepoint Approximation of a Continuous Distribution Func-

tion ,.. . 1374.G Complex Numbers in MATLAB 1384.H Gil-Pelaez Formula 1394.1 Proofs of the Results 139

4.1.1 Proof of Proposition 4.1.1 1394.1.2 Proof of Proposition 4.1.3 1404.1.3 Proof of Proposition 4.2.1 : 1414.1.4 Proof of Proposition 4.2.2 1414.1.5 Proof of Proposition 4.3.1 1424.1.6 Proof of Proposition 4.4.1 1434.1.7 Proof of Proposition 4.4.2 1434.1.8 Proof of Proposition 4.4.4 144

Bibliography . 144

5 Modeling Interest Rates 147Summary , 1475.1 Introduction 147

5.1.1 Vasicek Result 1475.2 Vasicek Model 148

5.2.1 Ornstein-Uhlenbeck Processes 149

Contents

5.2.2 Change of Measurement and Time Scales 1495.2.3 Properties of Ornstein-Uhlenbeck Processes 150

5.2.3.1 Moments of the Ornstein-Uhlenbeck Process 1505.2.3.2 Stationary Distribution of the Ornstein-

Uhlenbeck Process 1515.2.4 Value of Zero-Coupon Bonds under a Vasicek Model . 151

5.2.4.1 Vasicek Formula for the Value of a Bond . . 1525.2.4.2 Annualized Bond Yields 152

5.2.5 Estimation of the Parameters of the Vasicek Model Us-ing Zero-Coupon Bonds 1535.2.5.1 Measurement and Time Scales 1545.2.5.2 Duan Approach for the Estimation of Non Ob-

servable Data 1545.2.5.3 Joint Conditional Density of the Implied Rates 1555.2.5.4 Change of Variables Formula 1565.2.5.5 Application of the Change of Variable Formula

to the Vasicek Model 1565.2.5.6 Precision of the Estimation 158

5.3 Cox-Ingersoll-Ross (CIR) Model 1605.3.1 Representation of the Feller Process 160

5.3.1.1 Properties of the Feller Process 1625.3.1.2 Measurement and Time Scales 163

5.3.2 Value of Zero-Coupon Bonds under a CIR Model . . . 1635.3.2.1 Formula for the Value of a Zero-Coupon Bond

under the CIR Model 1645.3.2.2 Annualized Bond Yields 1655.3.2.3 Value of a Call Option on a Zero-Coupon Bond 1655.3.2.4 Put-Call Parity 166

5.3.3 Parameters Estimation of the CIR Model Using Zero-Coupon Bonds 1665.3.3.1 Measurement and Time Scales 1675.3.3.2 Joint Conditional Density of the Implied Rates 1675.3.3.3 Application of the Change of Variable Formula

for,the CIR Model 1685.3.3.4 Precision of the Estimation 169

5.4 Other Models for the Spot Rates 1705.4.1 Affine Models 171

5.5 Suggested Reading 1715.6 Exercises '. 1725.7 Assignment Questions - 1755. A Interpretation of the Stochastic Integral 1755.B Integral of a Gaussian Process 1765.C Estimation Error for a Ornstein-Uhlenbeck Process 1765.D Proofs of the Results 178

5.D.1 Proof of Proposition 5.2.1 178

Contents xi

5.D.2 Proof of Proposition 5.2.2 1785.D.3 Proof of Proposition 5.3.1 1795.D.4 Proof of Proposition 5.3.2 1805.D.5 Proof of Proposition 5.3.3 180

Bibliography 180

6 Levy Models 183Summary 1836.1 Complete Models 1836.2 Stochastic Processes with Jumps 184

6.2.1 Simulation of a Poisson Process over a Fixed Time In-terval 185

6.2.2 Jump-Diffusion Models 1856.2.3 Merton Model 1866.2.4 Kou Jump-Diffusion Model 1876.2.5 Weighted-Symmetric Models for the Jumps 187

6.3 Levy Processes 1886.3.1 Random Walk Representation 1886.3.2 Characteristics 1896.3.3 Infinitely Divisible Distributions 1906.3.4 Sample Path Properties 190

6.3.4.1 Number of Jumps of a Levy Process 1916.3.4.2 Finite Variation 191

6.4 Examples of Levy Processes 1926.4.1 Gamma Process 1926.4.2 Inverse Gaussian Process 193

6.4.2.1 Simulation of Ta3 1936.4.3 Generalized Inverse Gaussian Distribution 1946.4.4 Variance Gamma Process 1946.4.5 Levy Subordinators 195

6.5 Change of Distribution 1976.5.1 Esscher Transforms 1976.5.2 Examples of Application 198

6.5.2.1 Merton Model 1986.5.2.2 Kou Model 1996.5.2.3 Variance Gamma Process 1996.5.2.4 Normal Inverse Gaussian Process 199

6.5.3 Application to Option Pricing 1996.5.4 General Change of Measure 2006.5.5 Incompleteness - 201

6.6 Model Implementation and Estimation of Parameters . . . . 2036.6.1 Distributional Properties 204

6.6.1.1 Serial Independence 2046.6.1.2 Levy Process vs Brownian Motion 204

6.6.2 Estimation Based on the Cumulants 205

xii Contents

6.6.2.1 Estimation of the Cumulants 2066.6.2.2 Application 2076.6.2.3 Discussion 209

6.6.3 Estimation Based on the Maximum Likelihood Method 2096.7 Suggested Reading 2156.8 Exercises 2156.9 Assignment Questions 2166.A Modified Bessel Functions of the Second Kind 2176.B Asymptotic Behavior of the Cumulants 2186.C Proofs of the Results 219

6.C.I Proof of Lemma 6.5.1 . . . . . . r". 2196.C.2 Proof of Corollary 6.5.2 2196.C.3 Proof of Proposition 6.6.1 2206.C.4 Proof of Proposition 6.4.1 220

Bibliography 221

7 Stochastic Volatility Models 223Summary 2237.1 GARCH Models 223

7.1.1 GARCH(1,1) 2247.1.2 GARCH(p,q) 2267.1.3 EGARCH 2267.1.4 NGARCH 2277.1.5 GJR-GARCH 2277.1.6 Augmented GARCH 227

7.2 Estimation of Parameters 2287.2.1 Application for GARCH(p,q) Models 2297.2.2 Tests 2307.2.3 Goodness-of-Fit and Pseudo-Observations 2307.2.4 Estimation and Goodness-of-Fit When the Innovations

Are Not Gaussian 2327.3 Duan Methodology of Option Pricing 235

7.3.1 LRNVR Criterion 2357.3.2 Continuous Time Limit 237

7.3.2.1 A New Parametrization 2387.4 Stochastic Volatility Model of Hull-White 239

7.4.1 Market Price of Volatility Risk 2397.4.2 Expectations vs Partial Differential Equations 2407.4.3 Option Price as an Expectation . .' 2407.4.4 Approximation of Expectations . 242

7.4.4.1 Monte Carlo Methods 2427.4.4.2 Taylor Series Expansion 2427.4.4.3 Edgeworth and Gram-Charlier Expansions . 2437.4.4.4 Approximate Distribution 245

7.5 Stochastic Volatility Model of Heston 246

Contents xiii

7.6 Suggested Reading 2477.7 Exercises 2477.8 Assignment Questions 2497.A Khmaladze Transform 250

7.A.I Implementation Issues 2507.B Proofs of the Results 251

7.B.1 Proof of Proposition 7.1.1 2517.B.2 Proof of Proposition 7.4.1 2537.B.3 Proof of Proposition 7.4.2 254

Bibliography 254

8 Copulas and Applications 257Summary 2578.1 Weak Replication of Hedge Funds 257

8.1.1 Computation of g 2588.2 Default Risk 259

8.2.1 n-th to Default Swap 2598.2.2 Simple Model for Default Time 2608.2.3 Joint Dynamics of X* and Yi 2618.2.4 Simultaneous Evolution of Several Markov Chains . . 262

8.2.4.1 CreditMetrics 2628.2.5 Continuous Time Model 264

8.2.5.1 Modeling the Default Time of a Firm . . . . 2668.2.6 Modeling Dependence Between Several Default Times 266

8.3 Modeling Dependence 2668.3.1 An Image is Worth a Thousand Words 2678.3.2 Joint Distribution, Margins and Copulas 2698.3.3 Visualizing Dependence 269

8.4 Bivariate Copulas 2718.4.1 Examples of Copulas 2718.4.2 Sklar Theorem in the Bivariate Case 2728.4.3 Applications for Simulation : 2748.4.4 Simulation of (C/i,C/2) ~ C 2748.4.5 Modeling Dependence with Copulas 2758.4.6 Positive Quadrant Dependence (PQD) Order 276

8.5 Measures of Dependence 2768.5.1 Estimation of a Bivariate Copula 278

8.5.1.1 Precision of the Estimation of the EmpiricalCopula 278

8.5.1.2 Tests of Independence Based on the EmpiricalCopula 278

8.5.2 Kendall Function 2808.5.2.1 Estimation of Kendall Function 2818.5.2.2 Precision of the Estimation of the Kendall

Function 282

xiv Contents

8.5.2.3 Tests of Independence Based on the EmpiricalKendall Function 282

8.5.3 Kendall Tau 2868.5.3.1 Estimation of Kendall Tau 2868.5.3.2 Precision of the Estimation of Kendall Tau . 287

8.5.4 Spearman Rho 2878.5.4.1 Estimation of Spearman Rho 2888.5.4.2 Precision of the Estimation of Spearman Rho 288

8.5.5 van der Waerden Rho 2898.5.5.1 Estimation of van der Waerden Rho 2908.5.5.2 Precision of the Estimation of van der Waer-

den Rho 2908.5.6 Other Measures of Dependence 291

8.5.6.1 Estimation of p(J) 2918.5.6.2 Precision of the Estimation of p ( J ) 292

8.5.7 Serial Dependence 2928.6 Multivariate Copulas 293

8.6.1 Kendall Function 2948.6.2 Conditional Distributions 294

8.6.2.1 Applications of Theorem 8.6.2 2948.6.3 Stochastic Orders for Dependence 295

8.6.3.1 Frechet-Hoeffding Bounds 295 •8.6.3.2 Application 2968.6.3.3 Supermodular Order 296

8.7 Families of Copulas 2978.7.1 Independence Copula 2978.7.2 Elliptical Copulas 297

8.7.2.1 Estimation of p 2988.7.3 Gaussian Copula 298

8.7.3.1 Simulation of Observations from a GaussianCopula 299

8.7.4 Student Copula • 2998.7.4.1 Simulation of Observations from a Student

Copula 3008.7.5 Other Elliptical Copulas 3008.7.6 Archimedean Copulas 301

8.7.6.1 Financial Modeling 3018.7.6.2 Recursive Formulas 3018.7.6.3 Conjecture 3038.7.6.4 Kendall Tau for Archimedean Copulas . . . . 3038.7.6.5 Simulation of Observations from an Archimedean

Copula 3048.7.7 Clayton Family 304

8.7.7.1 Simulation of Observations from a ClaytonCopula 305

Contents xv

8.7.8 Gumbel Family 3058.7.8.1 Simulation of Observations from a Gumbel

Copula 3068.7.9 Frank Family 306

8.7.9.1 Simulation of Observations from a Frank Cop-ula 307

8.7.10 Ali-Mikhail-Haq Family 3088.7.10.1 Simulation of Observations from an Ali-

Mikhail-Haq Copula 3088.7.11 PQD Order for Archimedean Copula Families 3098.7.12 Farlie-Gumbel-Morgenstern Family r~. 3098.7.13 Plackett Family 3108.7.14 Other Copula Families 310

.8 Estimation of the Parameters of Copula Models 3118.8.1 Considering Serial Dependence 3118.8.2 Estimation of Parameters: The Parametric Approach . 312

8.8.2.1 Advantages and Disadvantages 3128.8.3 Estimation of- Parameters: The Semiparametric Ap-

proach 3128.8.3.1 Advantages and Disadvantages 313

8.8.4 Estimation of p for the Gaussian Copula 3138.8.5 Estimation of p and v for the Student Copula 3138.8.6 Estimation for an Archimedean Copula Family . . . . 3148.8.7 Nonparametric Estimation of a Copula 3148.8.8 Nonparametric Estimation of Kendall Function . . . . 315

i.9 Tests of Independence 3158.9.1 Test of Independence Based on the Copula 316

;.10 Tests of Goodness-of-Fit 3168.10.1 Computation of P-Values 3178.10.2 Using the Rosenblatt Transform for Goodness-of-Fit

Tests 3188.10.2.1 Computation of P-Values : 318

!.ll Example of Implementation of a Copula Model 3198.11.1 Change Point Tests 3208.11.2 Serial Independence 3208.11.3 Modeling Serial Dependence 320

8.11.3.1 Change Point Tests for the Residuals . . . . 3208.11.3.2 Goodness-of-Fit for the Distribution of Inno-

vations ' 3208.11.4 Modeling Dependence Between Innovations 321

8.11.4.1 Test of Independence for the Innovations . . 3218.11.4.2 Goodness-of-Fit for the Copula of the Innova-

tions 323S.12 Suggested Reading 325!.13 Exercises ' 326

xvi Contents •

8.14 Assignment Questions 3308. A Continuous Time Markov Chains 3318.B Tests of Independence 3328.C Polynomials Related to the Gumbel Copula 3338.D Polynomials Related to the Frank Copula 3348.E Change Point Tests 334

8.E.1 Change Point Test for the Copula 3358.F Auxiliary Results 3368.G Proofs of the Results 336

8.G.1 Proof of Proposition 8.4.1 3368.G.2 Proof of Proposition 8.4.2 . . . . " " . " ' 3378.G.3 Proof of Proposition 8.5.1 3388.G.4 Proof of Theorem 8.7.1 338

Bibliography 339

9 Filtering 345Summary 3459.1 Description of the Filtering Problem 3459.2 Kalman Filter 346

9.2.1 Model 3469.2.2 Filter Initialization 3479.2.3 Estimation of Parameters 348.9.2.4 Implementation of the Kalman Filter 348

9.2.4.1 Solution 3489.2.5 The Kalman Filter for General Linear Models 353

9.3 IMM Filter 3549.3.1 IMM Algorithm 3549.3.2 Implementation of the IMM Filter 356

9.4 General Filtering Problem 3569.4.1 Kallianpur-Striebel Formula 3569.4.2 Recursivity 3579.4.3 Implementing the Recursive Zakai Equation 3589.4.4 Solving the Filtering Problem 358

9.5 Computation of the Conditional Densities 3589.5.1 Convolution Method 3599.5.2 Kolmogorov Equation 360

9.6 Particle Filters 3609.6.1 Implementation of a Particle Filter 3609.6.2 Implementation of an Auxiliary Sampling/Importance

Resampling (ASIR) Particle Filter 3619.6.2.1 ASIRo 3639.6.2.2 ASIRi 3639.6.2.3 ASIR2 364

9.6.3 Estimation of Parameters 3659.6.3.1 Smoothed Likelihood 365

... Contents xvii

9.7 Suggested Reading 3669.8 Exercises 3679.9 Assignment Questions 3689.A Schwartz Model 3699.B Auxiliary Results 3709.C Fourier Transform 3719.D Proofs of the Results 371

9.D.1 Proof of Proposition 9.2.1 371Bibliography 372

10 Applications of Filtering 375Summary 37510.1 Estimation of ARMA Models 375

10.1.1 AR(p) Processes 37510.1.1.1 MA(q) Processes 376

10.1.2 MA Representation 37610.1.3 ARMA Processes and Filtering 377

10.1.3.1 Implementation of the Kalman Filter in theGaussian Case 378

10.1.4 Estimation of Parameters of ARMA Models 37910.2 Regime-Switching Markov Models 380

10.2.1 Serial Dependence 38010.2.2 Prediction of the Regimes 38110.2.3 Conditional Densities and Predictions 38210.2.4 Estimation of the Parameters 383

10.2.4.1 Implementation of the E-step 38310.2.5 M-step in the Gaussian Case 38410.2.6 Tests of Goodness-of-Fit 38510.2.7 Continuous Time Regime-Switching Markov Processes 388

10.3 Replication of Hedge Funds 38910.3.0.1 Measurement of Errors 390

10.3.1 Replication by Regression 39110.3.2 Replication by Kalman Filter 39110.3.3 Example of Application 391

10.4 Suggested Reading 39510.5 Exercises 39610.6 Assignment Questions 39710.A EM Algorithm '. 39810.B Sampling Moments vs Theoretical Moments 40110.C Rosenblatt Transform for the Regime-Switching Model . . . 40110.D Proofs of the Results 403

10.D.1 Proof of Proposition 10.1.1 40310.D.2 Proof of Proposition 10.1.2 404

Bibliography 404

xviii Contents

A Probability Distributions 407Summary 407A.I Introduction 407A.2 Discrete Distributions and Densities 408

A.2.1 Expected Value and Moments of Discrete Distributions 408A.3 Absolutely Continuous Distributions and Densities 410

A.3.1 Expected Value and Moments of Absolutely ContinuousDistributions 410

A.4 Characteristic Functions 412A.4.1 Inversion Formula . . . 413

A.5 Moments Generating Functions and Laplace Transform . . . 413A.5.1 Cumulants 414

A.5.1.1 Extension 415A.6 Families of Distributions 415

A.6.1 Bernoulli Distribution 415A.6.2 Binomial Distribution 416A.6.3 Poisson Distribution 416A.6.4 Geometric Distribution 417A.6.5 Negative Binomial Distribution 417A.6.6 Uniform Distribution 417A.6.7 Gaussian Distribution 418A.6.8 Log-Normal Distribution 418A.6.9 Exponential Distribution 419A.6.10 Gamma Distribution ". 420

A.6.10.1 Properties of the Gamma Function 420A.6.11 Chi-Square Distribution 421A.6.12 Non-Central Chi-Square Distribution 421

A.6.12.1 Simulation of Non-Central Chi-Square Vari-ables 421

A.6.13 Student Distribution 422A.6.14 Johnson SU Type Distributions 423A.6.15 Beta Distribution 423A.6.16 Cauchy Distribution 424A.6.17 Generalized Error Distribution 424A.6.18 Multivariate Gaussian Distribution 425

A.6.18.1 Representation of a Random Gaussian Vector 425A.6.19 Multivariate Student Distribution 426A.6.20 Elliptical Distributions 426A.6.21 Simulation of an Elliptic Distribution 429

A.7 Conditional Densities and Joint Distributions 429A. 7.1 Multiplication Formula 429A.7.2 Conditional Distribution in the Markovian Case . . . 430A.7.3 Rosenblatt Transform 430

A.8 Functions of Random Vectors 430A.9 Exercises 433

Contents xix

Bibliography 434

B Estimation of Parameters 435Summary 435B.I Maximum Likelihood Principle 435B.2 Precision of Estimators 437

B.2.1 Confidence Intervals and Confidence Regions 437B.2.2 Nonparametric Prediction Interval 437

B.3 Properties of Estimators 438B.3.1 Almost Sure Convergence . 438B.3.2 Convergence in Probability . . . .-, 438B.3.3 Convergence in Mean Square 438B.3.4 Convergence in Law 439

B.3.4.1 Delta Method 440B.3.5 Bias and Consistency 441

B.4 Central Limit Theorem for Independent Observations . . . . 441B.4.1 Consistency of the Empirical Mean 442B.4.2 Consistency of the Empirical Coefficients of Skewness

and Kurtosis 442B.4.3 Confidence Intervals I 445B.4.4 Confidence Ellipsoids 445B.4.5 Confidence Intervals II 445

B.5 Precision of Maximum Likelihood Estimator for Serially Inde-pendent Observations 446B.5.1 Estimation of Fisher Information Matrix 446

B.6 Convergence in Probability and the Central Limit Theorem forSerially Dependent Observations 448

B.7 Precision of Maximum Likelihood Estimator for Serially De-pendent Observations 448

B.8 Method of Moments 450B.9 Combining the Maximum Likelihood Method and the Method

of Moments 452B.10 M-estimators 453B.ll Suggested Reading 454B.12 Exercises \ 454Bibliography 454

Index 455