Design and Analysis of Simulation Experiments

Recent titles in the INTERNATIONAL SERIES IN OPERATIONS RESEARCH & MANAGEMENT SCIENCE Frederick S. Hillier, Series Editor, Stanford University Sethi, Yan & Zhang/ INVENTORY AND SUPPLY CHAIN MANAGEMENT WITH FORECAST

UPDATES Cox/ QUANTITATIVE HEALTH RISK ANALYSIS METHODS: Modeling the Human Health Impacts of

Antibiotics Used in Food Animals Ching & Ng/ MARKOV CHAINS: Models, Algorithms and Applications Li & Sun/ NONLINEAR INTEGER PROGRAMMING Kaliszewski/ SOFT COMPUTING FOR COMPLEX MULTIPLE CRITERIA DECISION MAKING Bouyssou et al/ EVALUATION AND DECISION MODELS WITH MULTIPLE CRITERIA: Stepping

stones for the analyst Blecker & Friedrich/ MASS CUSTOMIZATION: Challenges and Solutions Appa, Pitsoulis & Williams/ HANDBOOK ON MODELLING FOR DISCRETE OPTIMIZATION Herrmann/ HANDBOOK OF PRODUCTION SCHEDULING Axsäter/ INVENTORY CONTROL, 2nd Ed. Hall/ PATIENT FLOW: Reducing Delay in Healthcare Delivery Józefowska & Węglarz/ PERSPECTIVES IN MODERN PROJECT SCHEDULING Tian & Zhang/ VACATION QUEUEING MODELS: Theory and Applications Yan, Yin & Zhang/ STOCHASTIC PROCESSES, OPTIMIZATION, AND CONTROL THEORY

APPLICATIONS IN FINANCIAL ENGINEERING, QUEUEING NETWORKS, AND MANUFACTURING SYSTEMS

Saaty & Vargas/ DECISION MAKING WITH THE ANALYTIC NETWORK PROCESS: Economic, Political, Social & Technological Applications w. Benefits, Opportunities, Costs & Risks

Yu/ TECHNOLOGY PORTFOLIO PLANNING AND MANAGEMENT: Practical Concepts and Tools Kandiller/ PRINCIPLES OF MATHEMATICS IN OPERATIONS RESEARCH Lee & Lee/ BUILDING SUPPLY CHAIN EXCELLENCE IN EMERGING ECONOMIES Weintraub/ MANAGEMENT OF NATURAL RESOURCES: A Handbook of Operations Research Models,

Algorithms, and Implementations Hooker/ INTEGRATED METHODS FOR OPTIMIZATION Dawande et al/ THROUGHPUT OPTIMIZATION IN ROBOTIC CELLS Friesz/ NETWORK SCIENCE, NONLINEAR SCIENCE and INFRASTRUCTURE SYSTEMS Cai, Sha & Wong/ TIME-VARYING NETWORK OPTIMIZATION Mamon & Elliott/ HIDDEN MARKOV MODELS IN FINANCE del Castillo/ PROCESS OPTIMIZATION: A Statistical Approach Józefowska/JUST-IN-TIME SCHEDULING: Models & Algorithms for Computer & Manufacturing

Systems Yu, Wang & Lai/ FOREIGN-EXCHANGE-RATE FORECASTING WITH ARTIFICIAL NEURAL

NETWORKS Beyer et al/ MARKOVIAN DEMAND INVENTORY MODELS Shi & Olafsson/ NESTED PARTITIONS OPTIMIZATION: Methodology And Applications Samaniego/ SYSTEM SIGNATURES AND THEIR APPLICATIONS IN ENGINEERING RELIABILITY

* A list of the early publications in the series is at the end of the book *

Design and Analysis of Simulation

Experiments

Jack P.C. KleijnenTilburg University, Tilburg, the Netherlands

Jack P.C. Kleijnen Tilburg University Tilburg, The Netherlands Series Editor: Fred Hillier Stanford University Stanford, CA, USA Library of Congress Control Number: 2007926589 ISBN-13: 978-0-387-71812-5 e-ISBN-13: 978-0-387-71813-2 Printed on acid-free paper. © 2008 by Springer Science+Business Media, LLC All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now know or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks and similar terms, even if the are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. 9 8 7 6 5 4 3 2 1 springer.com

To my wife, Wilma

Preface

This book is the successor of several other books that I wrote on (roughly)the same topic. My first book consisted of two volumes, and was publishedin 1974/1975 (and translated into Russian in 1978). Its successor was pub-lished in 1987. In 1992, Willem van Groenendaal and I wrote a more generalbook on simulation, which included an update of parts of my 1987 book.So I thought that it was high time to write down all I know about thestatistical Design and Analysis of Simulation Experiments, which I abbre-viate to DASE (and pronounce as the girl’s name Daisy). This acronymis inspired by DACE, which stands for Design and Analysis of ComputerExperiments; the acronym DACE is popular in deterministic simulation.

In this book, I will focus on those DASE aspects that I have a certainexpertise in—I think.

Though I focus on DASE for discrete-event simulation (which includesqueueing and inventory simulations), I also discuss DASE for deterministicsimulation (applied in engineering, physics, etc.).

I discuss both computationally expensive and cheap simulations.I assume that the readers already have a basic knowledge of simulation;

e.g., they know concepts such as terminating simulation and steady-statesimulation. They should also have a basic understanding of mathematicalstatistics, including concepts such as distribution functions, averages, andvariances.

This book contains more than four hundred references. Yet, I have triedto eliminate older references that are mentioned in more recent references—unless the older reference is the origin of some important idea (so thereaders may get a historical perspective). To improve the book’s readability,

viii Preface

I try to collect references at the end of paragraphs—as much as seemsreasonable.

I recommend that the first three chapters be read in their implied order.The next chapters, however, are independent of each other, so they may beread in the order that best suits the interest of the individual reader.

I wrote this book in a foreign language (namely English, whereas Dutchis my mother tongue), so style, spelling, etc. may sometimes not be perfect:my apologies. Concerning style, I point out that I place redundant infor-mation between parentheses; the em-dash (or —) signals nonredundant,extra information. To enable readers to browse through the various chap-ters, I repeat the definition of an abbreviation in a given chapter—evenif that abbreviation has already been defined in a preceding chapter. Thebook contains paragraphs starting with the word “Note”, which upon firstreading may be skipped.

Each website address is displayed on a separate line, because a websiteaddress may be so long that it either runs over into the right margin of thepage or it must be hyphenated—but then the hyphen may be interpretedas part of the address. A comma or a period at the end of the address isnot part of the address!

I wrote this book in Scientific Workplace, which also helped me (throughits MuPAD computational engine) to solve some of the exercises that I for-mulated in this book. Winfried Minnaert (Tilburg University) introducedme to the basics of that text processor; Jozef Pijnenburg (Tilburg Univer-sity) helped me with some more advanced features.

I received valuable comments on preliminary versions of various chap-ters from the following colleagues: Ebru Angun (Galatasaray University,Istanbul), Russell Barton (Pennsylvania State), Victoria Chen (Universityof Texas at Arlington), Gabriella Dellino (Politecnico di Bari), Dick denHertog (Tilburg University), Tony Giunta (Sandia), Yao Lin (Georgia In-stitute of Technology), Carlo Meloni (Politecnico di Bari), Barry Nelson(Northwestern), William Notz (Ohio State), Huda Abdullah Rasheed (al-Mustansiriyah University, Baghdad), Wim van Beers (Tilburg University),Willem van Groenendaal (Tilburg University), Jim Wilson (North CarolinaState), and Bernard Zeigler (Arizona State).

I used a preliminary draft of this book to teach a course called “Simula-tion for Logistics” for the “Postgraduate International Program in LogisticsManagement Systems” at the Technical University Eindhoven. This helpedme to improve parts of the book. Students solved the exercises 1.6, 2.13,2.15. The names of these students are: Nicolas Avila Bruckner, Olla Gabali,Javier Gomes, Suquan Ju, Xue Li, Marıa Eugenia Martelli, Kurtulus Oner,Anna Otahalova, Pimara Pholnukulkit, Shanshan Wang, and Wei Zhang.I especially thank Xue Li and Shanshan Wang.

For that course, I also prepared PowerPoinT (PPT) slides that may alsobe downloaded in Portable Document Format (PDF) format from my webpage:

Preface ix

http://center.uvt.nl/staff/kleijnen/simwhat.html.My website also offers an update of this book, including corrections, newreferences, new exercises: visit

http://center.uvt.nl/staff/kleijnen/and click “Publications”.

Contents

1 Introduction 11.1 What is simulation? . . . . . . . . . . . . . . . . . . . . . . 11.2 What is DASE? . . . . . . . . . . . . . . . . . . . . . . . . . 71.3 DASE symbols and terms . . . . . . . . . . . . . . . . . . . 101.4 Solutions for exercises . . . . . . . . . . . . . . . . . . . . . 12

2 Low-order polynomial regression metamodels and theirdesigns: basics 152.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 162.2 Linear regression analysis: basics . . . . . . . . . . . . . . . 192.3 Linear regression analysis: first-order polynomials . . . . . . 27

2.3.1 First-order polynomial with a single factor . . . . . . 272.3.2 First-order polynomial with several factors . . . . . 28

2.4 Designs for first-order polynomials: resolution-III . . . . . . 362.4.1 2k−p designs of resolution-III . . . . . . . . . . . . . 362.4.2 Plackett-Burman designs of resolution-III . . . . . . 39

2.5 Regression analysis: factor interactions . . . . . . . . . . . . 402.6 Designs allowing two-factor interactions: resolution-IV . . . 422.7 Designs for two-factor interactions: resolution-V . . . . . . . 462.8 Regression analysis: second-order polynomials . . . . . . . . 492.9 Designs for second-degree polynomials: Central Composite

Designs (CCDs) . . . . . . . . . . . . . . . . . . . . . . . . . 502.10 Optimal designs and other designs . . . . . . . . . . . . . . 512.11 Validation of metamodels . . . . . . . . . . . . . . . . . . . 54

xii Contents

2.11.1 Coefficients of determination and correlationcoefficients . . . . . . . . . . . . . . . . . . . . . . . 54

2.11.2 Cross-validation . . . . . . . . . . . . . . . . . . . . 572.12 More simulation applications . . . . . . . . . . . . . . . . . 632.13 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . 662.14 Appendix: coding of nominal factors . . . . . . . . . . . . . 662.15 Solutions for exercises . . . . . . . . . . . . . . . . . . . . . 69

3 Classic assumptions revisited 733.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 733.2 Multivariate simulation output . . . . . . . . . . . . . . . . 74

3.2.1 Designs for multivariate simulation output . . . . . . 773.3 Nonnormal simulation output . . . . . . . . . . . . . . . . . 78

3.3.1 Realistic normality assumption? . . . . . . . . . . . 783.3.2 Testing the normality assumption . . . . . . . . . . . 793.3.3 Transformations of simulation I/O data, jackknifing,

and bootstrapping . . . . . . . . . . . . . . . . . . . 803.4 Heterogeneous simulation output variances . . . . . . . . . 87

3.4.1 Realistic constant variance assumption? . . . . . . . 873.4.2 Testing for constant variances . . . . . . . . . . . . . 883.4.3 Variance stabilizing transformations . . . . . . . . . 893.4.4 LS estimators in case of heterogeneous variances . . 893.4.5 Designs in case of heterogeneous variances . . . . . . 92

3.5 Common random numbers (CRN) . . . . . . . . . . . . . . 933.5.1 Realistic CRN assumption? . . . . . . . . . . . . . . 943.5.2 Alternative analysis methods . . . . . . . . . . . . . 943.5.3 Designs in case of CRN . . . . . . . . . . . . . . . . 96

3.6 Nonvalid low-order polynomial metamodel . . . . . . . . . . 973.6.1 Testing the validity of the metamodel . . . . . . . . 973.6.2 Transformations of independent and dependent

regression variables . . . . . . . . . . . . . . . . . . . 983.6.3 Adding high-order terms to a low-order polynomial

metamodel . . . . . . . . . . . . . . . . . . . . . . . 983.6.4 Nonlinear metamodels . . . . . . . . . . . . . . . . . 99

3.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . 993.8 Solutions for exercises . . . . . . . . . . . . . . . . . . . . . 100

4 Simulation optimization 1014.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 1014.2 RSM: classic variant . . . . . . . . . . . . . . . . . . . . . . 1054.3 Generalized RSM: multiple outputs and constraints . . . . . 1104.4 Testing an estimated optimum: KKT conditions . . . . . . . 1164.5 Risk analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 123

4.5.1 Latin Hypercube Sampling (LHS) . . . . . . . . . . 1264.6 Robust optimization: Taguchian approach . . . . . . . . . . 130

Contents xiii

4.6.1 Case study: Ericsson’s supply chain . . . . . . . . . 1354.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . 1374.8 Solutions for exercises . . . . . . . . . . . . . . . . . . . . . 138

5 Kriging metamodels 1395.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 1395.2 Kriging basics . . . . . . . . . . . . . . . . . . . . . . . . . . 1405.3 Kriging: new results . . . . . . . . . . . . . . . . . . . . . . 1475.4 Designs for Kriging . . . . . . . . . . . . . . . . . . . . . . . 149

5.4.1 Predictor variance in random simulation . . . . . . . 1515.4.2 Predictor variance in deterministic simulation . . . . 1525.4.3 Related designs . . . . . . . . . . . . . . . . . . . . . 154

5.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . 1555.6 Solutions for exercises . . . . . . . . . . . . . . . . . . . . . 156

6 Screening designs 1576.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 1576.2 Sequential Bifurcation . . . . . . . . . . . . . . . . . . . . . 160

6.2.1 Outline of simplest SB . . . . . . . . . . . . . . . . . 1606.2.2 Mathematical details of simplest SB . . . . . . . . . 1656.2.3 Case study: Ericsson’s supply chain . . . . . . . . . 1676.2.4 SB with two-factor interactions . . . . . . . . . . . . 169

6.3 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . 1716.4 Solutions for exercises . . . . . . . . . . . . . . . . . . . . . 172

7 Epilogue 173

References 175

Index 211