Rubin H. Landau, Manuel J. Páez, Cristian C. Bordeianu: Computational Physics —2015/5/5 — page VII — le-tex

VII

Contents

Dedication V

Preface XIX

1 Introduction 11.1 Computational Physics and Computational Science 11.2 This Book’s Subjects 31.3 This Book’s Problems 41.4 This Book’s Language: The Python Ecosystem 81.4.1 Python Packages (Libraries) 91.4.2 This Book’s Packages 101.4.3 The Easy Way: Python Distributions (Package Collections) 121.5 Python’s Visualization Tools 131.5.1 Visual (VPython)’s 2D Plots 141.5.2 VPython’s Animations 171.5.3 Matplotlib’s 2D Plots 171.5.4 Matplotlib’s 3D Surface Plots 221.5.5 Matplotlib’s Animations 241.5.6 Mayavi’s Visualizations Beyond Plotting 261.6 Plotting Exercises 301.7 Python’s Algebraic Tools 31

2 Computing Software Basics 332.1 Making Computers Obey 332.2 Programming Warmup 352.2.1 Structured and Reproducible Program Design 362.2.2 Shells, Editors, and Execution 372.3 Python I/O 392.4 Computer Number Representations (Theory) 402.4.1 IEEE Floating-Point Numbers 412.4.2 Python and the IEEE 754 Standard 472.4.3 Over and Underflow Exercises 482.4.4 Machine Precision (Model) 49

Rubin H. Landau, Manuel J. Páez, Cristian C. Bordeianu: Computational Physics —2015/5/5 — page VIII — le-tex

VIII Contents

2.4.5 Experiment: Your Machine’s Precision 502.5 Problem: Summing Series 512.5.1 Numerical Summation (Method) 512.5.2 Implementation and Assessment 52

3 Errors and Uncertainties in Computations 533.1 Types of Errors (Theory) 533.1.1 Model for Disaster: Subtractive Cancelation 553.1.2 Subtractive Cancelation Exercises 563.1.3 Round-off Errors 573.1.4 Round-off Error Accumulation 583.2 Error in Bessel Functions (Problem) 583.2.1 Numerical Recursion (Method) 593.2.2 Implementation and Assessment: Recursion Relations 613.3 Experimental Error Investigation 623.3.1 Error Assessment 65

4 Monte Carlo: Randomness, Walks, and Decays 694.1 Deterministic Randomness 694.2 Random Sequences (Theory) 694.2.1 Random-Number Generation (Algorithm) 704.2.2 Implementation: Random Sequences 724.2.3 Assessing Randomness and Uniformity 734.3 RandomWalks (Problem) 754.3.1 Random-Walk Simulation 764.3.2 Implementation: RandomWalk 774.4 Extension: Protein Folding and Self-Avoiding RandomWalks 794.5 Spontaneous Decay (Problem) 804.5.1 Discrete Decay (Model) 814.5.2 Continuous Decay (Model) 824.5.3 Decay Simulation with Geiger Counter Sound 824.6 Decay Implementation and Visualization 84

5 Differentiation and Integration 855.1 Differentiation 855.2 Forward Difference (Algorithm) 865.3 Central Difference (Algorithm) 875.4 Extrapolated Difference (Algorithm) 875.5 Error Assessment 885.6 Second Derivatives (Problem) 905.6.1 Second-Derivative Assessment 905.7 Integration 915.8 Quadrature as Box Counting (Math) 915.9 Algorithm: Trapezoid Rule 935.10 Algorithm: Simpson’s Rule 94

Rubin H. Landau, Manuel J. Páez, Cristian C. Bordeianu: Computational Physics —2015/5/5 — page IX — le-tex

IXContents

5.11 Integration Error (Assessment) 965.12 Algorithm: Gaussian Quadrature 975.12.1 Mapping Integration Points 985.12.2 Gaussian Points Derivation 995.12.3 Integration Error Assessment 1005.13 Higher Order Rules (Algorithm) 1035.14 Monte Carlo Integration by Stone Throwing (Problem) 1045.14.1 Stone Throwing Implementation 1045.15 Mean Value Integration (Theory and Math) 1055.16 Integration Exercises 1065.17 Multidimensional Monte Carlo Integration (Problem) 1085.17.1 Multi Dimension Integration Error Assessment 1095.17.2 Implementation: 10DMonte Carlo Integration 1105.18 Integrating Rapidly Varying Functions (Problem) 1105.19 Variance Reduction (Method) 1105.20 Importance Sampling (Method) 1115.21 von Neumann Rejection (Method) 1115.21.1 Simple Random Gaussian Distribution 1135.22 Nonuniform Assessment ⊙ 1135.22.1 Implementation ⊙ 114

6 Matrix Computing 1176.1 Problem 3: N–D Newton–Raphson; Two Masses on a String 1176.1.1 Theory: Statics 1186.1.2 Algorithm: Multidimensional Searching 1196.2 Why Matrix Computing? 1226.3 Classes of Matrix Problems (Math) 1226.3.1 Practical Matrix Computing 1246.4 Python Lists as Arrays 1266.5 Numerical Python (NumPy) Arrays 1276.5.1 NumPy’s linalg Package 1326.6 Exercise: Testing Matrix Programs 1346.6.1 Matrix Solution of the String Problem 1376.6.2 Explorations 139

7 Trial-and-Error Searching and Data Fitting 1417.1 Problem 1: A Search for Quantum States in a Box 1417.2 Algorithm: Trial-and-Error Roots via Bisection 1427.2.1 Implementation: Bisection Algorithm 1447.3 Improved Algorithm: Newton–Raphson Searching 1457.3.1 Newton–Raphson with Backtracking 1477.3.2 Implementation: Newton–Raphson Algorithm 1487.4 Problem 2: Temperature Dependence of Magnetization 1487.4.1 Searching Exercise 1507.5 Problem 3: Fitting An Experimental Spectrum 150

Rubin H. Landau, Manuel J. Páez, Cristian C. Bordeianu: Computational Physics — 2015/5/5 — page X — le-tex

X Contents

7.5.1 Lagrange Implementation, Assessment 1527.5.2 Cubic Spline Interpolation (Method) 1537.6 Problem 4: Fitting Exponential Decay 1567.7 Least-Squares Fitting (Theory) 1587.7.1 Least-Squares Fitting: Theory and Implementation 1607.8 Exercises: Fitting Exponential Decay, Heat Flow andHubble’s Law 1627.8.1 Linear Quadratic Fit 1647.8.2 Problem 5: Nonlinear Fit to a Breit–Wigner 167

8 Solving Differential Equations: Nonlinear Oscillations 1718.1 Free Nonlinear Oscillations 1718.2 Nonlinear Oscillators (Models) 1718.3 Types of Differential Equations (Math) 1738.4 Dynamic Form for ODEs (Theory) 1758.5 ODE Algorithms 1778.5.1 Euler’s Rule 1778.6 Runge–Kutta Rule 1788.7 Adams–Bashforth–Moulton Predictor–Corrector Rule 1838.7.1 Assessment: rk2 vs. rk4 vs. rk45 1858.8 Solution for Nonlinear Oscillations (Assessment) 1878.8.1 Precision Assessment: Energy Conservation 1888.9 Extensions: Nonlinear Resonances, Beats, Friction 1898.9.1 Friction (Model) 1898.9.2 Resonances and Beats: Model, Implementation 1908.10 Extension: Time-Dependent Forces 190

9 ODE Applications: Eigenvalues, Scattering, and Projectiles 1939.1 Problem: Quantum Eigenvalues in Arbitrary Potential 1939.1.1 Model: Nucleon in a Box 1949.2 Algorithms: Eigenvalues via ODE Solver + Search 1959.2.1 Numerov Algorithm for Schrödinger ODE⊙ 1979.2.2 Implementation: Eigenvalues viaODESolver+BisectionAlgorithm 2009.3 Explorations 2039.4 Problem: Classical Chaotic Scattering 2039.4.1 Model and Theory 2049.4.2 Implementation 2069.4.3 Assessment 2079.5 Problem: Balls Falling Out of the Sky 2089.6 Theory: Projectile Motion with Drag 2089.6.1 Simultaneous Second-Order ODEs 2099.6.2 Assessment 2109.7 Exercises: 2- and 3-Body Planet Orbits and Chaotic Weather 211

10 High-Performance Hardware and Parallel Computers 21510.1 High-Performance Computers 215

Rubin H. Landau, Manuel J. Páez, Cristian C. Bordeianu: Computational Physics —2015/5/5 — page XI — le-tex

XIContents

10.2 Memory Hierarchy 21610.3 The Central Processing Unit 21910.4 CPU Design: Reduced Instruction Set Processors 22010.5 CPU Design: Multiple-Core Processors 22110.6 CPU Design: Vector Processors 22210.7 Introduction to Parallel Computing 22310.8 Parallel Semantics (Theory) 22410.9 Distributed Memory Programming 22610.10 Parallel Performance 22710.10.1 Communication Overhead 22910.11 Parallelization Strategies 23010.12 Practical Aspects of MIMDMessage Passing 23110.12.1 High-Level View of Message Passing 23310.12.2 Message Passing Example and Exercise 23410.13 Scalability 23610.13.1 Scalability Exercises 23810.14 Data Parallelism and Domain Decomposition 23910.14.1 Domain Decomposition Exercises 24210.15 Example: The IBM Blue Gene Supercomputers 24310.16 Exascale Computing via Multinode-Multicore GPUs 245

11 Applied HPC: Optimization, Tuning, and GPU Programming 24711.1 General Program Optimization 24711.1.1 Programming for Virtual Memory (Method) 24811.1.2 Optimization Exercises 24911.2 Optimized Matrix Programming with NumPy 25111.2.1 NumPy Optimization Exercises 25411.3 Empirical Performance of Hardware 25411.3.1 Racing Python vs. Fortran/C 25511.4 Programming for the Data Cache (Method) 26211.4.1 Exercise 1: Cache Misses 26411.4.2 Exercise 2: Cache Flow 26411.4.3 Exercise 3: Large-Matrix Multiplication 26511.5 Graphical Processing Units for High Performance Computing 26611.5.1 The GPU Card 26711.6 Practical Tips for Multicore and GPU Programming ⊙ 26711.6.1 CUDA Memory Usage 27011.6.2 CUDA Programming ⊙ 271

12 Fourier Analysis: Signals and Filters 27512.1 Fourier Analysis of Nonlinear Oscillations 27512.2 Fourier Series (Math) 27612.2.1 Examples: Sawtooth and Half-Wave Functions 27812.3 Exercise: Summation of Fourier Series 27912.4 Fourier Transforms (Theory) 279

Rubin H. Landau, Manuel J. Páez, Cristian C. Bordeianu: Computational Physics —2015/5/5 — page XII — le-tex

XII Contents

12.5 The Discrete Fourier Transform 28112.5.1 Aliasing (Assessment) 28512.5.2 Fourier Series DFT (Example) 28712.5.3 Assessments 28812.5.4 Nonperiodic Function DFT (Exploration) 29012.6 Filtering Noisy Signals 29012.7 Noise Reduction via Autocorrelation (Theory) 29012.7.1 Autocorrelation Function Exercises 29312.8 Filtering with Transforms (Theory) 29412.8.1 Digital Filters: Windowed Sinc Filters (Exploration)⊙ 29612.9 The Fast Fourier Transform Algorithm ⊙ 29912.9.1 Bit Reversal 30112.10 FFT Implementation 30312.11 FFT Assessment 304

13 Wavelet and Principal Components Analyses: Nonstationary Signals andData Compression 307

13.1 Problem: Spectral Analysis of Nonstationary Signals 30713.2 Wavelet Basics 30713.3 Wave Packets and Uncertainty Principle (Theory) 30913.3.1 Wave Packet Assessment 31113.4 Short-Time Fourier Transforms (Math) 31113.5 TheWavelet Transform 31313.5.1 Generating Wavelet Basis Functions 31313.5.2 Continuous Wavelet Transform Implementation 31613.6 Discrete Wavelet Transforms, Multiresolution Analysis ⊙ 31713.6.1 Pyramid Scheme Implementation ⊙ 32313.6.2 Daubechies Wavelets via Filtering 32713.6.3 DWT Implementation and Exercise 33013.7 Principal Components Analysis 33213.7.1 Demonstration of Principal Component Analysis 33413.7.2 PCA Exercises 337

14 Nonlinear Population Dynamics 33914.1 Bug Population Dynamics 33914.2 The Logistic Map (Model) 33914.3 Properties of Nonlinear Maps (Theory and Exercise) 34114.3.1 Fixed Points 34214.3.2 Period Doubling, Attractors 34314.4 Mapping Implementation 34414.5 Bifurcation Diagram (Assessment) 34514.5.1 Bifurcation Diagram Implementation 34614.5.2 Visualization Algorithm: Binning 34714.5.3 Feigenbaum Constants (Exploration) 34814.6 Logistic Map Random Numbers (Exploration) ⊙ 348

Rubin H. Landau, Manuel J. Páez, Cristian C. Bordeianu: Computational Physics —2015/5/5 — page XIII — le-tex

XIIIContents

14.7 Other Maps (Exploration) 34814.8 Signals of Chaos: Lyapunov Coefficient and Shannon Entropy ⊙ 34914.9 Coupled Predator–Prey Models 35314.10 Lotka–Volterra Model 35414.10.1 Lotka–Volterra Assessment 35614.11 Predator–Prey Chaos 35614.11.1 Exercises 35914.11.2 LVM with Prey Limit 35914.11.3 LVM with Predation Efficiency 36014.11.4 LVM Implementation and Assessment 36114.11.5 Two Predators, One Prey (Exploration) 362

15 Continuous Nonlinear Dynamics 36315.1 Chaotic Pendulum 36315.1.1 Free Pendulum Oscillations 36415.1.2 Solution as Elliptic Integrals 36515.1.3 Implementation and Test: Free Pendulum 36615.2 Visualization: Phase-Space Orbits 36715.2.1 Chaos in Phase Space 36815.2.2 Assessment in Phase Space 37215.3 Exploration: Bifurcations of Chaotic Pendulums 37415.4 Alternate Problem: The Double Pendulum 37515.5 Assessment: Fourier/Wavelet Analysis of Chaos 37715.6 Exploration: Alternate Phase-Space Plots 37815.7 Further Explorations 379

16 Fractals and Statistical Growth Models 38316.1 Fractional Dimension (Math) 38316.2 The Sierpiński Gasket (Problem 1) 38416.2.1 Sierpiński Implementation 38416.2.2 Assessing Fractal Dimension 38516.3 Growing Plants (Problem 2) 38616.3.1 Self-Affine Connection (Theory) 38616.3.2 Barnsley’s Fern Implementation 38716.3.3 Self-Affinity in Trees Implementation 38916.4 Ballistic Deposition (Problem 3) 39016.4.1 Random Deposition Algorithm 39016.5 Length of British Coastline (Problem 4) 39116.5.1 Coastlines as Fractals (Model) 39216.5.2 Box Counting Algorithm 39216.5.3 Coastline Implementation and Exercise 39316.6 Correlated Growth, Forests, Films (Problem 5) 39516.6.1 Correlated Ballistic Deposition Algorithm 39516.7 Globular Cluster (Problem 6) 39616.7.1 Diffusion-Limited Aggregation Algorithm 396

Rubin H. Landau, Manuel J. Páez, Cristian C. Bordeianu: Computational Physics —2015/5/5 — page XIV — le-tex

XIV Contents

16.7.2 Fractal Analysis of DLA or a Pollock 39916.8 Fractals in Bifurcation Plot (Problem 7) 40016.9 Fractals from Cellular Automata 40016.10 Perlin Noise Adds Realism ⊙ 40216.10.1 Ray Tracing Algorithms 40416.11 Exercises 407

17 Thermodynamic Simulations and Feynman Path Integrals 40917.1 Magnets via Metropolis Algorithm 40917.2 An Ising Chain (Model) 41017.3 Statistical Mechanics (Theory) 41217.3.1 Analytic Solution 41317.4 Metropolis Algorithm 41317.4.1 Metropolis Algorithm Implementation 41617.4.2 Equilibration, Thermodynamic Properties (Assessment) 41717.4.3 Beyond Nearest Neighbors, 1D (Exploration) 41917.5 Magnets via Wang–Landau Sampling ⊙ 42017.6 Wang–Landau Algorithm 42317.6.1 WLS Ising Model Implementation 42517.6.2 WLS Ising Model Assessment 42817.7 Feynman Path Integral Quantum Mechanics⊙ 42917.8 Feynman’s Space–Time Propagation (Theory) 42917.8.1 Bound-StateWave Function (Theory) 43117.8.2 Lattice Path Integration (Algorithm) 43217.8.3 Lattice Implementation 43717.8.4 Assessment and Exploration 44017.9 Exploration: Quantum Bouncer’s Paths⊙ 440

18 Molecular Dynamics Simulations 44518.1 Molecular Dynamics (Theory) 44518.1.1 Connection to Thermodynamic Variables 44918.1.2 Setting Initial Velocities 44918.1.3 Periodic Boundary Conditions and Potential Cutoff 45018.2 Verlet and Velocity–Verlet Algorithms 45118.3 1D Implementation and Exercise 45318.4 Analysis 456

19 PDE Review and Electrostatics via Finite Differences and Electrostatics viaFinite Differences 461

19.1 PDE Generalities 46119.2 Electrostatic Potentials 46319.2.1 Laplace’s Elliptic PDE (Theory) 46319.3 Fourier Series Solution of a PDE 46419.3.1 Polynomial Expansion as an Algorithm 46619.4 Finite-Difference Algorithm 467

Rubin H. Landau, Manuel J. Páez, Cristian C. Bordeianu: Computational Physics —2015/5/5 — page XV — le-tex

XVContents

19.4.1 Relaxation and Over-relaxation 46919.4.2 Lattice PDE Implementation 47019.5 Assessment via Surface Plot 47119.6 Alternate Capacitor Problems 47119.7 Implementation and Assessment 47419.8 Electric Field Visualization (Exploration) 47519.9 Review Exercise 476

20 Heat Flow via Time Stepping 47720.1 Heat Flow via Time-Stepping (Leapfrog) 47720.2 The Parabolic Heat Equation (Theory) 47820.2.1 Solution: Analytic Expansion 47820.2.2 Solution: Time Stepping 47920.2.3 von Neumann Stability Assessment 48120.2.4 Heat Equation Implementation 48320.3 Assessment and Visualization 48320.4 Improved Heat Flow: Crank–Nicolson Method 48420.4.1 Solution of Tridiagonal Matrix Equations ⊙ 48720.4.2 Crank–Nicolson Implementation, Assessment 490

21 Wave Equations I: Strings and Membranes 49121.1 A Vibrating String 49121.2 The Hyperbolic Wave Equation (Theory) 49121.2.1 Solution via Normal-Mode Expansion 49321.2.2 Algorithm: Time Stepping 49421.2.3 Wave Equation Implementation 49621.2.4 Assessment, Exploration 49721.3 Strings with Friction (Extension) 49921.4 Strings with Variable Tension and Density 50021.4.1 Waves on Catenary 50121.4.2 Derivation of Catenary Shape 50121.4.3 Catenary and Frictional Wave Exercises 50321.5 Vibrating Membrane (2DWaves) 50421.6 Analytical Solution 50521.7 Numerical Solution for 2DWaves 508

22 Wave Equations II: Quantum Packets and Electromagnetic 51122.1 Quantum Wave Packets 51122.2 Time-Dependent Schrödinger Equation (Theory) 51122.2.1 Finite-Difference Algorithm 51322.2.2 Wave Packet Implementation, Animation 51422.2.3 Wave Packets in Other Wells (Exploration) 51622.3 Algorithm for the 2D Schrödinger Equation 51722.3.1 Exploration: Bound and Diffracted 2D Packet 51822.4 Wave Packet–Wave Packet Scattering 518

Rubin H. Landau, Manuel J. Páez, Cristian C. Bordeianu: Computational Physics —2015/5/5 — page XVI — le-tex

XVI Contents

22.4.1 Algorithm 52022.4.2 Implementation 52022.4.3 Results and Visualization 52222.5 E&MWaves via Finite-Difference Time Domain 52522.6 Maxwell’s Equations 52522.7 FDTD Algorithm 52622.7.1 Implementation 53022.7.2 Assessment 53022.7.3 Extension: Circularly Polarized Waves 53122.8 Application: Wave Plates 53322.9 Algorithm 53422.10 FDTD Exercise and Assessment 535

23 Electrostatics via Finite Elements 53723.1 Finite-Element Method⊙ 53723.2 Electric Field from Charge Density (Problem) 53823.3 Analytic Solution 53823.4 Finite-Element (Not Difference) Methods, 1D 53923.4.1 Weak Form of PDE 53923.4.2 Galerkin Spectral Decomposition 54023.5 1D FEM Implementation and Exercises 54423.5.1 1D Exploration 54723.6 Extension to 2D Finite Elements 54723.6.1 Weak Form of PDE 54823.6.2 Galerkin’s Spectral Decomposition 54823.6.3 Triangular Elements 54923.6.4 Solution as Linear Equations 55123.6.5 Imposing Boundary Conditions 55223.6.6 FEM 2D Implementation and Exercise 55423.6.7 FEM 2D Exercises 554

24 Shocks Waves and Solitons 55524.1 Shocks and Solitons in ShallowWater 55524.2 Theory: Continuity and Advection Equations 55624.2.1 Advection Implementation 55824.3 Theory: ShockWaves via Burgers’ Equation 55924.3.1 Lax–Wendroff Algorithm for Burgers’ Equation 56024.3.2 Implementation and Assessment of Burgers’ Shock Equation 56124.4 Including Dispersion 56224.5 Shallow-Water Solitons: The KdeV Equation 56324.5.1 Analytic Soliton Solution 56324.5.2 Algorithm for KdeV Solitons 56424.5.3 Implementation: KdeV Solitons 56524.5.4 Exploration: Solitons in Phase Space, Crossing 56724.6 Solitons on Pendulum Chain 567

Rubin H. Landau, Manuel J. Páez, Cristian C. Bordeianu: Computational Physics —2015/5/5 — page XVII — le-tex

XVIIContents

24.6.1 Including Dispersion 56824.6.2 Continuum Limit, the Sine-Gordon Equation 57024.6.3 Analytic SGE Solution 57124.6.4 Numeric Solution: 2D SGE Solitons 57124.6.5 2D Soliton Implementation 57324.6.6 SGE Soliton Visualization 574

25 Fluid Dynamics 57525.1 River Hydrodynamics 57525.2 Navier–Stokes Equation (Theory) 57625.2.1 Boundary Conditions for Parallel Plates 57825.2.2 Finite-Difference Algorithm and Overrelaxation 58025.2.3 Successive Overrelaxation Implementation 58125.3 2D Flow over a Beam 58125.4 Theory: Vorticity Form of Navier–Stokes Equation 58225.4.1 Finite Differences and the SOR Algorithm 58425.4.2 Boundary Conditions for a Beam 58525.4.3 SOR on a Grid 58725.4.4 Flow Assessment 58925.4.5 Exploration 590

26 Integral Equations of QuantumMechanics 59126.1 Bound States of Nonlocal Potentials 59126.2 Momentum–Space Schrödinger Equation (Theory) 59226.2.1 Integral to Matrix Equations 59326.2.2 Delta-Shell Potential (Model) 59526.2.3 Binding Energies Solution 59526.2.4 Wave Function (Exploration) 59726.3 Scattering States of Nonlocal Potentials ⊙ 59726.4 Lippmann–Schwinger Equation (Theory) 59826.4.1 Singular Integrals (Math) 59926.4.2 Numerical Principal Values 60026.4.3 Reducing Integral Equations to Matrix Equations (Method) 60026.4.4 Solution via Inversion, Elimination 60226.4.5 Scattering Implementation 60326.4.6 Scattering Wave Function (Exploration) 604

Appendix A Codes, Applets, and Animations 607

Bibliography 609

Index 615

Rubin H. Landau, Manuel J. Páez, Cristian C. Bordeianu: Computational Physics —2015/5/5 — page XVIII — le-tex