1 Statistical Physics: Is More than Statistical Mechanics ...... 1
Part I Modeling of Statistical Systems
2 Random Variables: Fundamentals of Probability Theory
and Statistics ............................................... 5
2.1 Probability and Random Variables ........................ 6
2.1.1 The Space of Events .............................. 6
2.1.2 Introduction of Probability ...................... 7
2.1.3 Random Variables ................................. 9
2.2 Multivariate Random Variables and Conditional
Probabilities .......................................... 13
2.2.1 Multidimensional Random Variables ............... 13
2.2.2 Marginal Densities .............................. 14
2.2.3 Conditional Probabilities and Bayes' Theorem .... 14
2.3 Moments and Quantiles .................................. 19
2.3.1 Moments ......................................... 19
2.3.2 Quantiles ....................................... 23
2.4 The Entropy ............................................ 25
2.4.1 Entropy for a Discrete Set of Events ............ 25
2.4.2 Entropy for a Continuous Space of Events ........ 26
2.4.3 Relative Entropy ................................ 27
2.4.4 Remarks ......................................... 28
2.4.5 Applications .................................... 28
2.5 Computations with Random Variables ..................... 32
2.5.1 Addition and Multiplication of Random
Variables ....................................... 32
2.5.2 Further Important Random Variables .............. 36
2.5.3 Limit Theorems .................................. 38
2.6 Stable Random Variables and Renormalization
Transformations ........................................ 40
2.6.1 Stable Random Variables ......................... 41
2.6.2 The Renormalization Transformation .............. 43
2.6.3 Stability Analysis .............................. 44
2.6.4 Scaling Behavior ................................ 46
2.7 The Large Deviation Property for Sums of Random
Variables .............................................. 47
3 Random Variables in State Space: Classical Statistical
Mechanics of Fluids ......................................... 53
3.1 The Microcanonical System .............................. 54
3.2 Systems in Contact ..................................... 57
3.2.1 Thermal Contact ................................. 57
3.2.2 Systems with Exchange of Volume and Energy ...... 65
3.2.3 Systems with Exchange of Particles and Energy ... 70
3.3 Thermodynamic Potentials ............................... 73
3.4 Susceptibilities ....................................... 79
3.4.1 Heat Capacities ................................. 79
3.4.2 Isothermal Compressibility ...................... 81
3.4.3 Isobaric Expansivity ............................ 82
3.4.4 Isochoric Tension Coefficient and Adiabatic
Compressibility ................................. 83
3.4.5 A General Relation Between Response Functions ... 83
3.5 The Equipartition Theorem .............................. 85
3.6 The Radial Distribution Function ....................... 87
3.7 Approximation Methods .................................. 94
3.7.1 The Virial Expansion ............................ 94
3.7.2 Integral Equations for the Radial Distribution
Function ....................................... 102
3.7.3 Perturbation Theory ............................ 104
3.8 The van der Waals Equation ............................ 107
3.8.1 The Isotherms .................................. 108
3.8.2 The Maxwell Construction ....................... 110
3.8.3 Corresponding States ........................... 112
3.8.4 Critical Exponents ............................. 114
3.9 Some General Remarks about Phase Transitions and
Phase Diagrams ........................................ 116
4 Random Fields: Textures and Classical Statistical
Mechanics of Spin Systems .................................. 121
4.1 Discrete Stochastic Fields ............................ 122
4.1.1 Markov Fields .................................. 123
4.1.2 Gibbs Fields ................................... 125
4.1.3 Equivalence of Gibbs and Markov Fields ......... 126
4.2 Examples of Markov Random Fields ...................... 127
4.2.1 Model with Independent Random Variables ........ 127
4.2.2 Auto Model ..................................... 128
4.2.3 Multilevel Logistic Model ...................... 129
4.2.4 Gauss Model .................................... 130
4.3 Characteristic Quantities of Densities for Random
Fields ................................................ 130
4.4 Simple Random Fields .................................. 133
4.4.1 The White Random Field or the Ideal
Paramagnetic System ............................ 133
4.4.2 The One-Dimensional Ising Model ................ 135
4.5 Random Fields with Phase Transitions .................. 139
4.5.1 The Curie-Weiss Model .......................... 139
4.5.2 The Mean Field Approximation ................... 141
4.5.3 The Two-Dimensional Ising Model ................ 146
4.6 The Landau Free Energy ................................ 152
4.7 The Renormalization Group Method for Random Fields
and Scaling Laws ...................................... 155
4.7.1 The Renormalization Transformation ............. 155
4.7.2 Scaling Laws ................................... 156
5 Time-Dependent Random Variables: Classical Stochastic
Processes .................................................. 161
5.1 Markov Processes ...................................... 162
5.2 The Master Equation ................................... 166
5.3 Examples of Master Equations .......................... 172
5.3.1 One-Step Processes ............................. 172
5.3.2 Chemical Reactions ............................. 175
5.3.3 Reaction-Diffusion Systems ..................... 176
5.3.4 Scattering Processes ........................... 177
5.4 Analytic Solutions of Master Equations ................ 178
5.4.1 Equations for the Moments ...................... 178
5.4.2 The Equation for the Characteristic Function ... 179
5.4.3 Examples ....................................... 180
5.5 Simulation of Stochastic Processes and Fields ......... 183
5.6 The Fokker-Planck Equation ............................ 191
5.6.1 Fokker-Planck Equation with Linear Drift Term
and Additive Noise ............................. 194
5.7 The Linear Response Function and the Fluctuation-
Dissipation Theorem ................................... 197
5.8 Approximation Methods ................................. 200
5.8.1 The Q Expansion ................................ 201
5.8.2 The One-Particle Picture ....................... 207
5.9 More General Stochastic Processes ..................... 210
5.9.1 Self-Similar Processes ......................... 210
5.9.2 Fractal Brownian Motion ........................ 211
5.9.3 Stable Levy Processes .......................... 212
5.9.4 Autoregressive Processes ....................... 213
6 Quantum Random Systems ..................................... 221
6.1 Quantum-Mechanical Description of Statistical
Systems ............................................... 221
6.2 Ideal Quantum Systems: General Considerations ......... 227
6.2.1 Expansion in the Classical Regime .............. 230
6.2.2 First Quantum-Mechanical Correction Term ....... 232
6.2.3 Relations Between the Thermodynamic Potential
and Other System Variables ..................... 233
6.3 The Ideal Fermi Gas ................................... 234
6.3.1 The Fermi-Dirac Distribution ................... 234
6.3.2 Determination of the System Variables at Low
Temperatures ................................... 237
6.3.3 Applications of the Fermi-Dirac Distribution ... 240
6.4 The Ideal Bose Gas .................................... 243
6.4.1 Particle Number and the Bose-Einstein
Distribution ................................... 243
6.4.2 Bose-Einstein Condensation ..................... 245
6.4.3 Pressure ....................................... 248
6.4.4 Energy and Specific Heat ....................... 250
6.4.5 Entropy ........................................ 252
6.4.6 Applications of Bose Statistics ................ 252
6.5 The Photon Gas and Black Body Radiation ............... 253
6.5.1 The Kirchhoff Law .............................. 259
6.5.2 The Stefan-Boltzmann Law ....................... 261
6.5.3 The Pressure of Light .......................... 262
6.5.4 The Total Radiative Power of the Sun ........... 264
6.5.5 The Cosmic Background Radiation ................ 266
6.6 Lattice Vibrations in Solids: The Phonon Gas .......... 267
6.7 Systems with Internal Degrees of Freedom: Ideal
Gases of Molecules .................................... 274
6.8 Magnetic Properties of Fermi Systems .................. 281
6.8.1 Diamagnetism ................................... 281
6.8.2 Paramagnetism .................................. 286
6.9 Quasi-Particles ....................................... 287
6.9.1 Models for the Magnetic Properties of Solids ... 290
6.9.2 Superfluidity .................................. 295
7 Changes of External Conditions ............................. 297
7.1 Reversible State Transformations, Heat, and Work ...... 297
7.1.1 Finite Transformations of State ................ 300
7.2 Cyclic Processes ...................................... 302
7.3 Exergy and Relative Entropy ........................... 306
7.4 Time Dependence of Statistical Systems ................ 309
Part II Analysis of Statistical Systems
8 Estimation of Parameters ................................... 317
8.1 Samples and Estimators ................................ 318
8.1.1 Monte Carlo Integration ........................ 321
8.2 Confidence Intervals .................................. 325
8.3 Propagation of Errors ................................. 328
8.3.1 Application .................................... 330
8.4 The Maximum Likelihood Estimator ...................... 331
8.5 The Least-Squares Estimator ........................... 336
9 Signal Analysis: Estimation of Spectra ..................... 347
9.1 The Spectrum of a Stochastic Process .................. 347
9.2 The Fourier Transform of a Time Series and the
Periodogram ........................................... 353
9.3 Filters ............................................... 358
9.3.1 Filters and Transfer Functions ................. 360
9.3.2 Filter Design .................................. 363
9.4 Consistent Estimation of Spectra ...................... 367
9.5 Cross-Spectral Analysis ............................... 375
9.6 Frequency Distributions for Nonstationary Time
Series ................................................ 380
9.7 Filter Banks and Discrete Wavelet Transformations ..... 383
9.7.1 Examples of Filter Banks ....................... 388
9.8 Wavelets .............................................. 392
9.8.1 Wavelets as Base Functions in Function
Spaces ......................................... 392
9.8.2 Wavelets and Filter Banks ...................... 397
9.8.3 Solutions of the Dilation Equation ............. 401
10 Estimators Based on a Probability Distribution for the
Parameters ................................................. 405
10.1 Bayesian Estimator and Maximum a Posteriori
Estimator ............................................. 405
10.2 Marginalization of Nuisance Parameters ................ 407
10.3 Numerical Methods for Bayesian Estimators ............. 412
11 Identification of Stochastic Models from Observations ...... 421
11.1 Parameter Identification for Autoregressive
Processes ............................................. 421
11.2 Granger Causality ..................................... 427
11.2.1 Granger Causality in the Time Domain ........... 429
11.2.2 Granger-Causality in the Frequency Domain ...... 430
11.3 Hidden Systems ........................................ 432
11.4 The Maximum a Posteriori Estimator for the Inverse
Problem ............................................... 435
11.4.1 The Least Squares Estimator as a Special
MAP Estimator .................................. 436
11.4.2 Strategies for Choosing the Regularization
Parameter ...................................... 438
11.4.3 The Regularization Method ...................... 441
11.4.4 Examples of the Estimation of a Distribution
Function by a Regularization Method ............ 444
11.5 Estimating the Realization of a Hidden Process ........ 447
11.5.1 The Viterbi Algorithm .......................... 447
11.5.2 The Kalman Filter .............................. 450
11.5.3 The Unscented Kalman Filter .................... 460
11.5.4 The Dual Kalman Filter ......................... 463
12 Estimating the Parameters of a Hidden Stochastic Model ..... 467
12.1 The Expectation Maximization Method (EM Method) ....... 468
12.2 Estimating the Parameters of a Hidden Markov Model .... 472
12.2.1 The Forward Algorithm .......................... 473
12.2.2 The Backward Algorithm ......................... 474
12.2.3 The Estimation Formulas ........................ 474
12.3 Estimating the Parameters in a State Space Model ...... 477
13 Statistical Tests and Classification Methods ............... 483
13.1 General Comments Concerning Statistical Tests ......... 483
13.1.1 Test Quantity and Significance Level ........... 483
13.1.2 Empirical Moments for a Test Quantity: The
Bootstrap Method ............................... 487
13.1.3 The Power of a Test ............................ 489
13.2 Some Useful Tests ..................................... 491
13.2.1 The z- and the t-Test .......................... 491
13.2.2 Test for the Equality of the Variances of Two
Sets of Measurements, the F-Test ............... 492
13.2.3 The χ2-Test .................................... 493
13.2.4 The Kolmogorov-Smirnov Test .................... 495
13.2.5 The F-Test for Least-Squares Estimators ........ 496
13.2.6 The Likelihood-Ratio Test ...................... 497
13.3 Classification Methods ................................ 499
13.3.1 Classifiers .................................... 500
13.3.2 Estimation of Parameters that Arise in
Classifiers .................................... 504
13.3.3 Automatic Classification (Cluster Analysis) .... 505
Appendix: Random Number Generation for Simulating
Realizations of Random Variables .............................. 509
Problems ...................................................... 511
Hints and Solutions ........................................... 533
References .................................................... 541
Index ......................................................... 547
|
