List of Figures .............................................. xiii
List of Tables ............................................... xvii
Foreword ...................................................... xix
Preface ....................................................... xxi
Acknowledgments ............................................... xxv
Authors ..................................................... xxvii
1 Introduction ................................................. 1
1.1 Channel Equalization .................................... 1
1.2 Source Separation ....................................... 5
1.3 Organization and Contents ............................... 7
2 Statistical Characterization of Signals and Systems ......... 11
2.1 Signals and Systems .................................... 13
2.1.1 Signals ......................................... 13
2.1.1.1 Continuous- and Discrete-Time
Signals ................................ 14
2.1.1.2 Analog and Digital Signals ............. 14
2.1.1.3 Periodic and Aperiodic/Causal and
Noncausal Signals ...................... 14
2.1.1.4 Energy Signals and Power Signals ....... 15
2.1.1.5 Deterministic and Random Signals ....... 16
2.1.2 Transforms ...................................... 16
2.1.2.1 The Fourier Transform of Continuous-
Time Signals ........................... 17
2.1.2.2 The Fourier Transform of Discrete-
Time Signals ........................... 18
2.1.2.3 The Laplace Transform .................. 18
2.1.2.4 The z-Transform ........................ 18
2.1.3 Systems ......................................... 19
2.1.3.1 SISO/SIMO/MISO/MIMO Systems ............ 20
2.1.3.2 Causal Systems ......................... 20
2.1.3.3 Invertible Systems ..................... 20
2.1.3.4 Stable Systems ......................... 20
2.1.3.5 Linear Systems ......................... 21
2.1.3.6 Time-Invariant Systems ................. 21
2.1.3.7 Linear Time-Invariant Systems .......... 21
2.1.4 Transfer Function and Frequency Response ........ 22
2.2 Digital Signal Processing .............................. 23
2.2.1 The Sampling Theorem ............................ 23
2.2.2 The Filtering Problem ........................... 24
2.3 Probability Theory and Randomness ...................... 25
2.3.1 Definition of Probability ....................... 25
2.3.2 Random Variables ................................ 27
2.3.2.1 Joint and Conditional Densities ........ 30
2.3.2.2 Function of a Random Variable .......... 32
2.3.3 Moments and Cumulants ........................... 33
2.3.3.1 Properties of Cumulants ................ 36
2.3.3.2 Relationships between Cumulants and
Moments ................................ 37
2.3.3.3 Joint Cumulants ........................ 37
2.4 Stochastic Processes ................................... 38
2.4.1 Partial Characterization of Stochastic
Processes: Mean, Correlation, and Covariance .... 39
2.4.2 Stationarity .................................... 41
2.4.3 Ergodicity ...................................... 43
2.4.4 Cyclostationarity ............................... 44
2.4.5 Discrete-Time Random Signals .................... 45
2.4.6 Linear Time-Invariant Systems with Random
Inputs .......................................... 46
2.5 Estimation Theory ...................................... 49
2.5.1 The Estimation Problem .......................... 50
2.5.1.1 Single-Parameter Estimation ............ 50
2.5.1.2 Multiple-Parameter Estimation .......... 50
2.5.2 Properties of Estimators ........................ 51
2.5.2.1 Bias ................................... 51
2.5.2.2 Efficiency ............................. 51
2.5.2.3 Cramer-Rao Bound ....................... 52
2.5.3 Maximum Likelihood Estimation ................... 53
2.5.4 Bayesian Approach ............................... 53
2.5.4.1 Maximum a Posteriori Estimation ........ 54
2.5.4.2 Minimum Mean-Squared Error ............. 56
2.5.5 Least Squares Estimation ........................ 57
2.6 Concluding Remarks ..................................... 59
3 Linear Optimal and Adaptive Filtering ....................... 61
3.1 Supervised Linear Filtering ............................ 64
3.1.1 System Identification ........................... 65
3.1.2 Deconvolution: Channel Equalization ............. 66
3.1.3 Linear Prediction ............................... 67
3.2 Wiener Filtering ....................................... 68
3.2.1 The MSE Surface ................................ 70
3.3 The Steepest-Descent Algorithm ......................... 77
3.4 The Least Mean Square Algorithm ........................ 81
3.5 The Method of Least Squares ............................ 85
3.5.1 The Recursive Least-Squares Algorithm .......... 87
3.6 A Few Remarks Concerning Structural Extensions ......... 89
3.6.1 Infinite Impulse Response Filters ............... 90
3.6.2 Nonlinear Filters ............................... 90
3.7 Linear Filtering without a Reference Signal ............ 91
3.7.1 Constrained Optimal Filters ..................... 92
3.7.2 Constrained Adaptive Filters .................... 95
3.8 Linear Prediction Revisited ............................ 96
3.8.1 The Linear Prediction-Error Filter as
a Whitening Filter .............................. 97
3.8.2 The Linear Prediction-Error Filter Minimum
Phase Property .................................. 98
3.8.3 The Linear Prediction-Error Filter as
a Constrained Filter ............................ 99
3.9 Concluding Remarks .................................... 100
4 Unsupervised Channel Equalization .......................... 103
4.1 The Unsupervised Deconvolution Problem ................ 106
4.1.1 The Specific Case of Equalization ............. 107
4.2 Fundamental Theorems .................................. 109
4.2.1 The Benveniste-Goursat-Ruget Theorem ........... 110
4.2.2 The Shalvi-Weinstein Theorem ................... 110
4.3 Bussgang Algorithms ................................... 1ll
4.3.1 The Decision-Directed Algorithm ................ 114
4.3.2 The Sato Algorithm ............................. 115
4.3.3 The Godard Algorithm ........................... 115
4.4 The Shalvi-Weinstein Algorithm ........................ 117
4.4.1 Constrained Algorithm .......................... 117
4.4.2 Unconstrained Algorithm ........................ 119
4.5 The Super-Exponential Algorithm ....................... 121
4.6 Analysis of the Equilibrium Solutions of
Unsupervised Criteria ................................. 125
4.6.1 Analysis of the Decision-Directed Criterion .... 126
4.6.2 Elements of Contact between the Decision-
Directed and Wiener Criteria ................... 127
4.6.3 Analysis of the Constant Modulus Criterion ..... 128
4.6.4 Analysis in the Combined Channel + Equalizer
Domain ......................................... 129
4.6.4.1 Ill-Convergence in the Equalizer
Domain ................................. 130
4.7 Relationships between Equalization Criteria ........... 132
4.7.1 Relationships between the Constant Modulus
and Shalvi-Weinstein Criteria .................. 132
4.7.1.1 Regalia's Proof of the Equivalence
between the Constant Modulus and
Shalvi-Weinstein Criteria .............. 133
4.7.2 Some Remarks Concerning the Relationship
between the Constant Modulus/Shalvi-Weinstein
and the Wiener Criteria ........................ 135
4.8 Concluding Remarks ..................................... 139
5 Unsupervised Multichannel Equalization ..................... 141
5.1 Systems with Multiple Inputs and/or Multiple
Outputs ............................................... 144
5.1.1 Conditions for Zero-Forcing Equalization of
MIMO Systems ................................... 146
5.2 SIMO Channel Equalization ............................. 148
5.2.1 Oversampling and the SIMO Model ..................... 150
5.2.2 Cyclostationary Statistics of Oversampled
Signals ........................................ 152
5.2.3 Representations of the SIMO Model .............. 153
5.2.3.1 Standard Representation ............... 153
5.2.3.2 Representation via the Sylvester
Matrix ................................ 154
5.2.4 Fractionally Spaced Equalizers and the MISO
Equalizer Model ................................ 156
5.2.5 Bezout's Identity and the Zero-Forcing
Criterion ...................................... 158
5.3 Methods for Blind SIMO Equalization ................... 160
5.3.1 Blind Equalization Based on Higher-Order
Statistics ..................................... 160
5.3.2 Blind Equalization Based on Subspace
Decomposition .................................. 161
5.3.3 Blind Equalization Based on Linear
Prediction ..................................... 165
5.4 MIMO Channels and Multiuser Processing ................ 168
5.4.1 Multiuser Detection Methods Based on
Decorrelation Criteria ......................... 169
5.4.1.1 The Multiuser Constant Modulus
Algorithm ............................. 170
5.4.1.2 The Fast Multiuser Constant Modulus
Algorithm ............................. 172
5.4.1.3 The Multiuser pdf Fitting Algorithm
(MU-FPA) .............................. 173
5.4.2 Multiuser Detection Methods Based on
Orthogonalization Criteria ..................... 176
5.4.2.1 The Multiuser Kurtosis Algorithm ....... 177
5.5 Concluding Remarks .................................... 179
6 Blind Source Separation .................................... 181
6.1 The Problem of Blind Source Separation ................ 184
6.2 Independent Component Analysis ........................ 186
6.2.1 Preprocessing: Whitening ....................... 188
6.2.2 Criteria for Independent Component Analysis .... 190
6.2.2.1 Mutual Information ........ 191
6.2.2.2 A Criterion Based on Higher-Order
Statistics ............................ 194
6.2.2.3 Nonlinear Decorrelation ............... 195
6.2.2.4 Non-Gaussianity Maximization .......... 196
6.2.2.5 The Infomax Principle and the
Maximum Likelihood Approach ........... 198
6.3 Algorithms for Independent Component Analysis ......... 200
6.3.1 Hérault and Jutten's Approach .................. 200
6.3.2 The Infomax Algorithm .......................... 201
6.3.3 Nonlinear PCA .................................. 202
6.3.4 The JADE Algorithm ............................. 204
6.3.5 Equivariant Adaptive Source Separation/
Natural Gradient ............................... 205
6.3.6 The FastICA Algorithm .......................... 206
6.4 Other Approaches for Blind Source Separation .......... 209
6.4.1 Exploring the Correlation Structure of the
Sources ........................................ 209
6.4.2 Nonnegative Independent Component Analysis ..... 210
6.4.3 Sparse Component Analysis ...................... 211
6.4.4 Bayesian Approaches ............................ 213
6.5 Convolutive Mixtures .................................. 214
6.5.1 Source Separation in the Time Domain ........... 215
6.5.2 Signal Separation in the Frequency Domain ...... 216
6.6 Nonlinear Mixtures .................................... 218
6.6.1 Nonlinear ICA .................................. 219
6.6.2 Post-Nonlinear Mixtures ........................ 220
6.6.3 Mutual Information Minimization ................ 222
6.6.4 Gaussianization ................................ 223
6.7 Concluding Remarks .................................... 224
7 Nonlinear Filtering and Machine Learning ................... 227
7.1 Decision-Feedback Equalizers .......................... 229
7.1.1 Predictive DFE Approach ........................ 231
7.2 Volterra Filters ...................................... 233
7.3 Equalization as a Classification Task ................. 235
7.3.1 Derivation of the Bayesian Equalizer ........... 237
7.4 Artificial Neural Networks ............................ 241
7.4.1 A Neuron Model ................................. 241
7.4.2 The Multilayer Perceptron ...................... 242
7.4.2.1 The Backpropagation Algorithm .......... 244
7.4.3 The Radial-Basis Function Network .............. 247
7.5 Concluding Remarks .................................... 251
8 Bio-Inspired Optimization Methods .......................... 253
8.1 Why Bio-Inspired Computing? ........................... 254
8.2 Genetic Algorithms .................................... 256
8.2.1 Fundamental Concepts and Terminology ........... 256
8.2.2 A Basic Genetic Algorithm ...................... 257
8.2.3 Coding ......................................... 258
8.2.4 Selection Operators ............................ 259
8.2.5 Crossover and Mutation Operators ............... 261
8.3 Artificial Immune Systems ............................. 266
8.4 Particle Swarm Optimization ........................... 269
8.5 Concluding Remarks .................................... 273
Appendix A: Some Properties of the Correlation Matrix ......... 275
A.l Hermitian Property .................................... 275
A.2 Eigenstructure ........................................ 275
A.3 The Correlation Matrix in the Context of Temporal
Filtering ............................................. 277
Appendix B: Kalman Filter ..................................... 279
B.1 State-Space Model ..................................... 279
B.2 Deriving the Kaiman Filter ............................ 280
References .................................................... 285
Index ......................................................... 303
|
