Fastiterative methods for least squares estimations

Michael K. NG*, Raymond H. CHAN

*Corresponding author for this work

Research output: Journal PublicationsJournal Article (refereed)peer-review

6 Citations (Scopus)


Least squares estimations have been used extensively in many applications, e.g. system identification and signal prediction. When the stochastic process is stationary, the least squares estimators can be found by solving a Toeplitz or near-Toeplitz matrix system depending on the knowledge of the data statistics. In this paper, we employ the preconditioned conjugate gradient method with circulant preconditioners to solve such systems. Our proposed circulant preconditioners are derived from the spectral property of the given stationary process. In the case where the spectral density function s(θ) of the process is known, we prove that if s(θ) is a positive continuous function, then the spectrum of the preconditioned system will be clustered around 1 and the method converges superlinearly. However, if the statistics of the process is unknown, then we prove that with probability 1, the spectrum of the preconditioned system is still clustered around 1 provided that large data samples are taken. For finite impulse response (FIR) system identification problems, our numerical results show that an nth order least squares estimator can usually be obtained in O(n log n) operations when O(n) data samples are used. Finally, we remark that our algorithm can be modified to suit the applications of recursive least squares computations with the proper use of sliding window method arising in signal processing applications.

Original languageEnglish
Pages (from-to)353-378
Number of pages26
JournalNumerical Algorithms
Issue number2
Publication statusPublished - Sept 1994
Externally publishedYes


  • circulant matrix
  • covariance matrix
  • finite impulse response (FIR) system identification
  • Least squares estimations
  • linear prediction
  • preconditioned conjugate gradient method
  • signal prediction
  • Toeplitz matrix
  • windowing methods


Dive into the research topics of 'Fastiterative methods for least squares estimations'. Together they form a unique fingerprint.

Cite this