Covariance, Correlation, and RMT
As previously exemplified by basket prediction, Quantivity is finding increasingly frequent use of random matrix theory (RMT), particularly in context of portfolio / basket covariance and correlation analysis across all trading frequencies. Although undoubtedly an exaggeration, RMT is beginning to feel like PCA did in the 1990s.
Below is a survey of RMT literature to whet readers’ appetite. Readers are requested to suggest additional relevant literature in comments. Subsequent posts may delve into more details on practical use of RMT, pending reader interest.
To begin, the following are modern book-length treatments of RMT:
- Random Matrices, Mehta (2004)
- Spectral Analysis of Large Dimensional Random Matrices, Bai and Silverstein (2010)
- An Introduction to Random Matrices, Anderson, Guionnet, and Zeitouni (2009)
The following are extended survey articles (focused on theory and numerical analysis, respectively), available online:
- Topics in Random Matrix Theory, Tao
- Random Matrix Theory, Edelman and Rao
The following are applied RMT articles, in published chronological order:
- Noise Dressing of Financial Correlation Matrices, Laloux et al. (1998)
- Universal and Non-universal Properties of Cross-Correlations in Financial Time Series, Plerou, et al. (1999)
- Random matrix theory and financial correlations, Utsugi et al. (1999)
- Identifying Business Sectors from Stock Price Fluctuations, Gopikrishnan, et al.(2000)
- A Random Matrix Approach to Cross-Correlations in Financial Data, Plerou, et al. (2001)
- Dynamics of correlations in the stock market, Drozdz, et al. (2001)
- Noisy Covariance Matrices and Portfolio Optimization, Pafka and Kondor (2001)
- Noisy Covariance Matrices and Portfolio Optimization II, Pafka and Kondor (2002)
- Random Matrix Theory Analysis of Cross Correlations in Financial Markets, Utsugi, Ino, and Oshikawa (2003)
- Signal and Noise in Financial Correlation Matrices, Burda and Jurkiewicz (2004)
- Exponential Weighting and Random-Matrix-Theory-Based Filtering of Financial Covariance Matrices for Portfolio Optimization, Pafka, Potters, and Kondor (2004)
- Random Matrix Theory for Portfolio Optimization: a Stability Approach, Sharifi et al. (2004)
- Financial Applications of Random Matrix Theory: Old Laces and New Pieces, Potters, Bouchaud, and Laloux (2005)
- Random Matrix Filtering in Portfolio Optimization, Papp et al. (2005)
- The Bulk of the Stock Market Correlation Matrix is Not Pure Noise, Kwapien et al. (2005)
- Asymmetric Matrices in an Analysis of Financial correlations, Kwapien, et al. (2006)
- Risk Evaluation with Enhanced Covariance Matrix, Urbanowicz, Richmond, and Holyst (2007)
- Random, but Not So Much: A Parameterization for the Returns and Correlation Matrix of Financial Time Series, Martins (2007)
- Topological Properties of Stock Networks Based on Random Matrix Theory in Financial Time Series, Eom, et al. (2007)
- Empirics versus RMT in Financial Cross-Correlations (2007), Drozdz, Kwapien, and Oswiecimka
- Maximal Spanning Trees, Asset Graphs and Random Matrix Denoising in the Analysis of Dynamics of Financial Networks, Heimo, Kaski, and Saramaki (2008)
- Detrended Cross-Correlation Analysis: A New Method for Analyzing Two Nonstationary Time Series, Podobnik and Stanley (2008)
- Financial Applications of Random Matrix Theory: a Short Review, Bouchaud and Potters (2009)
- Temporal Evolution of Financial Market Correlations, Fenn, et al. (2010)
- Cross-Correlation Dynamics in Financial Time Series, Conlon, et al. (2010)
These articles nicely illustrate the evolution in application of RMT, both in breadth and sophistication, over the past decade.
I currently learning about RMT and finding it very useful. I had read a few of these before, but this is a great resource to help direct some of my future research.
I have not yet seen a practical guide to RMT written, so I will be interested in your take on it.
In most things I have looked at, they are looking at a bunch of stocks and cleaning all the eigenvalues below some maximum. I had been working with a sample that had one very low-volatility asset and when I tried to do the same procedure I got forecasts for this one asset that made no sense (I ultimately just estimated this one separately). When I actually read more about RMT, it turns about that there are bounds where between them you can call them noise, not just some maximum. In practice, it would make sense for a bunch of stocks to only take the larger eigenvalues as factors and clean the rest, but in a diverse set of assets, one should be more careful.
I wonder how did the PCA feel in the 90′s ???