Understanding Estimation Error
Mark Kritzman, June 20, 2016
When investors build portfolios, they begin with a long history of returns of the assets to be included in the portfolio. They use these historical returns to compute volatilities and correlations, which they typically extrapolate to estimate future volatilities and correlations. Although they might also use these historical returns to guide their estimates of expected returns, it is uncommon to extrapolate historical means. More often, investors rely on fundamental analysis or other information to estimate expected returns.[1]
Sources of Error
Investors base their risk estimates on histories that are typically decades-long; however, the investment horizon they are attempting to characterize usually ranges from one to five years. This estimation process, therefore, exposes investors to three sources of error: small-sample error, independent-sample error, and interval error.
Small Sample Error and Independent-Sample Error
Small-sample error arises because the realization of volatilities and correlations from a small sample of returns will differ from the volatilities and correlations of the large sample from which it is selected. But investors are not concerned with a small sample within a large sample; rather, they care about the volatilities and correlations of a future small sample that is independent of the large historical sample. Therefore, investors also face independent-sample error because the volatilities and correlations of a future sample will differ from those of the prior historical sample, regardless of the size of the future sample. Finally, investors face a much-neglected error called interval error.
Interval Error
Interval error arises because investors usually base their calculations of volatilities and correlations on monthly returns and then extend these estimates to annual or multi-year horizons. In so doing, they assume that returns are independent and identically distributed, which implies that volatility scales with the square root of time such that the volatility of annual returns, for example, is equal to the volatility of monthly returns multiplied by the square root of 12. This assumption also implies that correlations are invariant to the return interval used to estimate them. Unfortunately, there is no evidence to support this assumption. Therefore, these mapping heuristics give a distorted estimate of longer-horizon volatilities and correlations. Together, these three sources of error cause out-of-sample volatilities and correlations to differ, often markedly, from their historical values.
Investors typically address estimation error in two ways. One approach is to blend individual estimates with their cross-sectional average or some other prior belief. This approach, called Bayesian shrinkage, has the effect of making the estimates of volatilities and correlations more similar to each other, which consequently causes the portfolios along the efficient frontier to be more self-similar. Another common approach for addressing estimation error is resampling. Resampling is a process by which optimal weights are generated many times from a distribution of inputs and then averaged to determine the final portfolio. Resampling also causes portfolios along the efficient frontier to be more alike.
Adjusting for Stability
We propose a new approach for dealing with estimation error called stability-adjusted optimization. Rather than reduce sensitivity to errors, we argue that investors should measure the relative stability of volatilities and correlations and treat this information as a distinct component of risk.
Here is how we proceed. We begin with a large sample of historical returns for the assets included in our portfolio. We then select all possible overlapping sub samples of a size equivalent to our investment horizon, and we compute covariance matrices[2] for all of these small samples using return intervals equal to the duration of our investment horizon.
Next we subtract the covariances in all of the sub samples from the covariances of that part of the original large sample that does not include the respective sub samples. We use the heuristics described earlier to convert the covariances of each sub sample’s complementary sample to the same interval of our investment horizon. This leaves us with error matrices for all of the sub samples.
Reflections
These error matrices reflect small-sample error because the sub samples are smaller than the complementary portion of the large sample. They reflect independent sample error because the small samples are independent of their large-sample complements. And they reflect interval error because the small-sample covariances are estimated from longer interval returns that account for lagged correlations, whereas the complementary-sample covariances are estimated from shorter-interval returns and extended to reflect longer-interval returns by using the heuristics described earlier.
We then select a base case small sample, which could be the median sub sample, for example, and we add the error matrices to the covariance matrix of the base case sub sample. Then, assuming normality, we generate return samples from all of the error-adjusted covariance matrices, and we combine them into a new large sample of returns, which by virtue of this process reflects the relative stability of the asset covariances.
This process for generating a stability-adjusted return distribution yields a distribution that has fatter tails than a normal distribution, because the variances of the error-adjusted small samples will differ from each other. These fatter tails shouldn’t present a problem to mean-variance optimization, though, as long as we can reasonably describe investor preferences with mean and variance. If that is not the case, however, we must resort to a portfolio construction process known as full-scale optimization. This process for generating a stability-adjusted return distribution is depicted in Exhibit 1.
Full-scale Optimization
Full-scale optimization identifies the optimal portfolio by trial and error. We start by selecting a particular utility function, which do not need to be well-approximated by mean and variance. Then, we choose a set of portfolio weights and apply them every period to the asset returns in the stability-adjusted return sample to compute the utility associated with those weights for every period. We then sum utility across all periods and record this value. Next, we choose a different set of portfolio weights and apply them to the sample returns to compute their total utility across all periods. We proceed in this fashion until we arrive at the portfolio composition that yields the highest utility across all periods. This full-scale approach to optimization may be computationally expensive; nonetheless, it accounts for every feature of the data, even beyond kurtosis and skewness.
Windham Labs' Windham Portfolio Advisor includes Full-scale Optimization features
Application Examples
We apply stability-adjusted optimization in two settings. We use it to allocate a portfolio across four assets classes: U.S. stocks, U.S. Treasuries, U.S. Corporates, and commodities based on monthly returns from February 1973 through December 2015.[3] We also apply it to replicate the S&P 500 Index using 20 randomly selected stocks, two from each GICS sector. We use weekly returns from January 2006 through December 2015. In both examples, we assume both power utility and kinked utility, which applies to investors who face thresholds.
Exhibit 2 below reveals that the stability-adjusted portfolio displayed significantly less volatility and downside risk during the global financial crisis than if we were to ignore errors or rely on Bayesian shrinkage.[4]
The third exhibit’s results are even more striking. It reveals that explicitly accounting for the relative stability of covariances markedly reduces quarterly tracking error and downside tracking error compared to an approach that either ignore errors or one that relies on Bayesian shrinkage.[5]
Summary
We described three sources of errors investors face when relying on a long historical sample of returns to characterize the risk of a future smaller sample. We then introduced a process for measuring the relative stability of asset covariances. Next, we showed how to use this information to create a stability-adjusted return sample. Finally, we applied this approach to construct a portfolio of asset classes as well as an index-replicating portfolio. We then presented evidence showing that adjusting for stability yields better behaved portfolios than commonly used alternative approaches. We hope to refine this research and to test stability-adjusted optimization across a wider set of applications.
Endmatter
This essay is based on the following article: Kritzman, M. and D. Turkington. “Stability-Adjusted Portfolios.” The Journal of Portfolio Management, Special QES Issue 2016, Vol. 42, No. 5: pp 113-122.
Covariances combine estimates of volatility and correlation.
We used the following indexes as proxies for the asset classes: S&P 500 Index, Barclays U.S. Treasury Index, Barclays U.S. Corporate Index, and the S&P/GSCI Commodities Index. Also we assumed we assumed the expected returns of U.S. stocks, U.S. Treasuries, U.S. corporates, and commodities were equal to 9%, 4%, 5%, and 5%, respectively.
This exhibit is taken from: Kritzman, M. and D. Turkington. “Stability-Adjusted Portfolios.” The Journal of Portfolio Management, Special QES Issue 2016, Vol. 42, No. 5: pp 113-122. Please refer to this article for greater detail about stability-adjusted optimization.
This exhibit is taken from: Kritzman, M. and D. Turkington. “Stability-Adjusted Portfolios.” The Journal of Portfolio Management, Special QES Issue 2016, Vol. 42, No. 5: pp 113-122. Please refer to this article for greater detail about stability-adjusted optimization.
Disclaimer
This material is not intended to provide professional or investment advice. You are advised to seek independent professional advice prior to investing in any products or strategies described herein or recommended by Windham Capital Management, LLC. This constitutes neither an offer to buy or sell any securities, nor a solicitation of an offer to buy or sell interests or shares in any fund or strategy. Past performance, including any projection or forecast, are not necessarily indicative of future or likely performance of any investment products. No assurance may be given that the strategies’ investment objectives will be achieved. Investments are subject to investment risks including possible loss of principal amount invested.
Get in touch with us to see how Windham Labs can help your practice!
Last updated
