Portfolio Analysis has been in existence since 1952. Extensive research has been carried out on the portfolio theory but the implementation has been minimal. Due to the "inputs" required the implementation of Portfolio Analysis has been less.
Optimum portfolios can be achieved if accurate data if data is available regarding expected future mean returns, the variances of the returns and the correlation of returns. The problem lies with obtaining the accurate data, particularly with respect to the matrix of correlation between securities.
Even though problem of estimating means and variances is not difficult but most work has been done on it, where as very less attention has been given to the problem of estimating correlations. There are two major reasons for this phenomenon. First, the problem lies with calculating with large number of estimate are to be calculated to obtain efficient portfolio, for 200 stocks there will be 19,900 correlation coefficients. Second, analysts mostly follow specific stocks in which organization of the analyst has some interest. Obtaining indirect estimates of correlation coefficients is not easy; the parameters of a single or multi-index model to obtain the correlation coefficients will need to be estimated. These parameters will contain implied forecasts of correlation coefficients. Most analysts do not think in these terms and their estimates will be necessarily superior to more mechanical procedures. Little or almost no attention has paid to the accuracy of techniques used for estimating the correlation structure of share prices.
This paper will deal with Estimating the Correlation Matrix of Stock Returns. The estimates calculated will be compared with respect to their ability to forecast correlation matrixes and with respect to their ability to choose portfolios which prove to be efficient in future period; these estimates will be based on historical data.
This paper is divided into two sections. Section I discusses the general types of forecasting models that could be used, their characteristics, and the variants of each model tested. Section II presents empirical results.
Section I
The Forecasting Models
The three types of models are discussed with different assumptions in each model and how to use each model for forecasting.
A. A Full Historical Model
The simplest forecasting model is the Full Historical Model, which uses historical data and assumes that "Past values are useful estimates of future values", estimates future correlation coefficients. Through historical data each pair's coefficient correlation are calculated and used to estimate future's coefficients. There are No assumptions regarding co movement of securities rather the co-movement of the securities itself is calculated.
B. Index Models
In Index Model a behavioral model is assumed of why securities move together, estimate the parameters of this model and then use this model to estimate the future correlation coefficients. There are two types of index models.
1. Single Index Model
This is the simplest behavioral model developed by Sharpe, assuming that securities move together only because of a common response to changes in an aggregate index.
The Sharpe's model can be written as:
Ri = Ai + Bi1 I1 + ei I1 = AN+1 + eN+1
E(eN+1 .e1) = 0 i = 1,…….,N
E(ei ej) = 0 i = 1,…….,N
Ri be the return on security i;
I1 be the return of the aggregate index;
Ai be the return of a security i that is independent of changes in the aggregate index;
Bi1 be a measure of the responsiveness of security i to changes in the aggregate index;
ei be a variable with mean of zero and variance ai2 which measures the variability of security i that is not attributable to changes in the aggregate index;
AN+1 be a constant equal to the expected return of the aggregate index;
eN+1 be a variable with mean of zero and variance of Æ¡N+12.Æ¡N+12 measures the variability of the aggregate index.
Estimates of the mean returns and variances (for individual securities) by Sharpe's model are similar to estimates produced by using Historical data. The estimates for correlation coefficients will differ. Letting E[ ] the expected value operator, covariance of security i and j can be written as:
E[{ri î º E(ri)} {rj î º E(rj)}] = Bi1Bj1Æ¡N+12+Bi1E(eN+1ej)+Bj1E(eN+1ei)+E(ei ej)
The correlation coefficient between security i and security j is the above divided by the product of the standard deviations of security i and security j.
If we calculate the parameters of the above model using least square regression than the second and third term will become zero but there is nothing that guarantees that the fourth term will be a zero. Rather we can assume it to be non-zero. In fact the assumption that E(eiej) = 0 is the only difference between estimation using the Sharpe model and estimation using full historical data. As the behavioral model is an approximation so a choice exists that if E(ei ej)/ơiơj is stable over time then estimating it from historical data will produce accurate results, if it is unstable then ignoring it will produce better results.
In this paper two different indexes were taken for testing purpose of Single Index Model. One, Standard and Poor Industrial Index adjusted for dividends written as Standard Single Index or SSI, second, specific sample of stocks written as F-1.
2. Multi Index Model
In the Multi Index Model it is assumed that the term E (eiej) has a stable component and some random noise. This model assumes that securities move together, partly because of economy wide charges and partly because of their association with some subgroup in the economy.
Ri = Ai + Bi1I1 + Bi2I2 + ……. + BimIm + ei i = 1,……., N
Ik = AN+k + eN+k k = 1,…….,m
E(eN+k .ei) = 0 i = 1,…….,N and k = 1,…….,m
E(ei ej) = 0 i = 1,…….,N and j = 1,…….,N and I ≠j
E(eN+k .eN+p) = 0 p = 1,…….,m k = 1,…….,m k ≠p
Where:
Bi1 is a measure of responsiveness of security i to changes in index k;
Ik is an aggregate index if k 1 and is an index of subgroup movements otherwise;
ei is a variable with mean of zero and variance Æ¡12.Æ¡12 measures the variability of security i that is not attributable to changes in any index;
eN+k is a variable with mean of zero and variance Æ¡N+12 . Æ¡N+12 measures the variability of the index k;
And other terms as before.
Once again estimates of expected returns and variances produced by this model would be identical to the estimates obtained using full historical estimation. So again letting E[ ] the expected value operator, covariance of security i and j can be written as:
E[{ri î º E(ri)} {rj î º E(rj)}] =
[Bi1Bj1ơN+12 + Bi2Bj2ơN+22+……+ BimBjmơN+m2] +
[Bi1Bj2E(eN+1eN+2) + Bi1Bj3E(eN+1eN+3)+……+ Bi N+k−1Bj N+kE(eN+K−1eN+k)] +
Bi1E(ejen+1)+……+ BiNE(ejen+1) + Bj1E(ejen+1) +……+ BjNE(ejen+1) +
E(ei ej)
The correlation between security i and security j is the above divided by Æ¡i Æ¡j. Above equation is divided into 4 parts and if we calculate parameters by least square regression method than again the Third term will be zero by construction. Also if the indices are orthogonal to each other than second term will also become zero.
The assumption that the fourth term E(ei ej) = 0 is again the only difference between Full Historical Model and Multi Index Model.
Single Index Model assumes no interaction except that caused by a common market movement or E (ei ej) = 0 where Multi Index Model splits this E(ei ej) into two parts, one due to movement of subgroups [Bi1Bj1ơN+12 + Bi2Bj2ơN+22+……+ BimBjmơN+m2] , and a residual E(ei ej) which is assumed zero. Now the performance of any of these models will depend on whether zero or the historical level is a better estimate of the future values of the subgroup indexes and the residuals.
In this paper three Multi Index Models were tested. The first two were a 3 and 8 index model which will be referred to as F-3 and F-8. The final multi-index model was determined by preserving those components whose eigen-value is greater than 1. In all cases this left us with 17 or 18 indices. Results from this model will be referred to as F-max.
C. Mean Models
Historical data only contains information concerning the mean correlation coefficients and that observed pair-wise differences are random or sufficiently unstable, so that zero is a better estimate (than their historical level) of their future value. The most aggregate averaging possible is to set every correlation coefficient equal to the average of all correlation coefficients. This is called Mean model. This could be thought of as a naive model which can be used to evaluate the more complex models discussed earlier. Alternatively, if the assumption of E (ei ej) = 0 is held, then the overall mean model is the same as a constrained form of the single index model. A more disaggregated mean model would be to assume that there is a common mean only within subgroups and that this mean could vary between subgroups.
Three forms of this model are tested. First, the SIC industrial classification represented homogeneous groups and averaged within and between these groups; we called this the Traditional Mean Model. The last two averaging models are similar to the Traditional Mean averaging model except that, rather than accepting traditional industries as homogeneous groups, multi-variate techniques were employed to determine which groups had behaved as homogeneous units. We called these homogeneous units pseudo-industries. In the traditional industries case, averaging was performed both within and between pseudo-industries. Two pseudo-industry models were tried. One model contained three pseudo-industries (identified as Pseudo-3 or P-3) and one involved seven pseudo-industries (identified as Pseudo-7 or P-7).
To review, we have examined 10 models:
Name Description
I. Historical Model Assumes future correlation coefficients are identical to past coefficients.
II. Single Index
A. SSI Index is S&P index adjusted for dividends.
B. F-1 Index is first component of principal component solution.
III. Multiple Index
A. F-3 Indices are first 3 components from principle component solution.
B. F-8 Indices are 8 components from principle component solution.
C. F-Max Indices are first 17 or 18 components from principle component solution.
IV. Mean Model
A. Overall Mean Assumes same correlation coefficient between all firms.
B. Traditional Mean Assumes same pattern of correlation coefficients within an industry.
C. P-3 Assumes same pattern of correlation coefficients within 3 pseudo-industries.
D. P-7 Assumes same pattern of correlation coefficients within 7 pseudo-industries.
Section II
EMPIRICAL RESULTS
The accuracy of forecasting are evaluated on the two criteria
Statistical significance
In statistical significance the forecast correlation matrices of every technique will be examined.
Economic significance
Economics significance will be analyzed by using the forecasted correlation coefficients as input to a portfolio selection model and examining the performance of portfolios to observe if the differences between techniques are of economic importance.
Two separate five year and three one year were studied to estimate of correlation matrices to test the accuracy over different time periods. To discuss the accuracy of forecast matrices, we will present result from each of these time period and also the combine result of both five year and one year sample.
A. Five Year Results-Statistical Information
Firstly we examine how well each model actually forecasts future correlation matrices. We employ tests of the statistical significance of the difference in absolute forecast error between techniques and comparisons of the cumulative frequency of forecast error.
The first model discussed was the full Historical model. 30 correlation matrices were formed for each of two five year forecast periods by randomly rearranging the entries in the Historical correlation matrix (shown in table 4). The mean absolute error for each of these 30 matrices for each of the two five year forecast periods is presented. The randomly arranged correlation matrices produce larger average absolute errors than the Historical correlation matrix and also produce forecasts which are considerably worse than any of the ten techniques under study.
In each of the five year and in the combined sample, the SSI model outperformed the full historical model at 5% level. This indicates that estimating correlation between stocks which is not due to correlation with the market at zero is better than estimating historical level. The SSI technique produces better forecasts of future correlation matrices than the full Historical correlation matrix.
In the cumulative frequency comparison the SSI and F1 outperformed the multi index models (F-3, F-8 and F-max). The single index model is statistically significant in both five years and combine five years sample. The multi index models explain a greater percentage of the historical correlation between the stocks in our sample, they lead to worse prediction. The correlation between stocks not due to common correlation with a single market index is zero is better than trying to predict it from the historical correlation of the residuals.
SSI outperforms F 1 because the index use for SSI is broadly based market index where F1 is sample of specific stock.
Three techniques produced better estimates of future correlation coefficients than SSI. These are Overall Mean, P-3, and Traditional Mean. Overall Mean outperformed SSI in both 5-year samples and the combined samples.
Allowing difference in the correlation between stocks to enter the model on either statistical ground and on industrial structure leads to an improvement in the results.
Full Historic, Overall Mean, Industry Mean, P-3, and P-7 all make the same estimate of the average correlation between stocks. The same is not true of the remaining models. The mean correlation produced by these models can be below or above the mean historic correlation coefficient.
The forecast method produce better mean level of the future correlation coefficients while they do a poor job estimating differences in the correlation between firms
5 year Results- Economic significance
In economic significance we differentiate correlation coefficient forecasting technique on their ability to support in the selection of more efficient portfolios at different risk level.
Now will find out, the estimates of correlation matrix which give rise to the selection of the most efficient portfolios.
The best three techniques are Overall Mean, Traditional Mean and P-3.Overall Mean is best with Traditional Mean and Pseudo-3 performing almost identically the difference in return from employing any of these techniques is quite small.
The Sharpe technique is ranked fourth under both evaluation procedures. The difference between using the Sharpe technique and the worst of the three averaging techniques is significant. There is crossing of performance for the four intermediate techniques (P-7,
F-1, F-8 and F-3) and the two worst techniques (Historic and F-Max) are clearly identified.
The fact that these techniques (for example historical and F-Max) perform worse in selecting efficient portfolios than they do in forecasting the correlation matrix would indicate that they do a relatively poorer job in estimating the correlation between stocks which are most promising for portfolios. We compare historical mean with traditional mean which yields a significance difference in performance. The return from the traditional mean model is 40% higher than the return from historical technique and 25% higher than by SSI.
The differences can be originated in the performance of the forecasting techniques under study. Statistically significant differences existed in the forecast accuracy of the techniques and the ranking of techniques was consistent from one 5-year sample to another and these differences were of economic significance. The return on a portfolio could be increased by as much as 50% by selecting portfolios on the basis of these techniques which forecast most accurately.
C. One Year Results-Statistical Significance
The forecasting techniques will be compared with the forecasting ability of 30 correlation matrices computed by randomly rearranging the elements in the Historical correlation matrix, as was done in the 5 year case.
The forecast error for 1-year is higher than the 5-year forecasts sample (shown in table 5). Average absolute forecast error of 1 year is two to three times larger than the 5-year forecasts. The reason for the large forecast error is due to some combination of errors in forecasting the average correlation between all stocks and errors in forecasting pair-wise deviation from this average.
Due to the year to year changes in the average correlation coefficient are observed and this contrasts to the stability of 5-year correlation coefficients. The deterioration in estimating short run correlation coefficient is due to error in estimating the mean correlation coefficient.
Furthermore, the large error in mean prediction should also cause the ranking among techniques to be in large part a reflection of their ability to estimate the mean.
The result for the 1 year forecasting ranking of the techniques is similar to the ranking for 5-year forecasts (both unadjusted and adjusted for mean). P-3, Industry Mean, and P-7 all provide better than the Overall Mean. SSI performs better than the Historical. The multi-index models prove mediocre to the single index models.
When we compare the forecasting on the unadjusted data the result would be more erratic.
The cumulative frequency functions of relative errors shows that there were eleven cases of complete dominance, where the corresponding 5-year period had twenty-seven
The ability of any technique to forecast the future 1-year correlation matrix depends mostly on how well it forecasts the average level of future correlations. An illustration of this we observe by examining Tables 9, 14 and 16. In the first 1-year sample, SSI moves from fifth place to first place as we remove the mean adjustment. From Table 9 we can see that this switch in position is due to a better forecast of the mean. On the other hand, in the third 1-year sample SSI moves from fifth to sixth place because it produces a poorer forecast of the average correlation coefficient.
The techniques do not show any consistency in their ability to forecast the mean. The models under study do not show a constant differential ability to forecast the average correlation coefficient.
We expect the lack of uniformity in results from year to year, the large forecast errors and the lack of statistical difference in performance which are shown by Table 14. The size of the errors produced by every technique and the inability to tell the difference between techniques on a statistically significant level makes us more cautious of conclusions drawn from 1-year forecasts as compare to 5-year forecasts.
One Year Forecast-Economic Significance
The ex-post performance of efficient portfolios selected by each forecasting technique, are shown at ex-post risk levels in Tables 17 and 18.
It is difficult to rank the techniques in terms of portfolio performance. Five of the ten techniques perform best for various ex-post risk levels. There is no significant difference between the efficiency of portfolios formed on the basis of 1-year correlation forecasts. When forecasting for a 1-year period as compare to 5 year result, very slight can be estimate about the relative performance of techniques on either statistical or economic grounds.
Conclusion
The principle of this paper was to identify accurate expectations about the future correlation of returns between each pair of securities. The problem lies in obtaining accurate inputs for applying the models discussed in this paper.
The three types of models are discussed with different assumptions in each model and how to use each model for forecasting, which included:
A Full Historical Model
Index Models
Mean Model
We studied two separate five year and three one year of estimates of correlation matrices to test the accuracy over different time period.
In each of the five year and in the combined sample, three techniques produced better estimates of future correlation coefficients than SSI on statistically significance results. These are Overall Mean, P-3, and Traditional Mean. Overall Mean is best with Traditional Mean and Pseudo-3 performing almost identically the difference in return from employing any of these techniques is quite small. The SSI model outperformed the full historical model and in the cumulative frequency comparison the SSI and F1 outperformed the multi index models (F-3, F-8 and F-max). The multi index models explain a greater percentage of the historical correlation between the stocks in our sample, they lead to worse prediction.
In the 5 year economic significance results, the best three techniques are Overall Mean, Traditional Mean and P-3.Overall Mean is best with Traditional Mean and Pseudo-3 performing almost identically the difference in return from employing any of these techniques is quite small.
In terms of one year statistically significance estimate, the P-3, Industry Mean and P-7 all provide better the results than the Overall Mean. SSI performs better than the Historical. The multi-index models prove mediocre to the single index models. The techniques do not show any consistency in their ability to forecast the mean. The models under study do not show a constant differential ability to forecast the average correlation coefficient.
The 1 year economic significance results, shows the difficult to rank the techniques in terms of portfolio performance. Five of the ten techniques perform best for various ex-post risk levels. There is no significant difference between the efficiency of portfolios formed on the basis of 1-year correlation forecasts. When forecasting for a 1-year period as compare to 5 year result, very slight can be estimate about the relative performance of techniques on either statistical or economic grounds.
Critique
The models and techniques which we have study, these techniques cannot be used as a generalized model for all time periods and all sort of data. An analyst would have to use a different model for estimating the correlation coefficient for one year time period and another one for five year time period making it inconvenient for the analyst. Also most of the these techniques are not performing better than historical model for one year time period , so what's the point of using a much more complicated technique rather than just estimating from historical data in one year. A lot of more work has to be done to estimate the correlation coefficient for short time period (e.g. 1 or 2 year). The theoretical and empirical research on estimation of correlation coefficient is not sufficient at this point a lot more work has to be done on this issue. As stated earlier in this topic the literature in finance has done almost no work on this topic so at this point and time these techniques can't be used at professional level.
