Introduction
When I investigate a topic for this blog, I usually have an idea of
what I expect to find. As I gather data,
usually things fall into place more or less as I expected. But sometime, as in today’s post, as I dig
deeper, the fundamental conclusion reverses itself. The deeper understanding results in the
opposite of what I expected.
Since the 1970s, the best advice on the stock market was to simply stay
in the market at all times, through highs and lows. Stock movements were believed to be random
and unpredictable. But a technique
developed by Nobel-prize winning economist Robert Shiller seemed to show
promise in reducing the noise in stock market returns and providing a reliable
way to predictably quantify the return on the market over long periods. I originally intended to simply replicate the
work published in Nate Silver’s book “The Signal and the Noise”. But after digging into the numbers, I realized
that there were real-world factors not considered in Silver’s analysis. After considering those factors, the
essential randomness of stock market returns re-emerged. The Shiller PE may provide some guidance
about when it is better or worse to be invested in stocks, but the correlation
is weak and returns are not well quantified.
I am back to the original advice of Burton Malkiel and Peter Lynch, to
stay invested in stocks at all times.
The Shiller PE
The inspiration for this post came from a series of charts in Nate
Silver’s remarkable book “The Signal and the Noise”. The charts show a remarkable predictive power
for the Shiller PE with respect to future stock market performance. The charts captured my attention, because
the power of prediction in the stock market is notoriously difficult and
immensely valuable. Nate Silver does
not permit reproduction of material from his book, so I gathered data from
Robert Shiller’s website and created new charts myself.
The PE, or price/earnings ratio, is the fundamental unit of
stock valuation. The current price of a
stock is divided by a measure of the annual profits, or earnings of the
company. The PE of stock shows whether
stocks are relatively cheap or expensive.
The PE ratio can also be calculated for a basket of stocks, such as the
Standard and Poor’s 500 company index, which represents the largest publically
traded companies in the market.
The Shiller PE, formally known as the Cyclically Adjusted Price
Earnings Ratio (CAPE), divides the price of a stock by the average of the past ten years of earnings, or annualized ten-year
trailing earnings. The use of ten years
of earnings removes temporary earnings anomalies and produces a better
appraisal of value.
Shiller PE versus Average Market Performance
So, if the Shiller PE is a better appraisal of value, let’s see how
well it works in predicting future stock market performance. We will look at data provided on Robert
Shiller’s website for stock market performance from 1871 to the present.
The following charts show the market performance of the
Standard and Poor 500 Index for varying periods of time, versus the Shiller
PE. The scale for each chart is the same
to facilitate comparison. Dividends are excluded from the calculation of
return. Return is calculated once per
year, January-to-January for each period, and all figures are adjusted for
inflation. Market return for multiple-year
periods is annualized by taking the average
of the one-year returns over the period.
We will see later how changing the way return is annualized significantly
changes the results.
We can see that the Shiller PE is almost useless in predicting the
short-term performance of the stock market.
The graph of one-year performance versus the Shiller PE looks like a
shotgun blast.
The graph of Shiller PE against average five year performance is
somewhat less scattered than the one-year graph.
Shiller PE versus Average Ten
Year Market Performance
The graph of average ten year market performance shows a little orderly
tendency in comparison to the Shiller PE.
Before we look at caveats to this result, let’s consider why the
Shiller PE works as a predictor of long-term stock market performance.
Noise Reduction by Repetition
Repetition is the key to the
predictive ability of the Shiller PE.
Repetition is a way to improve the quality of the signal in
noisy data. Geophysicists use this
principle in making seismic recordings for oil exploration. Geophysicists repeat their measurement of the
same subsurface point up to 120 times, using different angles of incidence and
reflection. The sum of these records is
called a “stack”. Random noise in one
seismic record is cancelled by countervailing noise in other records, and the
resulting data is much stronger and clearer than unstacked data.
There is a lot of noise in both short-term stock market
performance and in annual corporate earnings.
Stock prices go up and down according to the mood of the market; prices
are influenced by national politics, international crises, demographic changes,
and changes in the national economy.
Trends in the market affect the psychology of investors, so market
momentum is a feedback mechanism. This
feedback amplifies any upward or downward movement, producing booms and crashes
which are not justified by any change in fundamental information.
Earnings are also subject to noise. A company’s earnings can be severely impacted
by short-term events, including recessions, competition, market events, labor
disruptions, accidents or legal problems.
The measurement of both market performance and earnings are improved
with repetition. The Shiller PE, which
takes earnings over a ten-year interval, reduces noise through the repetition
of those earnings years. The Shiller PE
is a “stack” of ten years of earnings data.
Some years may be unusually high or low, but the ten year average will
be a better indication of the profitability of the company than any single
year. Likewise, taking a long-term view
of market performance improves the signal in that data. A five-year stack is better than a one-year
stack, and a twenty-year stack is better than a ten-year stack. Longer periods of market performance smooth
through the highs and lows, and provide a better picture of the return to the
investor over the longer term.
Real Market Return Compared to Average Annual Return
So, what is the problem with the correlation of the Shiller PE and
average long-term market return?
There are two problems.
“The
most powerful force in the universe is compound interest.” -- Albert Einstein
First, investors don’t receive an average of 20 years of annual
returns. Investors receive a real
return – the cumulative gain over twenty years. If we take the cumulative gain over a number
of years and annualize that gain by dividing by the number of years, we get a
different answer than if we simply take the average of a number of single year
returns. We can annualize the long-term
gain by taking the average of the one-year gains, or by taking the total gain
and dividing by the number of years.
How we annualize the return makes a difference, and the difference is
due to compounding.
For example, $1000 invested at a constant rate of 5% for ten years will
increase in value to $1628. If we divide
the real gain by the number of years, we have an annualized gain of 6.28%,
larger than the average annual gain of 5%.
If we change one year to a 35% gain, and another year to -30%, the
average gain will still be 5%, but the annualized real gain is now less than
4%. In the real world, the annualized
real gain may be greater or less than the average annual return.
I compared the average of 20 years of annual returns with real returns
for 20 years, over the period 1871 – 1995.
Here is the result.
Comparison of Average Annual
Return with Annualized Real Return on the S&P 500 Index.
So let’s compare the predictive power of the Shiller PE with the
annualized real gain over twenty years.
And for comparison, here again
is the Average 20 year return versus the Shiller PE
Real return shows greater scatter than the average annual return. In particular, returns are often higher at
the low end of the Shiller PE range, when stocks are cheap. In this correlation, returns are still
generally greater when stocks are cheap and smaller when stocks are expensive,
but there is little sense that returns can be quantified predictably.
Real Market Return with Saved Dividends
Of course, market performance, or capital gain, is only one component
of the return on owning stocks.
Dividends are a significant portion of the total return, even though
dividends are now small by historical standards. Dividend yield on the S&P 500 Index has
averaged 4.4% since 1871; while the current yield on the Index is a little over
2%.
History of the Dividend Yield on
the S&P 500 Index, 1871 – 2016
Suppose that we simply save the dividend yield, and recalculate the
return on stocks since 1871. Let’s look
at the predictive power of the Shiller PE compared to the real return, including
saved dividends.
Five Year Real Return with Saved
Dividends, versus the Shiller PE
Ten Year Real Return with Saved
Dividends, versus the Shiller PE
Twenty Year Real Return with
Saved Dividends, versus the Shiller PE
The graph of twenty year real return with saved dividends shows even
more scatter than the plot of real market return. Note that there are no negative returns over
the twenty-year time interval (although the 20 year performance of recent
markets, marked by high Shiller PE values, is not yet known). There is no particular trend to the data, and
a linear regression through the data is essentially flat.
Total Return with ReInvested Dividends
Finally, let’s consider the results of an investment program with reinvested
dividends.
Reinvested dividends allow market returns and compounding to contribute
and increase the contribution of dividend payments. Not surprisingly, the total return from real
market return, plus reinvested dividends and compounding exceeds previous
calculations of return.
Comparison of Various Methods of
Calculating Return over Twenty Year Intervals
The graph of Shiller PE versus total return including reinvested
dividends is again without apparent pattern.
All values are positive over twenty year intervals within the period
covered by the data, but the Shiller PE provides little predictability about
whether return will be high or low.
Twenty-Year Total Return, Including
Reinvested Dividends, versus Shiller PE, 1881 – 1996
So here is the result I did not expect.
The Shiller PE appears to have little power in predicting real-world
investment results for total return, including reinvested dividends.
Conclusion
The Shiller PE appears to have a good ability to predict the average
one-year market return over long periods.
But the parameter proves to be less useful for the thing that investors
really want to know: Is this a good time to invest in the market?”
If we test the ability of the Shiller PE to predict the real market
return on stocks, or the total return including dividends, or reinvested
dividends, the results are essentially random.
Within the history of the United States stock market, twenty-year
returns, including dividends, have always been positive (to date), after
adjusting for inflation. The Shiller PE
has very weak ability to predict the real market return, and even worse ability
to predict real returns including dividend payments, whether saved or invested.
From the 1970s, the best advice about stocks was to stay in the market,
regardless of whether the market was high or low. Burton Malkiel demonstrated this well in his
classic book, “A Random Walk Down Wall Street”, and Peter Lynch agreed, with
his adage to “Never Time the Market”.
Stock returns were interpreted as essentially random events, and timing
the market was judged to be a fools’ errand.
Despite the best efforts of
Nobel-prize winning economist Robert Shiller to devise an analytical tool which
would allow prediction of the stock market, stock market returns remain
essentially random. The best advice
seems to be to own stocks for the long term, and to avoid trying to time the
market.
An Important Caveat
The dataset used in this study was assembled by Robert Shiller, and
covers the history of the Standard and Poor’s 500 Index from 1871 to the
present. The conclusions which we can
draw from this experience are limited to the envelope of experience represented
in that history. From 1871 to the
present, there has been essentially no war on United States soil. There has been no revolution, no collapse of
the banking system, no devastating national health crisis, and no nationwide
natural or environmental disaster. The country has been on an upward
trajectory of increasing wealth and well-being for its citizens, and steadily
increasing economic productivity and technological innovation. We can see in other countries that such ideal
conditions do not necessarily hold, and there is no reason to believe that the
United States is immune to trouble beyond the range of what we have experienced
in the past. Any prudent financial plan
must include contingencies for disaster, of either personal or national scope.
References
Books
Benjamin Graham, 1949, 1986, 2005, The Intelligent
Investor, 640p.
Peter Lynch and John Rothchild, 1989, 2000, One Up on
Wall Street, 304p.
Peter Lynch and John Rothchild, 1993 Beating the Street,
336p.
Burton Malkiel, 1973, A Random Walk Down Wall Street,
456p.
Robert Shiller, 2000, 2005, 2015, Irrational Exuberance,
392p.
Nate Silver, 2012, The Signal and the Noise, 534p.
Internet sources:
Robert Shiller
Shiller PE, S&P
500 Index, S&P earnings
FRED Graph Observations
Federal Reserve Economic Data
Economic Research Division
Economic Research Division
Federal Reserve Bank of St. Louis
Inflation, Treasury Bond Yields
Nice post. These kinds of post help beginners to develop their skills. There are many classes are available for the investment of beginners like Stock Market Courses in Delhi which helps to develop our skills and knowledge for the bright future.
ReplyDelete