Can sentiment predict market direction?

Question: Is investor sentiment a useful indicator of market direction? If possible, I would like to trade on sentiment to make money.

Blogger Sentiment

Popular measures of investor sentment are reported in Barron's each week. For brief background reading, try the Investopedia article entitled "Investors Intelligence Sentiment Index." The article cites an academic study published in 2000 by Ken Fisher and Meir Statman that concluded: "We found the relationship between the sentiment of newsletter writers as measured by the Investors Intelligence survey and future S&P 500 returns to be negative but not statistically significant." As is the case with most (if not all?) fundamental and technical indicators, the prospect of using a simple sentiment index to trade and consistently realize excess profits does not look very encouraging.

However, instead of giving up so easily, let's have a look at a newer sentiment index that is being tracked by Ticker Sense, a financial blog of Birinyi Associates, run by former Salomon executive Laszlo Birinyi. Beginning in July of 2006, Ticker Sense has been reporting at the start of each trading week market sentiment figures resulting from a poll sent out to participating bloggers the prior Thursday. Bloggers state whether they are bullish, bearish or neutral on the S&P 500 for the upcoming 30 days. The chart below shows this sentiment data for the past 12 months.



We can develop a qualitative feel for how useful this type of sentiment data might be in a trading context by plotting the S&P 500 index alongside the weekly difference between the bull and bear sentiment percentages. The chart below shows that for the past couple of months blogger sentiment has correlated favorably with the directional movement of the S&P 500 index--the market moves down with negative bull-bear sentiment in early August, up with positive bull-bear sentiment from about mid-August through the market's recent highs in early October, and down on slightly negative sentiment last week. Hey, this is beginning to look promising. . . .



"Batting Average" Test

One barometer for gauging how helpful blogger sentiment can be in predicting market direction is to perform a "batting average" calculation. A baseball player's batting average is a number between zero and 1.000 (or zero and 1000, if we ignore the decimal point), indicating the ratio of hits to at-bats during a season. For example, during 2004 when Seattle Mariners all-star player Ichiro set a new all-time major league baseball record with 262 hits, his batting average in his 704 season at-bats was 262/704 = 0.372.

In an analogous fashion, we can define a sentiment "batting average" as being the time-average of the relevant (bull, bear or neutral) sentiment percentages corresponding to the actual up, down and flat market outcomes observed during all of the trials in a specified testing period:

Sentiment "Batting Average" = Sum(X_i)/N,

where

X_i = bullish (bearish, neutral) sentiment percentage at time i-1 if market return ends up in bullish (bearish, neutral) range at time i,

and i runs from 1 to N, inclusive, where N is the number of trials in the testing period.

(Note: Since the Ticker Sense sentiment poll is updated at a weekly frequency, for our analysis we pair each week's sentiment data with only the following week's market movement. In effect, sentiment predictions are "refreshed" each week, even though bloggers participating in the poll are asked for their opinion about the next 30 days.)

Here's a numerical example: If a particular week's sentiment poll gives bull-bear-neutral sentiment percentages of 50-30-20 and the market ends up falling into the bearish range the week after the poll is taken, this particular trial contributes an X-value of 0.300 to the average. Obviously, the highest batting average is attained when the actual market movement always matches the strongest prevailing sentiment (or highest percentage) among the three possible sentiment "states" (bull, bear and neutral).

Two extreme cases help to clarify the meaning of our sentiment "batting average": If all bloggers participating in the poll had complete clairvoyance, the sentiment percentages would always be 100-0-0 (bullish), 0-100-0 (bearish) or 0-0-100 (neutral), and each trial would always contribute an X-value of 1.000, resulting in a perfect time-average of 1.000. At the other extreme, if the bloggers were always completely split without any predominating opinion, the sentiment percentages would be 33-33-33 (rounded), and each trial would always contribute an X-value of 0.333, regardless of whether the market rises, falls or remains flat. Importantly, this batting average of 0.333 is also the expected outcome when collective opinion, whether skewed or split, is no better than a random guess at determining market direction.

In order to apply our batting-average methodology to the blogger sentiment data, we also need to define what we mean by bullish, bearish and neutral outcomes, and ideally these three "states" should be equally likely, so that no particular outcome is favored over the others. To take into account the possibility of "trending" regimes, rather than referencing static return ranges I use a 26-week moving window ending just prior to each weekly trial:

"Bullish": Above 67th percentile return of immediately prior 26 weeks

"Neutral": Between 33rd and 67th percentile return of immediately prior 26 weeks

"Bearish": Below 33rd percentile return of immediately prior 26 weeks

Though there is some variation from week to week, for the S&P 500 over the past year the 26-week moving averages of the 33rd and 67th percentile weekly returns have hovered around -0.3% and 1.2%, respectively, giving sentiment ranges approximately as follows:

"Bullish": Weekly return above 1.2%

"Neutral": Weekly return between -0.3% and 1.2%

"Bearish": Weekly return below -0.3%

The graph below shows the week-by-week contributions to the overall average. For example, for the week ending October 5, bullish sentiment (from the poll sent out Thursday of the prior week) was 50% and the market traded up, thereby contributing an X-value of 0.500. Note, however, that the overall average from the past one year is the smaller figure of 0.319, which is slightly worse than the 0.333 expected in the case of random guessing. In other words, the batting-average calculation suggests that blogger sentiment will probably not be very useful in predicting market direction.



Simulated Trading Test

Another way to visualize the sentiment data is to look at scatterplots of the bull-bear sentiment difference (at time i-1) versus S&P 500 returns for the following week (at time i). I divide the past year into two 26-week periods to enable us to run two simulated trading tests on the sentiment data.

For the first 26-week period, from October 2006 through April 2007, there is a somewhat negative correlation between sentiment and returns, suggesting that sentiment could be slightly contra-indicative of market direction.



However, over the next 26-week period, from April 2007 through October 2007, the correlation is essentially zero, indicating no apparent relationship between sentiment and subsequent returns.



We can trade on the sentiment data by using the weekly bull-bear sentiment differences to weight one-week trades on the S&P 500 index (or futures), going long when the difference is positive (net bullish) and going short when it is negative (net bearish). Trade size equals the absolute value of the bull-bear sentiment difference, so that we take larger trading positions when sentiment is more extremely bullish or bearish. The graph below shows the simulated outcome of such trading over our two 26-week periods. The initial 26-week trading period produces a loss, while the later 26-week trading period begins in negative territory but goes slightly positive from early August to early October when sentiment, as mentioned above, matches market direction.



Is There Hope?

Overall, with the batting-average test failing to show all-star-like performance above the 0.333 expected value of random guessing, and the trading test producing what appears to be little more than a random walk, it is difficult to place any confidence in blogger sentiment as a useful predictor of market direction.

But, perhaps we should not overlook how traders, investors and bloggers (myself included) all learn as we go, thereby opening up the possibility that the blogger sentiment index could be an example of a self-improving dynamic system engaged in an adaptive learning process. The most recent two months certainly show promise, and sentiment data from last Thursday's poll (bull-bear-neutral: 50-33-17), with bloggers turning bullish following last week's 4% sell-off, again matches the S&P 500's interim performance so far this week (up from Friday's 1501 close)--though, of course, it is too early to tell how the remainder of the week will turn out.

What, then, should we believe: the negative overall read from the past year, or the more promising performance of sentiment over the past few months? Rather than allowing our hopes to rise too high, I suggest keeping in mind a market truism: if there really is any predictive power in a well-publicized indicator, opportunistic traders will soon exploit this information, thereby squelching any excess returns that might have been available. Maybe, then, the trick is to be quick--exploit the winning "streak" before it vanishes! Good luck trading!

Source: http://lloydsinvestment.blogspot.com/2007/10/can-sentiment-predict-market-direction.html

forex trading strategies forex trading strategy forex trading system forex trading systems forex trading tips forex trading tutorial forex trading wiki