Most of the brilliant ideas my friends and I generate over beers die the next morning at the altar of my wife's relentless, data based questioning.

Live TV

Loading...

"So why exactly do you think this week long adventure trip with your buddies will help your book move forward ? You do know that despite the last 8 such trips the book is still at chapter 1…." You can imagine her narrowing eyes and the meaningful, suspicious look. And as I vacillate between amazement at how someone I normally feel so connected to, and generally find very intelligent, cannot grasp the simple genius of our idea, and anger at how every minute we talk we are not acting and airline ticket prices are increasing like volatility (VIX) in a crisis, some part of my brain starts telling me that we have been here before.

Eventually some of the words she is saying get through and yet again I realize the value of her opinion and remember why she has been appointed the "police" in our house. Yes I would be better off not going on a trip and just writing the damn book…

Sometimes our crazy ideas do sway her and she joins, other times she helps us rein it in. I am not a good listener - as most of my friends and wife will tell you, but that is exactly why even a little listening is incredibly helpful both to my portfolio and my life. Slowly but surely my friends and I have come to value skeptics. "Good to know", you might politely say, "but what does your wife have to do with investing ?"

Being researchers, my colleagues and I decided to investigate if the average investor is as bad at listening as we are and evaluate the effect it might have on their stock market trading. Using artificial intelligence (AI) we looked at 34 million message posts from 13,000 boards-yahoo stock message boards from Feb 1996 onwards and InvestorHub (iHub) from March, 2000 onwards. We found that 80 percent to 95 percent of the users had less than 50 posts and most of the traffic was about highly speculative nanocap (very small <$50 million market capitalization ) companies.

We then generated a dictionary by hand labeling a small number of posts (~ 4,500) for iHub, and using the 5 percent or so of Yahoo messages that are already labeled as "Strong Buy", "Buy", "Sell", "Strong Sell", and "Hold". Using that dictionary we then trained the computer via machine learning (ML) algorithms - a type of technique in the broad field of AI, to go through the rest of the posts and characterize the discussions. Our algorithms did a reasonable job with 72 percent accuracy for Yahoo and 81 percent for iHub.

We had three broad topics we wanted to understand and corresponding specific questions in mind:

Broad topic #1: Sentiment and Echo Chamber effect

Research question: Does sentiment decouple from external information?

Our Finding: Yes, sentiment does seem to de-couple from external information.

Users tend to be optimistic no matter what their favorite stock does- whether it goes up or down. On average there is no relation (as you can see below) between sentiment and stock return. On up days the correlation is moderately positive and on down days it is negative - indicating that users maintain their bias regardless of stock move.

Broad topic #2: Structure of communication

Research question: What roles do individuals play in the depth and length of specific discussions?

Our Finding: Reactors and Followers want and enforce positivity regardless of reality and abuse people who do not agree!

Even leaders start with slightly more bearish sentiment but reactors and followers dilute their own leaders' negative sentiment, and also exhibit animosity towards other users who attempt to say something negative about their favorite stock.

Broad topic #3: Effect on trading performance

Research question: Are these message boards useful for predicting returns and trading activity?

Our finding: No, message boards are not good for generating returns but are good for predicting activity.

Simply relying on message boards sentiment would lead 50-70 percent users to underperform in trading. However, using sentiment can help predict the volume or the amount of trading as well as the volatility. Traders simply relying on message board sentiments are more likely to be noise traders.

The other interesting finding is that the shortness of user tenure seems to line up with underperformance. Users new to these groups underperform the index by 70 percent vs. the ones that have been around for 10 years perform close to the index on average. However, we may have survivorship bias here- that is users who underperformed by a lot stopped posting.

We can see that the stocks that get more attention from investors have a much higher trading volume (as indicated by the blue line being on the right of the red line). Thus activity is unambiguously predictable. In separate results we are also able to predict volatility.

The echo chamber and how to avoid it?

Sustein, (2001) defines an Echo Chamber as a social discourse environment in which ideas tend to be mutually reinforced by a like-minded group of people- think of the current global political climate where we are less willing to listen than say 15 years ago. The economist, in a provocative article titled "The partisan brain" cites some interesting recent psychological studies that show that "a lot of reasoning is devoted to affirming your group's identity and your position within it." "It seems that giving consideration to the other sides' point of view hurts-literally."

Our research above implies that when we engage only with like-minded people, we will generally make worse decisions in trading. Thus, listening to people who have an opposing viewpoint while difficult or perhaps even physically painful might lead to better trading performance!

Apurv Jain (@theapurvj) is a visiting researcher at Harvard Business School working on using Big Data and AI in investing. He is an investor in and advisor to Kuvera.in: India’s first completely free Direct Mutual Fund investing platform. (@Kuverablog)

Have you signed up for Primo, our daily newsletter? It has all the stories and data on the market, business, economy and tech that you need to know.

First Published:Dec 14, 2018 1:59 PM IST