Insights
BIN There, Done That
|
|
Consilient Observer
|
• |
March 20, 2020
|
|
March 20, 2020
|
BIN There, Done That
BIN There, Done That |
|
How to Reduce the Sources of Forecasting Error
Anticipating revisions in market expectations is the key to generating long-term excess returns as an investor. Successful long-term investors see where expectations are headed based on their forecasts, but research shows that most experts do not make great forecasts.
Barbara Mellers, a professor of psychology at the University of Pennsylvania, says, “We find prediction really hard, but we find explanation fairly easy.”1 We tell stories to ourselves and others to paper over our poor predictions. But the news is not all gloomy. The U.S. intelligence community sponsored a forecasting tournament, which revealed that one-in-fifty “superforecasters” consistently made better predictions than did the other participants.
Part of the success of the superforecasters reflected their qualities, including numeracy, intellectual curiosity, probabilistic thinking, and open-mindedness. But part of their success relied on habits that can be “learned and cultivated by any intelligent, thoughtful, determined person.”2
Statisticians and psychologists examined the gap between “superforecasters” and regular forecasters using the “BIN Model,” where “B” refers to bias, “I” to information and “N” to noise.
When the researchers tallied why the superforecasters were so much better than the regular forecasters, they found the difference was roughly 50 percent from reducing noise, 25 percent from reducing bias and 25 percent from increasing information. It’s important to recognize that because noise and bias are independent sources of error, reducing either one of them improves forecasts.
Source: Counterpoint Global.
Source: Counterpoint Global.
When the researchers examined the impact of training on overall accuracy, they also found that the training improved accuracy more by reducing noise than by tamping down bias.
Noise. The three primary ways to reduce noise are combining judgments, using algorithms, and adopting the “Mediating Assessments Protocol” (MAP).
Combining forecasts is the core idea behind the wisdom of crowds, with the market a good example of a mechanism that aggregates views. Forecasting accuracy is compromised when there is no effective way to aggregate the information, or if the errors become correlated and noise transitions to collective bias.
Algorithms are simply sets of rules or procedures that allow you to achieve a goal. For example, a cake recipe is an algorithm. Research has found that one way to increase the acceptance of algorithms is to allow humans to adjust the final answer just a bit, what Daniel Kahneman calls “disciplined intuition.”
Finally, the Mediating Assessments Protocol (MAP) is a way to create intermediate ratings based on what’s critical in order to come to a final decision that is more informed than relying solely on intuition.
Bias. Debiasing methods introduce base rates to ground forecasts, premortems and red teams to open minds to alternative outcomes, and signposts to create a way to learn from mistakes. The methods produce spillover effects that help manage noise as well.
Information. Using information effectively means getting an informational advantage when you can, updating your view accurately and promptly to reflect new information, and placing the proper weight on the information at your disposal. Analysis of the forecasters in the tournament revealed that the frequency of belief updating was “the strongest single behavioral predictor of accuracy.”
Investors can use the BIN model to assess their decision-making process, with a heightened awareness of the large and potentially overlooked role of noise. Because noise tends to be overlooked or ignored, improving consistency can add a great deal of value, and identifying companies that adopt practices to reduce noise may be the source of excess returns.
RISK CONSIDERATIONS
Predictions are based on current market conditions, subject to change, and may not necessarily come to pass.
There is no assurance that the techniques mentioned in this article will be successful in helping improve the accuracy of predictions. There is no assurance that a Portfolio will achieve its investment objective. Portfolios are subject to market risk, which is the possibility that the market values of securities owned by the Portfolio will decline and that the value of Portfolio shares may therefore be less than what you paid for them. Market values can change daily due to economic and other events (e.g. natural disasters, health crises, terrorism, conflicts and social unrest) that affect markets, countries, companies or governments. It is difficult to predict the timing, duration, and potential adverse effects (e.g. portfolio liquidity) of events. Accordingly, you can lose money investing in this Portfolio. Please be aware that this Portfolio may be subject to certain additional risks. In general, equities securities’ values also fluctuate in response to activities specific to a company. Investments in foreign markets entail special risks such as currency, political, economic, market and liquidity risks. The risks of investing in emerging market countries are greater than risks associated with investments in foreign developed countries. Privately placed and restricted securities may be subject to resale restrictions as well as a lack of publicly available information, which will increase their illiquidity and could adversely affect the ability to value and sell them (liquidity risk). Derivative instruments may disproportionately increase losses and have a significant impact on performance. They also may be subject to counterparty, liquidity, valuation, correlation and market risks. Illiquid securities may be more difficult to sell and value than public traded securities (liquidity risk).