Insight Article Desktop Banner
Consilient Observer
March 20, 2020

BIN There, Done That

Insight Video Mobile Banner
March 20, 2020

BIN There, Done That

Consilient Observer

BIN There, Done That

Share Icon

March 20, 2020


How to Reduce the Sources of Forecasting Error


Anticipating revisions in market expectations is the key to generating long-term excess returns as an investor. Successful long-term investors see where expectations are headed based on their forecasts, but research shows that most experts do not make great forecasts.

Barbara Mellers, a professor of psychology at the University of Pennsylvania, says, “We find prediction really hard, but we find explanation fairly easy.”1 We tell stories to ourselves and others to paper over our poor predictions. But the news is not all gloomy. The U.S. intelligence community sponsored a forecasting tournament, which revealed that one-in-fifty “superforecasters” consistently made better predictions than did the other participants.

Part of the success of the superforecasters reflected their qualities, including numeracy, intellectual curiosity, probabilistic thinking, and open-mindedness. But part of their success relied on habits that can be “learned and cultivated by any intelligent, thoughtful, determined person.”2 

Statisticians and psychologists examined the gap between “superforecasters” and regular forecasters using the “BIN Model,” where “B” refers to bias, “I” to information and “N” to noise.

When the researchers tallied why the superforecasters were so much better than the regular forecasters, they found the difference was roughly 50 percent from reducing noise, 25 percent from reducing bias and 25 percent from increasing information. It’s important to recognize that because noise and bias are independent sources of error, reducing either one of them improves forecasts.

Display 1: Bias and Noise

Source: Counterpoint Global.


When the researchers examined the impact of training on overall accuracy, they also found that the training improved accuracy more by reducing noise than by tamping down bias.

Noise. The three primary ways to reduce noise are combining judgments, using algorithms, and adopting the “Mediating Assessments Protocol” (MAP).

Combining forecasts is the core idea behind the wisdom of crowds, with the market a good example of a mechanism that aggregates views. Forecasting accuracy is compromised when there is no effective way to aggregate the information, or if the errors become correlated and noise transitions to collective bias.

Algorithms are simply sets of rules or procedures that allow you to achieve a goal. For example, a cake recipe is an algorithm. Research has found that one way to increase the acceptance of algorithms is to allow humans to adjust the final answer just a bit, what Daniel Kahneman calls “disciplined intuition.”

Finally, the Mediating Assessments Protocol (MAP) is a way to create intermediate ratings based on what’s critical in order to come to a final decision that is more informed than relying solely on intuition.

Bias. Debiasing methods introduce base rates to ground forecasts, premortems and red teams to open minds to alternative outcomes, and signposts to create a way to learn from mistakes. The methods produce spillover effects that help manage noise as well.

Information. Using information effectively means getting an informational advantage when you can, updating your view accurately and promptly to reflect new information, and placing the proper weight on the information at your disposal. Analysis of the forecasters in the tournament revealed that the frequency of belief updating was “the strongest single behavioral predictor of accuracy.”

Investors can use the BIN model to assess their decision-making process, with a heightened awareness of the large and potentially overlooked role of noise. Because noise tends to be overlooked or ignored, improving consistency can add a great deal of value, and identifying companies that adopt practices to reduce noise may be the source of excess returns.

Article Team Image
Counterpoint Global’s culture fosters collaboration, creativity, continued development, and differentiated thinking.

Subscribe to Counterpoint Global Insights - Consilient Observer

Please enter the code sent to your email address.

Valid for 10 minutes only

Email Not Sent

1 “Want Better Forecasting Skills? Silence the Noise,” Knowledge@Wharton, November 26, 2019.

2 Philip E. Tetlock and Dan Gardner, Superforecasting: The Art and Science of Prediction (New York: Crown Publishers, 2015)



The views and opinions are those of the author as of the date of publication and are subject to change at any time due to market or economic conditions and may not necessarily come to pass. The views expressed do not reflect the opinions of all investment personnel at Morgan Stanley Investment Management (MSIM) and its subsidiaries and affiliates (collectively the Firm”), and may not be reflected in all the strategies and products that the Firm offers.

This material is for the benefit of persons whom the Firm reasonably believes it is permitted to communicate to and should not be forwarded to any other person without the consent of the Firm. It is not addressed to any other person and may not be used by them for any purpose whatsoever. It is the responsibility of every person reading this material to fully observe the laws of any relevant country, including obtaining any governmental or other consent which may be required or observing any other formality which needs to be observed in that country. 

This material is a general communication, which is not impartial, is for informational and educational purposes only, not a recommendation to purchase or sell specific securities, or to adopt any particular investment strategy. Information does not address financial objectives, situation or specific needs of individual investors. 

Any charts and graphs provided are for illustrative purposes only. Any performance quoted represents past performance. Past performance does not guarantee future results. All investments involve risks, including the possible loss of principal.

For the complete content and important disclosures, refer to the article's PDF.


This is a Marketing Communication.

It is important that users read the Terms of Use before proceeding as it explains certain legal and regulatory restrictions applicable to the dissemination of information pertaining to Morgan Stanley Investment Management's investment products.

The services described on this website may not be available in all jurisdictions or to all persons. For further details, please see our Terms of Use.

Privacy & Cookies    •    Your Privacy Choices Your Privacy Choices Icon    •    Terms of Use

©  Morgan Stanley. All rights reserved.