Insight Article Desktop Banner
 
 
Consilient Observer
  •  
March 20, 2020

BIN There, Done That

Insight Video Mobile Banner
 
March 20, 2020

BIN There, Done That


Consilient Observer

BIN There, Done That

Share Icon

March 20, 2020

 
 

How to Reduce the Sources of Forecasting Error

 
 

Anticipating revisions in market expectations is the key to generating long-term excess returns as an investor. Successful long-term investors see where expectations are headed based on their forecasts, but research shows that most experts do not make great forecasts.

Barbara Mellers, a professor of psychology at the University of Pennsylvania, says, “We find prediction really hard, but we find explanation fairly easy.”1 We tell stories to ourselves and others to paper over our poor predictions. But the news is not all gloomy. The U.S. intelligence community sponsored a forecasting tournament, which revealed that one-in-fifty “superforecasters” consistently made better predictions than did the other participants.

Part of the success of the superforecasters reflected their qualities, including numeracy, intellectual curiosity, probabilistic thinking, and open-mindedness. But part of their success relied on habits that can be “learned and cultivated by any intelligent, thoughtful, determined person.”2 

Statisticians and psychologists examined the gap between “superforecasters” and regular forecasters using the “BIN Model,” where “B” refers to bias, “I” to information and “N” to noise.

When the researchers tallied why the superforecasters were so much better than the regular forecasters, they found the difference was roughly 50 percent from reducing noise, 25 percent from reducing bias and 25 percent from increasing information. It’s important to recognize that because noise and bias are independent sources of error, reducing either one of them improves forecasts.

 
 
 
Display 1: Bias and Noise
 

Source: Counterpoint Global.

 
 

When the researchers examined the impact of training on overall accuracy, they also found that the training improved accuracy more by reducing noise than by tamping down bias.

Noise. The three primary ways to reduce noise are combining judgments, using algorithms, and adopting the “Mediating Assessments Protocol” (MAP).

Combining forecasts is the core idea behind the wisdom of crowds, with the market a good example of a mechanism that aggregates views. Forecasting accuracy is compromised when there is no effective way to aggregate the information, or if the errors become correlated and noise transitions to collective bias.

Algorithms are simply sets of rules or procedures that allow you to achieve a goal. For example, a cake recipe is an algorithm. Research has found that one way to increase the acceptance of algorithms is to allow humans to adjust the final answer just a bit, what Daniel Kahneman calls “disciplined intuition.”

Finally, the Mediating Assessments Protocol (MAP) is a way to create intermediate ratings based on what’s critical in order to come to a final decision that is more informed than relying solely on intuition.

Bias. Debiasing methods introduce base rates to ground forecasts, premortems and red teams to open minds to alternative outcomes, and signposts to create a way to learn from mistakes. The methods produce spillover effects that help manage noise as well.

Information. Using information effectively means getting an informational advantage when you can, updating your view accurately and promptly to reflect new information, and placing the proper weight on the information at your disposal. Analysis of the forecasters in the tournament revealed that the frequency of belief updating was “the strongest single behavioral predictor of accuracy.”

Investors can use the BIN model to assess their decision-making process, with a heightened awareness of the large and potentially overlooked role of noise. Because noise tends to be overlooked or ignored, improving consistency can add a great deal of value, and identifying companies that adopt practices to reduce noise may be the source of excess returns.

 
Article Team Image
 
Counterpoint Global consists of 42 people, including 21 investors, five disruptive change researchers, two consilient researchers and two sustainability researchers. Counterpoint Global’s culture fosters collaboration, creativity, continued development, and differentiated thinking.
 
 

Subscribe to Counterpoint Global Insights - Consilient Observer

Please enter the code sent to your email address.

Valid for 10 minutes only

Confirm
Email Not Sent
 
 
Submit
 
 
 

1 “Want Better Forecasting Skills? Silence the Noise,” Knowledge@Wharton, November 26, 2019.

2 Philip E. Tetlock and Dan Gardner, Superforecasting: The Art and Science of Prediction (New York: Crown Publishers, 2015)

 
 
 
 

DISTRIBUTION

This communication is only intended for and will only be distributed to persons resident in jurisdictions where such distribution or availability would not be contrary to local laws or regulations.

EMEA:

This marketing communication has been issued by MSIM Fund Management (Ireland) Limited. MSIM Fund Management (Ireland) Limited is regulated by the Central Bank of Ireland. MSIM Fund Management (Ireland) Limited is incorporated in Ireland as a private company limited by shares with company registration number 616661 and has its registered address at The Observatory, 7-11 Sir John Rogerson's Quay, Dublin 2, D02 VC42, Ireland.

U.S.:

NOT FDIC INSURED | OFFER NO BANK GUARANTEE | MAY LOSE VALUE | NOT INSURED BY ANY FEDERAL GOVERNMENT AGENCY | NOT A BANK DEPOSIT

Hong Kong: This document has been issued by Morgan Stanley Asia Limited for use in Hong Kong and shall only be made available to “professional investors” as defined under the Securities and Futures Ordinance of Hong Kong (Cap 571). The contents of this document have not been reviewed nor approved by any regulatory authority including the Securities and Futures Commission in Hong Kong. Accordingly, save where an exemption is available under the relevant law, this document shall not be issued, circulated, distributed, directed at, or made available to, the public in Hong Kong. Singapore: This document may not be circulated or distributed, whether directly or indirectly, to persons in Singapore other than to (i) an accredited investor (ii) an expert investor or (iii) an institutional investor as defined in Section 4A of the Securities and Futures Act, Chapter 289 of Singapore (“SFA”); or (iv) otherwise pursuant to, and in accordance with the conditions of, any other applicable provision of the SFA. This publication has not been reviewed by the Monetary Authority of Singapore. Australia: This publication is disseminated in Australia by Morgan Stanley Investment Management (Australia) Pty Limited AC: 122040037, AFSL No. 314182, which accept responsibility for its contents. This publication, and any access to it, is intended only for “wholesale clients” within the meaning of the Australian Corporations Act.

Japan: This document may not be circulated or distributed, whether directly or indirectly, to persons in Japan other than to (i) a professional investor as defined in Article 2 of the Financial Instruments and Exchange Act (“FIEA”) or (ii) otherwise pursuant to, and in accordance with the conditions of, any other allocable provision of the FIEA. This document is disseminated in Japan by Morgan Stanley Investment Management (Japan) Co., Ltd., Registered No. 410 (Director of Kanto Local Finance Bureau (Financial Instruments Firms)), Membership: the Japan Securities Dealers Association, The Investment Trusts Association, Japan, the Japan Investment Advisers Association and the Type II Financial Instruments Firms Association.

IMPORTANT INFORMATION

Investing involves risk including the possible loss of principal. There is no guarantee that any investment strategy will work under all market conditions, and each investor should evaluate their ability to invest for the long-term, especially during periods of downturn in the market. 

The views and opinions are those of the author as of the date of preparation of this material and are subject to change at any time due to market or economic conditions and may not necessarily come to pass. Furthermore, the views will not be updated or otherwise revised to reflect information that subsequently becomes available or circumstances existing, or changes occurring, after the date of publication. The views expressed do not reflect the opinions of all investment teams at Morgan Stanley Investment Management (MSIM) or the views of the firm as a whole, and may not be reflected in all the strategies and products that the Firm offers.

Certain information herein is based on data obtained from third party sources believed to be reliable. However, we have not verified this information, and we make no representations whatsoever as to its accuracy or completeness. The views expressed in the books and articles referenced in this whitepaper are not necessarily endorsed by Morgan Stanley Investment Management.

The information herein is a general communications which is not impartial and has been prepared solely for information and educational purposes and does not constitute an offer or a recommendation to buy or sell any particular security or to adopt any specific investment strategy. The material contained herein has not been based on a consideration of any individual client circumstances and is not investment advice, nor should it be construed in any way as tax, accounting, legal or regulatory advice. To that end, investors should seek independent legal and financial advice, including advice as to tax consequences, before making any investment decision.

Charts and graphs provided herein are for illustrative purposes only. Any securities referenced herein are solely for illustrative purposes only and should not be construed as a recommendation for investment.

This communication is not a product of Morgan Stanley’s Research Department and should not be regarded as a research recommendation. The information contained herein has not been prepared in accordance with legal requirements designed to promote the independence of investment research and is not subject to any prohibition on dealing ahead of the dissemination of investment research.

MSIM has not authorised financial intermediaries to use and to distribute this document, unless such use and distribution is made in accordance with applicable law and regulation. Additionally, financial intermediaries are required to satisfy themselves that the information in this document is appropriate for any person to whom they provide this document in view of that person’s circumstances and purpose. MSIM shall not be liable for, and accepts no liability for, the use or misuse of this document by any such financial intermediary.

The whole or any part of this work may not be directly or indirectly reproduced, copied, modified, used to create a derivative work, performed, displayed, published, posted, licensed, framed, distributed or transmitted or any of its contents disclosed to third parties without MSIM’s express written consent. This work may not be linked to unless such hyperlink is for personal and non-commercial use. All information contained herein is proprietary and is protected under copyright and other applicable law.

Morgan Stanley Investment Management is the asset management division of Morgan Stanley.

This document may be translated into other languages. Where such a translation is made this English version remains definitive. If there are any discrepancies between the English version and any version of this document in another language, the English version shall prevail.

 

It is important that users read the Terms of Use before proceeding as it explains certain legal and regulatory restrictions applicable to the dissemination of information pertaining to Morgan Stanley Investment Management's investment products.

The services described on this website may not be available in all jurisdictions or to all persons. For further details, please see our Terms of Use.

Not FDIC Insured—Offer No Bank Guarantee—May Lose Value
Not Insured By Any Federal Government Agency—Not A Deposit

Subscriptions    •    Privacy & Cookies    •    Terms of Use

©  Morgan Stanley. All rights reserved.

Morgan Stanley Distribution, Inc. Member FINRA/SIPC.