Tag Archives: experts

Intelligent openness

Photo credit: Tom

As an engineer and an academic, my opinion as an expert is sought often informally but less frequently formally, perhaps because I am reluctant to offer the certainty and precision that is so often expected of experts and instead I tend to highlight the options and uncertainties [see ‘Forecasts and chimpanzees throwing darts’ on September 2nd 2020].  These options and uncertainties will likely change as more information and knowledge becomes available.  An expert, who changes their mind and cannot offer certainty and precision, tends not to be welcomed by society, and in particular the media, who want simple statements and explanations.  One problem with offering certainty and precision as an expert is that it might appear you are part of a technocratic subset seeking to impose their values on the rest of society, as Mary O’Brien has argued.  The philosopher Douglas Walton has suggested that it is improper for experts to proffer their opinion when there is a naked assertion that the expert’s identity warrants acceptance of their opinion or argument.  Both O’Brien and Walton have argued that expert authority is legitimate only when it can be challenged, which is akin to Popper’s approach to the falsification of scientific theories – if it is not refutable then it is not science.  An expert’s authority should be acceptable only when it can be challenged and Onora O’Neill has argued that trustworthiness requires intelligent openness.  Intelligent openness means that the information being used by the expert is accessible and useable; the expert’s decision or argument is understandable (clearly explained in plain language) and assessable by someone with the time, expertise and access to the detail so that they can attempt to refute the expert’s statements.  In other words, experts need to be  transparent and science needs to be an open enterprise.

Sources:

Burgman MA, Trusting judgements: how to get the best out of experts, Cambridge: Cambridge University Press, 2016.

Harford T, How to make the world add up: 10 rules for thinking differently about numbers, London: Bridge Street Press, 2020.

O’Brien M, Making better environmental decisions: an alternative to risk assessment, Cambridge MA: MIT Press, 2000.

Walton D, Appeal to expert opinion: arguments from authority, University Park PA: Pennsylvania State University Press, 1997.

Royal Society, Science as an open enterprise, 2012: https://royalsociety.org/topics-policy/projects/science-public-enterprise/report/

Forecasts and chimpanzees throwing darts

During the coronavirus pandemic, politicians have taken to telling us that their decisions are based on the advice of their experts while the news media have bombarded us with predictions from experts.  Perhaps not unexpectedly, with the benefit of hindsight, many of these decisions and predictions appear to be have been ill-advised or inaccurate which is likely to lead to a loss of trust in both politicians and experts.  However, this is unsurprising and the reliability of experts, particularly those willing to make public pronouncements, is well-known to be dubious.  Professor Philip E. Tetlock of the University of Pennsylvania has assessed the accuracy of forecasts made by purported experts over two decades and found that they were little better than a chimpanzee throwing darts.  However, the more well-known experts seemed to be worse at forecasting [Tetlock & Gardner, 2016].  In other words, we should assign less credibility to those experts whose advice is more frequently sought by politicians or quoted in the media.  Tetlock’s research has found that the best forecasters are better at inductive reasoning, pattern detection, cognitive flexibility and open-mindedness [Mellers et al, 2015]. People with these attributes will tend not to express unambiguous opinions but instead will attempt to balance all factors in reaching a view that embraces many uncertainties.  Politicians and the media believe that we want to hear a simple message unadorned by the complications of describing reality; and, hence they avoid the best forecasters and prefer those that provide the clear but usually inaccurate message.  Perhaps that’s why engineers are rarely interviewed by the media or quoted in the press because they tend to be good at inductive reasoning, pattern detection, cognitive flexibility and are open-minded [see ‘Einstein and public engagement‘ on August 8th, 2018].  Of course, this was well-known to the Chinese philosopher, Lao Tzu who is reported to have said: ‘Those who have knowledge, don’t predict. Those who predict, don’t have knowledge.’

References:

Mellers, B., Stone, E., Atanasov, P., Rohrbaugh, N., Metz, S.E., Ungar, L., Bishop, M.M., Horowitz, M., Merkle, E. and Tetlock, P., 2015. The psychology of intelligence analysis: Drivers of prediction accuracy in world politics. Journal of experimental psychology: applied, 21(1):1-14.

Tetlock, P.E. and Gardner, D., 2016. Superforecasting: The art and science of prediction. London: Penguin Random House.