Category Archives: uncertainty

Near earth objects make tomorrow a little less than certain

Photograph of earth from spaceA couple of weeks ago, I wrote about unattainable uncertainty [see ‘Uncertainty is unattainable and near-uncertainty unaffordable’ on May 12th, 2021] and you might have thought that some things are certain, such as that tomorrow will follow today.  However, even that’s not certain – it has been estimated that there is a 1 in 300,000 chance in the next one hundred years of an asteroid impact on earth resulting in more than one million fatalities.  It might seem like a very small probability that you will not be around tomorrow due to an asteroid impact; however, as Sir David Spiegelhalter has pointed out, if that probability of fatalities related to an industrial installation, then it would be considered an intolerable risk by UK Health and Safety Executive.  By the way, if you want a more accurate estimate of the probability that an asteroid impact will prevent you seeing tomorrow then NASA provides information about the next Near Earth Object (NEO) to pass within 10 lunar distances (the distance between the moon and the earth which is 384,000 km) at https://cneos.jpl.nasa.gov/.  121 NEOs came within one lunar distance during the last twelve months of which the largest had a diameter of between 88 m and 200m, which is about the size of an Olympic Stadium, and came within 310,000 km; while the closest came within 8000 km, less than a Earth’s diameter which is 12,742 km, and was between 4.8 m and 11 m in diameter or about the size of two double-decker buses.  Spiegelhalter reassures us by telling us that there is no record of anyone, except a cow, being killed by an asteroid whereas tragically the same cannot be said about double decker buses!

Sources:

Reinhardt JC, Chen X, Liu W, Manchev P, Pate-Cornell ME. Asteroid risk assessment: a probabilistic approach. Risk Anal. 36:244–61, 2016.

Spiegelhalter D. Risk and uncertainty communication. Annual Review of Statistics and Its Application. 4:31-60, 2017.

Certainty is unattainable and near-certainty unaffordable

The economists John Kay and Mervyn King assert in their book ‘Radical Uncertainty – decision-making beyond numbers‘ that ‘economic forecasting is necessarily harder than weather forecasting’ because the world of economics is non-stationary whereas the weather is governed by unchanging laws of nature. Kay and King observe that both central banks and meteorological offices have ‘to convey inescapable uncertainty to people who crave unavailable certainty’. In other words, the necessary assumptions and idealisations combined with the inaccuracies of the input data of both economic and meteorological models produce inevitable uncertainty in the predictions. However, people seeking to make decisions based on the predictions want certainty because it is very difficult to make choices when faced with uncertainty – it raises our psychological entropy [see ‘Psychological entropy increased by ineffective leaders‘ on February 10th, 2021].  Engineers face similar difficulties providing systems with inescapable uncertainties to people desiring unavailable certainty in terms of the reliability.  The second law of thermodynamics ensures that perfection is unattainable [see ‘Impossible perfection‘ on June 5th, 2013] and there will always be flaws of some description present in a system [see ‘Scattering electrons reveal dislocations in material structure‘ on November 11th, 2020].  Of course, we can expend more resources to eliminate flaws and increase the reliability of a system but the second law will always limit our success. Consequently, to finish where I started with a quote from Kay and King, ‘certainty is unattainable and the price of near-certainty unaffordable’ in both economics and engineering.

Predicting the future through holistic awareness

Decorative picture: view along hotel corridorIt is traditional at the start of the year to speculate on what will happen in the new year.  However, as Niels Bohr is reputed to have said ‘Prediction is very difficult, especially about the future’.  Some people have suggested that our brains are constantly predicting the future. We weigh up the options for what might happen next before choosing a course of action. Our ancestors might have watched a fish swimming near a river bank and predicted where it would be a moment later when their spear entered the water. Or on a longer timescale, they predicted that seeds planted at a particular time of year would yield a crop some months later. Our predictions are not always correct but our life depends on enough of them being reliable that we have evolved to be good predictors of the immediate future. In Chinese thought, a distinction is made between predicting the near and distant future because the former is possible and latter is impossible, at least with any degree of confidence (Simandon, 2018).  Wisdom can be considered to be understanding the futility of trying to predict the distant future while being able to sense the near future through an acute awareness and immersion in one’s surroundings. This implies that a wise person can go beyond the everyday predictions of the immediate future, made largely unconsciously by our brains, and anticipate events on a slightly longer timescale, the near future. In engineering terms, events in the near future are short-term behaviour dominated by the current status of the system whereas events in the distant future are largely determined by external interactions with the system. This seems entirely consistent with the Chinese concept of wisdom arising from ‘vanishing into things’ which means to become immersed in a situation and hence to be able sense the current status of the system and reliably anticipate the near future. Some engineers might call it intuition which has been defined as ‘judgments that arise through rapid, non-conscious and holistic associations’ (Dane & Pratt, 2007).  So, in 2021 I hope to continue to exercise my intuition and remain immersed in a number of issues but I am not going to attempt to predict any distant events.

References:

Dane, E. and Pratt, M.G., 2007. Exploring intuition and its role in managerial decision making. Academy of management review, 32(1), pp.33-54.

Simandan, D., 2018. Wisdom and foresight in Chinese thought: sensing the immediate future. Journal of Futures Studies, 22(3), pp.35-50.

 

Credible predictions for regulatory decision-making

detail from abstract by Zahrah ReshRegulators are charged with ensuring that manufactured products, from aircraft and nuclear power stations to cosmetics and vaccines, are safe.  The general public seeks certainty that these devices and the materials and chemicals they are made from will not harm them or the environment.  Technologists that design and manufacture these products know that absolute certainty is unattainable and near-certainty in unaffordable.  Hence, they attempt to deliver the service or product that society desires while ensuring that the risks are As Low As Reasonably Practical (ALARP).  The role of regulators is to independently assess the risks, make a judgment on their acceptability and thus decide whether the operation of a power station or distribution of a vaccine can go ahead.  These are difficult decisions with huge potential consequences – just think of the more than three hundred people killed in the two crashes of Boeing 737 Max airplanes or the 10,000 or so people affected by birth defects caused by the drug thalidomide.  Evidence presented to support applications for regulatory approval is largely based on physical tests, for example fatigue tests on an aircraft structure or toxicological tests using animals.  In some cases the physical tests might not be entirely representative of the real-life situation which can make it difficult to make decisions using the data, for instance a ground test on an airplane is not the same as a flight test and in many respects the animals used in toxicity testing are physiologically different to humans.  In addition, physical tests are expensive and time-consuming which both drives up the costs of seeking regulatory approval and slows down the translation of new innovative products to the market.  The almost ubiquitous use of computer-based simulations to support the research, development and design of manufactured products inevitably leads to their use in supporting regulatory applications.  This creates challenges for regulators who must judge the trustworthiness of predictions from these simulations.  [see ‘Fake facts & untrustworthy predictions‘ on December 4th, 2019]. It is standard practice for modellers to demonstrate the validity of their models; however, validation does not automatically lead to acceptance of predictions by decision-makers.  Acceptance is more closely related to scientific credibility.  I have been working across a number of disciplines on the scientific credibility of models including in engineering where multi-physics phenomena are important, such as hypersonic flight and fusion energy [see ‘Thought leadership in fusion energy‘ on October 9th, 2019], and in computational biology and toxicology [see ‘Hierarchical modelling in engineering and biology‘ on March 14th, 2018]. Working together with my collaborators in these disciplines, we have developed a common set of factors which underpin scientific credibility that are based on principles drawn from the literature on the philosophy of science and are designed to be both discipline-independent and method-agnostic [Patterson & Whelan, 2019; Patterson et al, 2021]. We hope that our cross-disciplinary approach will break down the subject-silos that have become established as different scientific communities have developed their own frameworks for validating models.  As mentioned above, the process of validation tends to be undertaken by model developers and, in some sense, belongs to them; whereas, credibility is not exclusive to the developer but is a trust that needs to be shared with a decision-maker who seeks to use the predictions to inform their decision [see ‘Credibility is in the eye of the beholder‘ on April 20th, 2016].  Trust requires a common knowledge base and understanding that is usually built through interactions.  We hope the credibility factors will provide a framework for these interactions as well as a structure for building a portfolio of evidence that demonstrates the reliability of a model. 

References:

Patterson EA & Whelan MP, On the validation of variable fidelity multi-physics simulations, J. Sound & Vibration, 448:247-258, 2019.

Patterson EA, Whelan MP & Worth A, The role of validation in establishing the scientific credibility of predictive toxicology approaches intended for regulatory application, Computational Toxicology, 17: 100144, 2021.

Image: Extract from abstract by Zahrah Resh.