Tag Archives: simulation

Credible predictions for regulatory decision-making

detail from abstract by Zahrah ReshRegulators are charged with ensuring that manufactured products, from aircraft and nuclear power stations to cosmetics and vaccines, are safe.  The general public seeks certainty that these devices and the materials and chemicals they are made from will not harm them or the environment.  Technologists that design and manufacture these products know that absolute certainty is unattainable and near-certainty in unaffordable.  Hence, they attempt to deliver the service or product that society desires while ensuring that the risks are As Low As Reasonably Practical (ALARP).  The role of regulators is to independently assess the risks, make a judgment on their acceptability and thus decide whether the operation of a power station or distribution of a vaccine can go ahead.  These are difficult decisions with huge potential consequences – just think of the more than three hundred people killed in the two crashes of Boeing 737 Max airplanes or the 10,000 or so people affected by birth defects caused by the drug thalidomide.  Evidence presented to support applications for regulatory approval is largely based on physical tests, for example fatigue tests on an aircraft structure or toxicological tests using animals.  In some cases the physical tests might not be entirely representative of the real-life situation which can make it difficult to make decisions using the data, for instance a ground test on an airplane is not the same as a flight test and in many respects the animals used in toxicity testing are physiologically different to humans.  In addition, physical tests are expensive and time-consuming which both drives up the costs of seeking regulatory approval and slows down the translation of new innovative products to the market.  The almost ubiquitous use of computer-based simulations to support the research, development and design of manufactured products inevitably leads to their use in supporting regulatory applications.  This creates challenges for regulators who must judge the trustworthiness of predictions from these simulations.  [see ‘Fake facts & untrustworthy predictions‘ on December 4th, 2019]. It is standard practice for modellers to demonstrate the validity of their models; however, validation does not automatically lead to acceptance of predictions by decision-makers.  Acceptance is more closely related to scientific credibility.  I have been working across a number of disciplines on the scientific credibility of models including in engineering where multi-physics phenomena are important, such as hypersonic flight and fusion energy [see ‘Thought leadership in fusion energy‘ on October 9th, 2019], and in computational biology and toxicology [see ‘Hierarchical modelling in engineering and biology‘ on March 14th, 2018]. Working together with my collaborators in these disciplines, we have developed a common set of factors which underpin scientific credibility that are based on principles drawn from the literature on the philosophy of science and are designed to be both discipline-independent and method-agnostic [Patterson & Whelan, 2019; Patterson et al, 2021]. We hope that our cross-disciplinary approach will break down the subject-silos that have become established as different scientific communities have developed their own frameworks for validating models.  As mentioned above, the process of validation tends to be undertaken by model developers and, in some sense, belongs to them; whereas, credibility is not exclusive to the developer but is a trust that needs to be shared with a decision-maker who seeks to use the predictions to inform their decision [see ‘Credibility is in the eye of the beholder‘ on April 20th, 2016].  Trust requires a common knowledge base and understanding that is usually built through interactions.  We hope the credibility factors will provide a framework for these interactions as well as a structure for building a portfolio of evidence that demonstrates the reliability of a model. 

References:

Patterson EA & Whelan MP, On the validation of variable fidelity multi-physics simulations, J. Sound & Vibration, 448:247-258, 2019.

Patterson EA, Whelan MP & Worth A, The role of validation in establishing the scientific credibility of predictive toxicology approaches intended for regulatory application, Computational Toxicology, 17: 100144, 2021.

Image: Extract from abstract by Zahrah Resh.

Digital twins could put at risk what it means to be human

Detail from abstract by Zahrah ReshI have written in the past about my research on the development and use of digital twins.  A digital twin is a functional representation in a virtual world of a real world entity that is continually updated with data from the real world [see ‘Fourth industrial revolution’ on July 4th, 2018 and also a short video at https://www.youtube.com/watch?v=iVS-AuSjpOQ].  I am working with others on developing an integrated digital nuclear environment from which digital twins of individual power stations could be spawned in parallel with the manufacture of their physical counterparts [see ‘Enabling or disruptive technology for nuclear engineering’ on January 1st, 2015 and ‘Digitally-enabled regulatory environment for fusion power-plants’ on March 20th, 2019].  A couple of months ago, I wrote about the difficulty of capturing tacit knowledge in digital twins, which is knowledge that is generally not expressed but is retained in the minds of experts and is often essential to developing and operating complex engineering systems [see ‘Tacit hurdle to digital twins’ on August 26th, 2020].  The concept of tapping into someone’s mind to extract tacit knowledge brings us close to thinking about human digital twins which so far have been restricted to computational models of various parts of human anatomy and physiology.  The idea of a digital twin of someone’s mind raises a myriad of philosophical and ethical issues.  Whilst the purpose of a digital twin of the mind of an operator of a complex system might be to better predict and understand human-machine interactions, the opportunity to use the digital twin to advance techniques of personalisation will likely be too tempting to ignore.  Personalisation is the tailoring of the digital world to respond to our personal needs, for instance using predictive algorithms to recommend what book you should read next or to suggest purchases to you.  At the moment, personalisation is driven by data derived from the tracks you make in the digital world as you surf the internet, watch videos and make purchases.  However, in the future, those predictive algorithms could be based on reading your mind, or at least its digital twin.  We worry about loss of privacy at the moment, by which we probably mean the collation of vast amounts of data about our lives by unaccountable organisations, and it worries us because of the potential for manipulation of our lives without us being aware it is happening.  Our free will is endangered by such manipulation but it might be lost entirely to a digital twin of our mind.  To quote the philosopher Michael Lynch, you would be handing over ‘privileged access to your mental states’ and to some extent you would no longer be a unique being.  We are long way from possessing the technology to realise a digital twin of human mind but the possibility is on the horizon.

Source: Richard Waters, They’re watching you, FT Weekend, 24/25 October 2020.

Image: Extract from abstract by Zahrah Resh.

35 years later and still working on a PhD thesis

It is about 35 years since I graduated with my PhD.  It was not ground-breaking although, together with my supervisor, I did publish about half a dozen technical papers based on it and some of those papers are still being cited, including one this month which surprises me.  I performed experiments and computer modelling on the load and stress distribution in threaded fasteners, or nuts and bolts.  There were no digital cameras and no computer tomography; so, the experiments involved making and sectioning models of nuts and bolts in transparent plastic using three-dimensional photoelasticity [see ‘Art and Experimental Mechanics‘ on July 17th, 2012].  I took hundreds of photographs of the sections and scanned the negatives in a microdensitometer.  The computer modelling was equally slow and laborious because there were no graphical user interfaces (GUI); instead, I had to type strings of numbers into a terminal, wait overnight while the calculations were performed, and then study reams of numbers printed out on long rolls of paper.  The tedium of the experimental work inspired me to work on utilising digital technology to revolutionise the field of experimental mechanics over the following 15 to 20 years.  In the past 15 to 20 years, I have moved back towards computer modelling and focused on transforming the way in which measurement data are used to improve the fidelity of computer models and to establish confidence in their predictions [see ‘Establishing fidelity and credibility in tests and simulations‘ on July 25th, 2018].  Since completing my PhD, I have supervised 32 students to successful completion of their PhDs.  You might think that was a straightforward process of an initial three years for the first one to complete their research and write their thesis, followed by one graduating every year.  But that is not how it worked out, instead I have had fallow years as well as productive years.  At the moment, I am in a productive period, having graduated two PhD students per year since 2017 – that’s a lot of reading and I have spent much of the last two weekends reviewing a thesis which is why PhD theses are the topic of this post!

Footnote: the most cited paper from my thesis is ‘Kenny B, Patterson EA. Load and stress distribution in screw threads. Experimental Mechanics. 1985 Sep 1;25(3):208-13‘ and this month it was cited by ‘Zhang D, Wang G, Huang F, Zhang K. Load-transferring mechanism and calculation theory along engaged threads of high-strength bolts under axial tension. Journal of Constructional Steel Research. 2020 Sep 1;172:106153‘.

Tacit hurdle to digital twins

Tacit knowledge is traditionally defined as knowledge that is not explicit or that is difficult to express or transfer from someone else.  This description of what it is not makes the definition itself tacit knowledge which is not very helpful.  Management guides resolve this by giving examples, such as aesthetic sense, or innovation and leadership skills which are elusive skills that are hard to explain [see ‘Innovation out of chaos‘ on June 29th 2016 and  ‘Clueless on leadership style‘ on June 14th, 2017].  In engineering, there are a series of skills that are hard to explain or teach, including creative problem-solving [see ‘Learning problem-solving skills‘  on October 24th, 2018], artful design [see ‘Skilled in ingenuity‘ on August 19th, 2015] and elegant modelling [see ‘Credibility is in the eye of the beholder‘ on April 20th, 2016].  In a university course we attempt to lay the foundations for this tacit engineering knowledge; however, much of it is gained in work through experience and becomes regarded by organisations as part of their intellectual assets – the core of their competitiveness and source of their sustainable technology advantage.  In our work on integrated nuclear digital environments, from which digital twins can be spawned, we would like to capture both explicit and tacit knowledge about complex systems throughout their life cycle which will extend beyond the working lives of their designers, builders and operators.  One of the potential advantages of digital twins is as a knowledge management system by duplicating the life of the physical system and thus allowing its safer and cheaper operation in the long-term as well as its eventual decommissioning.   However, besides the very nature of tacit knowledge that makes its capture difficult, we are finding that its perceived value as an intellectual asset renders stakeholders reluctant to discuss it with us; never mind consider how it might be preserved as part of a digital twin.  Research has shown that tacit knowledge sharing is influenced by environmental factors including national culture, leadership characteristics and social networks [Cai et al, 2020].  I suspect that all of these factors were present in the heyday of the UK civil nuclear power industry when it worked together to construct advanced and complex systems; however, it has not built a power station since 1995 and, at the moment, new power stations are cancelled more often than built, which has almost certainly depressed all of these factors.  So, perhaps we should not be surprised by the difficulties encountered in establishing an integrated nuclear digital environment despite its importance for the future of the industry.

Reference: Cai, Y., Song, Y., Xiao, X. and Shi, W., 2020. The Effect of Social Capital on Tacit Knowledge-Sharing Intention: The Mediating Role of Employee Vigor. SAGE Open, 10(3), p.2158244020945722.