Category Archives: Soapbox

Credible predictions for regulatory decision-making

3 Replies

detail from abstract by Zahrah ReshRegulators are charged with ensuring that manufactured products, from aircraft and nuclear power stations to cosmetics and vaccines, are safe.  The general public seeks certainty that these devices and the materials and chemicals they are made from will not harm them or the environment.  Technologists that design and manufacture these products know that absolute certainty is unattainable and near-certainty in unaffordable.  Hence, they attempt to deliver the service or product that society desires while ensuring that the risks are As Low As Reasonably Practical (ALARP).  The role of regulators is to independently assess the risks, make a judgment on their acceptability and thus decide whether the operation of a power station or distribution of a vaccine can go ahead.  These are difficult decisions with huge potential consequences – just think of the more than three hundred people killed in the two crashes of Boeing 737 Max airplanes or the 10,000 or so people affected by birth defects caused by the drug thalidomide.  Evidence presented to support applications for regulatory approval is largely based on physical tests, for example fatigue tests on an aircraft structure or toxicological tests using animals.  In some cases the physical tests might not be entirely representative of the real-life situation which can make it difficult to make decisions using the data, for instance a ground test on an airplane is not the same as a flight test and in many respects the animals used in toxicity testing are physiologically different to humans.  In addition, physical tests are expensive and time-consuming which both drives up the costs of seeking regulatory approval and slows down the translation of new innovative products to the market.  The almost ubiquitous use of computer-based simulations to support the research, development and design of manufactured products inevitably leads to their use in supporting regulatory applications.  This creates challenges for regulators who must judge the trustworthiness of predictions from these simulations.  [see ‘Fake facts & untrustworthy predictions‘ on December 4th, 2019]. It is standard practice for modellers to demonstrate the validity of their models; however, validation does not automatically lead to acceptance of predictions by decision-makers.  Acceptance is more closely related to scientific credibility.  I have been working across a number of disciplines on the scientific credibility of models including in engineering where multi-physics phenomena are important, such as hypersonic flight and fusion energy [see ‘Thought leadership in fusion energy‘ on October 9th, 2019], and in computational biology and toxicology [see ‘Hierarchical modelling in engineering and biology‘ on March 14th, 2018]. Working together with my collaborators in these disciplines, we have developed a common set of factors which underpin scientific credibility that are based on principles drawn from the literature on the philosophy of science and are designed to be both discipline-independent and method-agnostic [Patterson & Whelan, 2019; Patterson et al, 2021]. We hope that our cross-disciplinary approach will break down the subject-silos that have become established as different scientific communities have developed their own frameworks for validating models.  As mentioned above, the process of validation tends to be undertaken by model developers and, in some sense, belongs to them; whereas, credibility is not exclusive to the developer but is a trust that needs to be shared with a decision-maker who seeks to use the predictions to inform their decision [see ‘Credibility is in the eye of the beholder‘ on April 20th, 2016].  Trust requires a common knowledge base and understanding that is usually built through interactions.  We hope the credibility factors will provide a framework for these interactions as well as a structure for building a portfolio of evidence that demonstrates the reliability of a model. 

References:

Patterson EA & Whelan MP, On the validation of variable fidelity multi-physics simulations, J. Sound & Vibration, 448:247-258, 2019.

Patterson EA, Whelan MP & Worth A, The role of validation in establishing the scientific credibility of predictive toxicology approaches intended for regulatory application, Computational Toxicology, 17: 100144, 2021.

Image: Extract from abstract by Zahrah Resh.

Puzzles and mysteries

2 Replies

Detail from abstract by Zahrah ReshPuzzles and mysteries are a pair of words that have taken on a whole new meaning for me since reading John Kay’s and Mervyn King’s book called ‘Radical uncertainty: decision-making for an unknowable future‘ during the summer vacation [see ‘Where is AI on the hype curve?‘ on August 12th, 2020]. They describe puzzles as well-defined problems with knowable solutions; whereas mysteries are ill-defined problems, that have no objectively correct solution and are imbued with vagueness and indeterminacy.  I have written before about engineers being creative problems-solvers [see ‘Learning problem-solving skills‘ on October 24th, 2018] which leads to the question of whether we specialise in solving puzzles or mysteries, or perhaps both types of problems.  The problems that I set for students to solve for homework to refine and evaluate their knowledge of thermodynamics [see ‘Problem-solving in thermodynamics‘ on May 6th, 2015] clearly fall into the puzzle category because they are well-defined and there is a worked solution available.  Although for many students these problems might appear to be mysteries, the intention is that with greater knowledge and understanding the mysteries will be transformed into mere puzzles.  It is also true that many real-world mysteries can be transformed into puzzles by research that advances the collective knowledge and understanding of society.  Part of the purpose of an engineering education is to equip students with the skills to make this transformation from mysteries to puzzles.  At an undergraduate level we use problems that are mysteries only to the students so that success is achievable; however, at the post-graduate level we use problems that are perceived as mysteries to both the student and the professor with the intention that the professor can guide the student towards a solution.  Of course, some mysteries are intractable often because we do not know enough to define the problem sufficiently that we can even start to think about possible solutions.  These are tricky to tackle because it is unreasonable to expect a research student to solve them in limited timeframe and it is risky to offer to solve them in exchange for a research grant because you are likely to damage your reputation and prospects of future funding when you fail.  On the other hand, they are what makes research interesting and exciting.

Image: Extract from abstract by Zahrah Resh.

Digital twins could put at risk what it means to be human

4 Replies

Detail from abstract by Zahrah ReshI have written in the past about my research on the development and use of digital twins.  A digital twin is a functional representation in a virtual world of a real world entity that is continually updated with data from the real world [see ‘Fourth industrial revolution’ on July 4th, 2018 and also a short video at https://www.youtube.com/watch?v=iVS-AuSjpOQ].  I am working with others on developing an integrated digital nuclear environment from which digital twins of individual power stations could be spawned in parallel with the manufacture of their physical counterparts [see ‘Enabling or disruptive technology for nuclear engineering’ on January 1st, 2015 and ‘Digitally-enabled regulatory environment for fusion power-plants’ on March 20th, 2019].  A couple of months ago, I wrote about the difficulty of capturing tacit knowledge in digital twins, which is knowledge that is generally not expressed but is retained in the minds of experts and is often essential to developing and operating complex engineering systems [see ‘Tacit hurdle to digital twins’ on August 26th, 2020].  The concept of tapping into someone’s mind to extract tacit knowledge brings us close to thinking about human digital twins which so far have been restricted to computational models of various parts of human anatomy and physiology.  The idea of a digital twin of someone’s mind raises a myriad of philosophical and ethical issues.  Whilst the purpose of a digital twin of the mind of an operator of a complex system might be to better predict and understand human-machine interactions, the opportunity to use the digital twin to advance techniques of personalisation will likely be too tempting to ignore.  Personalisation is the tailoring of the digital world to respond to our personal needs, for instance using predictive algorithms to recommend what book you should read next or to suggest purchases to you.  At the moment, personalisation is driven by data derived from the tracks you make in the digital world as you surf the internet, watch videos and make purchases.  However, in the future, those predictive algorithms could be based on reading your mind, or at least its digital twin.  We worry about loss of privacy at the moment, by which we probably mean the collation of vast amounts of data about our lives by unaccountable organisations, and it worries us because of the potential for manipulation of our lives without us being aware it is happening.  Our free will is endangered by such manipulation but it might be lost entirely to a digital twin of our mind.  To quote the philosopher Michael Lynch, you would be handing over ‘privileged access to your mental states’ and to some extent you would no longer be a unique being.  We are long way from possessing the technology to realise a digital twin of human mind but the possibility is on the horizon.

Source: Richard Waters, They’re watching you, FT Weekend, 24/25 October 2020.

Image: Extract from abstract by Zahrah Resh.

My Engineering Day

1 Reply

Photograph of roof tops and chimneys in Liverpool.Today is ‘This is Engineering’ day organised by the Royal Academy of Engineering to showcase what engineers and engineering really look like, celebrate our impact on the world and shift public perception of engineering towards an appreciation that engineers are a varied and diverse group of people who are critical to solving societal challenges.  You can find out more at https://www.raeng.org.uk/events/online-events/this-is-engineering-day-2020.  I have decided to contribute to ‘This is Engineering’ day by describing what I do on a typical working day as an engineer. 

Last Wednesday was like many other working days during the pandemic.  I got up about 7am went downstairs for breakfast in our kitchen and then climbed back upstairs to my home-office in the attic of our house in Liverpool [see ‘Virtual ascent of Moel Famau’ on April 8th, 2020].  I am lucky in that my home-office is quite separate from the living space in our house and it has a great view over the rooftops.  I arrived there at about 7.45am, opened my laptop, deleted the junk email, and dealt with the emails that were urgent, interesting or could be replied to quickly.  At around 8am, I closed my email and settled down to write the first draft of a proposal for funding to support our research on digital twins [see ‘Tacit hurdle to digital twins’ on August 26th, 2020].  I had organised a meeting earlier in the week with a group of collaborators and now I had the task of converting the ideas from our discussion into a coherent programme of research.  Ninety caffeine-fuelled minutes later, I had to stop for a Google Meet call with a collaborator at Airbus in Toulouse during which we agreed the wording on a statement about the impact our recent research efforts.  At 10am I joined a Skype call for a progress review with a PhD student on our dual PhD programme with National Tsing Hua University in Taiwan, so we were joined by his supervisor in Taiwan where it was 6pm [see ‘Citizens of the World’ on November 27th, 2019].  The PhD student presented some very interesting results on evaluating the waviness of fibres in carbon-fibre composite materials using ultrasound measurements which he had performed in our laboratory in Liverpool.  Despite the local lockdown in Liverpool due to the pandemic, research laboratories on our campus are open and operating at reduced occupancy to allow social distancing.

After the PhD progress meeting, I had a catch-up session with my personal assistant to discuss my schedule for the next couple of weeks before joining a MS-Teams meeting with a couple of colleagues to discuss the implications of our current work on computational modelling and possible future directions.  The remaining hour up to my lunch break was occupied by a conference call with a university in India with whom we are exploring a potential partnership.  I participated in my capacity as Dean of the School of Engineering and joined about twenty colleagues from both institutions discussing possible areas of collaboration in both research and teaching.  Then it was back downstairs for a half-hour lunch break in the kitchen. 

Following lunch, I continued in my role as Dean with a half-hour meeting with Early Career Academics in the School of Engineering followed by internal interviews for the directorship of one of our postgraduate research programmes.  At 3.30pm, I was able to switch back to being a researcher and meet with a collaborator to discuss the prospects for extending our work on tracking synthetic nanoparticles into monitoring the motion of biological entities such as viruses and bacteria [see ‘Modelling from the cell through the individual to the host population’ on May 5th 2020].  Finally, as usual, I spent the last two to three hours of my working day replying to emails, following up on the day’s meetings and preparing for the following day.  One email was a request for help from one of my PhD students working in the laboratory who needed a piece of equipment that had been stored in my office for safekeeping.  So, I made the ten-minute walk to campus to get it for her which gave me the opportunity to talk face-to-face with one of the post-doctoral researchers in my group who is working on the DIMES project [see ‘Condition-monitoring using infra imaging‘ on June 17th, 2020].  After dinner, my wife and I walked down to the Albert Dock and along the river front to Princes Dock and back up to our house.

So that was my Engineering Day last Wednesday!

 

Logos of Clean Sky 2 and EUThe DIMES project has received funding from the Clean Sky 2 Joint Undertaking under the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 820951.  The opinions expressed in this blog post reflect only the author’s view and the Clean Sky 2 Joint Undertaking is not responsible for any use that may be made of the information it contains.