Category Archives: Engineering

Ancient models and stochastic parrots

1 Reply

Decorative image of a parrot in the parkIn 2021 Emily Bender and her colleagues published a paper suggesting that the Large Language Models (LLMs) underpinning many Artificial Intelligence applications (AI apps) were little more than stochastic parrots.  They described LLMs as ‘a system for haphazardly stitching together sequences of linguistic forms it has observed in its vast training data, according to probabilistic information about how they combine, but without any reference to meaning’.  This has fuelled the ongoing debate about the real capabilities of AI apps versus the hype from the companies trying persuade us to use them.  Most AI apps are based on statistical analysis of data as stated by Bender et al; however, there is a trend toward physics-based machine learning in which known laws of physics are combined with machine-learning algorithms trained on data sets [see for example the recent review by Meng et al, 2025].  We have been fitting data to models for a very long time.  In the fifth century BC, the Babylonians made perhaps one of the greatest breakthroughs in the history of science, when they realized that mathematical models of astronomical motion could be used to extrapolate data and make predictions.  They had been recording astronomical observations since 3400 BC and the data was all collated in cuneiform in the library at Nineveh belonging to King Ashurbanipal who ruled from 669-631 BC.  While our modern-day digital storage capacity in data centres might far exceed the clay tablets with cuneiform symbols found in Ashurbanipal’s library, it seems unlikely that they will survive five thousand years as part of the data from the Babylonians’ astronomical observations has done and still be readable.

References:

Bender, E.M., Gebru, T., McMillan-Major, A. and Shmitchell, S., 2021, March. On the dangers of stochastic parrots: Can language models be too big?🦜. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency (pp. 610-623).

Meng C, Griesemer S, Cao D, Seo S, Liu Y. 2025. When physics meets machine learning: A survey of physics-informed machine learning. Machine Learning for Computational Science and Engineering. 1(1):20.

Wisnom, Selena, The library of ancient wisdom.  Penguin Books, 2025.

Image: Parrot in the park – free stock photo by Pixabay on Stockvault.net

Passive nanorheology measurements

Leave a reply

What do marshmallows, jelly (or Jell-O), cream cheese and Chinese soup dumplings have in common?  They are often made with gelatin.  Gelatin is derived from the skin and bones of cattle and pigs through the partial hydrolysis of collagen.  Gelatin is a physical hydrogel meaning that it consists of a three-dimensional network of polymer molecules in which a large amount of water is absorbed, as much as 90% in gelatin.  These polymer molecules are cross-linked by hydrogen bonds, hydrophobic interactions and chain entanglements.  External stimuli, such as temperature, can change the level of cross-linking causing the material to transition between its solid, liquid and gel states.  This is why jelly sets in the fridge and melts when it’s heated up – the cross-links holding the molecules together break down.  This type of responsive behaviour allows the properties of hydrogels to be controlled at the micro and sub-micron scale for a host of applications including tissue engineering, drug delivery, water treatment, wearable technologies, and supercapacitors.  However, the design and manufacture of soft hydrogels can be challenging due to the lack of technology for measuring the local properties.  Current quantitative techniques for measuring the properties of hydrogels usually focus on bulk properties and provide little data about local variations or real-time responses to external stimuli.  My colleagues and I have used gold nanoparticles as probes in hydrogels to map the properties at the microscale of thermosensitive hydrogels undergoing real-time transition from the solid to gel phases [see ‘Passive nanorheological tool to characterise hydrogels’].  This is an extension, or perhaps more accurately an application, of our earlier work on tracking nanoparticles through the vitreous humour of the eye [see ‘Nanoparticle motion-through heterogeneous hydrogels’ on November 6th, 2024].  The novel technique, which yields passive nanorheological measurements, allows us to evaluate local viscosity, identify time-varying heterogeniety and monitor dynamic phase transitions at the micro through to nano scale.  The significant challenges of other techniques, such as weak signals due to high water content and the dynamism of hydrogels, are overcome with a fast, inexpensive and user-friendly technology.  Although, even with these advantages, you are unlikely to use it when you are making jelly or roasting marshmallows over the campfire; however, it is really useful for understanding the transport of drugs through biological hydrogels or designing manufacturing processes for artificial tissue.

Reference

Moira Lorenzo Lopez, Victoria R. Kearns, Eann A. Patterson & Judith M. Curran, Passive nanorheological tool to characterise hydrogels, Nanoscale, 2025,17, 15338-15347.

Image: Figure 5 from the above reference showing a hydrogel transitioning to a gel phase as result of an increase in temperature with 100 nm diameter gold nanoparticles with some particles (yellow arrows) at the interface between phases.  The image was taken in an inverted optical microscope set up for tracking the nanoparticles.

Star sequence minimises distortion

Leave a reply

It is some months since I have written about engineering so this post is focussed on some mechanical engineering.  The advent of pneumatic and electric torque wrenches has made it impossible for the ordinary motorist to change a wheel because it is very difficult to loosen wheel nuts by hand when they have been tightened by a powered wrench which most of us do not have available.  This has probably made motoring safer but also means we are more likely to need assistance when we have a flat tire.  It also means that the correct tightening pattern for nuts and bolts is less widely known.  A star-shaped sequence is optimum, i.e., if you have six bolts numbered sequentially around a circle then you start with #1, move across the diameter to #4, then to #2 followed by #5 across the diameter, then to #3 and across the diameter to #6.  This sequence is optimum for flanges, bolted joints in the frames of buildings and joining machine parts as well as wheel nuts.  We have recently discovered that it works in reverse, in the sense that it is the optimum sequence for releasing parts made by additive manufacturing (AM) from the baseplate of the AM machine (see ‘If you don’t succeed try and try again’ on September 29th, 2021).  Additive manufacturing induces large residual stresses as a consequence of the cycles of heat input to the part during manufacturing and some of these stresses are released when it is removed from the baseplate of the AM machine, which causes distortion of the part.  Together with a number of collaborators, I have been researching the most effective method of building thin flat plates using additive manufacturing (see ‘On flatness and roughness’ on January 19th, 2022).  We have found that building the plate vertically layer-by-layer works well when the plate is supported by buttresses on its edges.  We have used two in-plane buttresses and four out-of-plane buttresses, as shown in the photograph, to achieve parts that have comparable flatness to those made using traditional methods.  It turns out that optimum order for the removal of the buttresses is the same star sequence used for tightening bolts and it substantially reduces distortion of the plate compared to some other sequences.  Perhaps in retrospect, we should not be surprised by this result; however, hindsight is a wonderful thing.

The current research is funded jointly by the National Science Foundation (NSF) in the USA and the Engineering and Physical Sciences Research Council (EPSRC) in the UK and the project was described in ‘Slow start to an exciting new project on thermoacoustic response of AM metals’ on September 9th 2020.

Image: Photograph of a geometrically-reinforced thin plate (230 x 130 x 1.2 mm) built vertically layer-by-layer using the laser powder bed fusion process on a baseplate (shown removed from the AM machine) with the supporting buttresses in place.

Sources:

Patterson EA, Lambros J, Magana-Carranza R, Sutcliffe CJ. Residual stress effects during additive manufacturing of reinforced thin nickel–chromium plates. IJ Advanced Manufacturing Technology;123(5):1845-57, 2022.

Khanbolouki P, Magana-Carranza R, Sutcliffe C, Patterson E, Lambros J. In situ measurements and simulation of residual stresses and deformations in additively manufactured thin plates. IJ Advanced Manufacturing Technology; 132(7):4055-68, 2024.

Reproducibility in science and technology

Leave a reply

Schematic diagram from cited paper in Open Research EuropeIt has been suggested that there is crisis in science concerning the reproducibility of data [1].  New research findings are usually published based on data collected only by the group reporting the new findings, which raises the probability of bias in the results as well as reducing their likely validity.  It also creates a temptation to tamper with or falsify data given the incentives to publish.  It is unlikely that any prestigious journal would publish work that simply demonstrates that previously published findings can be reproduced consistently.  Yet, when they have tried to reproduce published data from experiments, many researchers have been unable to do so [2], which perhaps perversely makes the attempt to reproduce results publishable.  However, if no one has attempted to reproduce a published dataset then it stands until demonstrated to not be reproducible, which implies that much of the data in the published literature could be irreproducible and hence of dubious value.  This is a bigger problem than it might seem, because most scientific and technological innovation is built on the findings of fundamental research.  Hence, we are building on shaky foundations if results are not reproducible. Similarly, the transition from prototypes to reliable products is dependent on achieving reproducibility in the real-world of results obtained with a prototype in the laboratory.  I have been discussing these issues with a close collaborator for a number of years and last week we published a letter, in Open Research Europe, summarizing our views.  In ‘Achieving reproducibility in the innovation process’ [3], we propose that a different approach to reproducibility is required for each phase of the innovation process, i.e., discovery, translation and application, because reproducibility has different implications in each phase.  The diagram, reproduced from the paper (CC-BY-4.0), shows our ideas schematically but follow the link to read and comment on them.

References

[1] Baker, M. (2016). Reproducibility crisis. Nature, 533(26), 353-66.

[2] Camerer, C. F., Dreber, A., Holzmeister, F., Ho, T. H., Huber, J., Johannesson, M., … & Wu, H. (2018). Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nature Human Behaviour, 2(9), 637-644.

[3] Whelan M & Patterson EA, (2025). Achieving reproducibility in the innovation process, Open Research Europe, 5:25. https://doi.org/10.12688/openreseurope.19408.1