Category Archives: Soapbox

Ancient models and stochastic parrots

In 2021 Emily Bender and her colleagues published a paper suggesting that the Large Language Models (LLMs) underpinning many Artificial Intelligence applications (AI apps) were little more than stochastic parrots. They described LLMs as ‘a system for haphazardly stitching together sequences of linguistic forms it has observed in its vast training data, according to probabilistic information about how they combine, but without any reference to meaning’. This has fuelled the ongoing debate about the real capabilities of AI apps versus the hype from the companies trying persuade us to use them. Most AI apps are based on statistical analysis of data as stated by Bender et al; however, there is a trend toward physics-based machine learning in which known laws of physics are combined with machine-learning algorithms trained on data sets [see for example the recent review by Meng et al, 2025]. We have been fitting data to models for a very long time. In the fifth century BC, the Babylonians made perhaps one of the greatest breakthroughs in the history of science, when they realized that mathematical models of astronomical motion could be used to extrapolate data and make predictions. They had been recording astronomical observations since 3400 BC and the data was all collated in cuneiform in the library at Nineveh belonging to King Ashurbanipal who ruled from 669-631 BC. While our modern-day digital storage capacity in data centres might far exceed the clay tablets with cuneiform symbols found in Ashurbanipal’s library, it seems unlikely that they will survive five thousand years as part of the data from the Babylonians’ astronomical observations has done and still be readable.

References:

Bender, E.M., Gebru, T., McMillan-Major, A. and Shmitchell, S., 2021, March. On the dangers of stochastic parrots: Can language models be too big?🦜. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency (pp. 610-623).

Meng C, Griesemer S, Cao D, Seo S, Liu Y. 2025. When physics meets machine learning: A survey of physics-informed machine learning. Machine Learning for Computational Science and Engineering. 1(1):20.

Wisnom, Selena, The library of ancient wisdom. Penguin Books, 2025.

Image: Parrot in the park – free stock photo by Pixabay on Stockvault.net

Staying connected to reality via literature

Leave a reply

For most of this year, I have not been a frequent visitor to bookshops so I am not suffering from tsundoku [see ‘Tsundoku’ on May 24^th, 2017]. Instead, I have been unable to resist borrowing books from people when visiting them for weekends [see ‘Fictional planetary emergencies’ on June 4^th, 2025]. This has allowed me to enjoy Open Water by Caleb Azumah Nelson, Oranges Are Not the Only Fruit by Jeanette Winterson, Fen by Daisy Johnson, and Eight Months on Ghazzah Street by Hilary Mantel. The last one describes the experiences of the narrator living in a Middle Eastern country while her husband works as civil engineer on a lucrative employment contract. It is a thriller but the cultural differences between life in a Middle Eastern kingdom and the West for a professional woman are shocking and perhaps should be a ‘must-read’ for anyone tempted by lucrative job offers in the Middle East. A month or so later, I borrowed from the same bookshelf Hope and Other Dangerous Pursuits by Laila Lalami and The Optician of Lampedusa by Emma Jane Kirby. ‘Hope’ describes a boat journey across the Straits of Gibraltar from Morocco to Spain by migrants and the back stories of the migrants that induced them to take the extraordinary risks of paying a people trafficker for the crossing in an overcrowded small boat. The ‘Optician’ is a first person account of someone who, when cruising in their boat with a group of friends, rescued dozens of migrants from the Mediterranean Sea after their boat sank. However, the rescue was too late for hundreds of men, women and children. The book deals with the grief of the rescuers and their shock at the response of the Italian authorities. In a world in which many people are becoming increasingly tribal and insular, within their own bubble [see ‘You’re all weird!’ on February 8^th, 2017], it is crucial that WEIRD (Western, Educated, Industrialised, Rich, Democratic) people stay connected with the realities created by our addiction to fossil fuels and the deep inequalities of wealth – literature can help us connect, especially literature based on real-life experience.

References:

Caleb Azumah Nelson, Open Water, Penguin Books, 2022.

Daisy Johnson, Fen, Penguin Books, 2017.

Emma Jane Kirby, The Optician of Lampedusa, 2017.

Hilary Mantel, Eight Months on Ghazzah Street, Harper Collins Publishers, 2004.

Jeanette Winterson, Oranges Are Not the Only Fruit, Penguin Books, 2025.

Laila Lalami, Hope and Other Dangerous Pursuits, Algonquin Books, New York.

Image: Painting by Sarah Evans owned by the author.

Fictional Planetary Emergencies

1 Reply

A little while ago, when looking for something to read when visiting someone’s house, I picked up ‘The Complete Short Stories: volume 2’ of JG Ballard and started reading from the last story in the collection, ‘Report from an Obscure Planet’. I was surprised to discover its similarity to a fictional piece I posted on this blog last year, see ‘Where has the blue planet gone?’ on July 3^rd 2024. Then I was shocked to realize that some readers of my blog might have thought I had plagiarised Ballard’s short story, whereas I was completely unaware of it when I wrote the post. In Ballard’s story, a rescue mission has just landed on a remote planet from which frantic emergency signals had been received; however, their aerial reconnaissance of hundreds of cities spread across the planet found no inhabitants. They accidentally activate the planet’s extensive, and apparently undamaged, computer networks when broadcasting a signal of greeting and friendship. The networks react with ‘a sudden show of alarm, as if well used to mistrusting these declarations of good intent’. The visitors’ research reveals that war was the most popular sport of the inhabitants, with nations maintaining huge arsenals. They conclude that the computer networks sent out the emergency signals in an attempt to save themselves from a danger that was about to overwhelm their planet. In my version, the rescue mission finds a planet transformed by a climate change and mass extinct induced by an asteroid strike or the activities of the inhabitants. Ballard wrote his story in 1992, so more than thirty years before me, and perhaps twenty years after the first data centre had been built by IBM. The first convincing evidence of the warming effect of carbon dioxide was found in the 1960s and scientists started ringing the alarm bells in the late 1980s and early 1990s, for instance at the 1988 Toronto Conference on the Changing Atmosphere – so perhaps too early to feature in Ballard’s story. Of course, I could also have written about artificial intelligence being the only sign of life found on the planet but that really would have looked like wholesale plagiarism!

Reference:

JG Ballard, ‘The Complete Short Stories: volume 2’, Harper Perennial, London, 2006.

Subconscious awareness of the erosion of individuality

Realize Engineering

An engineering commentary for everyone on the first Wednesday of every month