Tag Archives: creativity

Beyond language with stochastic parrots

Leave a reply

Decorative image of a summer flowerSome months ago, I wrote in unflattering terms about artificial intelligence applications (AI apps) and large language models (LLMs), (see ‘Ancient models and stochastic parrots‘ on October 1st, 2025).  My view is changing, probably as AI apps develop and my user skills improve.  I have started using a couple of different free AI apps as research assistants in three ways.  First, when I am writing administrative documents, such as a job description for a Coordinator of AI in Education, for which a job title was sufficient for the app to generate a first draft that only required light editing and tailoring to the specific context.  Second, using a different AI app, to answer questions about phenomena which have allowed me to construct explanations for observations made of new and, or, complex systems – I could have delved into textbooks and monographs or searched research articles but AI does this much more quickly.  The third way I have used AI apps is to identify gaps in knowledge that could be fruitful topics for research.  This is a more difficult task because AI apps only know about stuff they can find on the internet in the form of language or text.  Hence, I have to ask questions with answers that reveal something unknown or not understood.  This is not straightforward because LLMs are fundamentally constrained by language.  In ‘The Years’, Annie Ernaux wrote that ‘language will continue to put the world into words’.  Yann LeCun, Meta’s former chief scientist, has suggested that to understand how the world works, a model would need to learn from videos and spatial data, not just language, and that without this type of learning human-level intelligence is impossible.  He has set up a new company, Advanced Machine Intelligence Labs, to do just that.  Language is used by people to describe the world from their perspective which might be inaccurate, incomplete or distorted and that can mislead LLMs.  However, using AI apps we can also ‘distort’ videos of the world, so that machine intelligence will have to be based on direct observation of the real-world, which after all is the approach that science attempts to use.

Source:

Yann LeCun, Intelligence is really about learning. FT Weekend, 3-4 January 2026

Annie Ernaux, The Years, Fitzcarraldo Editions, London, 2018.

Webs of expertise and knowledge

1 Reply

I am writing this post while I am in the middle of leading a breakout activity for more than a hundred PhD students from our Centres for Doctoral Training in nuclear science and engineering, GREEN and SATURN.  We have asked them to construct a knowledge network for a start-up company commissioned to build either a fusion energy power station or a power station based on small modular reactors (SMRs).  A knowledge network is a web of expertise and information whose value comes from the connections and interactions within and outside an organization.  Our aim is to encourage students to think beyond science and engineering and consider the interactions required to deliver safe, economic nuclear power.

We have brought the students together in York from six universities located in the North of England (Lancaster, Leeds, Liverpool, Manchester & Sheffield) and Scotland (Strathclyde).  This is an annual event usually held in the first working week of the New Year (see ‘Nuclear Winter School’ on January 23rd 2019).

The breakout activity has three one-hour time-slots on three consecutive days.  In today’s time-slot, we have divided the students into twenty groups of seven and given them paper, pencils, and a circle stencil plus an eraser with which to draw knowledge networks.  We are hoping for creativity, lively discussions, and some fun.  In yesterday’s one-hour slot, they had briefings from the Chief Manufacturing Engineer for a company building SMRs and the Deputy Chief Engineer of a company developing a fusion power station, as well as from a Digital Knowledge Management Consultant whose PhD led to a paper on knowledge networks, which we shared with the students last month (see ‘Evolutionary model of knowledge management’ on March 6th, 2024).  Tomorrow, one person from each group will have two minutes to present their knowledge network, via a portable visualiser, to an audience of several hundred.  What can go wrong?  Twenty two-minute presentations in one hour with one minute for questions and changeover.

GREEN (Growing skills for Reliable Economic Energy from Nuclear) is co-funded by a consortia of industrial organisations and the UK EPSRC (grant no. EP/S022295/1).

SATURN (Skills And Training Underpinning a Renaissance in Nuclear) succeeded GREEN and is also co-funded by a consortia of industrial organisations and the UK EPSRC (grant no. EP/Y034856/1).

Image shows thumbnail of figure from shared paper with knowledge networks for an engineering consultancy company and an electricity generator, follow this link for full size image.

More than human

Leave a reply

Decorative imageIn his recent book, ‘The Place of Tides’, James Rebanks writes ‘the age of humans will pass.  Perhaps the end has already begun though it may take a long time to play out’.  I grew up when nuclear armageddon appeared to be the major threat to the future of life on Earth and it remains a major threat, especially given current tensions between nations.  However, other threats have gained prominence including both a massive asteroid impact, on the scale of the one that caused the extinction of the dinosaurs 66 million years ago, and climate change, which caused the largest mass extinction, killing 95% of all species, about 252 million years ago.  The current extinction rate is between 100 and 1000 times greater than the natural rate and is being driven by the overexploitation of the Earth’s resources by humans leading to habitat destruction and climate change.  Humans are part of a complex ecosystem, or system of systems, including soil systems with interactions between microorganisms, plants and decaying matter; pollination systems characterised by co-dependence between plants and pollinators; and, aquatic systems connecting rivers, lakes and oceans by the movement of water, nutrients and migratory species.  The overexploitation of these systems to support our 21st century lifestyle is starting to cause systemic failures that are the underlying cause of the increasing rate of species extinction and it is difficult, if not impossible, to predict when it will be our turn.  In his 1936 book, ‘Where Life is Better: An Unsentimental American Journey’, James Rorty observes that the most dangerous fact he has come across is ‘the overwhelming fact of our lazy, irresponsible, adolescent inability to face the truth or tell it’.  Not much has changed in nearly one hundred years, except that the global population has increased fourfold from about 2.2 billion to 8.2 billion with a corresponding increase in the exploitation of the Earth for energy, food and satisfying our materialistic desires.  A recent exhibition at the Design Museum in London, encouraged us to think beyond human-centred design and to consider the impact of our designs on all the species on the planet.  A process sometimes known as life-centred design or interspecies design.  What if designs could help other species to flourish, as well as humans?

References:

Rebanks, James, The place of tides, London: Penguin, 2025.

Rorty, James, Where life is better: an unsentimental journey.  New York, Reynal & Hitchcock, 1936.  (I have not read this book but it was quoted by Joanna Pocock in ‘Greyhound’, Glasgow: Fitzcarraldo Editions, HarperCollins Publishers, 2025, which I have read and enjoyed).

Image: Photograph of Pei yono uhutipo (Spirit of the path) by Sheraonawe Hakihiiwe, a member of the Yanomami Indigenous community who live in the Venezuelan and Brazilian Amazon. One of a series of his paintings in the ‘More than Human‘ exhibition at the Design Museum which form part of an archive of Yanomami knowledge that reflects the abundance of life in the forest.

Ancient models and stochastic parrots

1 Reply

Decorative image of a parrot in the parkIn 2021 Emily Bender and her colleagues published a paper suggesting that the Large Language Models (LLMs) underpinning many Artificial Intelligence applications (AI apps) were little more than stochastic parrots.  They described LLMs as ‘a system for haphazardly stitching together sequences of linguistic forms it has observed in its vast training data, according to probabilistic information about how they combine, but without any reference to meaning’.  This has fuelled the ongoing debate about the real capabilities of AI apps versus the hype from the companies trying persuade us to use them.  Most AI apps are based on statistical analysis of data as stated by Bender et al; however, there is a trend toward physics-based machine learning in which known laws of physics are combined with machine-learning algorithms trained on data sets [see for example the recent review by Meng et al, 2025].  We have been fitting data to models for a very long time.  In the fifth century BC, the Babylonians made perhaps one of the greatest breakthroughs in the history of science, when they realized that mathematical models of astronomical motion could be used to extrapolate data and make predictions.  They had been recording astronomical observations since 3400 BC and the data was all collated in cuneiform in the library at Nineveh belonging to King Ashurbanipal who ruled from 669-631 BC.  While our modern-day digital storage capacity in data centres might far exceed the clay tablets with cuneiform symbols found in Ashurbanipal’s library, it seems unlikely that they will survive five thousand years as part of the data from the Babylonians’ astronomical observations has done and still be readable.

References:

Bender, E.M., Gebru, T., McMillan-Major, A. and Shmitchell, S., 2021, March. On the dangers of stochastic parrots: Can language models be too big?🦜. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency (pp. 610-623).

Meng C, Griesemer S, Cao D, Seo S, Liu Y. 2025. When physics meets machine learning: A survey of physics-informed machine learning. Machine Learning for Computational Science and Engineering. 1(1):20.

Wisnom, Selena, The library of ancient wisdom.  Penguin Books, 2025.

Image: Parrot in the park – free stock photo by Pixabay on Stockvault.net