Tag Archives: examinations

Inducing chatbots to write nonsense

Leave a reply

titanium dental implant face profile technical pictureThe chatbot, ChatGPT developed by OpenAI, has been in the news recently and is the subject of much discussion in universities primarily because of its potential use by students to complete their coursework assignments but also the positive uses to which it might be applied. After last week’s invitations to edit two special issues in different journals on cosmetic dentistry and wire arc additive manufacturing (WAAM) [‘Wire arc additive manufacturing and cosmetic dentistry?‘ on February 8th, 2023], I did a little research in the scientific literature to find out if anyone had published research on using WAAM to make parts for cosmetic dentistry but found nothing.  I was not surprised – the level of precision achievable with WAAM is about 1 millimetre which would be insufficient for most applications in cosmetic dentistry.  Then, I signed up for a free trial with ChatGPT and conducted an experiment by asking it to write about wire arc additive manufacturing and cosmetic dentistry.  The chatbot produced 128 words about how WAAM is becoming increasing popular in cosmetic dentistry because of its accuracy and precision also because a wide range of materials can be used allowing a match to the colour and texture of teeth.  I repeated the experiment and the chatbot produced 142 different words, again stating that dental prostheses can be produced using WAAM with high precision and accuracy to match a patient’s existing teeth in colour so that restorations appear natural and undetectable.  In each case the six or seven sentences were well-written and included some facts that were used to construct a set of false statements, which superficially appeared reasonable; however, only a modicum of knowledge would be required to identify the fallacious rationale.  Some of my colleagues are already exploring incorporating the chatbot into students’ coursework by asking students to use it to generate a description of a technical topic and then asking them to critique its output in order to assess their understanding of the topic.  I expect chatbots will improve rapidly but for the moment it is relatively easy to induce them to write nonsense.

Bibliography

Li Y, Su C, Zhu J. Comprehensive review of wire arc additive manufacturing: Hardware system, physical process, monitoring, property characterization, application and future prospects. Results in Engineering,100330, 2021.

Image: www.authoritydental.org CC BY 2.0 downloaded from https://www.flickr.com/photos/dental-photos/50730990757

A sign of normality returning

Leave a reply

I am in the midst of marking examination scripts.  I have about two weeks to award a maximum of about 26,000 marks which is a huge number of decisions to make in a relatively short time [see ‘Depressed by exams‘ on January 31st 2018].  Although the pile of examination scripts is tall and the task can feel overwhelming, it represents a return to normality following the pandemic when we conducted on-line, open-book examinations [see ‘Limited bandwidth’ on June 2nd, 2021].  We have been teaching 100% on-campus for the whole semester and all of our examinations have returned to their pre-pandemic format, i.e., the majority have been in-person, closed-book and invigilated.  I have enjoyed teaching thermodynamics in a huge lecture-theatre filled with students and it is relief that I do not have to set examination questions whose answers cannot be found using a search engine or solved using a programme.  Anyway I need to pick up my red pen and return to my marking so only a brief post this week.

Meta-knowledge: knowledge about knowledge

5 Replies

As engineers, we like to draw simple diagrams of the systems that we are attempting to analyse because most of us are pictorial problem-solvers and recording the key elements of a problem in a sketch helps us to identify the important issues and select an appropriate solution procedure [see ‘Meta-representational competence’ on May 13th, 2015].  Of course, these simple representations can be misleading if we omit parameters or features that dominate the behaviour of the system; so, there is considerable skill in idealising a system so that the analysis is tractable, i.e. can be solved.  Students find it especially difficult to acquire these skills [see ‘Learning problem-solving skills‘ on October 24th, 2018] and many appear to avoid drawing a meaningful sketch even when examinations marks are allocated to it [see ‘Depressed by exams‘ on January 31st, 2018].  Of course, in thermodynamics it is complicated by the entropy of the system being reduced when we omit parameters in order to idealise the system; because with fewer parameters to describe the system there are fewer microstates in which the system can exist and, hence according to Boltzmann, the entropy will be lower [see ‘Entropy on the brain‘ on November 29th, 2017].  Perhaps this is the inverse of realising that we understand less as we know more.  In other words, as our knowledge grows it reveals to us that there is more to know and understand than we can ever hope to comprehend [see ‘Expanding universe‘ on February 7th, 2018]. Is that the second law of thermodynamics at work again, creating more disorder to counter the small amount of order achieved in your brain?

Image: Sketch made during an example class

Creating an evolving learning environment

10 Replies

A couple of weeks ago, I wrote about marking examinations and my tendency to focus on the students that I had failed to teach rather than those who excelled in their knowledge of problem-solving with the laws of thermodynamics [see my post ‘Depressed by exams‘ on January 31st, 2018].  One correspondent suggested that I shouldn’t beat myself up because ‘to teach is to show, to learn is to acquire‘; and that I had not failed to show but that some of my students had failed to acquire.  However, Adams and Felder have stated that the ‘educational role of faculty is not to impart knowledge; but to design learning environments that support knowledge acquisition‘.  My despondency arises from my apparent inability to create a learning environment that supports and encourages knowledge acquisition for all of my students.  People arrive in my class with a variety of formative experiences and different ways of learning, which makes it challenging to generate a learning environment that is effective for everyone.   It’s an on-going challenge due to the ever-widening cultural gap between students and their professors, which is large enough to have warranted at least one anthropological study (see My Freshman Year by Rebekah Nathan). So, my focus on the weaker exam scripts has a positive outcome because it causes me to think about evolving the learning environment.

Sources:

Adams RS, Felder RM, Reframing professional development: A systems approach to preparing engineering educators to educate tomorrow’s engineers. J. Engineering Education, 97(3):230-240, 2008.

Nathan R, My freshman year: what a professor learned by becoming a student, Cornell University Press, Ithaca, New York, 2005