Tag Archives: technology

Reproducibility in science and technology

Leave a reply

Schematic diagram from cited paper in Open Research EuropeIt has been suggested that there is crisis in science concerning the reproducibility of data [1].  New research findings are usually published based on data collected only by the group reporting the new findings, which raises the probability of bias in the results as well as reducing their likely validity.  It also creates a temptation to tamper with or falsify data given the incentives to publish.  It is unlikely that any prestigious journal would publish work that simply demonstrates that previously published findings can be reproduced consistently.  Yet, when they have tried to reproduce published data from experiments, many researchers have been unable to do so [2], which perhaps perversely makes the attempt to reproduce results publishable.  However, if no one has attempted to reproduce a published dataset then it stands until demonstrated to not be reproducible, which implies that much of the data in the published literature could be irreproducible and hence of dubious value.  This is a bigger problem than it might seem, because most scientific and technological innovation is built on the findings of fundamental research.  Hence, we are building on shaky foundations if results are not reproducible. Similarly, the transition from prototypes to reliable products is dependent on achieving reproducibility in the real-world of results obtained with a prototype in the laboratory.  I have been discussing these issues with a close collaborator for a number of years and last week we published a letter, in Open Research Europe, summarizing our views.  In ‘Achieving reproducibility in the innovation process’ [3], we propose that a different approach to reproducibility is required for each phase of the innovation process, i.e., discovery, translation and application, because reproducibility has different implications in each phase.  The diagram, reproduced from the paper (CC-BY-4.0), shows our ideas schematically but follow the link to read and comment on them.

References

[1] Baker, M. (2016). Reproducibility crisis. Nature, 533(26), 353-66.

[2] Camerer, C. F., Dreber, A., Holzmeister, F., Ho, T. H., Huber, J., Johannesson, M., … & Wu, H. (2018). Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nature Human Behaviour, 2(9), 637-644.

[3] Whelan M & Patterson EA, (2025). Achieving reproducibility in the innovation process, Open Research Europe, 5:25. https://doi.org/10.12688/openreseurope.19408.1

Professor soars through the landscape

3 Replies

When I was younger, I often had dreams when I was asleep in which I raised my arms and flew effortlessly across the landscape.  I had the opportunity to have a similar experience while awake when I was in Taiwan earlier this year.  I am fairly frequent visitor to Taiwan [see ‘‘Crash’ in Taipei: an engineer’s travelogue‘ on November 19th, 2014 and ‘Citizens of the world‘ on November 27th, 2019].  I often go with colleagues from the UK who have not been before and almost without fail we visit the amazing National Palace Museum.  On my last visit in January [see: ‘Ancient standards‘ on January 29th, 2020] there was an exciting blend of art and technology in an exhibit that allowed the visitor to fly through the landscape of a painting.  I stood in front of a projection of the picture on a large screen and lifted my arms for a moment to allow the computer system to register my position before starting to fly into the picture, tilting left or right to turn, and lowering and raising my arms to slow down or speed up.  Although there was no mask or headphones to wear, the experience was absorbing and realistic.  You can watch me flying with my ‘jetpack’ in this video.