Tag Archives: Goodhart’s Law

Tsundoku

I used to suffer from tsundoku but now I am almost cured…  Tsundoku is a Japanese word meaning ‘the constant act of buying books but never reading them’.  I still find it hard to walk into a good bookshop and leave without buying a small pile of books.  I did it early this month in the Camden Lock Books and left with ‘The New Leaders‘ by Daniel Goleman, ‘What we talk about when we talk about love‘ by Raymond Carver and ‘The Fires of Autumn‘ by Irène Némirowsky.  I will probably read all of these three books over the coming months so it was not really an act of tsundoku.  But, it’s perhaps only because there are so few really good bookshops left that I don’t  buy more in a year than I can read.  Although this is not quite true in my professional life, because I have started buying books on-line and the pile of unread books in my office is growing; so I am not completely cured of tsundoku.  Actually, all researchers are probably suffering from it because we collect piles of research papers that we never read – in part because we can’t keep up with the 2.5 million papers published every year.  And, it’s growing by about 5% per annum, according to Sarah Boon; perhaps, because there are more than 28,000 scholarly journals publishing peer-reviewed research.  Of course, that’s what happens if you measure research productivity in terms of papers published – it’s a form of Goodhart’s law [see my post entitled ‘Goodhart’s Law‘ on August 6th, 2014].

Goodhart’s law

blueskyWe used to talk about R&D, i.e. research and development. In broad terms, most research happened in universities and national labs while most development was undertaken by companies. Nowadays we are being pressed to research and innovative. Nearly, every application for research funding from government agencies must include a section on the likely impact of the proposed research. This emphasis on impact is a global trend that was identified by Dr Helen Neville, Vice-President at Procter & Gamble for Global Open Innovation, in a recent talk I heard her give on trends in international research collaboration. The focus of university research used to be blue-sky, i.e. research with no pre-conceived application. We are exploiting the blue-sky research of twenty or thirty years ago now. And by only funding research with identifiable impacts our successors are likely to be short on breakthroughs to exploit in the middle of the century. It is analogous to a forester harvesting trees planted by his parents and not planting any for his children.

Attempting to evaluate the potential impact of a piece of research whose outcome, by definition, is not yet known is problematic and a matter of judgement rather than measurement.  Even for a piece of university research performed twenty years ago it is not possible to make a precise measurement of its impact. There are no international standards against which to make the measurement, as there is for the metre or the kilogram.  Consequently, the impact of research is probably one of those cultural measures that are subject to Goodhart’s law.  In 1975, Charles Goodhart postulated that once a measure is chosen for making policy decisions it begins to lose its value as a measure.  This is because people adjust their behaviour to optimise the value of the measure, e.g. university researchers tend towards research with short-term impact rather than focussing on discovery followed by dissemination and, or development.

Source: Measuring culture.  Robert P Crease in Physics World, April 2013.