Metrics in science – Performance Measurement and the world of scientific research
In a recent article published in the weekly international scientific journal ‘Nature‘, under the section ‘Science metrics’ the use of performance measurement in science is thoroughly investigated. According with the article, suggestive entitled ‘A profusion of metrics‘, performance measurement in science has captured an increased role since it was first started to be used in this particular area of human activity and interest (Richard van Noorden, 2010).
As van Noorden (2010) acknowledges, it was American psychologist James McKeen Cattle who first popularized the idea of ranking scientists according with their performance, 100 years ago. In one of his works published in 1910 ‘American Men of science: A Biographical Directory‘, the scholar acknowledged the importance of measuring the science researchers performance for promoting and encouraging the development of the field in a systematic and scientific way.
The first performance measurement attempts were based on a simple survey in which the experts were asked to rank the best performers in a specific scientific field by merits. In comparison, today a different situation revolves.
It seems that in the last few decades, the field of science witnessed an increased attraction, in terms of the use of performance measurement. A new set of objective metrics and indicators was developed that help quantify and capture almost all the aspects in the scientific field from a diversity of perspective: quality, impact or prestige. Even more, the technological advancements and the development of the online databases such as Web of Science from Thomson Reuters, Scopus from Elsevier or Google Scholar further fueled the development of ever more sophisticated measures (van Noorden, 2010). Some of the most important and used measures are shortly presented below:
• # Citations: Measures the number of times a researcher or research paper is cited by other authors
o # Citations in top journals
o # Citations per publication
o # Citations by scientific field
• # H-index: Measures the number of publications authored that are cited by at least a specific number of times
• # Impact factor: Measures the average frequency with which an article published in a journal gets cited
• # Research paper online accesses: Measures the total number of times a research paper is accessed online
Van Noorden (2010) doesn’t stop at simply describing the metrics that have impacted the most the measurement of the scientific field, but he further argues that there is a need for a thoroughly investigation on how this performance measures can be best used for monitoring different aspects of the scientific field and scientists activity.
While some of the measures, like the ‘Impact factor’ are best fitted for measuring the popularity of a journal, other measures like the ‘H-index’ best captures the individual performance of the scientists. Same happens in all scientific fields. One has to acknowledge that not all metrics best fit to capture the performance in all scientific fields and that a thoroughly selection and analysis must be made before proceeding with measuring certain aspects. More than that, the amalgam and mixture of old and new metrics, of simple or more sophisticated ones, need a thoughtful selection of the ones that can best capture and reflect the efficiency and effectiveness of scientists or scientific field.
Most of the times, simple measures can better reflect specific performance aspects than more sophisticated and complicated measures. Also when having a large number of metrics at your disposal it can become easy to monitor same thing with more measures. As van Noorden (2010) acknowledges, in the scientific field today there are ‘many metrics that correlate strongly with one another, capturing much of the same information about the data they describe’. Accordingly, more and more researcher voices are asking for reflection and consolidation in the field in regards with performance measurement aspects.
To explore further examples of metrics that capture research & development aspects, innovation or knowledge management visit Key Performance Indicators for Knowledge and Innovation Functional Area, available at smartKPIs.com.
References:
- van Noorden, Richard 2010, A profusion of measures, Nature, vol 465, 17 June 2010
- smartkpis.com 2010, Key Performance Indicators in Knowledge and Innovation Functional Area
Image by David Parkins, @ 2010 Nature Publishing Group
Tags: KPI, KPI in Practice, Performance Measurement, Research
