Individual performance management and the use of metrics in the world of scientific research
In a recent blog post, ‘Metrics in science – Performance Measurement and the world of scientific research’, we have outlined the most important performance measures for the scientific field as presented by the researcher Richard van Noorden (2010) in the weekly international scientific journal ‘Nature’.
The post concluded that the provision of metrics that invaded the field of science in the last decades needs closer review before being used, as oftentimes simple measures can better reflect scientific performance aspects than more sophisticated and complicated measures. Acknowledgment for the need of reflection and consolidation in the field of scientific performance measurement was also emphasized.
Continuing this line of inquiry, a new question is explored in more detail below:
How does performance measurement influence the careers of researchers in regards to matters such as hire, promotion or tenure?
The premise that “no scientific career can be summarized by a number” is the starting point of the research study initiated by the Nature magazine. It argues that there are other things that can recommend a good scientist, which need to be taken in consideration when making an individual performance assessment in addition to:
• # Published papers
• # Impact factor of the journals published in
• # Citation frequency of the published papers
• $ Amount of grant money earned, or
• # H-index
However, from the 150 researchers that responded to the research poll, a vast number considered that the metrics of scientific performance represent a major factor in hiring decisions, tenure decisions, promotions or performance reviews / appraisal.
The study also revealed that the most important criteria for scientist evaluation, in the poll respondent’s view, were grants and income, number of publication in high impact journals and citations of published research. These findings are doubled up by a high proportion of respondents that were unsatisfied by the way some of the measures are used in performance assessments, considering that too much credit is given to objective measurement and less to qualitative reviews, such as letters of recommendation from people in the field, or reviews of the work by peers outside the department or institution.
Surprisingly however, when asked to rate the most important criteria that should be used to evaluate researchers, the poll revealed that number of publication in high impact journals, grants earned, training and mentoring students and the number of citations on published research stood out in the respondents choice.
On the other side of the barricades, the administrators and evaluators, when interviewed in regards with the same matters insisted that metrics don’t matter nearly as much for hiring, promotion and tenure as the poll respondents seemed to think.
Despite these mixed opinions, the bottom line, as revealed by the Nature study remains that:
• 51% of respondents said that they have changed their behavior because of the way they are evaluated;
• 71% of respondents said that they are concerned their colleagues can ‘game’ or ‘cheat’ the systems for evaluation in their institutions.
Summing all these findings, one conclusion that can be reached at is that the problem doesn’t necessarily sit with the use of metrics in researcher’s performance evaluation process but more with the way these measures are used. Thus the challenge, as it is acknowledged also by the study authors is not to reduce the reliance on metrics, but to give more clarity, consistency and transparency to the performance appraisal process.
Integrating the individual performance evaluation in a more comprehensive context of a performance management system and aligning the objectives from the individual level with those from the operational and strategic level can be one of the solutions to strengthen and give more clarity, consistency and direction to the individual performance appraisal process viewed from an integrated, organizational performance perspective.
Question such as: Do metrics matter? What can be done to streamline the performance monitoring and appraisal process at the individual level? continue to generate interest in both researcher and practitioner communities.
References:
Image Source:- Image by David Parkins, @ 2010 Macmillan Publishers Limited
- Nature, Do metrics matter?, @ 2010 Macmillan Publishers Limited
Additional resources:
- smartkpis.com 2010, Metrics in science – Performance Measurement and the world of scientific research, KPIs Blog
- smartkpis.com 2010, Key Performance Indicators in Knowledge and Innovation Functional Area
Tags: Performance Measurement, Richard van Noorden, Science, Scientific Research
