UHN is a signatory to DORA, the Declaration of Research Assessment, which recognizes the need to improve the way the impact of research and researchers are evaluated. Many of the below metrics have been in use for years, but organizations including UHN are moving away from solely relying on them to assess research. This is especially true for the use of journal-based metrics to evaluate the individual papers in a journal. To see examples of good practices, policies and guidance for assessing or evaluating research impact and researchers, check out the Policies and Guidance in the DORA Resource Library.
UHN's DORA-compliant measures of research quality and impact template includes links to the full report and policy brief developed by UHN's DORA Advisory Group, and can guide reporting the impact of your research.
The generic term metrics or traditional metrics are used in bibliometric reports or bibliometric research to refer to measures like citation count, h-index, journal impact factor, etc. These are measures of how many researchers could have seen and/or used the research in question. Traditional metrics usually require a wait of a couple of years before they are worth calculating, because it takes time for other researchers to do, write up, submit, and publish articles that cite the article in question.
Altmetrics, alt-metrics or alternative metrics are used to describe measures like number of shares on social media, number of mentions in news stories, how many downloads, etc. These are measures of interest, awareness and possible impact in the wider world. Altmetrics are usually very fast, involving a flood of attention when the article in question is first published, and then slowing down as discussion moves on to newer topics.
Both traditional and alternative metrics have their weaknesses as tools for measuring the importance and impact of a research study, a researcher or a research institute.
When a measure becomes a target, it ceases to be a good measure.
Find out more:
Michael Fire, Carlos Guestrin, Over-optimization of academic publishing metrics: observing Goodhart’s Law in action, GigaScience, Volume 8, Issue 6, June 2019.
Will Koehrsen, How to Mind Goodhart’s Law and Avoid Unintended Consequences, Built In, Oct. 19, 2021.
Mattson C, Bushardt RL, Artino AR Jr. "When a Measure Becomes a Target, It Ceases to be a Good Measure". J Grad Med Educ. 2021 Feb;13(1):2-5. doi: 10.4300/JGME-D-20-01492.1. Epub 2021 Feb 13.
