What are Citation Metrics?
The extent to which a research paper or academic text is cited in other pieces of academic work such as papers, thesis, dissertations, or other texts has now become the ‘gold standard’ for measuring the outcome and impact of a piece of research. These citation metrics or bibliometrics, are now referenced as conclusive evidence of academic quality by research journals, and governments, institutions, corporations, and funding agencies look to them as evidence of top academic performance.
More Is Better
Publishing companies are increasingly leveraging the availability of this data to propose even more allegedly conclusive proof of the worldwide influence of a research paper. Thomson Reuters’ InCites software, for example, claims to offer measurements of: “productivity, influence, efficiency, relative impact, and specialization, and can be applied to an individual, group, institution, subject area or geographic region.”
However, while the acknowledgment of the work of your peers in references or bibliographies may show mutual respect, can those citations really be counted on for the derivation of evaluative data? There is no doubt that information on each article or paper can be categorized, and the number of outputs by author, journal and topic can be collated, but making the leap to assessments of quality as indicated by volume of citation is less conclusive.
The Need to be Comprehensive
Citation metrics are no different than any other attempts at basic quantification and analysis. The results are only going to be as good as the initial information provided. Listing the basic “who, what, why, where, when, and how,” will allow appropriate classification, but making extrapolations based solely on the number of journals and the number of citations per journal makes the enormous assumption that the data on every journal is being captured. This is typically not the case. To then infer measures of productivity and efficiency from that number of citations makes another spurious assumption that all research papers are equal, which is also not the case.
Fulfilling a Need for Justification
In the competitive world of academic research, the decision to fund Project A over Project B has to be justified, irrespective of whether that funding is coming from the government, a corporation, an institution, or a non-profit agency. Referencing the citation metrics for the research team and/or the institution at which the research will be conducted can help to justify the choice being made. However, extending the reach of those metrics beyond those parameters would be highly questionable.
By only capturing the number of times an article or paper is referenced, bibliometrics are unable to offer any equally important information that might alter the perceived value of that research team dramatically.
For example, if the submitted paper was required to go through multiple revisions at the peer review stage, wouldn’t that reduce its’ perceived value when measured against an article that was accepted first time? Should a paper with multiple authors who collaborated equally carry a better rating than one where several of the authors were included based on rank within the department rather than on contribution? At best, citation metrics should be used for trend analysis or anomaly detection within the data points collected, but going beyond that projects value that is undeserved.
