H-index — Is it Enough to Score Scientific Excellence?

Publishing your work is great for your career and future funding opportunities. It will increase your ranking as a scientist too. But how do you measure your contributions to science? The citation impact factor is criticized as being a “popularity” ranking rather than an indication of the quality of research. For this reason, Jorge Hirsch came up with the H-index, which aims to measure the productivity as well as the impact of research. You have probably noticed it on Scopus or ISI Web of Science. Does the H-index give the research impact measure it intended?

How the H-index Works

The H-index is the number of your publications, that have been each cited by the same number of times. For example, if you have published 16 papers, but the first 12 papers have each been cited at least 12 times, and the remaining 4 articles have only been cited 10 times each, then your H-index is 12. Since authors are required to cite the first publication of an idea, citations are used as a measure of a relevant contribution to the field. The argument is that the H-index is more objective than a citation impact factor because if your publications generally only get cited a few times, but one of your publications receives a huge number of citations, the latter paper will not skew your ranking metrics.

Criticism of the H-index

Just because my H-index is bigger than yours doesn’t necessarily mean I’m doing better research than you. Unfortunately, it is difficult to compare the H-index between researchers because it does not give an indication of:

1. Career stage: If you have not published much over your many years of being a professor, your H-index could be the same as that of a younger professor who has published a few highly cited articles.
2. Which author you are: If your name is on the paper, your H-index will go up, regardless whether you’re first author or 12^th If you’re lucky enough to work on a large collaboration with a well-known researcher, your H-index will go up.
3. Research quality: Although the H-index aims to measure the quality and quantity of your research, it does not say whether your citations are positive or whether other authors are citing you because they disagree with your work.
4. Journal quality: The quality of the journal that published your research is not taken into account.
5. Your contribution to science: It only takes your publications into account, not whether you are an active member of the scientific community. Do you mentor junior scientists, do you perform peer reviews, or take part in scientific discussions and scientific outreach?

Does the H-index Really Score Impact?

Nobel laureate chemist Harry Kroto is not ranked highly in his field according to the H-index, yet he won a Nobel prize. A single publication earned him the award. Clearly, he is a distinguished scientist and should be highly regarded. This demonstrates that the H-index does not always give a very accurate ranking of the impact a published work has had in the field. It merely states how many times a scientist has published and how many times it has been cited. A young scientist that is brilliant will score a low H-index, purely because they do not have time on their side. Of course, there is also the temptation for researchers to artificially boost their H-index through self-citations. In short, the H-index is simply too basic.

H-index Inventor Agrees it is Not Perfect

H-index inventor, Jorge Hirsch says, “about half the scientific community loves the H-index and half hates it”. He further states that although the H-index is a good measure of scientific ranking, it has some unintended negative consequences:

1. A young researcher may not question the concepts taught by someone with a high H-index, since they assume a high score makes them an expert in their field. This could result in unquestioned and perhaps incorrect concepts being passed down through scientific generations. The more a paper is cited, the more difficult it is to question its validity.
2. The H-index is sensitive to science in fashion. Researchers aiming for a high H-index may only pursue or cite popular fields of science.
3. Different disciplines have differences in productivity and therefore cannot be compared using the H-index.

H-index is Still Opinion-Based

It would be great if you could make a decision of who to award grant funding to or which researcher to employ in your research group, based on a number. Often there are so many worthy applicants that it may be difficult to choose.

Although the H-index is a better measure than a citation impact factor, it is still based on the opinions of other authors. Therefore, it should not be used in isolation. Performance in the research environment is more intricate than just publication history. When comparing applicants for funding, promotions or prizes, other factors must be considered. These include age, career stage, and the field of a scientist. In addition, self-citations should be noted. All in all, the H-Index does not solve the “popularity” problem that other scientific metrics are criticized for.

However, all is not lost. I believe that additional scientific metrics that address the shortfalls mentioned above could make the H-index one of a set of valuable measures to determine scientific excellence. Further information can be obtained from metrics such as the m-value. This measures the rate of increase of the H-index over time by dividing the H-index by the number of years since the first publication. A rating that takes exceptional science into account should be included. Just as in the Nobel laureate example above, a single research article that describes groundbreaking work should be considered. At the end of the day, any rating will have its drawbacks and perhaps one should not try to rank scientists by mathematics alone.