Peer review is a critical part of the scholarly publishing process. Apart from their busy schedules, researchers spend countless hours in peer-reviewing manuscripts for free. This valuable contribution by scientists often goes unnoticed and is always under-appreciated. ReviewerCredits aims to facilitate the review process by recognizing and certifying peer review activity and rewarding researchers for their hard work. In addition, ReviewerCredits also recognizes and maintains a unique verified database of conference talks. As part of our interview series on Connecting Scholarly Publishing Experts and Researchers, we had the opportunity to speak with Giacomo Bellani and Robert Fruscio from ReviewerCredits.
(From L-R: Giacomo Bellani and Robert Fruscio)
Could you share with our readers the inspiration behind launching ReviewerCredits? How did this journey begin?
Robert and I have been friends since the early years of medical school. We both undertook an academic career after our graduation. As scientists and researchers, we began to receive several invitations to perform peer reviews. As junior scientists, this is very exciting, however, with increasing number of activities and commitments, time constraints become a major problem. It is often difficult to take time out from writing your own manuscript and spend it on reviewing another researcher’s manuscript. Of course, peer review is a service scientists’ provide to the community, but in academia, we are primarily measured on our scientific productivity (grants, career progression, etc.). For the time being, peer review is not accounted for in any metric. So, after chatting several times about this issue, the idea of developing an interesting metric for peer review emerged!
When was ReviewerCredits launched and what were some of the challenges faced by you while developing the platform?
We conceived our idea back in February 2016. We spent many hours one night envisioning our goals, how the platform should look like, how it should function, etc. That night, we registered our domain reviewercredits.com. In the following months, we began the platform development with a small company called Joins (www.joins.ch), which we knew through other previous experiences.
The first challenge was to explain the process of scholarly publishing and peer review to someone out of the field. It might look strange, but if you try to explain the peer review process to a non-scientist, you will notice that they have a confused reaction. Once the platform became operational, we had to start from scratch with members, journals, and so forth. In the beginning, we perceived an understandable diffidence, especially from editors. Eventually, most of them slowly appreciated that we were not working AGAINST journals, but FOR journals. At the same time, we had to transform what had begun as a personal initiative into a “real” company. The process of becoming a university spin-off took several months of bureaucracy. However, we are now very proud of being a spin-off company endorsed by the University of Milan-Bicocca.
The latest major challenge was overcome last summer/fall when we completely re-engineered the website. Again, this was worth the effort as we now provide a state-of-the-art platform to our users. You can read more about this here.
Could you share with our readers an example of a reviewer profile on the platform?
This is my own profile: https://www.reviewercredits.com/user/giacbell/. On the profile page, the activity of a peer reviewer is highlighted by the Reviewer Index. It also shows connections to social profiles, including ORCiD. Finally, the talks given at recent conferences are highlighted in a separate section.
Moreover, the user can also download a PDF certificate of their peer review activity.
Reviewers on this platform receive credits for each verified peer review. How does this differ from ReviewerIndex? What is the purpose of these credits?
The Reviewer Index reflects the activity of a scientist as a peer reviewer. The concept is very simple and straightforward. For each review performed, our members get 1 point added to their Reviewer Index, which is, therefore, the number of peer reviews performed and confirmed by the journal’s editorial office. The strength of this metric is that it includes only certified peer reviews, and because this certification comes directly from the journals, it is robust and reliable. The credits system is totally different. The aim of creating it was to try giving our members a tangible benefit. Credits are received upon confirmation that the review has been performed. The number of credits depends on the status of the journal in our website (with or without an account and, soon, with a premium account, which will assign more credits). We are working hard to allow the possibility of exchanging these credits for real benefits, like discounts on APCs, subscriptions to journals, and possibly, small research grants. We firmly believe that this system will bring real advantages to reviewers, who will finally see their hard work being rewarded. This will also benefit journals, because scientists will be more inclined towards accepting peer review requests. Once this system is established and running, we also plan to allow journals to rate reviewers, as this will further improve the process and, ultimately scientific progress.
Recently, a new feature of a database on conference talks was launched on the website. Please share with our readers the process of updating such information about various conferences?
We are very proud of this new feature. Most scientists are actively engaged in scientific dissemination. They are selected to give a talk on a specific topic on the basis of their professionalism and their ability to present literature data or innovative research in a straightforward and engaging way. Today, several databases collect journal articles, abstracts, monographs, and book chapters. However, the existence of one unique database, able to answer the question “who-spoke-about-what” was still missing until we created it. This database is populated in two ways. On one side, scientists can record the talks which they gave (or plan to give), and after verification, it is added to their personal profile. Moreover, conference organizers can open a profile and upload the program of any conference edition whose talks are then linked to the individual scientist’s profile.
How can researchers, journals, and conference organizers join ReviewerCredits to create their own profiles?
Joining ReviewerCredits is free and very simple. It is done through our signup page. Initially, it is necessary to select the type of profile (reviewer, journal, or conference) and then fill a short form, with a minimal dataset of mandatory data. Several other data (pictures, bio, and social profiles) can be filled in afterward, from the profile page. Reviewer profiles are immediately active, whereas those for journals and conferences require approval for activation. This process aims to verify that the email provided in the signup form can be univocally linked to the journal/conference, in order to avoid malicious registrations. This process normally does not take more than 24 hours.
Could you share some member journal highlights? For example, their index status, impact factor, etc.?
We now have around 150 journal profiles on our website. These journals originate from different geographical locations and cover a variety of subject areas (from medicine to agriculture and chemistry to geography). About 80% of these journals are indexed in Scopus and WoS and some even have very high impact factors like the American Journal of Respiratory and Critical Care Medicine, to quote one. In addition, publishers can open a profile on our website and arrange all their published journals under one umbrella. Finally, we want to highlight that even without creating a profile more than 3000 journals have decided to collaborate with our platform, by approving peer review claims.
How is ReviewerCredits different from its competitors?
There are three main differences when compared to our competitors.
FIRST: We have a robust system for verification of the information added to our website so that our metrics are fully reliable. We do not accept a “thank you mail” as a proof of peer review for different reasons. This is because it is a simple text-based email that can be easily edited by changing the date, name of the journal, and the manuscript ID. Another reason is that we are very respectful of the privacy of peer reviewers. The “thank you” mail usually contains the name of the authors, and the title of the manuscript reviewed and sharing this data is unacceptable to us. Finally, we are trying to obtain an active involvement of journals in the process of certification of the peer reviewers’ work. Indeed, in more than one instance, we have received this type of response from editors, contacted to verify a claim: “We have no track of this individual in our records.“ These types of responses reinforced our certainty in the validity of our system.
SECOND: We are an academic spinoff, totally independent from publishers. Sometimes we wonder what journals editors, who collaborate with our competitors, feel about giving their data to large companies that are also active in the field of publications. Data is the new gold!
THIRD: We also provide certification for talks given at conferences, a feature unique to our website.
Can ReviewerCredits benefit researchers in ESL regions like China, Korea, Japan, and Taiwan?
Of course! There is a large number of high-quality research emerging from these regions, but honestly, our penetration in these countries is still quite limited. From the very beginning, we wanted to develop a system that would work for researchers from every country; indeed we hope to see the numbers from China, Korea, Japan, and Taiwan growing fast in the coming months!
Could you tell us more about the technology powering ReviewerCredits and its upcoming API?
The ReviewerCredits website is powered by the Italian developer 4Science, our key technological partner. 4Science is a highly specialized company that develops products and solutions particularly for open access and open data, based on open source technologies and open standards. 4Science has been providing technical contributions to the communities of DSpace, DSpace-CRIS, IIIF, and OJS. We are currently developing a bi-directional REST API (Application Programming Interface) that allows the automated transferal of peer review data from publishing platforms. This will further simplify the process and avoid the need for “verification of claims.” Moreover, this bi-directional API will not only let our website receive data from publishing platforms, but will also query and send data to other databases. Most institutions and universities have repositories and tools to track and measure the publication metrics of their employees. By using our API, it will be possible (upon authorization from the members) to integrate data on peer review or conference activities into these databases, similar to what currently happens for publications. It is our belief that these data should be taken into account when deciding university rankings, like the excellent one published by Quacquarelli Symonds.
What are the primary focus areas for ReviewerCredits in 2018?
We have several ideas in the pipeline. On one hand, there is the technological development mentioned above. We will soon undertake the development of APIs, to enable direct transferal of data from journals willing to collaborate with us. Moreover, we are very keen to support the world of open access publishing. To this end and thanks to the great experience of 4Science, we intend to develop a plugin for Open Journal System (OJS), to be released for free. This will entail all journals using OJS to interconnect with ReviewerCredits. We would also like to empower an interconnection with ORCiD, in order to allow our users to transfer their data towards and from this key platform in the field of scholarly publishing. Finally, we will activate the “redeemability” of credits. On the other hand, there is the strategic development. We intend to increase our visibility among researchers, publishers, journals, conferences, universities, and research institutions. Having a large user base is essential to create a momentum. Finally, we need to grow as a company, with an efficient, streamlined, and sustainable organization.
What are some of the other initiatives being undertaken to broaden the scope of partnership with publishers, journals, congresses, repositories, and research organizations?
This is a crucial aspect in our plans for 2018. Indeed, we are talking to publishers, proposing them to open corporate accounts for their journals. We aim to let publishers and editors understand that they would greatly benefit from collaborating with ReviewerCredits. The conference feature has just been launched, so we have just started to contact conferences. In this respect, we would welcome partnerships with event managing agencies, which could easily provide, for free, an additional service to their customers. Finally, we are actively looking for collaborations with organizations that support, in the broad sense, the world of research. The recently announced collaboration with Enago is a great example of this!
In your opinion, how is technology changing the landscape of academic publishing? What are the benefits and challenges according to you?
Over the past decade, technology has surely led to radical changes in scientific communication. These changes are mostly positive, as now there are many journals that cater to very specific topics thereby helping authors improve their chances of getting their work published. The other side of the coin is that, with the increasing number of articles published, it is becoming difficult to estimate the value of a work. In this context, the work of peer reviewers is more crucial than ever. However, the paradox is that there are many manuscripts submitted but very few scientists willing to review them. In such situations, journals tend to trust the judgment not of the BEST reviewers, but of the FIRST AVAILABLE ones, which are typically young inexperienced scientists. This is another reason why we believe that the recognition of the work as a peer reviewer today is essential.
The peer review process is an important part of scientific communication. However, it is mostly under-valued and un-rewarded. What according to you will be the after-effects of initiatives such as reviewer credits, rewards, and recognition programs?
No metrics or rewards exist to recognize and reward the performance of peer reviewers. Peer review is a crucial phase in the process of publication for journals and publishers. However, relying only on a voluntary effort with a lack of any form of recognition has led to two major consequences. Firstly, while established journals with higher impact factor have their own pool of trusted reviewers, many other journals struggle to find available scientists for peer review.
Secondly, because reviewers work for free, they are often not motivated to meet the expected deadlines, and the overall quality of their evaluation relies only on individual integrity and cannot be evaluated. Obviously, this leads to heavy delays in the publication cycle and decreases the efficiency and effectiveness of the system. It is clear that journals, publishers, and editors would tremendously benefit from a satisfactory rewarding system for peer reviewers. Reviewers will be more motivated to accept invitations and provide excellent reports while editors will be in the position to select the BEST peer reviewers not just those who happen to be available.
Reviewer Credits – Rewarding peer review activity from Reviewer Credits on YouTube
How do you verify and filter out predatory conferences and journals?
Typically, predatory journals do not evaluate manuscripts by peer review, and therefore they should not be interested in our initiative. However, since we have received this question from the very beginning of our activity, we are thinking about different criteria that must be fulfilled in order to create an account on our website. These criteria could include, for example, the presence of an editorial board, the publication of a minimum number of articles per year, and, most importantly, the verification of the existence of the peer review process during the manuscripts evaluation. We are not so sure if we will be entitled to define a journal as predatory or not, but what we can certify without any doubt is the peer review activity employed by the journal, from the numbers of claims submitted for that journal by our members.
What are some important tips that you would give early-stage researchers wishing to peer review for journals?
First: Create a profile on Reviewercredits.com.
Second: Ask your mentors to help you with peer reviews. Discuss with them your comments on the paper. Maybe they might recommend your name to the editors if (as very often happens) they are too busy to accept an invitation.
Third: Don’t be shy! Editors are actively looking for peer reviewers. Propose yourself and simply drop them an email with your CV.
Fourth: Do all your best to do a great job!
Fifth: If you haven’t done so… go back to the first point!
Can authors use this platform to select reviewers if their journal has asked them to? If yes, would you have any tips for these authors?
Of course, this is one of the purposes of our platform! You may look for those individuals who are more active as peer reviewers in your field.
As experienced authors yourselves, please share some important tips and advice for early-stage ESL researchers undergoing their first peer review.
At first, think about peer review right from the very beginning of your experiment/study. If there is any flaw, be honest to yourself. Otherwise, sooner or later someone will spot it, and all your efforts could be wasted. Never ever indulge in the idea of altering or manufacturing data even if, after months of hard work, the results are not what you expected. Sometimes there is a strong temptation to make some changes even though it might appear innocent in your eyes, however, this could provide false results to the scientific community and could easily ruin your career forever. Do not pretend that you already know how to write a manuscript: Take a class, or use the resources on Enago Academy!
Ask your colleagues to peer review your manuscript first. You will receive criticisms and that (again, after hard work) is never pleasant. However, rather than defending your own point of view and trying to explain that you are right, think that they are probably right, and maybe your paper is not as clear or flawless as you think. Peer reviewers will not be more condescending than your colleagues. Be humble, especially if you are an ESL author. Have your manuscript reviewed for language improvement! Finally, when you receive criticisms from peer reviewers, think of this as one of the best ways to improve your manuscript. Always consider that someone somewhere took some free time to read and comment on your manuscript!
Do you think there is a need to conduct training workshops for budding reviewers?
The peer review process is the cornerstone of scientific communication and performing a peer review carries its weight of responsibility. It is usually a demanding task that requires significant time and effort. Education of peer reviewers is mandatory to improve the general process; however, little attention has been devoted to this. One great resource is Nature Masterclasses; however, more could be done. Organizing a more structured educational process with classes and workshops would be ideal in this respect. Maybe Enago Academy, which is conducting different types of workshops for authors, would like to step in?
We noticed you added Enago Academy under your resources section (and thank you for that!). Which resources did you find especially helpful for your members?
We simply love the Enago Academy platform. The “Academic Writing” section provides so many useful resources and tips. These are useful both for junior researchers (we always point our younger colleagues to Enago Academy while writing their first papers), as well as more experienced ones (if we may call ourselves so!). The topics are very broad and cover most of the challenges we face when writing—even those we are not yet aware of!
I drastically changed my approach to writing paper titles after reading this informative article. A second unmissable section is the “Featured Interviews.” Interviews give a rapid overview on the viewpoint of different actors in the field of publication, and make us aware of issues that one does not realize in their daily routine, but are a crucial part of the scholarly publishing process. One great example is the interview with Stephanie Kinnan. She talks about what authors know (or believe to know) very well, i.e. the publication cycle, but under a very different perspective and knowing this perspective is tremendously useful when you are submitting a manuscript!
You can check out the ReviewerCredits website and also follow them on Twitter, Facebook, and LinkedIn.
It was a great pleasure to talk to Giacomo Bellani and Robert Fruscio. We sincerely thank them for taking the time to be a part of this interview and also wish them the very best in their future endeavors!
(This interview is a part of our interview series on Connecting Scholarly Publishing Experts and Researchers.)
