Open Science in Times of Coronavirus: Introducing the Concept of Real-Time Publication
From Firenze University Press Journal: Substantia
Alexandre Hocquet, Université de Lorraine, Université de Strasbourg, CNRS, AHP-PReST, F-54000 Nancy, France
Who doesn’t have an opinion about hydroxychloroquine?
The recent developments of the latest research in Marseilles on the potential of this antimalarial drug to reduce the viral load of SARS-CoV-2 have been heating up. Obviously, the current pandemic is a sudden and unprecedented health crisis. Unexpectedness and scale are turning the outbreak into widespread panic: science is summoned to find solutions as soon as possible. In a sense, the worldwide situation is a way of asking how fast can science go.
The famed publication from Didier Raoult’s group allows us to highlight an evolution in peer review practices, and this trend allows us to question what it means to be “open” in science. Drug development studies take years, even decades. The publication process that follows usually takes months or even years. It is increasingly difficult for scientific journals to find reviewers and to get the reviewers comments in time. The task of reviewing is burdensome. It usually is anonymous and voluntary, yet it is a liability since the reviewers’ comments are supposed to certify the validity of the publication. The volume of manuscripts submitted has been increasing exponentially. Finding adequate reviewers and getting comments reported back in time has become a dauntless task for many journal editors.
Yet, in times of urgent crisis, with means and relations, a preliminary study can be done in a fortnight and published very quickly. After having submitted a manuscript to a journal on March 18, Raoult’s famous paper got its reviews back for publication… on March 20! In exceptional circumstances, exceptional publication times are key. The two reviewers therefore analyzed the publication and delivered their comments in less than two days. Indeed, science seems to be able to speed up its process when emergency calls. What’s new is that the critique of science can itself go very fast. In times of global panic, publications that deal with potential treatments for coronavirus are actually read, and even scrutinized by an attentive and numerous readership.
Among them are colleagues, competitors, doctors, pharmacists, microbiologists, statisticians, bio-informaticians, mere curious readers, some enthusiastic, others malicious. A myriad of potential reviewers from a diversity of backgrounds, in sharp contrastwith the officially appointed reviewers, who are seldom thematically far away from the authors. For scientific journals, peer review has been playing the role of certification (does the article deserve to be published?) but it also takes the role of evaluation (is the article good enough for such or such journal?). Consequently, an article is considered cutting-edge science or not depending primarily on the prestige of the journal in which it appears, to the detriment of the evaluation of its content itself. Journals prestige is key in defining a tiered system of publication. “Publish or perish” has become “impact or perish”.
In contrast, PubPeer is a grassroots web platform launched in 2012, whose purpose is to potentially open a discussion forum about any scientific article. It was designed by its creators as the online version of a journal club(a sort of lab meeting where publications are critiqued) and was imagined out of the frustration that stems from the virtual impossibility to critique articles within the journals themselves.PubPeer is a form of post-publication peer-review. It is radical in the sense that anyone can comment, and can even comment anonymously. This “anyone can edit” principle is similar to Wikipedia’s principles, where a contribution can indeed be criticized or reverted by anyone, the difference being that in PubPeer, articles are commented but not edited. Not unlike in Wikipedia, the most affected articles are those that shine under the media spotlight.
Comments from people from different backgrounds entail multifaceted critique. Inthe PubPeer thread dedicated to Raoult’s article in our example,7one can find a flurry of comments about such themes as the publication process, the compliance with the state of the art, medical ethics, statistical methodology, statistical processing…among many others.What’s new is that it includes reanalyses, i.e. analyses that the commentator has inferred from the data provided by the authors, rather that from the publication text. Reanalyses are carried out with the aim to design new calculations, new graphs, new interpretations, in order to put the “results and discussions” of the paper to the test. They can actually achieve this goal only if the data is provided by the authors, which is still far from systematic. The publication is thus becoming a sort of “live” paper, instead of being set in stone, just as a Wikipedia article follows, by definition, a never-ending dynamics.In this regard, PubPeer looks like a social network.
Critics, reanalyses, and potential answers from the authors to their critics are discussed, debated. The article review itself is becoming a live process. In times of crisis, it may even turn into a bursting of interactions. The publication from Didier Raoult’s group has been dissected in less time than it took to publish it: a hundred or so comments on PubPeer between mid and late March made the article look like a “real-time” publication. PubPeeris criticized among scientists for its anonymity and (sometimes rightly) for its denunciatory atmosphere… just as anonymity in Wikipedia is criticized.
In both cases, anonymity is seen by the community as a condition to enhance participation (since it allows to escape from the pressure of hierarchy or competition). On the other hand, it is viewed as cowardice by people who fear that anonymity may lead to the promotion of abuse. As Michel Dubois and Catherine Guaspare point out, the tension lies in what constitutes a “peer”. How can a platform warrant potential commentators to feel free enough to contribute in a hierarchical and competitive scientific world, while at the same time protecting from malicious attacks? As in Wikipedia, PubPeer possesses rules to ensure that attention is focused on content rather than on individuals, but as in Wikipedia, malicious attacks do exist. Amusingly enough, the exact same tensions and debates emerged when referees were becoming anonymous in mid-XIXth century. Another criticism of PubPeer (and Wikipedia) is the pervasive obsession of contributors about technical details (such as image editing) rather than the substance of the articles.
Pubpeer founders respond that no faking of data is benign enough to be delegitimized as irrelevant.What happens on PubPeer is also performative: contributors play the role of moral entrepreneurs by defining through commentary what is acceptable or deviant. In a way, the thematic diversity of comments about Didier Raoult’s publication does PubPeer justice to its “journal club” credentials. Commentators from a wide range of backgrounds suddenly take an interest in hydroxychloroquine. What is key here is a situation of global emergency combined with the inclusiveness of access to critique. The situation allows for the emergence of a globalized, real-time journal club. It would be naive to see PubPeer as the enchanted world of “self-correcting” science progressing through Disputatio.
Conversely, it would be equally naive to think that the publication system as it exists readily lends itself to constructive criticism. The vast majority of journal articles cannot be commented upon. The function of a publication is more about recording antecedence than to engage in debate, which the conservative structure of scientific journals discourages. As a matter of fact, PubPeer receives bad press in the quarters of scientific institutions. Those institutions entice researchers into publishing their papers in “open access” and to make their data available, yet they criticize those open initiatives that they fail to control. Everybody wants open science, but the scientific community finds it difficult to realize that encouraging open data (in the name of transparency requirements) results in the possible reuse of these data, and thus, among other things, in the availability of those data for post-publication critique, a critique that has been almost non-existent until now. Some scientific journals are trying to change peer reviewing rules. Several ideas are devised to separate evaluation from certification: reviews can be made before or after publication, they can be confidential or transparent, they can be spontaneous or invited, anonymous or identified. Some new editorial practices transform a “version of record” into a record of versions, according to Bianca Kramer’s phrasing.
Unfortunately, the most prestigious journals, those that make or break careers, are often maintaining the statu quo. The scientific landscape of journals is diverse, though. The situation of journals in the humanities, for example, is far from this depiction. Pubpeer is actually most active in the field of life sciences where the likes of Cell and Nature concentrate the criticism from the scientific community.In this sense, the PubPeer platform not only undermines the conservatism of scientific journals. By questioning the peer review process, it questions the very utility of the concept of publication and scientific journals.
Even without going as far as extreme cases of retraction, the risk for the publication is to be contested in its role as a reference document, and the scientific journal its role as a gatekeeper contested too. Alternative viewpoints about openness put inclusiveness at the centre of their concerns. It poses problems (such as the question of legitimacy) while trying to solve others, but it has the merit of raising awareness that some ways of being open are more complex and challenge more things than we imagine… like Wikipedia does.
