Skip to main content
SearchLoginLogin or Signup

Proposal for a “Next-Generation” Scientific Publication System

We proposed a system that creates a social network among scientists to facilitate collaboration and to reduce publication-induced stress.

Published onFeb 01, 2018
Proposal for a “Next-Generation” Scientific Publication System

Proposal for a “Next-Generation” Scientific Publication System

Yu Wang1,2 and George M. Church 1,2

1.     Wyss Institute for Biologically Inspired Engineering, Harvard University, Boston, Massachusetts, USA

2.     Department of Genetics, Harvard Medical School, Boston, Massachusetts, USA

Correspondence: Yu Wang ([email protected]) and George M. Church ([email protected])

    The idea of ‘open science’ is gaining attraction among the scientific community, with an increasing number of journals becoming open-access.  However, open science is not only about the negotiation between scientists and publishers, but also about how “open” scientists should be before their scientific work is published on journals. One key challenge we see in the journal-based system is that competition between scientists for publication in high-profile journals impedes open and rapid sharing of scientific work. In order to generate novel but also significant results that can meet the criteria for high-profile journals, researchers typically spend years in one project to follow a specific hypothesis and generate a compelling scientific ‘story’. As a result, it is not uncommon to see phenomena like keeping project as confidential for years, irreproducible results with biased experiment design and data interpretation to favor certain hypotheses, and researchers stressed out with the concern of getting ‘scooped’. In an attempt to alleviate publication pressure for researchers and build an e-network among scientists to promote collaboration, we propose an online scientific data and results sharing system that complements the current journal-based system. Our system is inspired by the Gordon conference and Github. The ultimate goal of this system is to build a community in which researchers with similar interest collaboratively solve scientific questions, just as the Human Genome Sequencing Project.

Publicly sharing scientific data and results is essential for promoting the research community as a whole by offering opportunities for engagement with peer scientists throughout the world. Academic journals are traditionally the mainstream approach to publish scientific work. It is undeniable that publication on scientific journals has become one of the most critical factors to evaluate a researcher’s scientific achievement and capability, which directly affects their career advancement. It also appears that researchers are more likely to be acknowledged by publishing their work in high profile journals.

    It becomes problematic being that high profile journals are limited. With an increasing number of scientific works being generated nowadays, those journals have to keep raising the bar and be more selective about the work they publish. This naturally creates a competition among researchers who are involved in the system. Sadly, many of us have started to do science in a way that it is more likely to be published in high profile journals, instead of being driven by pure scientific interest. We are constantly exposed to the pressure of getting positive, novel and significant results in order to make a compelling “story.” Unfortunately, the nature of science determines that we only get negative or positive yet incremental results. Worse still, we have to keep our work confidential until it is close to publication or has been published, which could take months or even years.

    This has led to several issues. Firstly, in order to complete a project for publication on scientific journals within a reasonable amount of time, researchers are more likely to be biased on experimental design and data interpretation that favors their original hypotheses. Worse case scenario, some researchers may even falsify data to get expected results. Secondly, even though researchers hold a fair attitude towards their work, science is complicated and sensitive to the technologies, methods and reagents used in the experiments. There is a chance that experiment results are pure artifacts that cannot be reproduced by others labs. This can be from contaminated cell lines or wrong antibodies used in the project, or lack of expertise in skill-demanding fields such as statistic and microscopy. A good example is the use of p-values for statistical significance tests in biological experiments, which has been heavily criticized by statisticians and even banned by several journals1,2. Unfortunately, because the scientific work is hidden for confidential purposes, some of these artifacts may not be realized for years depending on when the work is published. Thirdly, even in the case that everything is on the right track, researchers may worry their work can be ‘scooped’ by other researchers. This can be devastating for researchers because most high profile journals list novelty as one of their major criteria for publication. Finally, not only can this publication pressure lead to scientific irreproducibility, but also contribute to scientists’ mental health issues that have recently gained a high level of awareness3.

    In order to alleviate this publication pressure and promote collaboration among researchers, we need a faster but also reliable scientific data and results sharing system that can complement the current journal-based publication system. Inspired by the Gordon conference, Github, and many other open science platforms, we here propose a new publication scheme that takes advantage of fast-developing Internet platforms, with the goal to build a scientifically rigorous and socially joyful research community.

    The proposed system consists of two inter-connected components: an online open-access data/results deposition platform (e.g. Pubpub, Biorxiv) as well as an online networking system (e.g. an integrated system from Researchgate, Facebook and Google Scholar) (Figure1). One key difference of our proposed system from the current publication system is that we encourage researchers to publish project milestones instead of complete scientific ‘stories.’ Project milestones can be scientific ideas or experiment results that have been rigorously validated at any length. The format of published milestones can be short communications, including a brief introduction for background or project rationale, results, and more importantly, detailed methods and original data that are the key for quality check and result reproduction. The online networking system is used to facilitate the sharing of scientific work and encourage peer scientists to evaluate and give feedback on the work. In this online platform, researchers can establish their own profiles with real identity, and directly communicate with peer scientists by following each other’s profiles. Moreover, researchers are allowed to select their research interest as tags to form research circles. Each published project milestone will be labeled with keywords as tags by the authors. The online system will match the tags and automatically send the milestone to all researchers who have the same tag. Each published milestone will also come with a publicly trackable interactive discussion board where authors and peer scientists can leave and reply to feedback on the work. In contrast to the current peer-review system that depends on opinions from one to three reviewers. This system enables a cloud based peer review that obtains opinions from thousands of peer scientists with similar interest from around the world. When finished, researchers can organize multiple deposited milestones into a more whole article and submit it to their favorite scientific journals. This online system will automatically generate a report including the metrics and the ‘cloud’ peer review, which can be used by journal editors to make a fast decision on the manuscript. 

Figure 1. Schematic of Proposed Scientific Data and Result Sharing System. Step1: Researcher 001 finishes a project milestone and writes it as a short communication. Step2: The short communication is deposited on public domains such as Pubpub and Biorxiv. The public appearance serves to protect the author’s intellectual property. Step3: The short communication is shared to other researchers through two ways: one is via posting by Researcher 001 on his/her scientific project blog on the Online networking platform to the project followers; the other is via keywords matching by the system to scientists who share the same research interest. Step4: Peer researchers give feedback on the work through an online interactive discussion board, functioning as a ‘cloud’ peer review system. Step5: Researcher 001 responds to the feedback and gets inspired to carry on the project. Step6: Researcher 001 organizes multiple related project milestones into a full article and submits it to a journal. Step7: Researcher 001 requests a report of the cloud peer review generated by the system to be sent to the journal editor. Step8: The journal editor makes a decision on the publication based on the cloud peer review. Step9: The full article gets accepted by the journal and released to the entire research community.

    This proposed system is motivated to build an online platform that mimics Gordon conference where researchers are encouraged to share scientific thoughts and results openly and comfortably. A few features of the system should be noted: firstly, releasing project milestones instead of full-length articles will greatly accelerate the process of scientific results sharing. Researchers do not have to spend years to follow a specific hypothesis and prove it ‘right and significant.’ Negative and incremental results can be written as project milestones and published. In addition, by sharing results early in a project, researchers can get prompt feedback from peer scientists. This can help researchers to quality-check their current research as well as to develop future research plans. Secondly, unlike Gordon conference where unpublished results are only shared based on the trust between scientists within the same research field. This proposed system utilizes public domains, such as Pubpub, to serve as time records to protect researchers’ intellectual properties. This potentially enables even more open discussion among a wider range of scientists. Thirdly, with the increasing trend of interdisciplinary research, it has become more  difficult to find reviewers who have the expertise in all aspects of a paper. The ‘cloud’ peer review includes opinions from peer scientists with different backgrounds, allowing a more thorough evaluation of the paper. Although it might be arguable, we believe that this ‘cloud’ peer review should be less biased in contrast to the current system where editors based on suggestions from authors choose anonymous reviewers. Lastly, this proposed system still allows researchers to organize multiple milestones into a full-length article to submit to scientific journals. We include this component mainly because the journal-based publication system is so deeply ingrained in the research community that it is challenging to change it completely at once. It, in our perspective, is also the reason that open science initiatives have hitherto not been very successful. We hope this proposed system can function as an intermediate stage to allow funding agencies, career committees and scientists to gradually transition to open science.

    With fast-developing Internet technologies, we consider that there is no significant technical barrier for this proposed system. In fact, most of the components in the proposed system are already available (e.g. Pubpub, Biorxiv, Researchgate, Google scholar article alert). In our opinion, different open science platforms should coordinate with each other to construct a single integrated system that provides researchers the most convenience, instead of forcing researchers to shuffle between different platforms. In addition, to make researchers more willing to openly share their work, funding agencies and career committees should examine beyond publication on journals. This system could potentially generate metrics based on researchers’ activities to provide additional references for funding agencies and career committees to evaluate. With a joint effort between academia (i.e. thousands of researchers who are actively engaged in the publication system, and funding agencies) and technology companies (e.g. Facebook, Google and Microsoft), this proposed system and other open science systems would greatly benefit the entire scientific research community, by making it scientifically rigorous but also socially joyful.

    Lastly, we have been actively discussing this proposed system with our colleagues, including established faculties, young investigators, postdocs and graduate students, and encountered some frequently asked questions. We have listed them and put our thoughts as a Q&A section here:

1.     If we deposit our milestones in public, how can we prevent other labs from ‘stealing’ the idea to perform similar experiments and get them published in scientific journals before we do?

    We think this is the key element that drives the entire publication pressure. That is, scientific journals are the only way for researchers to protect their intellectual properties. It is also the key problem we are trying to address in our proposed system. Deposition in public domains such as Pubpub can serve as a time stamp for original authors to claim credits. In addition, the milestones will be automatically sent to thousands of scientists with the same interest. It will very likely cover most of researchers who work in the same field with original authors.  This will quickly establish the idea among the scientific community that those original authors make the initial discovery, which is more important than just publishing in specific journals.

    In addition, the initial motivation for developing this new system is to encourage our research community to be more collaborative and less competitive. Competition in science is less valuable when compared to the competition in business. Once we make the initial discovery and deposit it into the online system, we should encourage peer scientists to follow up our work and make a joint effort to decipher the science behind the discovery. One potential way to promote collaboration is to describe current or proposed work for the project at the end of the deposited milestone. By doing this, authors can potentially get feedback from peer scientists on the direction of the project. Meanwhile, other scientists can avoid doing the same work.

    From a practical perspective, a significant amount of follow-up work can be done during the preparation of milestone papers. The ‘pirates’ will actually run the risk of getting ‘scooped’ by follow-up milestones from original authors. In addition, this online networking system allows faster and wider communication throughout the entire research community. This also means any dishonest behaviors such as releasing work with false information to prevent others to follow up the work or stealing others’ work can be quickly and easily recognized.

2.     How can we prevent the system from abuse to claim credits by publishing ill-performed studies?

    We agree this could potentially be a downside for open sharing of scientific results before significant peer review. However, the system comes with a quality check mechanism, that is, the ‘cloud’ peer review. We encourage researchers to include detailed methods and original data along with the deposited milestone for evaluation. Any ill-performed studies (e.g. manipulating or even falsifying data) will soon be flagged, and the authors run the risk of losing their reputation by publishing ill-performed studies. In addition, scientific credits will not be simply given to someone just because they published first. Assignment of scientific credits has to be carefully justified and broadly acknowledged.

 

3.     The comment section for papers has existed for journals and preprint service (e.g. Biorxiv) for a long time. However, it has not been well used. How are you going to promote this ‘cloud’ peer review idea?

    We have also noted this phenomenon. Although we have anecdotally heard of authors’ papers published on Biorxiv receiving comments through emails, it is not sufficient for the ‘cloud’ peer review. We think the underlying reason for missing useful comments in the current system is that researchers do not have the motivation to leave and respond to comments publicly. Papers that are deposited in Biorxiv are mostly full articles that are likely under review or soon-to-be reviewed at peer-review journals. Paper authors will likely only take comments from journal reviewers seriously. As a result, leaving comments publicly on Biorxiv by peer scientists will not gain much attention from authors. On the contrary, in our proposed system, the feedback on deposited milestones from peer scientists may directly affect downstream experiments. Therefore, it is critical for authors to have an active discussion with peer scientists. In addition, we encourage journal editors to take this ‘cloud’ peer review into consideration when they make a decision for publishing. This will encourage peer scientists to leave comments in the system as well as somewhat ‘nudge’ authors to respond to comments. We think once more researchers get used to the open discussion platform, it will become a habit without further motivation.

 

4.     What is the relationship between this proposed system and the current journal-based system? Will it replace the current journal-based system?

    We think this proposed system complements the current journal-based system. We don’t foresee it replacing the current system for some time. However, it may modify the current one. For example, the ‘cloud’ peer review may replace the close-door anonymous peer-review, which may actually expedite the entire paper publication process.

    In our opinion, scientific journals, especially high profile journals, have a broad range of audience from different backgrounds. Therefore, it can serve as a good “advertising” platform to bring scientific work to a larger number of researchers who do not share the same scientific interest tags.

 

5.     How is this system different from other open science platforms that allow scientists to openly share lab notebooks?

    Openly sharing lab notebooks is one of the ultimate goals of ‘open science initiative’. However, it may not be feasible for every researcher at current stage. In addition, a key challenge we see in openly sharing lab notebooks is that lack of organized information makes peer-review from other scientists challenging. It should still be coupled with releasing work in a ‘paper-like’ format.

 

 Acknowledgement

We acknowledge members in Dr. George Church’s and Dr. Peng Yin’s laboratories, Dr. Constance Cepko, Dr. Bernardo Sabatini, Dr. Wayne Marasco for fruitful discussion. We acknowledge Justin Granchelli and Noah Donoghue for their comments on the manuscript.

 

Reference

 

1          Baker, M. Statisticians issue warning over misuse of P values. Nature 531, 151, doi:10.1038/nature.2016.19503 (2016).

2          Nuzzo, R. Scientific method: statistical errors. Nature 506, 150-152, doi:10.1038/506150a (2014).

3          Pain, E. Ph.D. students face significant mental health challenges. Science, doi:10.1126/science.caredit.a1700028 (2017).

 

 

 

Comments
0
comment
No comments here
Why not start the discussion?