Forum:Mapping God Found ‘Scientifically Dishonest’ By Anonymous Peer Reviewers

Question

[johnblue81]

Very interesting discussion about a peer review at bioinformatics. One reviewer called Heng Li ‘scientifically dishonest’ and rejected one of his papers.

But how did it come that far?!?

• Was the paper not good enough?
• Was the described tool not good enough?
• Is a great publication record enough to publish every manuscript?
• It the peer review process itself the problem?

Many questions are raised at this blog... what is your opinion? Do you think that an open reviewing process would fix the problem? Would the voice of one reviewer heard, if hundreds of Heng Li fans rate all of his papers as 'expert prooved'? My opinion: In a public reviewing process, the paper would have been published just because of the name 'Heng Li'. Good, or bad???

read more...

[Devon Ryan]

That blog post is a nice example of sensationalist headline grabbing (I know you didn't write it, so that's not directed at you!). Heng Li wasn't actually "found" anything. He just got in a spat with a reviewer who started calling him names (i.e., "scientifically dishonest").

[Daniel]

It would sound very different if the headline was: Mapping God Called 'Scientifically Dishonest’ by Anonymous Peer Reviewers

[Devon Ryan]

Or even just, "Author and reviewer have a row" :P

[Istvan Albert]

Disgruntled Reviewer Sinks Paper, News at 11

[Devon Ryan]

If it were actually a nightly news promo, it'd be more like, "What you don't know this reviewer could KILL you! News at 11". I guess that'd bump the sensationalism up a notch.

Answer 1

[Istvan Albert]

From the discussion it is clear that this is more of a situation where the reviewer had an axe to grind. The author of a tool (or a close associate) took offense to their tool being penalized in a benchmark. Alas in science this is not a unique or even particularly uncommon situation. Moreover it is not clear how one could even solve this type of conflict of interest, short of sending the paper to a different journal.

Reviewing is a very subjective proposition anyhow.

If anything papers are not scrutinized enough.

Answer 2

[David Langenberger]

I think this discussion is not worth it. The reviewer is anonymous, the review is not published and everything written is based on Heng Li's view or gossip spread by a blog.

---

Pro anonymous peer review:

• Reviewer can comment a publication without being afraid of big names coming back on them. For not so famous researcher it might be a problem to publicly blame some experts.

Pro open peer review:

• Everyone can see if there is a conflict of interest and more eyes will find more mistakes or problems.

Perhaps it would be a good idea to blacken the authors (even though it would be useless for a tool like bwa-mem) and use a mixture of open and anonymous reviewers?

[Istvan Albert]

but there is a middle ground there, publishing the anonymous reviews as well, and having the ability to comment on them,

I think that even if the reviews stayed anonymous the mere fact of having them published would lead to an increase of their quality.

At the same time of course this in turn might have a negative impact on the rate at which people accept reviewing duties. It could be discouraging to some as it might feel as one would need to do more work.

Answer 3

[johnblue81]

So you think the problem is the reviewer? He wants to avoid the publication of the manuscript, because his/her tool does not look as nice, as he/she wants it to? So his comments are not verified in your eyes?

AND: Isn't it the editor, who makes the final decision? As Heng Li described it, it was 2:1 for publication. Is the editor of Bioinformatics the bad guy? For me, the whole story seems a bit shady. I mean... the editor is not anonymous, but he/she still made that decision against the reviewers voting.

[Istvan Albert]

(if this is relates to my post then you should add it as a comment rather than a new post)

the problem is clearly with the reviewer and the system where one hostile reviewer can preclude the publication of a paper.

For what is worth the situation is many orders of magnitude more painful for grant reviewing. The processing times are usually far longer and one gets no chance to respond to reviewers within that cycle, then in the next cycle one will get completely different reviewers that may not even care about the prior problems and just get hung up on new issues. And, unlike picking a different journal it is not that easy to find a different avenue for funding.

My advice to anyone in the same situation, if you feel that you would be enraged by an obviously unfair decision then you need to pick a different field of work. Scientific work evaluation is subjective and opinionated and usually depends on a small number of people so you can' even expect the outcomes to even out on long term.

[Devon Ryan]

Who are you replying to? It's often the case that all the reviewers need to agree for a publication to actually get accepted. Editors are often risk-averse, so this isn't exactly an unusual way for things to go down. It looks like the actual problem was that the reviewer and Heng were talking past each other and not watching their wording. Apparently that's since been cleared up, which is always nice to hear.

Answer 4

[zyan2005]

As of today, this paper could become the most cited bioinformatics paper rejected by reviewer who called the author dishonest. https://arxiv.org/abs/1303.3997 Cited by 7882 times

It is obvious that the reviewer either are not qualified or dishonest him/herself