Forum:Portal of choice for literature commenting

Question

[cdsouthan]

Within the general domain of post-pub commenting we now have open choices between Research Gate, Publons, PubPeer (both of which cold-pinged me), Kudos and PubMed commons. I have joined in with the latter and I note Kudos has just gained the implicit recommendation of ORCID.

Given that life is not long, which of these would folks recommend on the basis of detailed engagement (even for just one of course)?

P.S. if representatives from any of these see fit to make comparative factual comments that's fine but please declare your interests.

[Devon Ryan]

I moved this to forum since it's probably a better fit there (if anyone disagrees they should feel free to move it back to the questions section).

BTW, I also changed the title. I assume you meant "Portal of choice" or "Site of choice" rather than "Paralysis of choice", since I think we'd all prefer "no paralysis".

Answer 1

[Istvan Albert]

My personal opinion is that commenting should be available on the site that published the work.

Something like DISQUS that integrates with each publisher so that authorship and identities are verified. In parallel a culture needs to be developed for providing the author(s) with incentive and recognition for participating in these discussions.

Some other personal musings: I have in the past looked at a few of the sites that you list there, Research Gate, Publons and I am always greatly annoyed by the "walled in" style of these sites. Typically one cannot access any of the useful information unless they "join" which of course means giving up a lot of personal information and thus opening the floodgates of spam (the notorious ResearchGate invitations) and basically becoming a product. That is what keeps me and many people from joining these sites and I don't see that changing anytime soon.

[cdsouthan]

Points taken - so this indicates PubMed Commons as best choice (no walls plus open attributions)?

[Istvan Albert]

there is a lot of "froth" in this area with too many players and it will take some time until it settles down

ResearchGate has the massive private funding and knows how to monetize, PubMed Commons has the insider track to being featured on PubMed but they are not all that well known for creating user friendly interfaces, the other players have various commercial interests,

the last to be consulted about how this should work will be the people for whom these services should be rendered for

[Obi Griffith]

I definitely empathize with your personal musings. I think to be fair though it should be pointed out that (unlike ResearchGate) Publons, PubPeer, and Kudos do make their content available without sign-in and don't seem to pester you to create an account until you want to contribute content of your own. The latter seems necessary to me. Even if you want to allow anonymous comments you still need to have accounts for a variety of reasons. I have a feeling that ResearchGate is by far the most successful of these services so far. This is probably because it has been around longer but also because it has adopted a relentless (and often obnoxious) networking strategy. As much as it annoys me it is also true that such services aren't really that useful until a large momentum has been established.

[Istvan Albert]

heh, an amusingl/maddening usability fail right there with Publons,

I tried it before writing my piece above and could not get to the actual review, that's why I thought it does not allow me to see it.

If you are on the page for "Paper X" and scroll down and click on the title that says a "Review of: Paper X" it reloads the page and takes the you back to the paper itself, the same page you just viewed and scrolled down on. It is hard to put into words just how confusing and user-unfriendly that behavior is. In addition a lower banner pops up an blocks the bottom of the page where there may be some nuggets of interesting information.

Then the view is dominated by oversized and useless buttons to various redundant sections such as Start A Discussion, Pre-Publication-Discussion, Post-Publication-Discussion, etc. One needs to go all the way to the right and click Details for each review separately to get to the review itself, when in fact the whole purpose of being on this site is the reviews. If one wanted to read the next review they'd have to redo the whole process.

The page seems to be designed by people that themselves are not scientists and do not understand how science works, and themselves do not need to use the site to get something out of it. And that is the fundamental problem with it. Using a site like Publons feels like hard work and effort, as if we didn't already had enough stuff to put up with - that's why it has very low chance for success.

Research Gate is successful for two reasons it pesters people a lot but it is also very easy to use, incomparably better than any of the other options.

[Obi Griffith]

I have had the exact same experience that you describe of chasing my tail on publons and not actually finding a review I was looking for. Having to click that 'Details' link is NOT intuitive. And, that was when trying to find a review that I myself put there! So, I knew to keep digging but others might get frustrated and move on. Such UI flaws need to be fixed. On the other hand, I have been impressed with how easy it is to upload my past reviews into the system. For a bunch of them, I actually just forwarded old emails with the editor correspondence and the publons staff did the heavy lifting of entering them into the system. A pure non-profit probably couldn't provide such a service. Finally, I do like that you have easy/total control for how the review is published from essentially anonymous to completely open. Here's my profile as an example:

https://publons.com/author/107773/obi-lee-griffith

It took me maybe 2-3 hours of effort over two days to get all 11 of these reviews into publons and the staff were very responsive to glitches that I reported during the process. Most of that time was actually digging through my archived emails. Uploading a new review would take just 1-2 minutes which is negligible compared to the actual work of the review. I would definitely consider using publons for reviews going forwards. But, none of this addresses the more important question of whether even the perfect review management system will contribute to an academic culture where this work is in any way recognized.

Answer 2

[Mary]

I find the most interesting discussions happen on twitter. And I wish some of the folks who comment would put it at a site--any site--but they rarely do. They also snipe at blog posts the same way, never commenting at the site where it might actually do some good to drive discussion. Alas. You can't make people comment.

I like PubMedCommons for the notes people are leaving about their software availability, which may have changed since publication. I've already found that handy a few times. But--PubMedCommons doesn't have a bunch of plant science stuff I need. Also sometimes the computer science stuff may not be included either. So a more general resource is really still needed. But I can't say I've really warmed up to any of them.

The publishers sites with all sorts of different logins is a drag. If it all went disqus it might not be so awful.

[Istvan Albert]

I think the explanation is that on Twitter one has is a unambiguous ownership and control of the content that one produces. It is also very simple to keep track of responses without having to visit each site separately or getting emails from them. On the other hand they are so fleeting and leave no mark on the world.

[Mary]

I think this one is gonna leave a mark. But it was actually made on the permanent record:

Some skeptical comments about the recent “8 kinds of schizophrenia” study. http://t.co/JgCWQu4ggi via @betenoire1

— carlzimmer (@carlzimmer) September 29, 2014

[cdsouthan]

Problem is people are already gaming the system, eg as above and below

PubMed Commons ‏@PubMedCommons Sep 19

Dealing with multiple agents in #clinicaltrials: A Messori highlights role of Bayesian network meta-analysis. http://1.usa.gov/1tXqKSG 

Christopher Southan ‏@cdsouthan

Hmm @PubMedCommons IMCO this provides an example of blatent self-promotion mini-pubs masquerading as comments that you need to stamp out.

[Devon Ryan]

I'm not sure how much gaming is really going on there. It's often near impossible to get "letters to the editor" published, which the long comment from Gerome Breen on the schizophrenia article would otherwise constitute. Regarding the self promotion, it'd be difficult to prevent that in any system.

[cdsouthan]

I feel a new author index coming on ...... "comments:PMIDs". I would look much "worse" than AM or GB , unless the comment word count was factored in. Suggestions for a name? Kardashian Comment Index?

[Istvan Albert]

Frankly I don't think any commenting system could be a sustainable avenue for posting rebuttals or questioning the findings of a paper.

It is the system that does not reward this type of contributions - not the medium.