Is Sleuth a good package?

Question

[scchess]

Sleuth is an R-package for k-mer isoform differential analysis tool for Rna-Seq Kallisto. Unlike Kallisto, Sleuth is not published in any respectful journal. It's not even listed on Bioconductor.

Both edgeR and DESeq2 have been tested extensively, so I'm confident working with them. However, my experience with Sleuth is mixed; the fold-change values it reports are suspicious.

Q: What does the Bioinformatics community think about Sleuth? Would you like it be part of your pipeline?

[GenoMax]

Having something published is not necessarily a criteria of being "good". FastQC (which many use for initial QC of data is unpublished) and BBMap (one of the many tools is currently under review in a paper) are still unpublished but are one of the "best of the breed" packages.

If you logically agree with the approach/implementation and results from an package then you should be able to defend and include the results in your publication. Ultimately results from sleuth are helping generate new hypotheses that you will independently test with some other experimental technique. They would not be an end result themselves.

[igor]

You can also add Picard to a list of highly respected tools that are not published.

[davedeto]

Sleuth is also just super-new. I'd imagine it's being reviewed somewhere and will be published soon.

[igor]

The preprint was posted in June, so it was likely submitted for review around the same time.

Preprint: http://biorxiv.org/content/early/2016/06/10/058164

[james.lloyd]

In addition to it being published as a "pre-print" in the bioRxiv, as igor pointed out, it is a useful and powerful piece of software.

It is the only package (to my knowledge) that can take advantage of the bootstrapping that kallisto and salmon both do. This is one of the main selling points of these tools (in addition to speed).

Also, edgeR and DESeq were both designed for gene level analysis, and I am not sure they handle transcript level analysis appropriately. This is what Sleuth was designed for. So if you want to use the power of bootstraps and get isoform level analysis done, Sleuth is probably the right tool.

Answer 1

[Devon Ryan]

As genomax2 correctly noted, being published is a meaningless criterion in bioinformatics. I'll add to his list "bwa mem" as another excellent and extremely commonly used tool that will never be published.

Regarding Sleuth, from the paper and the various comparisons people have done it seems to be a good package. My only concern with it is that you appear to not actually be gaining much from using it for (at least for the use-cases that I normally care about), so there's little personal motivation to change what already works well.

[igor]

Why is "bwa mem" not publishable?

[Devon Ryan]

It's completely publishable, but won't ever be aside from what's in bioarxiv. The "tldr" history behind this is that Heng Li submitted a paper a while back and the reaction in review was...not great. He basically then said, "it's not like I really need the added 1000 citations anyway..." and posted what he wrote to bioarxiv. To be frank, he's probably the most well known person in bioinformatics anyway, so it really doesn't matter if he publishes a paper on his stuff or not (his C.V. could just be, "If you're qualified to make hiring decisions, you already know what I've done.").

[igor]

Who would reject bwa mem? Definitely would be a boost to any journal's impact factor.