How is FPKM variability handled?

Question

[NHEJ]

I have seen a recurring trend that FPKM values tend to be highly variable/volatile quantities across samples. For example, for the same transcript of a given gene, an FPKM value for this transcript could be 10 or 15 or 5 (across three samples analyzed by Cufflinks, for example).

How does one computationally handle the variability in FPKM value? How "real" are these values and how much can one trust them in a lab setting?

Answer 1

[Vishaka Datta]

How much can one trust them : The cufflinks output files (genes.fpkm_tracking, isoforms.fpkm_tracking) will contain confidence intervals for each of the FPKM estimates (under the headings FPKM_conf_lo and FPKM_conf_hi). Similarly, cuffdiff output also contains the FPKM estimates and confidence intervals used for differential expression testing. These intervals will give you an idea as to the extent of variability within each sample.

You could also do a pair-wise scatter plot of the FPKM values of each of your samples. That is, plot the FPKM values from sample 1 along the axis, and FPKM values from sample 2 on the y axis to look at how much variability is present in your data. The scatter should appear "elliptical", with the major axis along the y = x line if the variability between replicates is "low".

How "real" are these values : I didn't quite follow your question. FPKM values are estimates, and have errors associated with them due to many reasons. The confidence interval is an estimate of that error. Was that what you meant?

[NHEJ]

+1 for great response. Yes, when you say that FPKM values are estimates that have errors associated with them due to many reasons, could you please name some of those reasons (and cite a relevant publication)?

[Vishaka Datta]

Reasons for errors - Off the top of my head :

1. Sequence reads might align to multiple locations within the genome. This makes assigning a given read to a particular gene ambiguous. Cufflinks has a few ways of dealing with it (the multi-read correct option, for instance). See the cufflinks manual for this.
2. The fragments in the library may be very short, or from a particular location within the gene, etc. This means that the FPKM values of the gene that gave rise to these fragments might be underestimated. Again, cufflinks gives you an option on whether you want to correct for this (Paper : http://genomebiology.com/2011/12/3/R22/ref )
3. The supplement of the original cufflinks paper gives you details on the various approximations, assumptions made in computing FPKM estimates. I can't list all of them here.

[Sean Davis]

See this figure from the DESeq paper and, of course, the accompanying paper: http://genomebiology.com/2010/11/10/R106/figure/F7

Answer 2

[Devon Ryan]

The best practice is to avoid using FPKM for anything. If you need to use an expectation maximization algorithm in your workflow, then keep things in estimated counts. One of the (many) issues with FPKM/RPKM is that they lose precision information. Thus, an FPKM of 1 in samples 1 and 2 will likely have completely different errors associated with them and tracking/handling this is very non-trivial (I think cuffdiff converted to a more DESeq-like method internally a while back, perhaps because of this).

[NHEJ]

Could you please point me to a relevant publication explaining why estimated counts are a superior measure than FPKM? So is DESeq superior to Cufflinks then, in this regard?

[Devon Ryan]

Not off-hand, search pubmed. This has been more heavily discussed in blog posts. BTW, it's unclear what cuffdiff actually uses internally these days (its method seems to have changed drastically over the years) for computation.

[NHEJ]

If it's not formally published that estimated counts are a superior measure to FPKM, then it's just personal preference (particularly in blogs, where people mostly speak of their experience with their data that they analyzed). If that claim is not objectively benchmarked (as seen in high-impact papers that compare methods), then it is just a claim, plain and simple.

[Devon Ryan]

That's not how it works in bioinformatics. Many things in this field never get fully published, but reside on blogs and bioarxiv or even only in presentations. If you want publications, you should be able to find a few on pubmed. I don't have sufficient time to perform trivial searches for you.

[Vishaka Datta]

There are comparisons between FPKM and counts with respect to which measures and methods of modelling RNA-seq data tend to give better differential expression results.

[Manvendra Singh]

I think Devon Ryan is absolutely right.

This issue has been discussed so many times on this forum. And if you put some effort to search the comparison of FPKM with other methods on pubmed. I am sure you would get more papers to get your answer.

I am not sure that anyone would like to waste their time to provide you all the evidences and citations, for what they are answering your issues.

[Kevin Blighe]

Indeed, and apart from the specific discussion in this thread, FPKM/RPKM should not be used for differential expression analysis - there is no cross-sample normalisation performed when deriving these expression units.

Please read this: A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis

The Total Count and RPKM [FPKM] normalization methods, both of which are still widely in use, are ineffective and should be definitively abandoned in the context of differential analysis.

Also, by Harold Pimental: What the FPKM? A review of RNA-Seq expression units

The first thing one should remember is that without between sample normalization (a topic for a later post), NONE of these units are comparable across experiments. This is a result of RNA-Seq being a relative measurement, not an absolute one.