Mark duplicates fails using samtools view to remove unmapped reads

Question

[jsgounot]

Hi,

I've got an error using Picard MarkDuplicates function. It worked fine on my data before I add an other step in my pipeline, where I use samtools view -F4 -h to remove unmapped reads in the alignment. If I do this, an error occurred during Picard treatment:

Exception in thread "main" htsjdk.samtools.SAMFormatException: SAM validation error: ERROR: Record 6, Read name FCC63KDACXX:3:1115:14357:43781#GCGGAACT_TCTTTCCC, Mate unmapped flag should not be set for unpaired read.

Do you have any ideas of how to fix that ? I understand the error but I don't know how to make it works. Previously I 'manually' removed unmapped reads (reads for which an * appears in the 3rd column in the sam files) but I read that's not the good way to do that and that the best way is to use samtools with the command line above.

Thanks

Answer 1

[John]

Doing -F 4 will result in reads that were not mapped being removed from the file - but if their mate was mapped, the mate will be left in there as a singleton. Picard expects these singletons to look like singletons - not to look like half a pair, with the other half just mysteriously gone.

I imagine that running FixMateInformation will sort this out, otherwise you can use -F 12 instead of -F 4 when filtering the BAM. I can't remember if samtools does AND or OR when using bitflags, so that will either remove both reads in a pair if the read AND the mate is unmapped, or both reads in a pair if the read OR the mate is unmapped (what you probably want).

[jsgounot]

Thanks ! It works perfectly now !