Markduplicates: Value Was Put Into Pairinfomap More Than Once

Question

[lifuqiang0611]

When I use pircard MarkDuplicates to remove Duplicates from the BWA result, I get this error in MarkDuplicates: Exception in thread "main" net.sf.picard.PicardException: Value was put into PairInfoMap more than once. 1: CTTGTA_L006:HWI-ST1148:55:C1CKAACXX:6:1308:16351:60596 Does anyone know how to solve it?

Answer 1

[Raony Guimarães]

Prefix the read IDs with the lane number before merging the files.

MarkDuplicates will not consider two reads to be duplicates if they have different read group IDs, so assigning different read group IDs will address the immediate problem

Fix your SM tag before merging the BAM files

Source: http://sourceforge.net/mailarchive/forum.php?thread_name=50993D76.8060400%40broadinstitute.org&forum_name=samtools-help

Answer 2

[karl.stamm]

This problem might have been solved two years ago with Raony's suggestion. It's a common problem when the FASTQ are merged from more than one lane, the read names may overlap and Picard gets mad if a read is in the BAM twice.

I've had this PicardException arise for another reason. Samples from MiSeq with some low quality bases, run with BWA-MEM, can be placed into the BAM more than once with equivalent quality scores. Then Picard MD sees for example two forward reads and one reverse with the same read name. While BWA-MEM should be reporting only the best quality alignment, (at least for version 0.7.3) it can report more than one alignment when soft clipping allows for equivalent quality scores.

Funny that MarkDuplicates cant handle technical duplicates, but the solution is to filter with samtools to keep only properly paired reads.

[thomas.smith2]

This is a separate issue with BWA. You need to specify the "Mark Secondary Hits" option with BWA (-M)

See http://bio-bwa.sourceforge.net/bwa.shtml

[rohitsatyam102]

In my case, the issue didn't get resolved by using -M option. I have posted my query at GATK. Would anyone like to offer a suggestion on what must be done?

https://gatk.broadinstitute.org/hc/en-us/community/posts/360071534472-SAMException-Value-was-put-into-PairInfoMap-error-even-after-using-M-flag

[karl.stamm]

have you tried the solution I suggested " filter with samtools to keep only properly paired reads. "

[rohitsatyam102]

Hi!!

Actually, for every read, I have two copies in my Fastq Files. That's weird. Also, both of them are properly paired as shown below

A00804:41:HNJ53DSXX:1:1121:10746:16282  81      chr1    9996    0       111S40M chr2    32916252        0       CCCCCCCCCACCCCCACCCCCCCCCCCACCCCCCCCACCCCCCACACCTTTACCCTTAACAATACGCATAACGCTATTCTGATGAGTGACTGGATTTCTTACGTTTGCTCTTCCGATATCCCTAACCCTAACCCTAACCCTAACCCTAACC FF:,FFFF,,FFF:,,,FFF,,FFFFF,:F,FFF,,,:::F,,,,,,,,,:,,F,,,:,,,,F:,,,,::,F,,,FF,F,,,,::F:,FF,F,F,,,F,,FFF,,,,F,FFF:FF,F,:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF NM:i:1  MD:Z:7A32       MC:Z:107S44M    AS:i:35 XS:i:34 RG:Z:HNJ53DSXX:1        XA:Z:chr3,+198173832,34M117S,0;chr12,-108091,118S33M,0;chr5,-10004,119S32M,0;chr1,-180805,119S32M,0;chr1,-78128742,119S32M,0;chr3,-10519,119S32M,0;chr1,-10052,119S32M,0;chr1,-10354,119S32M,0;chr4,+190123043,32M119S,0;chr1,-10258,119S32M,0;chr1,-10046,119S32M,0;chr1,-10111,119S32M,0;chr3,-10005,119S32M,0;chr1,-180749,119S32M,0;chr2,-181275796,119S32M,0;chr10,+133787391,36M115S,1;chr3_KI270784v1_alt,-61695,117S34M,0;chr12_GL877875v1_alt,-102106,118S33M,0;
A00804:41:HNJ53DSXX:1:1121:10746:16282  81      chr1    9996    0       111S40M chr2    32916252        0       CCCCCCCCCACCCCCACCCCCCCCCCCACCCCCCCCACCCCCCACACCTTTACCCTTAACAATACGCATAACGCTATTCTGATGAGTGACTGGATTTCTTACGTTTGCTCTTCCGATATCCCTAACCCTAACCCTAACCCTAACCCTAACC FF:,FFFF,,FFF:,,,FFF,,FFFFF,:F,FFF,,,:::F,,,,,,,,,:,,F,,,:,,,,F:,,,,::,F,,,FF,F,,,,::F:,FF,F,F,,,F,,FFF,,,,F,FFF:FF,F,:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF NM:i:1  MD:Z:7A32       MC:Z:107S44M    AS:i:35 XS:i:34 RG:Z:HNJ53DSXX:1        XA:Z:chr3,+198173832,34M117S,0;chr12,-108091,118S33M,0;chr1,-10354,119S32M,0;chr3,-10005,119S32M,0;chr1,-10258,119S32M,0;chr5,-10004,119S32M,0;chr4,+190123043,32M119S,0;chr1,-10111,119S32M,0;chr1,-180805,119S32M,0;chr2,-181275796,119S32M,0;chr3,-10519,119S32M,0;chr1,-10052,119S32M,0;chr1,-10046,119S32M,0;chr1,-180749,119S32M,0;chr1,-78128742,119S32M,0;chr10,+133787391,36M115S,1;chr3_KI270784v1_alt,-61695,117S34M,0;chr12_GL877875v1_alt,-102106,118S33M,0;
A00804:41:HNJ53DSXX:1:1121:10746:16282  161     chr2    32916252        0       107S44M chr1    9996    0       CCCTAACCCTAACCCTAACCCTAACCCTAACCATAACGCAATAGCGACGTCTAGGCAAAAAGGGTAGGATACGGGGTAGCACTAGCGGTTCCCCGTCGCATAACACTCGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF,,F,F,,,F,,,F,,,,,,FF,,,,::,,,,:,:,:,,:,,,,:,:,,,,,,,,,:,,,,:,,,,,:,,,,,,,,,,:,,,,,:,:,,::::,,:,:::F::F:F,::FF,::::FF,F NM:i:0  MD:Z:44 MC:Z:111S40M    AS:i:44 XS:i:43 RG:Z:HNJ53DSXX:1        SA:Z:chr4,190122941,-,118S33M,0,0;      XA:Z:chr2,+32916514,108S43M,0;chr2,+32916291,108S43M,0;chr2,+32916358,108S43M,0;
A00804:41:HNJ53DSXX:1:1121:10746:16282  161     chr2    32916252        0       107S44M chr1    9996    0       CCCTAACCCTAACCCTAACCCTAACCCTAACCATAACGCAATAGCGACGTCTAGGCAAAAAGGGTAGGATACGGGGTAGCACTAGCGGTTCCCCGTCGCATAACACTCGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF,,F,F,,,F,,,F,,,,,,FF,,,,::,,,,:,:,:,,:,,,,:,:,,,,,,,,,:,,,,:,,,,,:,,,,,,,,,,:,,,,,:,:,,::::,,:,:::F::F:F,::FF,::::FF,F NM:i:0  MD:Z:44 MC:Z:111S40M    AS:i:44 XS:i:43 RG:Z:HNJ53DSXX:1        SA:Z:chr4,190122941,-,118S33M,0,0;      XA:Z:chr2,+32916514,108S43M,0;chr2,+32916358,108S43M,0;chr2,+32916291,108S43M,0;
A00804:41:HNJ53DSXX:1:1121:10746:16282  2225    chr4    190122941       0       118H33M chr1    9996    0       TGGTTAGGGTTAGGGTTAGGGTTAGGGTTAGGG       ,FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF       NM:i:0  MD:Z:33 MC:Z:111H40M    AS:i:33 XS:i:32 RG:Z:HNJ53DSXX:1        SA:Z:chr2,32916252,+,107S44M,0,0;
A00804:41:HNJ53DSXX:1:1121:10746:16282  2225    chr4    190122941       0       118H33M chr1    9996    0       TGGTTAGGGTTAGGGTTAGGGTTAGGGTTAGGG       ,FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF       NM:i:0  MD:Z:33 MC:Z:111H40M    AS:i:33 XS:i:32 RG:Z:HNJ53DSXX:1        SA:Z:chr2,32916252,+,107S44M,0,0;