How to fix .bam file that was diagnosed as "MISSING_READ_GROUP" by Picard ValidateSamFile

Question

[freddiejung]

Hi, I am using Pilon to polish and elongate my PacBio assembly.

But I realized Pilon always generate error like shown in the bottom.

I found that one of my .bam file have something wrong and I performed Picard ValidateSamFile.

This program diagnosed this .bam is missing group(↓).

## HISTOGRAM    java.lang.String
Error Type  Count
ERROR:MISSING_READ_GROUP    1
WARNING:RECORD_MISSING_READ_GROUP   22324694

How to fix this .bam file? Any comments would be appreciated.

Best, Jung

---

Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.simontuffs.onejar.Boot.run(Boot.java:340)
at com.simontuffs.onejar.Boot.main(Boot.java:166)
Caused by: java.lang.UnsupportedOperationException: Cannot query stream-based BAM file
at htsjdk.samtools.BAMFileReader.query(BAMFileReader.java:410)
at htsjdk.samtools.SamReader$PrimitiveSamReaderToSamReaderAdapter.query(SamReader.java:498)
at htsjdk.samtools.SamReader$PrimitiveSamReaderToSamReaderAdapter.query(SamReader.java:503)
at htsjdk.samtools.SamReader$PrimitiveSamReaderToSamReaderAdapter.queryOverlapping(SamReader.java:365)
at org.broadinstitute.pilon.BamFile.readsInRegion(BamFile.scala:329)
at org.broadinstitute.pilon.BamFile.recruitBadMates(BamFile.scala:357)
at org.broadinstitute.pilon.GapFiller$$anonfun$recruitJumps$1.apply(GapFiller.scala:380)
at org.broadinstitute.pilon.GapFiller$$anonfun$recruitJumps$1.apply(GapFiller.scala:379)
at scala.collection.immutable.List.foreach(List.scala:381)
at org.broadinstitute.pilon.GapFiller.recruitJumps(GapFiller.scala:379)
at org.broadinstitute.pilon.GapFiller.recruitReads(GapFiller.scala:391)
at org.broadinstitute.pilon.GapFiller.assembleAcrossBreak(GapFiller.scala:51)
at org.broadinstitute.pilon.GapFiller.fixBreak(GapFiller.scala:45)
at org.broadinstitute.pilon.GenomeRegion$$anonfun$identifyAndFixIssues$4.apply(GenomeRegion.scala:383)
at org.broadinstitute.pilon.GenomeRegion$$anonfun$identifyAndFixIssues$4.apply(GenomeRegion.scala:381)
at scala.collection.immutable.List.foreach(List.scala:381)
at org.broadinstitute.pilon.GenomeRegion.identifyAndFixIssues(GenomeRegion.scala:381)
at org.broadinstitute.pilon.GenomeFile$$anonfun$processRegions$4.apply(GenomeFile.scala:120)
at org.broadinstitute.pilon.GenomeFile$$anonfun$processRegions$4.apply(GenomeFile.scala:109)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at scala.collection.parallel.ParIterableLike$Foreach.leaf(ParIterableLike.scala:972)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply$mcV$sp(Tasks.scala:49)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$class.tryLeaf(Tasks.scala:51)
at scala.collection.parallel.ParIterableLike$Foreach.tryLeaf(ParIterableLike.scala:969)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.internal(Tasks.scala:169)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.internal(Tasks.scala:443)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.compute(Tasks.scala:149)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:443)
at scala.concurrent.forkjoin.RecursiveAction.exec(RecursiveAction.java:160)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

[oriolebaltimore]

Hi Pierre and other experts,

I used Picard ValidateSamFile and found the following errors. about mate not found and missing read group.

I used tophat instead of BWA. Is there any difference between Tophat and BWA derived BAM files.

Also what to do abut Mate not found in a PE reads. It should have been a warning instead of Error. Any significance of this error wrt. RNA-Seq Tophat BAM file.

Thanks Adrian

## HISTOGRAM    java.lang.String
Error Type      Count
ERROR:MATE_NOT_FOUND    3157873
ERROR:MISSING_READ_GROUP        1
WARNING:RECORD_MISSING_READ_GROUP       140564013

[finswimmer]

Hello oriolebaltimore ,

You should know that the old 'Tuxedo' pipeline of Tophat(2) and Cufflinks is no longer the "advisable" tool for RNA-seq analysis. The software is deprecated/ in low maintenance and should be replaced by HISAT2, StringTie and ballgown. See this paper: Transcript-level expression analysis of RNA-seq experiments with HISAT, StringTie and Ballgown. There are also other alternatives, including alignment with STAR and bbmap, or pseudo-alignment using salmon.

Please stop using Tophat https://t.co/Es4ohxOEyx Cole and I developed the method in *2008*. It was greatly improved in TopHat2 then HISAT & HISAT2. There is no reason to use it anymore. I have been saying this for years yet it has more citations this year than last #methodsmatter

— Lior Pachter (@lpachter) December 2, 2017

Answer 1

[Pierre Lindenbaum]

you need to add some read group to your BAM, either using your originial mapper , or using picard https://broadinstitute.github.io/picard/command-line-overview.html#AddOrReplaceReadGroups

furthemore, this error:

Caused by: java.lang.UnsupportedOperationException: Cannot query stream-based BAM file

means that you're trying to random access a stream (stdin) to get a specific region, which is not possible.

Answer 2

[mittu1602]

Hi Jung, Missing read groups occur because either your machine could not capture the reads, or the quality cutoffs are too high where those reads were removed. You can try decreasing your quality score.

This is best to my knowledge.

[WouterDeCoster]

Hi mittu1602,

Thank you for contributing, but I have the impression you are confused. There is no connection between sam-format read groups and the sequencing machine or quality cutoffs.

Cheers,
Wouter