sam to bam conversion problem

Question

[jinxinhao1988]

I use bwa mem to align reads. And got several log info at the head of the samfile. Like this:

[M::main_mem] read71286 sequences(10000200 bp)...
[M::mem_pestat]# candidate unique pairs for(FF, FR, RF, RR):(0,5009,0,0) 
[M::mem_pestat]skip orientation FF as there are not enough pairs 
[M::mem_pestat]analyzing insert size distribution for orientation FR...
[M::mem_pestat](25,50,75) percentile:(160,170,174)
[M::mem_pestat]low and high boundaries for computing mean and std. 
[M::mem_pestat]mean and std. dev:(165.99,9.13)
[M::mem_pestat]low and high boundaries for proper pairs:(118,216) 
[M::mem_pestat]skip orientation RF as there are not enough pairs 
[M::mem_pestat]skip orientation RR as there are not enough pairs 
[M::mem_pestat]processed71286 reads in6.210 CPU sec,6.219 real sec

Then I want to convert the sam file into bam, when I use samtools view to do that, It has an error :

# [samopen] no @SQ lines in the header.
# [sam_read1] missing header? Abort!

so I cut the first 11 lines in the head, and then run it again, and I got another error,

# [samopen] SAM header is present: 25 sequences.
# Parse error at line 71433: sequence and quality are inconsistent

So there are mutiple this log info scattered in the sam file ,so I can not convert into bam.

How could this happen? is there other ways I can convert to bam file?

Thank you in advance.

[WouterDeCoster]

Please add the commands you used for bwa mem.

[jinxinhao1988]

I use the basic command of mem align.

bwa mem test.fasta 1.fq 2.fq > aln_pe.sam

[WouterDeCoster]

Sure you didn't use nohup or '&>` or something similar? Because this looks like combining stderr and stdout together.

[jinxinhao1988]

I use nohup and & in the bwa mem running. but don't use in the samtools running cause I want to see the error info. Sorry I don't quite understand the meaning of stderr and stdout combination.

[WouterDeCoster]

I use nohup and & in the bwa mem running.

Well there's your problem.

I use the basic command of mem align.

bwa mem test.fasta 1.fq 2.fq > aln_pe.sam

Next time, when someone asks you about the commands you used, give the commands you used and not a piece of it. You left out important information.

[Tm]

WouterDeCoster means standard log message displayed by BWA got concatenated with your standard .sam output file instead of adding to nohup file. Thus you were able to see BWA log message as first 11 lines of your .sam file.

I think you should rerun the BWA mem again without nohup or anything similar followed by samtools view.

And again if you get this error

[samopen] SAM header is present: 25 sequences.

Parse error at line 71433: sequence and quality are inconsistent

then there is surely something wrong in your reads file as error "sequence and quality are inconsistent" means number of total bases in your sequence is not same as number of quality representing those bases

Answer 1

[mittu1602]

Try following the commands below with same parameters:\

bwa mem ucsc.hg19.fasta 1.fastq 2.fastq -M   > test.alnpe.sam
samtools view -Sb test.alnpe.sam >test_bam.alnpe.bam
samtools "sort" -@ 24 test_bam.alnpe.bam >test_bam.alnpe.sort.bam

Good luck!!

[WouterDeCoster]

Simplified, without intermediate files:

bwa mem ref.fasta R1.fastq R2.fastq | samtools sort -o sorted_result.bam -

[gubrins]

sorry, but how this samtools command is transforming a SAM to a BAM file? You don't use view, just sort. I would appreacite the help, thanks!

[GenoMax]

Samtools is smart and will figure out the output format needed based on the extension (.bam) used.

[gubrins]

last question, what is the - symbol at the end? If I do that, a lot of temporary files are created

[GenoMax]

Temp files are for the sorting process. They will be deleted once the sorting is done. - indicates STDIN i.e. read from there. bwa mem writes to STDOUT, which becomes STDIN for the samtools.

Answer 2

[Tm]

you have not mentioned about the parameters you are using for sam to bam conversion, however you can try using simple basic command:

samtools view -bS in.sam > out.bam

Or you can check out similar post addressing the same issue here

[jinxinhao1988]

The bwa align I use basic command line : bwa mem test.fasta 1.fq 2.fa > aln_pe.sam

and the conversion I use exact like the one you write in the comments.

[mittu1602]

one of your file is in fastq and the other in fasta? or just a typo error?

[jinxinhao1988]

Sorry, that's a typo error. I am trying to run the bwa mem without nohup as toralmanvar said. I think maybe that's the problem.