Please help me with samtools

Question

[seta]

Hi friends,

Please help me with the samtools. I used the command of ./samtools view -b -u /home/seta/software/files1_1/8_test.sam | samtools sort -@ 7 -m 10G -T temp.bam -o 8_test.bam for converting the SAM mapping file (generated by bowtie2) to BAM followed by sorting. But after few second, it gives me an error that sort: invalid option -- 'T'

[bam_header_read] EOF marker is absent. The input is probably truncated.
[bam_header_read] invalid BAM binary header (this is not a BAM file).
Segmentation fault (core dumped)

This is my first experience on samtools, even reading the corresponding manual did not help me much. I would be highly appreciated for your help to figure out the error and solve it.

[Renesh]

See this thread.

Answer 1

[Pierre Lindenbaum]

"(this is not a BAM file)"

Tell samtools to read a SAM, not a BAM with option -S:

./samtools view -S -bu /home/seta/software/files1_1/8_test.sam

[seta]

Thanks Pierre for being here

[seta]

Sorry for many question. I'm trying to convert and sort with

./samtools view -S -bu /home/seta/software /file1_1/8_test.sam | samtools sort -o 8_test.sorted.bam -m 10G -@ 7 8_test.bam

but the error says that:

open: No such file or directory
[bam_sort_core] fail to open file 8_test.sorted.bam

I read the manual several times, but fail to understand the error. please help me how to solve it?

[Renesh]

Try below commands, These should work

./samtools view -S -bu /home/seta/software /file1_1/8_test.sam -o 8_test.bam
./samtools sort -n -m 10G -@ 7 -l 9 8_test.bam 8_test.sorted

[seta]

Thanks for your comment. Actually, I tried the command of

./samtools view -S -bu /home/seta/software/file1_1/8_test.sam -o 8_test.bam | samtools sort -m 10G -@ 4 -l 9 8_test.bam 8_test.sorted.bam

But, it generated the 8_test.bam file with about 97 Gb (the sam file was 120 Gb) and the error that says

open: No such file or directory
[bam_sort_core] fail to open file 8_test.bam

It sound that the error refer to second part of command (sorting), yes? how to solve it?

However, based on what I read, I thought that the bam file should be very smaller than sam file. Please kindly let me know if the 97 GB bam file is usual in my case?

This is my first experience with samtools, so I would like to make sure about the results. Many thanks in advance.

[Devon Ryan]

Use the commands as written by PyPerl. Your incorrect usage of pipes are what caused the problem.

[seta]

Thanks. Actually I used the command of Pyperl as pipes that I prefer. Could you please correct my command with using pipes?

Devon, please kindly tell me resulting 97 GB bam file from 120 GB sam file is usual?

[Devon Ryan]

samtools view -Su /home/seta/software/file1_1/8_test.sam | samtools sort -m 10G -@ 4 - 8_test.sorted

This won't create the large BAM file. Your previous attempt didn't work because you were piping nothing into samtools sort and thereby running it at the same time as the samtools view command (I imagine you got the error almost immediately). The BAM file was so large because it was uncompressed.

[seta]

Unfortunately, it sounds that it did not work as the usage of program was appeared. why??

[Devon Ryan]

Try copying and pasting, or removing the "-m 10G" part. That command will not produce an error if typed correctly and the SAM file exists and can be read.

[seta]

Hi Devon, I copy and pasted your suggested command and removed -m 10G as you kindly told me. But, after few second, these appeared:

[bam_header_read] EOF marker is absent. The input is probably truncated.
[samopen] SAM header is present: 87358 sequences

However, the tool works and after generating a sorted.bam file with about 16 G the work was stopped with the following error:

[bam_header_read] EOF marker is absent. The input is probably truncated.
[bam_sort_core] merging from 177 files...

I really confused what is wrong. Please help me out to solve the problem and finally create a sorted bam file.

[Devon Ryan]

The "EOF" thing can be ignored. I see no actual error, check with samtools -c if the expected number of entries are there.

[seta]

It was OK. Thank you very much for helping me to solve the problem.

[seta]

Unfortunately, it sounds that it did not work as the usage of samtools sort program was appeared. why??

Answer 2

[John Marshall]

You are reading the manual for current samtools (version 1.2), but you are using samtools 0.1.18 or so from several years ago. (This is indicated by the various non-existent options and the "this is not a BAM file" message -- with current samtools, SAM/BAM/CRAM is automatically detected and you don't need view -S.)

So you will want to build and install up to date tools.

[seta]

How you said the version of 0.1.18, I'm working on current version (1.2).

[John Marshall]

The error message that you reported seeing ("invalid BAM binary header (this is not a BAM file)") exists in samtools 0.1.8 through 0.1.19. It does not exist in samtools 1.0 and later.

Therefore you are in fact not running the current version. You may have built the current version, but it is not (always) being used. So you need to use which samtools and check your $PATH settings.

[seta]

Thanks John, just the current version of samtools (1.2) has been installed on server. However, this error was solved. But the another error appeared that refer to using pipes as Devon also mentioned. I posted it in the comments of this post. I would be highly appreciate if you could help me to solve it.