samtools sort- no output file generated

Question

[mirza]

Hi,

I am doing variant calling and am using samtools sort o sort my .bam files. I have 40 threads in my workstation [Thread(s) per core:2; Core(s) per socket:10; Socket(s):2]. So, I used the following command

$ samtools sort brs11b.bam -o brs11_sorted.bam -@ 10

-@= no. of threads

I could see the temporary files being generated but finally I don't have any output file and the terminal looks like this,

I removed the -@ argument and used the following command,

$ samtools sort brs11b.bam -o brs11_sorted.bam

My input file is of 7.1GB and now, I have more than 50 small sorted.bam files. Is it right? How should I proceed, should I merge all these files in one?

[nita_gaura]

Hello,

Could you may be tell me how did you downloaded samttols1..3.1?

[ATpoint]

Please open a new question, add necessary details and, with all due respect, put some effort in spelling and typos. Most of us are non-native english speakers, but you can at least spell the tool correctly when asking unpaid volunteers for help.

Answer 1

[IP]

Hi,

You are not giving the arguments correctly to samtools sort.

If you type samtools sort in the terminal you will see the usage: Usage: samtools sort [options...] [in.bam]

In yoy command, you are specifying first the bam to sort, and then you are giving the options. Which is incorrect. You have to specify first the arguments and then give the input.bam file.

The following command should work for you:

samtools sort  -o brs11_sorted.bam -@ 10 brs11b.bam

P.D: This will work if you are using samtools last version. The order of the samtools sort command was changed in version 1.3

[mirza]

Yes, I am using version 1.3.1 and when I used the command you are suggsting, it says,

open: No such file or directory [bam_sort_core] fail to open file sortbrs13.bam.

thats why I used,

$ samtools sort brs11b.bam -o brs11_sorted.bam

and it worked. But, now I have another query, My input file is of 7.1GB and after running my command (samtools sort brs11b.bam -o brs11_sorted.bam) now, I have more than 50 small sorted.bam files. Is it right? How should I proceed, should I merge all these files in one?

[mastal511]

Samtools sort usually merges all the output files when it finishes, and you just get one output file. I think samtools could be running out of memory and stopping before it has completed the sort.

[GenoMax]

How much memory do you have? If you use more than one thread then each thread requires a certain amount of RAM (~4G seems common). So you should add -m 4G and reduce number of threads accordingly, if you do not have ~40G of free RAM available.

[mirza]

The free -m command tells me that,

I have total mem= 257726, shared mem= 23400 and free mem= 10275 and swap= 124999 and free swap= 124999

now what should I do, how much memory and threads I should give in the command?

[GenoMax]

You can confirm with free -mh but looks like you have 256G of RAM so -@ 10 -m 4G should be fine.

It is odd that you were not able to get the sorting to complete the first time around. As @mastal pointed out, on successful completion of sorting all those small temp sort files should be deleted automatically.

[mirza]

Yes, I have 256Gb RAM and right now, 245G available memory, yet the sorting didn't complete when I used your suggestion i.e.

-@ 10 -m 4G

should I reduce the no. of threads or what should I do??

[GenoMax]

I have a feeling that you have a corrupt BAM file. Since you have plenty of resources available there should be no issues with sorting the file. I am going to suggest that you validate your BAM file using this.

[mirza]

I had the same feeling coz' after generating a few temp files, gibberish characters start running in my terminal (I did add the pic in my original question above). thanks. I'll validate and let you know.

Answer 2

[mirza]

I am posting my solution to the issue. It might help a newbie like me facing the same problem. I deleted all older versions and kept only samtools1.3.1 containing the executable file, made sure that this directory is in the PATH and instead of using just,

$ samtools sort

I gave the complete path to samtools executables, i.e.

$ /home/bin/samtools1.3.1/samtools sort

and it's working fine now.