I am converting the fq.gz. files (which are the results of the mgi study) to bam files to view on igv.

Question

[canavar]

Hey everyone, before i start apologies for the inconvenience cause of my wrong or inappropriate use of terms. I take some fails of bwa mem lately. As i mentioned in the post title i'm working with fastq files. And i get:

[E::main_mem] fail to open file '/mnt/MGI-R4/L01/(sample_name)_(barcode_number)_1.fq.gz'. 

and

[main_samview] fail to read the header from "-".

And here is my command:

bwa mem -M -t 8 ~/genome/hg19.fa /mnt/MGI-R4/L01/(samplename)_(lane)_(barcodenumber)_1.fq.gz /mnt/MGI-R4/L01/(samplename)_(barcodenumber)_2.fq.gz | samtools view -o (samplename)_(lane)_(barcodenumber).bam -

i know that my command works because i used to convert. Can you please help me?

[Pierre Lindenbaum]

is it the real command ? I mean, are the files literally named "(samplename)_(lane)_(barcodenumber)_1.fq.gz" (?!!)

[GenoMax]

Even if they are not canavar seems to be using them as such in command line (unless they have gone to the trouble of obfuscating the file name even in the error message).

canavar : ( ) are place holders. You need to use real file names for the command to work. For example with a sample named Sample1 your command will look like this:

bwa mem -M -t 8 ~/genome/hg19.fa /mnt/MGI-R4/L01/Sample1_1.fq.gz /mnt/MGI-R4/L01/Sample1_2.fq.gz | samtools view -o Sample1.bam -

[canavar]

I am sorry i didn't really know that file names matters. There's nothing important to hide. I thought this makes it easier. Sorry again. Let me post it all again:

bwa mem -M -t 8 ~/genome/hg19.fa /mnt/MGI-R4/L01/111414_L01_41_1.fq.gz /mnt/MGI-R4/L01/111414_L01_41_2.fq.gz | samtools view -o 111414_L01_41_1.bam -

[GenoMax]

There is no need to view. Following should work to create a sorted BAM directly.

bwa mem -M -t 8 ~/genome/hg19.fa /mnt/MGI-R4/L01/111414_L01_41_1.fq.gz /mnt/MGI-R4/L01/111414_L01_41_2.fq.gz | samtools sort -o 111414_L01_41_1.bam -

[canavar]

Thank you so much, i'll try this command.

[canavar]

As i mentioned before i am really a beginner. I had no such purpose. And thank you all for your answers already.

[canavar]

by the way i am using this command to sort:

samtools sort -o 111414_L01_41_1.sorted.bam 111414_L01_41_1.bam

and it goes like this:

samtools sort --threads 7 -o 111414_L01_41_1.sorted.bam 111414_L01_41_1.bam

[canavar]

to index the file:

samtools index 111414_L01_41_1.sorted.bam 

and after that there comes rm and cp phases

[GenoMax]

Please use ADD REPLY/ADD COMMENT when replying to existing posts. Do not use SUBMIT ANSWER unless you are posting a new answer for original question.

You do not need to sort the BAM file if you use the command above since it will already be sorted. samtools is smart about that. You will just need to Index the resulting sorted BAM file.

[canavar]

Thank you and apologies.. Yes, you are right there's no need to sort for bam files. I am sorting files for bai files. And i have fix the error btw i guess that was related to my 1tb storage. Thanks alot again. Have a healty day kind sir.

[jkbonfield]

You can also now add --write-index to modern Samtools versions so the index is written at the same time as the BAM (etc) file. It needs to be modern though! We've fixed a couple bugs in that code when combined with multi-threading. (It can be quite challenging to get indices to work there due to migrating reads to the next block if they don't fit.)