Parallelizing Bwa On Multiple Cpus

Question

[Alex Paciorkowski]

Has anyone successfully parallelized BWA alignment on multiple CPUs? Do the reads contained in a fastq file depend upon one another, or can one break the fastq file up, align the pieces using multiple CPUs, and then reassemble the resulting sam files? My suspicion is the answer is no, but I don't know and have not found anything in the BWA documentation. Does anyone have any experience with (parallel BWA)? Thanks.

Answer 1

[Sean Davis]

Yes, you can split the reads into multiple fastq files, align, and then merge results. The reads are aligned independently of each other.

[Alex Paciorkowski]

Thanks, Sean. We've just indeed confirmed this. The split reads can be concatenated without a problem after the alignment.

[Sean Davis]

Just to clarify for posterity sake, the FASTQ files can be split into chunks, aligned on separate machines independently, and the results merged; this is equivalent to aligning one big FASTQ file.

Answer 2

[Zev.Kronenberg]

Check out the -t [n-cpu] option. It allows you to use multiple processors... Is that what you are going for?

[Alex Paciorkowski]

Thanks, Zev. I believe the -t option refers to multithreading, not parallelizing across multiple processors. We're just investigating how to make efficient use of multiple nodes on a cluster, and identifying which stages of NGS alignment/variant calling can be truly parallelized.

[Chris Miller]

I think he wants to split among multiple physical processors or machines.

[Zev.Kronenberg]

@Chris_Miller: What do you think the time trade off is for splitting the fastqs and then doing bwa?

[Zev.Kronenberg]

Oh, okay. I have also done this. I used template toolkit to write automated PBS submission scripts for whole genomic data (across many fastq). On our cluster I hauled through whole genome data. I was using 20 nodes with 12 CPU per core. I ran into one hitch: I didn't have priority on the cluster so I had to write a script to check that the alignments finished. May you have 'publication in premiere Scientific journal'.

Answer 3

[Sukhi Singh]

Try pBWA specifically made for that.

Cheers