Using parallel with Mem

Question

[always_learning]

Hi All.

We are trying to us MEM with GNU parallel with paired fastQ files, but its saying that can't accept more then one file. What could be the possible reasons? What if we concatante both file as single one and then use that with mem? Does Mem command support multi threading too ?

Thanks

Syed

[Pierre Lindenbaum]

show us the cmd-line please.

[always_learning]

this exact commant we are using:

${bwa} mem -t 8 -R \\"@RG\\tID:${ursId}_pairend\\tSM:${sampleName}\\tLB:${libraryId}\\tPL:${platform}\\" ${refGenomeFastaFile} \${seqReadsDir}/${ursId}_pairend1.fastq.gz ${seqReadsDir}/${ursId}_pairend2.fastq.gz

[Pierre Lindenbaum]

but how do you invoke parallel ?

[always_learning]

Sorry as I didn't able to post complete command. Here it goes as we are trying to convert SAM files to BAM files at same time only

${bwa} mem -t 8 -R "@RG\\tID:${ursId}_pairend\\tSM:${sampleName}\\tLB:${libraryId}\\tPL:${platform}" ${refGenomeFastaFile} \
${seqReadsDir}/${ursId}_pairend1.fastq.gz ${seqReadsDir}/${ursId}_pairend2.fastq.gz | ${samtools} view -bhS - > ${bamDir}/${ursId}.bam" > \
${seqReadsDir}/${ursId}_pairend.txt; parallel -a ${seqReadsDir}/${ursId}_pairend.txt&#39

Answer 1

[Matt Shirley]

This command is telling bwa mem to use 8 threads (`-t`) so you will have some level of parallelism. Your command above does not invoke GNU parallel, and to do so you would use something like:

parallel bwa mem -t 8 -R "@RG\t@ID:blahblahblah" reference.fasta {1} {2} > {1.}.sam ::: *R1.fastq.gz ::: *R2.fastq.gz

See here: https://www.gnu.org/software/parallel/man.html#example__use_multiple_inputs_in_one_command and here: Gnu Parallel - Parallelize Serial Command Line Programs Without Changing Them

[Ron]

Hi Matt,

I am using gnu parallel as explained by you for cufflinks.Here is my post.

Cufflinks output with filename prefix

I want to add the prefix to my cufflink output files using filenames from bam samples.

Answer 2

[Korsocius]

There is snytax for parallel with bwa mem, but I can't load ID and SM from INPUT. Do you know, where could be a problem?

ls *R1_001.fastq | parallel 'bwa mem -k 19 -A 1 -B 4 -O 6 -L 5 -R '@RG\tID:'{.}'\tSM:'{.}'\tLB:'Trusight_custom_amplicon_CARR'\tPL:'ILLUMINA'\tPI:150' $REFERENCE {} {= s/_R1_001/_R2_001/ =} > {= s/_R1_001.fastq/.sam/ =}'

[Pierre Lindenbaum]

ask this as a new question please.

[Korsocius]

Paralle bwa mem - problem with -R argument for ID and SM