Snakemake bwa mem

Question

[priya.bmg]

Hello

I am trying to run bwa-mem in cluster. I have written the rules in snakemake. I keep getting error in the rule file. It would be of great help, if someone can give suggestions to correct the error

snakefile

samples = ['DE98NGSUKBD117612_1', 'DE71NGSUKBD117613_3']
rule all:
    input: expand("Alignment/{sample}.sam", sample = samples)

rule mapFASTQ:
    input: 
        f1 =  "/scicore/home/cichon/GROUP/test_workflow/data/samples/{sample}.1.fq", 
        f2 = "/scicore/home/cichon/GROUP/test_workflow/data/samples/{sample}.2.fq", 
        ref = "/scicore/home/cichon/GROUP/test_workflow/data/gch38.fa"
    output: temp("Alignment/{sample}.sam")
    threads: 8
    params: time = 120, mem = "8g"
    shell:
     """
     bwa mem -@ {threads} -R "@RG\\tID:{wildcards.sample}\\tSM:{wildcards.sample}" {input.ref} {input.f1} {input.f2} > {output}
     """

Command used to execute above snakefile

snakemake -s snakefile --use-conda --cluster 'sbatch -t {params.time} --mem={params.mem} -c {threads}' -j 10

Error:

Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cluster nodes: 10
Job stats:
job         count    min threads    max threads
--------  -------  -------------  -------------
all             1              1              1
mapFASTQ        2              1              1
total           3              1              1

Select jobs to execute...

[Tue Aug 31 15:46:49 2021]
rule mapFASTQ:
    input: /scicore/home/cichon/GROUP/test_workflow/data/samples/DE98NGSUKBD117612_1.1.fq, /scicore/home/cichon/GROUP/test_workflow/data/samples/DE98NGSUKBD117612_1.2.fq, /scicore/home/cichon/GROUP/test_workflow/data/gch38.fa
    output: Alignment/DE98NGSUKBD117612_1.sam
    jobid: 1
    wildcards: sample=DE98NGSUKBD117612_1
    resources: tmpdir=/scratch

Submitted job 1 with external jobid 'Submitted batch job 58209199'.

[Tue Aug 31 15:46:53 2021]
rule mapFASTQ:
    input: /scicore/home/cichon/GROUP/test_workflow/data/samples/DE71NGSUKBD117613_3.1.fq, /scicore/home/cichon/GROUP/test_workflow/data/samples/DE71NGSUKBD117613_3.2.fq, /scicore/home/cichon/GROUP/test_workflow/data/gch38.fa
    output: Alignment/DE71NGSUKBD117613_3.sam
    jobid: 2
    wildcards: sample=DE71NGSUKBD117613_3
    resources: tmpdir=/scratch

Submitted job 2 with external jobid 'Submitted batch job 58209201'.
[Tue Aug 31 15:46:59 2021]
Error in rule mapFASTQ:
    jobid: 2
    output: Alignment/DE71NGSUKBD117613_3.sam
    shell:

     bwa mem -@ 8 -R "@RG\tID:DE71NGSUKBD117613_3\tSM:DE71NGSUKBD117613_3" /scicore/home/cichon/GROUP/test_workflow/data/gch38.fa /scicore/home/cichon/GROUP/test_workflow/data/samples/DE71NGSUKBD117613_3.1.fq /scicore/home/cichon/GROUP/test_workflow/data/samples/DE71NGSUKBD117613_3.2.fq > Alignment/DE71NGSUKBD117613_3.sam

        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
    cluster_jobid: Submitted batch job 58209201

Error executing rule mapFASTQ on cluster (jobid: 2, external: Submitted batch job 58209201, jobscript: /scicore/home/cichon/GROUP/test_workflow/bwa_mem_snakemake/.snakemake/tmp.i5j1sgtc/snakejob.mapFASTQ.2.sh). For error details see the cluster log and the log files of the involved rule(s).
[Tue Aug 31 15:47:09 2021]
Error in rule mapFASTQ:
    jobid: 1
    output: Alignment/DE98NGSUKBD117612_1.sam
    shell:

     bwa mem -@ 8 -R "@RG\tID:DE98NGSUKBD117612_1\tSM:DE98NGSUKBD117612_1" /scicore/home/cichon/GROUP/test_workflow/data/gch38.fa /scicore/home/cichon/GROUP/test_workflow/data/samples/DE98NGSUKBD117612_1.1.fq /scicore/home/cichon/GROUP/test_workflow/data/samples/DE98NGSUKBD117612_1.2.fq > Alignment/DE98NGSUKBD117612_1.sam

        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
    cluster_jobid: Submitted batch job 58209199

Error executing rule mapFASTQ on cluster (jobid: 1, external: Submitted batch job 58209199, jobscript: /scicore/home/cichon/GROUP/test_workflow/bwa_mem_snakemake/.snakemake/tmp.i5j1sgtc/snakejob.mapFASTQ.1.sh). For error details see the cluster log and the log files of the involved rule(s).
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /scicore/home/cichon/GROUP/test_workflow/bwa_mem_snakemake/.snakemake/log/2021-08-31T154633.723402.snakemake.log

Thanks

[Jeremy Leipzig]

show us what the snakemake.log says

[priya.bmg]

log contains the same error message which I have copied above in the question

Answer 1

[curious]

this is an error with your shell command in a rule. typically I test run w/o using slurm first then if i get any error like this i run again w/ --printshellcmds arg then manually enter whatever it prints manually in the terminal open to the top level dir of the project

Answer 2

[N15]

Is "-@" standard for bwa mem? I haven't seen that before

[ATpoint]

Seems like OP is confusing it with -t. The -@ is the thread option in samtools.

[priya.bmg]

I copied the script from this link: https://www.sichong.site/2019/10/17/how-to-run-snakemake-pipeline-on-hpc/

[ATpoint]

Random scripts in the internet are not free from flaws, this is why you should always test commands manually in a terminal on a small example set of reads to ensure it runs properly before wrapping into a workflow manager or script.

$ bwa mem -@ 2 test.fa test.fq
mem: illegal option -- @

$ bwa mem -t 2 test.fa test.fq
[M::bwa_idx_load_from_disk] read 0 ALT contigs
@SQ SN:chr1 LN:38
@PG ID:bwa  PN:bwa  VN:0.7.17-r1188 CL:bwa mem -t 2 test.fa test.fq
[M::process] read 1 sequences (19 bp)...
(...)