Snakemake error in bwahaha pipeline

Question

[kamanovae]

Hi! My snakemake code is throwing this error.

SyntaxError in line 53 of /storage1/GatkBwaTest/SnakemakeDir/tumor_pipeline/snakefile:
invalid syntax. Perhaps you forgot a comma?

53 line is first line in shell. I suspect it has something to do with the characters in the group. I would be glad for any help!

rule bwamem:
    input:
        ref=config["ref"], forward_reads=rules.repair.output.forward_fastrepair,
        reverse_fastrepair=rules.repair.output.reverse_fastrepair
    output:
        bam=config["out"] + "/{sample}/{sample}.sorted.mkdup.bam"
    params:
        bwa_threads=config["bwamem_threads"], fixmate_threads=config["fixmate_threads"],
        per_thread_sort_mem=config["per_thread_sort_mem"], markdup_threads=config["markdup_threads"]
    log:
        bwa=config["out_logs"]+"/{sample}/{sample}.bwa.log", fixmate=config["out_logs"]+"/{sample}/{sample}.fixmate.log",
        sort=config["out_logs"]+"/{sample}/{sample}.sort.log", markdup=config["out_logs"]+"/{sample}/{sample}.markdup.log"
    shell:
        " {bwa_mem} mem -t {params.bwa_threads} {input.ref} {input.forward_reads} {input.reverse_fastrepair} "
        " -R \'@RG\\tID:{wildcards.sample}\\tPU:unit1\\tSM:{wildcards.sample}\\tPL:Illumina\\tLB:lib1\' 2>{log.bwa_threads} | "
        "  samtools fixmate -@ {params.fixmate_threads} -m - -  2>{log.fixmate_threads}| "
        " samtools sort -T "sort_temorary" -@ {params.sort_threads} -m {params.per_thread_sort_mem} 2>{log.sort}| "
        " samtools markdup -@ {params.markdup_threads} - {output.bam} 2>{log.markdup} "

Thank you

[antonioggsousa]

Hi,

I think you need to use single-quotes in the sort_temorary (add instead ...sort -T 'sort_temorary' -@...) parameter and not double-quotes. Double-quotes define the beginning and end of the shell commands in this case.

I hope this helps,

António

[kamanovae]

Thank you! But now I have another error without description(

Error in rule bwamem:
    jobid: 3
    output: out//NIST7035_TAAGGCGA/NIST7035_TAAGGCGA.sorted.mkdup.bam
    log: out/out_logs//NIST7035_TAAGGCGA/NIST7035_TAAGGCGA.bwa.log, out/out_logs//NIST7035_TAAGGCGA/NIST7035_TAAGGCGA.fixmate.log, out/out_logs//NIST7035_TAAGGCGA/NIST7035_TAAGGCGA.sort.log, out/out_logs//NIST7035_TAAGGCGA/NIST7035_TAAGGCGA.markdup.log (check log file(s) for error message)
    shell:
         /storage1/GatkBwaTest/bin/bwa-mem2-2.2.1_x64-linux/bwa-mem2 mem -t 4 /storage1/GatkBwaTest/SnakemakeInput/ref.fa out//NIST7035_TAAGGCGA/NIST7035_TAAGGCGA_fastrepair_R1.fastq out//NIST7035_TAAGGCGA/NIST7035_TAAGGCGA_fastrepair_R2.fastq  -R '@RG\tID:NIST7035_TAAGGCGA\tPU:unit1\tSM:NIST7035_TAAGGCGA\tPL:Illumina\tLB:lib1' 2>out/out_logs//NIST7035_TAAGGCGA/NIST7035_TAAGGCGA.bwa.log |   samtools fixmate -@ 1 -m - -  2>out/out_logs//NIST7035_TAAGGCGA/NIST7035_TAAGGCGA.fixmate.log|  samtools sort -T 'sort_temorary' -@ sort_threads -m 10 2>out/out_logs//NIST7035_TAAGGCGA/NIST7035_TAAGGCGA.sort.log|  samtools markdup -@ 4 - out//NIST7035_TAAGGCGA/NIST7035_TAAGGCGA.sorted.mkdup.bam 2>out/out_logs//NIST7035_TAAGGCGA/NIST7035_TAAGGCGA.markdup.log
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /storage1/GatkBwaTest/SnakemakeDir/tumor_pipeline/.snakemake/log/2022-02-15T175028.487260.snakemake.log

[antonioggsousa]

Not sure if the following mistake is a parsing problem of the error message or it is actually a mistake, but some of your paths have two slashes // like in here: out/out_logs//NIST7035_TAAGGCGA/NIST7035_TAAGGCGA.markdup.log. Just edit your code to remove one slash character.

The error seems to be related with the command given. So, it does not seem related with snakemake anymore.

Running snakemake with the option --verbose might provide further details about what is going on.

I hope this helps,

António

Answer 1

[dariober]

A couple of suggestions that may not fix your issue but still worth considering.

Instead of concatenating strings in the shell command use raw string triple quotes (r""" ... """), like:

shell:
    r"""
    {bwa_mem} mem ... -R '@RG\tID:{wildcards.sample}\tPU:unit1\tSM:{wildcards.sample}\tPL:Illumina\tLB:lib1' 2>{log.bwa_threads} \
    | etc...
    """

In this way you don't need to escape special characters and you are less likely to make syntax errors.

Another thing, I prefer to use os.path.join(dir, filename) to make paths to files rather than concatenating strings (dir + '/' + filename). In this way you avoid double slashes that would confuse snakemake.

[kamanovae]

thanks for the advice!

I realized that the cause of the error in the snakemake, but I don't know how to fix it For some reason the rule bwamem does not start the samtools markdup

[main] unrecognized command 'markdup'

[dariober]

That is probably unrelated to snakemake... Try to show more contex. Is there any chance you are using a version of samtools so old that it doesn't have markdup? (Unlikely, I think)

[kamanovae]

Еhat's right, I used the old version of the samtools

Thanks!