Snakemake - Missing output files

Question

[am29]

Hello, I've been trying to run fastGWA (GCTA) with Snakemake. I'm keeping getting the same error, for the same commands that work when I submit them with slurm without Snakemake, as a shell script.

Here is my command:

rule all:
        input:
                'results/fastGWA_Chr1'

rule fastgwa:
        input:
                bfile='Chr1.bed',
        pheno='phenotypes',
        grmid='grm_sparse.grm.id',
        grmsp='grm_sparse.grm.sp'
        output:
                out='results/fastGWA_Chr1'
        log: 'logs/fastgwa/fastGWA_Chr1.log'
        benchmark: 'logs/fastgwa/fastGWA_Chr1.bench'
        threads: 10
        resources:
                partition = "medium",
                mem_gb=50,
                time=60,
                temp='${TMP_LOCAL}'
        params:
                bfile="Chr1",
        grm="grm_sparse"
        shell:
                """
                (
                /gcta64 \
        --bfile {params.bfile} \
        --grm-sparse {params.grm} \
        --fastGWA-mlm \
        --pheno {input.pheno} \ 
        --maf 0.01 \
        --thread-num {threads} \
        --out {output.out}
                )
                """

The error I keep getting is:

[Thu Sep 12 11:22:03 2024] rule fastgwa:
    input: Chr1.bed, Chr1.bim, Chr1.fam, phenotypes, grm_sparse.grm.id, grm_sparse.grm.sp
    output: results/fastGWA_Chr1
    log: logs/fastgwa/fastGWA_Chr1.log
    jobid: 1
    benchmark: logs/fastgwa/fastGWA_Chr1.bench
    reason: Missing output files: results/fastGWA_Chr1
    threads: 10
    resources: mem_mb=86066, mem_mib=82079, disk_mb=86066, disk_mib=82079, tmpdir=<TBD>, partition=medium, mem_gb=50, time=60, temp=${TMP_LOCAL}

When I look into log file from GCTA I get:

Error: missing the --out option
An error occurs, please check the options or data

Answer 1

[dariober]

The error I'm keep getting is:

    [Thu Sep 12 11:22:03 2024] rule fastgwa: input: Chr1.bed, Chr1.bim, Chr1.fam, phenotypes, grm_sparse.grm.id, grm_sparse.grm.sp output: results/fastGWA_Chr1 log: logs/fastgwa/fastGWA_Chr1.log jobid: 1 benchmark: logs/fastgwa/fastGWA_Chr1.bench reason: Missing output files: results/fastGWA_Chr1 threads: 10 resources: mem_mb=86066, mem_mib=82079, disk_mb=86066, disk_mib=82079, tmpdir=<TBD>, partition=medium, mem_gb=50, time=60, temp=${TMP_LOCAL}

This is not an error, it is snakemake informing you why rule fastgwa needs to be executed (see reason: Missing output files...)

---

Error: missing the --out option An error occurs, please check the options or data

This appears to be the actual error. Your shell directive looks a bit odd to me and in general also the formatting of your snakefile, at leats as posted here. Try with this:

shell:
    r"""
    gcta64 --bfile {params.bfile} --grm-sparse {params.grm} --fastGWA-mlm --pheno {input.pheno} --maf 0.01 --thread-num {threads} --out {output.out}
    """

You may have a space after a newline break \ and this trips bash. In fact I see a trailing space in line --pheno {input.pheno} \

Also, I'm not sure why/if you need to wrap in round brackets and use /gcta64 instead of gcta64 (without /).

[am29]

This worked! Thank you!

Answer 2

[tim.booth]

For any vim users reading this, add this to your .vimrc:

highlight default ExtraWhitespace ctermbg=darkgreen guibg=#216000
match ExtraWhitespace '\s\+$'

No more invisible whitespace on the end of lines! No more of these nasty bugs!