how to speed up snakemake workflow

Question

[dew]

Dear guys,

Are there ways to speed up the snakemake rnaseq workflow? After 4h running, still in the install package status...

Thanks a lot!strong text

Downloading and installing remote packages.
Environment for https://github.com/snakemake/snakemake-wrappers/raw/0.77.0/bio/star/index/environment.yaml created (location: .snakemake/conda/ac8f19afa7afaa9828ae54d2d1e62573_)
Creating conda environment https://github.com/snakemake-workflows/rna-seq-star-deseq2/raw/v1.2.0/workflow/envs/gffutils.yaml...
Downloading and installing remote packages.
Environment for https://github.com/snakemake-workflows/rna-seq-star-deseq2/raw/v1.2.0/workflow/envs/gffutils.yaml created (location: .snakemake/conda/533adc3d3517303afab58bc67cbde5fe_)
Creating conda environment https://github.com/snakemake-workflows/rna-seq-star-deseq2/raw/v1.2.0/workflow/envs/rseqc.yaml...
Downloading and installing remote packages.
Environment for https://github.com/snakemake-workflows/rna-seq-star-deseq2/raw/v1.2.0/workflow/envs/rseqc.yaml created (location: .snakemake/conda/ad4d7671d9f68682c910f27087482a51_)
Creating conda environment https://github.com/snakemake-workflows/rna-seq-star-deseq2/raw/v1.2.0/workflow/envs/deseq2.yaml...
Downloading and installing remote packages.

[Eric Lim]

I assume you're not using mamba? Install the module and try running the workflow with --conda-frontend mamba. Hopefully it'll speed things up considerably.

https://snakemake.readthedocs.io/en/stable/executing/cli.html#conda

[dew]

Thanks a lot! Bless!

[ATpoint]

Here’s a tutorial in case you are new to Docker/Podman Building Docker images

Answer 1

[ATpoint]

Make a container once, use that. It’s more reproducible anyway since all dependencies and the OS is pinned.

[dew]

Thanks a lot! Best!

Answer 2

[LauferVA]

consider also whether Docker or Singularity might be an option for you. what is your expected usage and what type of environment?

[dew]

Thanks a lot!

Yeah, Docker and Singularity would be good options. Trying to make the analysis pipeline easier/reproducible to use on the HPC, say, just prepare the samplesheet.csv and units.csv files, then submit the job, and maybe after the time of a cup of coffee, all the results are ready for download and summarize.