best way to submit hpc job using snakemake

Question

[QX]

Hi all,

I try to create a snakemake workflow for HPC. I know that snakemake have options for cluster ifself, but can we just put the shell: sbatch --optios in the rules, which is alot easier?

Do you know what is the pros and cons for the 2 approaches?

Best,

[colindaven]

You should actually learn to do it properly as suggested here and in the docs: https://stackoverflow.com/questions/76166705/how-do-i-set-up-snakemake-on-slurm-properly

I don't think using sbatch or srun in the command is a viable or sensible solution.

Answer 1

[i.sudbery]

If you just put sbatch in the rules, then snakemake will run the command, which will report success almost immediately as sbatch reports that the job is successfully submitted (not successfully completed). Snakemake will them move immediately on to the next rule without waiting for the jobs from the previous one to finish, and will be upset when the files that were the output of rule 1, and the input to rule 2, don't yet exist.

[QX]

if output1 from rule1 is the input2 for rule2; is rule2 will while until rule1 output1 finished, or it will execute immediately after input1 is generated (which then could be a problem as they are not whole file)?

also, for analysis with many files, is it better to use snakemake for each file through whole process; or it is better to run all file at the same time for each step?

[i.sudbery]

If run properly snakemake will wait for rule1 output1 is finished, as signaled by SLURM returning a successful completion code.

If you run sbatch in a rule, then rule2 might execute before rule1 has even started, let alone finihsed.

is it better to use snakemake for each file through whole process; or it is better to run all file at the same time for each step?

The purpose of snake make is not to worry about this sort of thing. You tell it you need this 10 or 20 steps done to that 10 or 20 files and it works it out for itself, and you come back to all files having been run for all steps.

[QX]

thank you!