temp directory as output in snakemake

Question

[hamarillo]

Hi, does anyone know of the effects of using both temp() and directory() in snakemake's output?

e.g.

rule:
  output: temp(directory("bowtie/{samp}/global_alignment"))

Answer 1

[Ming Tommy Tang]

if you use temp(), the folder will be deleted after snakemake finishes the whole run

Answer 2

[Mauricio]

After all rules that take in the output as input are ran then it get deleted, not at the end of the whole run. So if you only use "bowtie/{samp}/global_alignment" as input in rule A, then after rule A finishes the files will be deleted You can read more here: https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#protected-and-temporary-files

[Jesse]

Yep, this behavior is easy to confirm with a couple of test rules and something like shell: "sleep 60" in the last one, so you can see the temp directory outputs get removed before the whole workflow is done running. That distinction (versus removal at the very end) can matter if you have big workflows with multiple batches of temporary files.

I'm pretty sure that Snakemake implicitly sets rule priorities to try to get temp outputs cleared out sooner rather than later, too, but I can't find this in the docs. (Maybe implied by the help text for --scheduler where it "aims to reduce runtime and hdd usage by best possible use of resources"? I could have sworn I saw it described somewhere but can't find it now.)