How to use Ruffus to create and use directories not based on input files.

Question

[sym24]

I am making a pipeline using two external bioinformatics tools. I decided to use Ruffus. It is powerful python library. However, I am having trouble using python decreators.

What I would like to do is use argparse get the work directory from user and generate the directory if it does not exsit. To do this, I used: @follows(defaultwd, mkdir(work_dir)) while defaultwd() is a function get default work directory, and work_dir = args.workdir. I have not tested this.

To my understanding, formatter() or regex() or suffix() gets the string from input and convert input components into variables within the decorator function, which can be conviently used as variables when specifying output format. The challenge is how can I use external variables to specify output format?

For example, I have fastq libraries stored in directory /work/library/A11111.gz.fq, I will use this as input file. I would like my output file to be at /cluster/project/output/A11111/anaylised.fq. It is clearly two different directories. I know I can simply write /cluster/project/output/A11111/anaylised.fq as output, however, in my case this will be determined by user and stored in a variable called work_dir. How do I deal with this in Ruffus?

The reason I use Ruffus is that it will check whether the file exits and safe time if one step of the pipeline crashed. To accomplish this goal, the first step is to let the program know where to look for output files. Therefore, it is important to specifiy correct output file directories.

Thanks in advance. Sorry for the long question.

Answer 1

[Ram]

Is using Python file path/string processing an option? That seems like the easier way to deal with this to me.

[sym24]

Do you mean things like os.path.basename()? I use those to get the file name and directory. Yeah, you are right. I can generate a string contains output file names and pass the string to the decorator as output.

[Ram]

Not sure what a decorator is, but yeah, os.path should help crack it.