Queuing jobs for a synthetic read generator python script

Question

[matt81rd]

Hi all,

I have written a farly simple script that will take an input.tsv with some genes and variant inforfmation to create a VCF file that is then used as an input for a synthetic read genetor tool called NEAT (https://github.com/ncsa/NEAT). I'm goinog to have to create quite a few synthetic genomes and as this process takes around 15 mins per one genome i will need to be able to queue jobs for this script but have no done that before.

Can anyone suggest any methods or popular options for job queueing and maybe some thinsg i should think about before i trial some options? Or would a simple shell script that would loop through all the .tsv files in a given directory work well enough?

Many thanks

Answer 1

[Istvan Albert]

Use GNU parallel:

• Gnu Parallel - Parallelize Serial Command Line Programs Without Changing Them