Core utilization with Galaxy

Question

[anorman07]

Hi

I'm a complete Galaxy newbie, and am trying to configure a 16-core workstation to run, with an emphasis on speed-per-job. In other words, I'd like to dedicate as many cores to a given job as possible, and will run jobs sequentially.

This example job_conf.xml file was very well laid out and explained (from https://biostar.usegalaxy.org/p/10158/#23245), and I've attempted to modify it to run 14 cores on the listed tools. However, it's not working, and it looks like only one core is working when I run Cufflinks (for example). Any idea what I'm doing wrong? Are there any other configuration files that I need to tweak? I have a feeling I'm doing some that is obviously wrong, and I apologize if I am.

<job_conf> <plugins> <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="2"/> <plugin id="multilocal" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="1"/> </plugins>

<handlers> <handler id="main"/> </handlers>

<destinations default="local"> <destination id="local" runner="local"/> <destination id="multicore14" runner="multilocal"> <param id="local_slots">14</param> </destination> </destinations>

<tools> <tool id="cuffdiff" destination="multicore14"/> <tool id="cufflinks" destination="multicore14"/> <tool id="cuffmerge" destination="multicore14"/> <tool id="fastq_groomer" destination="multicore14"/> <tool id="fastqc" destination="multicore14"/> <tool id="rgrnastar" destination="multicore14"/> <tool id="trimmomatic" destination="multicore14"/> </tools>

</job_conf>

[GenoMax]

You should post this over at Galaxy Biostars.

Not every program is able to use multiple cores so don't think you would be able to accelerate every program by making it use multiple cores.

[martenson]

genomax is right, to use more threads the feature has to be supported by both the software and the Galaxy tool. Galaxy tool authors specify it e.g. as --num-threads="\${GALAXY_SLOTS:-4}" in their tool wrapper. Only if both these conditions are fulfilled you can use the job_conf and set up a multi-thread destination for such tools.

check out https://galaxyproject.org/admin/config/galaxy_slots/