Running Bowtie As Service Or Daemon?

Question

[Alex Reynolds]

Is it correct that each time bowtie is run, that it loads the index into memory to perform alignment? If so, is there a way to run bowtie as a service, which keeps a particular index-of-interest in memory and is therefore ready to perform alignments without the startup overhead? Thanks for your advice.

Answer 1

[Ido Tamir]

It is possible:

 --shmem   Use shared memory to load the index, rather than normal C file I/O. Using shared memory allows many concurrent bowtie processes on the same computer to share the same memory image of the index (i.e. you pay the memory overhead just once). This facilitates memory-efficient parallelization of bowtie in situations where using -p is not desirable. Unlike --mm, --shmem installs the index into shared memory permanently, or until the user deletes the shared memory chunks manually. See your operating system documentation for details on how to manually list and remove shared memory chunks (on Linux and Mac OS X, these commands are ipcs and ipcrm). You may also need to increase your OS's maximum shared-memory chunk size to accomodate larger indexes; see your OS documentation.

So with shmem it loads into shared memory. Is it faster?

>time bowtie --shmem /projects/solexadst/genomes/worm/bowtie_chr/ce6 -f test.fasta 
# reads processed: 1
# reads with at least one reported alignment: 0 (0.00%)
# reads that failed to align: 1 (100.00%)
No alignments

real    0m0.016s
user    0m0.004s
sys    0m0.008s

>time bowtie  /projects/solexadst/genomes/worm/bowtie_chr/ce6 -f test.fasta 
# reads processed: 1
# reads with at least one reported alignment: 0 (0.00%)
# reads that failed to align: 1 (100.00%)
No alignments

real    0m0.083s
user    0m0.000s
sys    0m0.056s

Very scientific analysis with one datapoint for each condition gives me a speedup of 5x. I could not load larger indices into shared memory.

[Maximilian Haeussler]

On our server, Centos 6.8, the -shmem option hangs. I got around that by putting the index into /dev/shm and using --mm, which should be the same thing.

Answer 2

[fperraudeau]

Do you know if it is possible to do a similar thing with Bowtie2 ?

For Bowtie2, option --shmem does not seem to exist.

Thank you

[Maximilian Haeussler]

In a more current bowtie2 version, this exists now.