Entering edit mode
8.7 years ago
michealsmith
▴
800
#!/bin/bash
#BSUB -n 8
#/share/pkg/bwa/0.7.12/bwa aln -t 16 -q 15 g1k_37.fasta 013_1.fastq.gz > 013_1.sai
For the bwa-aln, multi-threading is allowed; as in my script above, I request 8 cores, and set -t 16 to require 16 thread to run the program.
This run is successful, but raises my question, so this example shows each thread for bwa-aln won't occupy the whole core? Maybe say 0.2 or 0.5 core?
Also, if the requested core cannot run multi-thread, then what will happen? For example, only 10 threads are allowed, but I set 16 thread, then what will happen?
I doubt you are getting more than 8 cores if you requested only 8 with BSUB directive. You are also not using
-R "span[hosts=1]"option (unless you were requesting exclusive access to a node) so your jobs are probably not running very efficiently if the threads are spread across physical servers (your admins may be preventing that by LSF config).On some clusters the hyperthreading may be turned off to avoid the type of situation that @John described above. You will need to check on your cluster config.
No, I did use
-R "span[hosts=1]"But actually I don't quite understand what that means
That was appropriate. That option means - "all processors allocated to this job must be on the same host".
Note that bwa-aln uses a (often limiting) singlethreaded phase for conversion to sam. bwa-mem is fully multithreaded, and much faster anyway.
Why not just run 8 threads with 8 cores, or 16 cores with 16 threads? I'm not sure there's an advantage of running more cores than you have threads. It's like 16 people trying to eat soup at the same time with only 8 spoons. Not everyone can eat at the same time. This is what's going on with your situation, there's not enough cores to run all of the threads simultaneously. Some have to wait while others are executed.
You might be mixing up cores and threads and hyperthreading. A single CPU core can run multiple threads, without the need for hyperthreading. However, a single core cannot run multiple threads simultaneously. Hyperthreading is a special feature that provides a logical core to allow a degree of parallel execution of two threads on a single physical core. You don't need hyperthreading to run multiple threads on a single core.
Do you mean to ask what happens if you allocate 10 cores for your program that needs 16 cores? I'm guessing your batch system would kill the job.
Thanks. Actually I mean I request 8 cores, but require set 16 threads in my script, and the program runs quite well without error, should I worry the quality of the output bam files?
Should I re-run requesting 16 cores for 16 threads?
Yes, I understood you perfectly. No, there will be no errors in the BWA output. This is just a lesson in efficient/good practices for running jobs on a HPC system.
The batch system, BSUB in your case, will give you 8 cores to use and BWA will start 16 threads on each core.
There's no need to rerun now, This will just be good to know in the future. I would generally try and stick to 1:1 threads:cores. If you can only get 8 cores, just run 8 threads.
The tl;dr of my response was that if:
A process or thread does not require it's own core 100% of the time to simply exist. Right now on my dual-core laptop, i have about 100 processes running, each probably having anywhere from 1 to 10 threads. This is possible because the OS allocates each process a bit of time on a core, giving the impression to the user of simultaneous execution of all 100 processes.
However, it doesn't make sense to run more CPU-intense processes than the number of cores you have available, because there is overhead for the OS switching processes in and out like this. It is better to have 1 intensive process occupying 1 core for 99% of the time, than two intensive-processes occupying a core for 47% of the time each.
So, in conclusion, give bwa the same number of processors as it has threads. Hyperthreading makes a single core look like two cores to the OS, which I presume is all abstracted away from BSUB.
Also, for a more definitive answer, try it and see.
Thanks John, yes now I'm using same number of processors as threads.
But for the bam files already generated requesting 8 cores yet requiring 16 threads, should I trust the quality of the output (No errors generated in std_out and std_err)? Or should I re-run everything requesting 16 cores for 16 threads?
The number of processors/threads used shouldn't have any effect on the output - just the time taken to run :)