GATB Multithread always one core

Question

[lys8918]

Hi,

I'm writing some codes using GATB. First, I write a sample code to test the multithread. But, why this code always run on one core? Any problems of my code?

The code from examples provided by GATB (multithreading1.cpp). I add some code in Functor to waste times.

 #include <gatb/gatb_core.hpp>
using namespace std;
/********************************************************************************/
/*                    Multithreaded iteration of an integer range               */
/********************************************************************************/
// We define a functor that will be cloned by the dispatcher
struct Functor { void operator() (int i)
{
    double s = 0;
    char str[100];
    sprintf(str, "%d.txt", i);
    FILE *fp = fopen(str, "w");
    for (int i = 0; i < 100000; i++) {
        s += i/10000.0;
        s *= 3.0;
        fprintf(fp, "%lf\n", s);
    }
    fclose(fp);
    cerr<<i<<"\n";
    // In this instruction block, we are executing in one of the nbCores threads
    // created by the dispatcher. Note that 'i' is one value of our range
}};
/********************************************************************************/
int main (int argc, char* argv[])
{
    // We get the number of cores to be used.  If we don't give any number,
    // we set to 0 which implies the usage of all available cores
    size_t nbCores = (argc >=2 ? atoi(argv[1]) : 0);
    // We create an iterator over an integer range
    Range<int>::Iterator it (1,1000);
    // We create a dispatcher configured for 'nbCores' cores.
    Dispatcher dispatcher (nbCores);
    // We dispatch the range iteration with the dispatcher.
    // This will create nbCores threads and each thread will be fed with
    // one value of the defined range
    // NOTE: we could also use lambda expression (easing the code readability)
    IDispatcher::Status status = dispatcher.iterate (it, Functor());
    // We dump some information about the dispatching
    cout << "nbCores=" << status.nbCores << "  time=" << status.time << endl;
    // IMPORTANT: usage of Dispatcher has sense only if the iterated items
    // can be processed independently from each other.
    // The point to understand with the Dispatcher is that it can
    // iterate any instance of Iterator class. If you have any set of items
    // that can be enumerated through an Iterator implementation, then you
    // can parallelize the iteration with a Dispatcher instance
}

[Rayan Chikhi]

indeed, I can confirm the issue. For some reason the dispatcher only uses one core in that setting. Will see how to suggest fix.

Answer 1

[Rayan Chikhi]

Hi, We've found the issue here, thank to @grizk. Please change

 IDispatcher::Status status = dispatcher.iterate (it, Functor());

to

 IDispatcher::Status status = dispatcher.iterate (it, Functor(), 1);

This is because the third argument of iterate() is a batch size, i.e. a number of iterations that are batched into a single job in a thread. By default it is set to 1000, as the elements that are iterated on are typically kmers from a large set. Setting a large batch size avoids the overhead of switching threads for each k-mer. Here, by setting a batch size of 1, each Functor() is executed as soon as a thread is available.