"orthomcl" python multithreading loading to mysql database table

Question

[seok1213neo]

while using orthomcl,

i am planning to load my local data into the similarsequence table.

but the problem is that my data is too large, it will take forever to load my data.

so i tried the code below to do some multithreading.

import threading
import os

class Worker(threading.Thread):
    def __init__(self, name):
        super().__init__()
        self.name = name

    def run(self):
        os.system ('/bin/bash -c "./orthomclLoadBlast ./orthomcl.config ./similarSequence_ar.txt"')

threads = []
for i in range(3):
    thread = Worker(i)
    thread.start()
    threads.append(thread)

for thread in threads:
    thread.join()

would this work for multithreading? increase my cpu usage and decrease my working time?

please help me i am running outta time

thank you!

Answer 1

[Philipp Bayer]

Your code will always execute the same piece of bash script, just three times, you would need to split the ./similarSequence_ar.txt into three pieces and then give each worker one piece. You could also use PorthoMCL, a parallel re-implementation of OrthoMCL to speed things up: https://github.com/etabari/PorthoMCL

I would personally recommend Orthofinder, it's much much much much faster than OrthoMCL, and doesn't require MySQL.

[dariober]

I would personally recommend Orthofinder

I was going to say that. I spent quite some time trying to make OrthoMCL work to no avail. Besides, I think the development of OrthoMCL is kind of dead.