Entering edit mode
6 months ago
kerianaleerivera
▴
10
Hello! I have been trying to run a Nanopore 16s pipeline I found online for a while but have not been able to. I have come to the conclusion that the hpc at my facility is not up to date with many of the requirements. Are there any other instutions that allow to use their hpc without charge? Thank you in advance!
Without charge? Nope, compute costs money. HPC, own server, AWS, etc. all cost money for the lab I work in.
Also, how did you reach such a conclusion? Just because you can't get something to compile or something is out of date doesn't mean you can't work around it.
I reached that conclusion after a thorough review of the programs and resources available on our HPC. Additionally, other labs have experienced similar issues with their pipelines and chose to either pay for external services or seek assistance from other institutions. Only then were they able to successfully run their data. Thank you for your reply, I will keep on looking.
If you are not able to run a 16S pipeline then what you likely have can't be classified as HPC. It is possible that the hardware you have is not compatible. Which pipeline are you referring to?
Closest to "free" compute may be "CyVerse" : https://cyverse.org/ You will need to create an account.
Hello! I am referring to this pipeline: NanoRTax. I have been able to run it, but the pipeline completes with multiple errors. When I check the error logs, all of them originate from specific programs (modules) that are already installed and available in our hpc (e.g., fastp). Thank you for the suggestion about CyVerse. I will look into creating an account and exploring its capabilities.
So it sounds more like this is an issue that you need to address with local system administrators. A brief look at the pipeline you linked does not suggest anything out of ordinary in terms of compute infrastructure needs (if you truly have a cluster or multi-core, multi-GB memory servers).
krakendatabase is also for just 16S so that should not be very large. Only thing that will likely not work is the visualization app. Depending on how your HPC is set up your admins will likely not let you run web servers on it.Thank you for the feedback, I am sure that the visualization app will not work, however I will contact the local system administrators again to see if there's something they can do about the other errors. According to them they provide over 2240 compute cores, and 200 terabytes of high-performance storage served over a QDR Infiniband and 10G Ethernet backbone. Thank you!
That sounds plenty capable hardware and should have no issues running the pipeline once the software errors are addressed. Hopefully your admins will help you figure the issues out.
You could also look at 16S pipeline provided by Oxford Nanopore as an alternate: https://github.com/epi2me-labs/wf-16s
I took a look at the pipeline you mentioned and saw the specifications you provided about your HPC. Since we don't know the exact error, we can't simply fix it. However, Fastp is a tool used for quality assessment and cleaning of raw reads. Some versions have bugs, and one of them can cause the tool to hang indefinitely. You might be encountering this known bug. Check out the Fastp releases page on GitHub (here) and see if there are documented bugs similar to the one you're experiencing.
Upgrading to the latest version of Fastp and updating your pipeline to use it could potentially resolve the issue. Specifying the exact error message will be crucial for further troubleshooting if the problem persists. In most cases, running the pipeline on a regular computer (not an HPC) might just take longer but could still work.