Processing Fast5 files from ONT for direct RNA

Question

[Shagnik]

Hi, Am trying to process some sequencing data from ONT. I have fast5, fastq, fasta files. I have also generated bam files with minimap2 and samtools. However, when I cam trying to run Nanopolish, ELIGOS2, DRUMMER, Tombo - all tools are giving me errors. I cannot process my reads. My guess is that some plugin needs to be installed and CONFIGURED CORRECTLY to make the programs read the fast5 current signals along with the reference genome and bam files. Can anyone help me out ? I am doing ONT data processing for the first time and it is a bit urgent. Thanks! :)

[Shagnik]

This is the error from tombo resquiggle command :

Traceback (most recent call last):
  File "/usr/bin/tombo", line 33, in <module>
    sys.exit(load_entry_point('ont-tombo==1.5.1', 'console_scripts', 'tombo')())
  File "/usr/lib/python3/dist-packages/tombo/__main__.py", line 230, in main
    from . import resquiggle
  File "/usr/lib/python3/dist-packages/tombo/resquiggle.py", line 44, in <module>
    from . import tombo_stats as ts
  File "/usr/lib/python3/dist-packages/tombo/tombo_stats.py", line 84, in <module>
    HALF_NORM_EXPECTED_VAL = stats.halfnorm.expect()
  File "/home/snu/.local/lib/python3.10/site-packages/scipy/stats/_distn_infrastructure.py", line 2927, in expect
    dub = integrate.quad(fun, d, ub, **kwds)[0]
  File "/home/snu/.local/lib/python3.10/site-packages/scipy/integrate/_quadpack_py.py", line 411, in quad
    retval = _quad(func, a, b, args, full_output, epsabs, epsrel, limit,
  File "/home/snu/.local/lib/python3.10/site-packages/scipy/integrate/_quadpack_py.py", line 525, in _quad
    return _quadpack._qagie(func,bound,infbounds,args,full_output,epsabs,epsrel,limit)
  File "/home/snu/.local/lib/python3.10/site-packages/scipy/stats/_distn_infrastructure.py", line 2904, in fun
    return x * self.pdf(x, *args, **lockwds)
  File "/home/snu/.local/lib/python3.10/site-packages/scipy/stats/_distn_infrastructure.py", line 2094, in pdf
    place(output, cond, self._pdf(*goodargs) / scale)
  File "/home/snu/.local/lib/python3.10/site-packages/scipy/stats/_continuous_distns.py", line 3820, in _pdf
    return np.sqrt(2.0/np.pi)*np.exp(-x*x/2.0)
**FloatingPointError: underflow encountered in exp**

[Shagnik]

This is the error sample from nanopolish call methylation command :

HDF5-DIAG: Error detected in HDF5 (1.10.6) thread 140457856218688:
  #000: H5L.c line 825 in H5Lexists(): not a location
    major: Invalid arguments to routine
    minor: Inappropriate type
  #001: H5Gloc.c line 246 in H5G_loc(): invalid object ID
    major: Invalid arguments to routine
    minor: Bad value
[warning] fast5 file is unreadable and will be skipped

[colindaven]

• Why are you using Tombo ? I have been doing ONT analysis for years and never had to use that.
• What is your goal?
• Why not just use guppy or megalodon for methylation analysis ?
• Which data do you have ?

[Shagnik]

My goal is to analyse RNA base modifications from direct RNA sequencing using ONT. I am doing it for the first time and trying multiple tools for the same, with no success in any yet. I have fast5 files, base called fastq files as well. I need to analyse the current change patterns to detect the methylation pattern changes in treated and control samples. Any guidance on this will be really helpful. Thanks :)

[Shagnik]

Hey, Am trying megalodon now for rna methylation analysis. Could you help me out with this error?

Command :

 megalodon /media/snu/5TB_nano/SO_10875_Nanopore_Rawdata/cat_compress_tar_fast5_pass/fast5_pass/A1/ --rna --outputs mods mod_mappings --output-directory /home/snu/Anirudh/megalodon_A1/ --overwrite --reference /home/snu/Anirudh/***** --mod-motif a RRACH 3 --processes 10 --write-mods-text --guppy-config ./ont-guppy-cpu/data/rna_r9.4.1_70bps_fast.cfg --guppy-server-path ./ont-guppy-cpu/bin/guppy_basecall_server

Error :

[guppy/error] basecall_service::BasecallClient::worker_loop: Connection error. [basecall_config_unavailable] could not retrieve details for config from server: ./ont-guppy-cpu/data/rna_r9.4.1_70bps_fast
****************************************************************************************************
    ERROR: Error connecting to Guppy server. Undefined error: "Basecalling config unavailable: <result.basecall_config_unavailable: 6>"

[colindaven]

I think you're misspecifying the configs. Also, I would combine the reference genome/contigs into one file and specify that directly. Megalodon seems to like full paths. Heres' what I use for dna from promethion (nextflow code, variables are full paths, using GPU as needed for Megalodon).

megalodon $fast5s --output-directory $params.outdir --guppy-params "--chunks_per_runner 512 --chunk_size 3072 --gpu_runners_per_device 8 -d /path/to/rerio/basecall_models/" --guppy-config res_dna_r941_prom_modbases_5mC_v001.cfg --outputs per_read_mods mod_mappings mods --reference $params.fastammi --processes 8 --devices 0 --overwrite --guppy-server-path /path/to/programs/ont-guppy/bin/guppy_basecall_server --num-read-enumeration-threads 8 --num-extract-signal-processes 8 --guppy-timeout 1000

[colindaven]

Also I would ask the ONT devs via the Megalodon github repo what current best practices are for rna, think it's v tricky.

This list might help get you started too https://github.com/GoekeLab/awesome-nanopore

[Shagnik]

Thanks! I tried tombo and am still doing with no success.

[Shagnik]

Which config file to use? My guess is megalodon shows the error due to this reason only!

[barbarahufnagel]

Hello, Could you solve your problem? I am having the same problem over here.

Answer 1

[dariober]

Since you say

My guess is that some plugin needs to be installed and CONFIGURED CORRECTLY

See if this helps Dealing with nanopore fast5 files compressed with vbz

[Shagnik]

I tried this tutorial. I am using the mamba command. Shows me this error :

mamba install ont_vbz_hdf_plugin
Command 'mamba' not found, did you mean:
  command 'samba' from deb samba (2:4.15.9+dfsg-0ubuntu0.3)
Try: sudo apt install <deb name>

Also, when I am using h5dump on a fast5 file, i am getting a long output on terminal with data information. What to do?

[dariober]

Command 'mamba' not found: You need to install mamba or conda and configure for bioconda or install the plugin in some other way (I would go for mamba though). Google "bioconda installation" for details.

[Shagnik]

Looking for: ['ont_vbz_hdf_plugin']

pkgs/r/linux-64 No change pkgs/main/linux-64 No change pkgs/main/noarch No change pkgs/r/noarch No change

Pinned packages:

• python 3.9.*

Could not solve for environment specs Encountered problems while solving:

• nothing provides requested ont_vbz_hdf_plugin

The environment can't be solved, aborting the operation

Still cannot recognize and fetch the plugin! :(

[dariober]

If you haven't configured conda for working with bioconda, you should, otherwise try this:

conda install -c bioconda ont_vbz_hdf_plugin
# Or better:
mamba install -c bioconda ont_vbz_hdf_plugin

If it doesn't work, give more detail of what you have done.

[Shagnik]

Hi, I installed the plugin by mamba. It downloaded the packages fine. But tombo resquiggle still shows error.

Command was :

tombo resquiggle ./cat_compress_tar_fast5_pass/fast5_pass/A1/ Hydra_v3.fasta --processes 4 --num-most-common-errors 5

Error message :

Traceback (most recent call last):
  File "/usr/bin/tombo", line 33, in <module>
    sys.exit(load_entry_point('ont-tombo==1.5.1', 'console_scripts', 'tombo')())
  File "/usr/lib/python3/dist-packages/tombo/__main__.py", line 230, in main
    from . import resquiggle
  File "/usr/lib/python3/dist-packages/tombo/resquiggle.py", line 44, in <module>
    from . import tombo_stats as ts
  File "/usr/lib/python3/dist-packages/tombo/tombo_stats.py", line 84, in <module>
    HALF_NORM_EXPECTED_VAL = stats.halfnorm.expect()
  File "/home/snu/.local/lib/python3.10/site-packages/scipy/stats/_distn_infrastructure.py", line 2927, in expect
    dub = integrate.quad(fun, d, ub, **kwds)[0]
  File "/home/snu/.local/lib/python3.10/site-packages/scipy/integrate/_quadpack_py.py", line 411, in quad
    retval = _quad(func, a, b, args, full_output, epsabs, epsrel, limit,
  File "/home/snu/.local/lib/python3.10/site-packages/scipy/integrate/_quadpack_py.py", line 525, in _quad
    return _quadpack._qagie(func,bound,infbounds,args,full_output,epsabs,epsrel,limit)
  File "/home/snu/.local/lib/python3.10/site-packages/scipy/stats/_distn_infrastructure.py", line 2904, in fun
    return x * self.pdf(x, *args, **lockwds)
  File "/home/snu/.local/lib/python3.10/site-packages/scipy/stats/_distn_infrastructure.py", line 2094, in pdf
    place(output, cond, self._pdf(*goodargs) / scale)
  File "/home/snu/.local/lib/python3.10/site-packages/scipy/stats/_continuous_distns.py", line 3820, in _pdf
    return np.sqrt(2.0/np.pi)*np.exp(-x*x/2.0)

FloatingPointError: underflow encountered in exp

Answer 2

[kenneditodd]

Tombo is extremely error prone now that it's no longer being maintained. I had to do multiple file conversions, downgrade a couple softwares, and install the hd5f plugin and then i finally got it working. However, someone told me the Master of Pores git repo has a container you can run it in.