Tophat and Bowtie installed with Conda not working on test data.

Question

[ariel.balter]

I created a python 2.7 environment using conda, and installed tophat and bowtie. I'm trying the test data as specified here:

https://ccb.jhu.edu/software/tophat/tutorial.shtml#inst

However, I get the following error:

balter@server3:/.../users/balter/tophat/test_data$ source activate p27
(p27) balter@server3:/.../users/balter/tophat/test_data$ python -V
Python 2.7.12
(p27) balter@server3:/.../users/balter/tophat/test_data$ conda install tophat bowtie2
Fetching package metadata .............
Solving package specifications: ..........

# All requested packages already installed.
# packages in environment at /home/.../Applications/miniconda3/envs/p27:
#
bowtie2                   2.3.0                    py27_0    bioconda
tophat                    2.1.1                    py27_0    bioconda
(p27) balter@server3:/.../users/balter/tophat/test_data$ ls
reads_1.fq  test_ref.1.bt2   test_ref.2.bt2   test_ref.3.bt2  test_ref.fa         test_ref.rev.1.ebwt  test_ref.rev.2.ebwt
reads_2.fq  test_ref.1.ebwt  test_ref.2.ebwt  test_ref.4.bt2  test_ref.rev.1.bt2  test_ref.rev.2.bt2   tophat_out
(p27) balter@server3:/.../users/balter/tophat/test_data$ tophat -r 20 test_ref reads_1.fq reads_2.fq

[2017-01-17 10:32:00] Beginning TopHat run (v2.1.1)
-----------------------------------------------
[2017-01-17 10:32:00] Checking for Bowtie
          Bowtie version:    2.3.0.0
[2017-01-17 10:32:00] Checking for Bowtie index files (genome)..
    Found both Bowtie1 and Bowtie2 indexes.
[2017-01-17 10:32:00] Checking for reference FASTA file
[2017-01-17 10:32:00] Generating SAM header for test_ref
[2017-01-17 10:32:00] Preparing reads
     left reads: min. length=75, max. length=75, 100 kept reads (0 discarded)
    right reads: min. length=75, max. length=75, 100 kept reads (0 discarded)
[2017-01-17 10:32:00] Mapping left_kept_reads to genome test_ref with Bowtie2
    [FAILED]
Error running:
/home/.../Applications/miniconda3/envs/p27/bin/bam2fastx --all ./tophat_out/tmp/left_kept_reads.bam|/home/.../Applications/miniconda3/envs/p27/bin/bowtie2 -k 20 -D 15 -R 2 -N 0 -L 20 -i S,1,1.25 --gbar 4 --mp 6,2 --np 1 --rdg 5,3 --rfg 5,3 --score-min C,-14,0 -p 1 --sam-no-hd -x test_ref -|/home/.../Applications/miniconda3/envs/p27/bin/fix_map_ordering --bowtie2-min-score 15 --read-mismatches 2 --read-gap-length 2 --read-edit-dist 2 --read-realign-edit-dist 3 --index-outfile ./tophat_out/tmp/left_kept_reads.mapped.bam.index --sam-header ./tophat_out/tmp/test_ref_genome.bwt.samheader.sam - ./tophat_out/tmp/left_kept_reads.mapped.bam ./tophat_out/tmp/left_kept_reads_unmapped.bam
(p27) balter@server3:/.../users/balter/tophat/test_data$

[WouterDeCoster]

Not what you ask for, but Tophat is no longer considered the best tool on the shelf for mapping, it's even deprecated. Instead, you should look at its successor hisat2 or alternatives such as STAR or BBmap.

[ariel.balter]

Thanks. My group is discussing this right now. I'm actually not the fusion person, but I was trying to get a working conda recipe going for them. Turns out to be two things: 1) python 3 incompatibility in conda recipe and 2) bowtie2 + tophat version incompatibility. https://github.com/bioconda/bioconda-recipes/issues/3510

But your comment is definitely in the right direction.

[Devon Ryan]

I have a pull request to implement the changes. Can you confirm that using the correct version of bowtie2 still doesn't work under python3?

[ariel.balter]

I get the usual I/O error with using python 3

balter@server:...users/balter/tophat/test_data$ conda create --name bowtietest bowtie2=2.2.5 tophat
Fetching package metadata .............
Solving package specifications: ..........

Package plan for installation in environment .../Applications/miniconda3/envs/bowtietest:

The following NEW packages will be INSTALLED:

    bowtie2:         2.2.5-py35_2       bioconda
    ca-certificates: 2016.9.26-0        conda-forge
    certifi:         2016.9.26-py35_0   conda-forge
    libgcc:          5.2.0-0
    ncurses:         5.9-10             conda-forge
    openssl:         1.0.2h-3           conda-forge
    perl:            5.22.0.1-0         conda-forge
    perl-threaded:   5.22.0-pl5.22.0_12 bioconda
    pip:             9.0.1-py35_0       conda-forge
    python:          3.5.2-4            conda-forge
    readline:        6.2-0              conda-forge
    setuptools:      32.3.1-py35_0      conda-forge
    sqlite:          3.13.0-1           conda-forge
    tk:              8.5.19-1           conda-forge
    tophat:          2.1.1-py35_0       bioconda
    wheel:           0.29.0-py35_0      conda-forge
    xz:              5.2.2-0            conda-forge
    zlib:            1.2.8-3            conda-forge

Proceed ([y]/n)? y

Linking packages ...
[      COMPLETE      ]|############################################################################################################################################################################| 100%
#
# To activate this environment, use:
# > source activate bowtietest
#
# To deactivate this environment, use:
# > source deactivate bowtietest
#

balter@server:...users/balter/tophat/test_data$ tophat -r 20 test_ref reads_1.fq reads_2.fq

[2017-01-17 14:40:39] Beginning TopHat run (v2.1.1)
-----------------------------------------------
Traceback (most recent call last):
  File ".../Applications/miniconda3/bin/tophat", line 4107, in <module>
    sys.exit(main())
  File ".../Applications/miniconda3/bin/tophat", line 3899, in main
    run_log = open(logging_dir + "run.log", "w", 0)
ValueError: can't have unbuffered text I/O
balter@server:...users/balter/tophat/test_data$ source activate bowtietest
(bowtietest) balter@server:...users/balter/tophat/test_data$ tophat -r 20 test_ref reads_1.fq reads_2.fq

[2017-01-17 14:41:15] Beginning TopHat run (v2.1.1)
-----------------------------------------------
Traceback (most recent call last):
  File ".../Applications/miniconda3/envs/bowtietest/bin/tophat", line 4107, in <module>
    sys.exit(main())
  File ".../Applications/miniconda3/envs/bowtietest/bin/tophat", line 3899, in main
    run_log = open(logging_dir + "run.log", "w", 0)
ValueError: can't have unbuffered text I/O

Note: I also created this: https://github.com/bioconda/bioconda-recipes/issues/3510

[Devon Ryan]

I've merged the fix for this, thanks for letting everyone know about the issue!

[ariel.balter]

Hi @Devon--that's great! Just to clarify: is this for the tophat code, the bowtie2 code, or the conda recpie?

[Devon Ryan]

This is the conda recipe, so what you get with conda install tophat. This will hopefully no longer work at all in python3 (you should get some sort of "no such package" error), though I haven't checked.

[ariel.balter]

Looks like it is still trying:

balter@server:~$ which python
/home/.../Applications/miniconda3/bin/python
balter@server:~$ python -V
Python 3.5.2
balter@server:~$ conda install tophat
Fetching package metadata .............
Solving package specifications: ..........

Package plan for installation in environment /home/.../Applications/miniconda3:

The following NEW packages will be INSTALLED:

    tophat: 2.1.1-py35_0 bioconda

Proceed ([y]/n)?

[Devon Ryan]

I've asked one of the bioconda core people to check into this. My update isn't even appearing yet under python 2.7!

[Devon Ryan]

OK, the normal python3 sorts of errors. I'll fix my PR and merge it in today after talking to the core folks about what to do with the python3 packages already available in conda.