Testing Tophat installation

Question

[erikfas]

I'm just starting out with RNA-seq, and I'm trying to get the installation of Tophat to work. I'm following the "Getting started" instructions on the site, and am currently testing the installation. When I run

tophat -r 20 test_ref reads_1.fq reads_2.fq

... this is what I get:

[2014-06-20 13:01:22] Beginning TopHat run (v2.0.11)
-----------------------------------------------
[2014-06-20 13:01:22] Checking for Bowtie
          Bowtie version:     2.2.3.0
[2014-06-20 13:01:22] Checking for Samtools
        Samtools version:     0.1.19.0
[2014-06-20 13:01:22] Checking for Bowtie index files (genome)..
    Found both Bowtie1 and Bowtie2 indexes.
[2014-06-20 13:01:22] Checking for reference FASTA file
[2014-06-20 13:01:22] Generating SAM header for test_ref
[2014-06-20 13:01:22] Preparing reads
     left reads: min. length=75, max. length=75, 100 kept reads (0 discarded)
    right reads: min. length=75, max. length=75, 100 kept reads (0 discarded)
[2014-06-20 13:01:22] Mapping left_kept_reads to genome test_ref with Bowtie2
    [FAILED]
Error running:
/Users/erikfasterius/bin/bam2fastx --all ./tophat_out/tmp/left_kept_reads.bam|/Users/erikfasterius/bin/bowtie2 -k 20 -D 15 -R 2 -N 0 -L 20 -i S,1,1.25 --gbar 4 --mp 6,2 --np 1 --rdg 5,3 --rfg 5,3 --score-min C,-14,0 -p 1 --sam-no-hd -x test_ref -|/Users/erikfasterius/bin/fix_map_ordering --bowtie2-min-score 15 --read-mismatches 2 --read-gap-length 2 --read-edit-dist 2 --read-realign-edit-dist 3 --index-outfile ./tophat_out/tmp/left_kept_reads.mapped.bam.index --sam-header ./tophat_out/tmp/test_ref_genome.bwt.samheader.sam - ./tophat_out/tmp/left_kept_reads.mapped.bam ./tophat_out/tmp/left_kept_reads_unmapped.bam

... and I don't really understand the error. It doesn't say what type of error it is, just says [FAILED] and give the long string of whatever caused the error, which I don't understand.

Any help would be greatly appreciated!

Erik

[Devon Ryan]

You'll have to run

/Users/erikfasterius/bin/bam2fastx \
    --all ./tophat_out/tmp/left_kept_reads.bam|/Users/erikfasterius/bin/bowtie2 \
    -k 20 \
    -D 15 \
    -R 2 \
    -N 0 \
    -L 20 \
    -i S,1,1.25 \
    --gbar 4 \
    --mp 6,2 \
    --np 1 \
    --rdg 5,3 \
    --rfg 5,3 \
    --score-min C,-14,0 \
    -p 1 \
    --sam-no-hd \
    -x test_ref \
    -|/Users/erikfasterius/bin/fix_map_ordering \
    --bowtie2-min-score 15 \
    --read-mismatches 2 \
    --read-gap-length 2 \
    --read-edit-dist 2 \
    --read-realign-edit-dist 3 \
    --index-outfile ./tophat_out/tmp/left_kept_reads.mapped.bam.index \
    --sam-header ./tophat_out/tmp/test_ref_genome.bwt.samheader.sam \
    - \
    ./tophat_out/tmp/left_kept_reads.mapped.bam \
    ./tophat_out/tmp/left_kept_reads_unmapped.bam

from the directory yourself to see what the actual error was.

[erikfas]

Ok, I did that, and this is what it says:

100 reads; of these:
  100 (100.00%) were unpaired; of these:
    59 (59.00%) aligned 0 times
    41 (41.00%) aligned exactly 1 time
    0 (0.00%) aligned >1 times
41.00% overall alignment rate
[sam_read1] missing header? Abort!

[Devon Ryan]

Interesting, given that the command tells bowtie2 to not provide a header, it's surprising that fix_map_ordering would be expecting one. What happens if you remove the --sam-no-hd portion from that line?

[erikfas]

This:

/Users/erikfasterius/bin/bowtie2-align-s \
    --wrapper basic-0 \
    -k 20 \
    -D 15 \
    -R 2 \
    -N 0 \
    -L 20 \
    -i S,1,1.25 \
    --gbar 4 \
    --mp 6,2 \
    --np 1 \
    --rdg 5,3 \
    --rfg 5,3 \
    --score-min C,-14,0 \
    -p 1 \
    --bowtie2-min-score 15 \
    --read-mismatches 2 \
    --read-gap-length 2 \
    --read-edit-dist 2 \
    --read-realign-edit-dist 3 \
    --index-outfile ./tophat_out/tmp/left_kept_reads.mapped.bam.index \
    --sam-header ./tophat_out/tmp/test_ref_genome.bwt.samheader.sam \
    - \
    ./tophat_out/tmp/left_kept_reads.mapped.bam \
    ./tophat_out/tmp/left_kept_reads_unmapped.ba
bowtie2-align-s: unrecognized option `--bowtie2-min-score'

Followed by what seems to be the bowtie help (i.e. lots of rows of info on parameters), and then this at the end:

Error: Encountered internal Bowtie 2 exception (#1)
Command: /Users/erikfasterius/bin/bowtie2-align-s --wrapper basic-0 -k 20 -D 15 -R 2 -N 0 -L 20 -i S,1,1.25 --gbar 4 --mp 6,2 --np 1 --rdg 5,3 --rfg 5,3 --score-min C,-14,0 -p 1 --bowtie2-min-score 15 --read-mismatches 2 --read-gap-length 2 --read-edit-dist 2 --read-realign-edit-dist 3 --index-outfile ./tophat_out/tmp/left_kept_reads.mapped.bam.index --sam-header ./tophat_out/tmp/test_ref_genome.bwt.samheader.sam - ./tophat_out/tmp/left_kept_reads.mapped.bam ./tophat_out/tmp/left_kept_reads_unmapped.ba

[Devon Ryan]

I'm guessing that you missed a pipe, try:

/Users/erikfasterius/bin/bam2fastx \
    --all \
    ./tophat_out/tmp/left_kept_reads.bam|/Users/erikfasterius/bin/bowtie2 \
    -k 20 \
    -D 15 \
    -R 2 \
    -N 0 \
    -L 20 \
    -i S,1,1.25 \
    --gbar 4 \
    --mp 6,2 \
    --np 1 \
    --rdg 5,3 \
    --rfg 5,3 \
    --score-min C,-14,0 \
    -p 1 \
    -x test_ref \
    -|/Users/erikfasterius/bin/fix_map_ordering \
    --bowtie2-min-score 15 \
    --read-mismatches 2 \
    --read-gap-length 2 \
    --read-edit-dist 2 \
    --read-realign-edit-dist 3 \
    --index-outfile ./tophat_out/tmp/left_kept_reads.mapped.bam.index \
    --sam-header ./tophat_out/tmp/test_ref_genome.bwt.samheader.sam \
    - \
    ./tophat_out/tmp/left_kept_reads.mapped.bam \
    ./tophat_out/tmp/left_kept_reads_unmapped.bam

[erikfas]

That gives the exact same as above (i.e. "100 reads, of these:" etc). (I'm very much appreciating the help, btw, even tho we/you haven't found the solution yet =P)

[Devon Ryan]

At this point I haven't a clue. If you post the reads you're using and let me know the genome they're from then I'll have a look. Barring that, you might try posting this on the google group.

[erikfas]

The reads are reads_1.fq and reads_2.fq from the test_data that can be downloaded from the Tophat tutorial, which also contains the reference (test_ref.*.bt2).

[Devon Ryan]

For what it's worth, I can also reproduce this. Presumably this is specific to the test dataset, since otherwise I imagine someone would have reported this already. Either way, do ask on the google group that I linked to, since this really does seem to be a bug.

As an aside, I can recommend using STAR instead. It's vastly faster and produces equivalent to superior results.

Answer 1

[doctorbrassica]

I ran into this problem too and found a suggested fix here. It is a on-line amendment to the tophat script that worked for me and others: https://groups.google.com/forum/#!topic/tuxedo-tools-users/vG5Rn0IgxoA

[erikfas]

Ah, interesting. Exactly how do I change the Tophat code? Which file, where is it, etc? Do I have to somehow change the source code and compile myself, or is it some other way to do it?

[Devon Ryan]

You can edit the tophat file itself in any text editor (it's just a python script). At line 1415, change this:

        if read_params.color:
            bowtie_header_cmd.append('-C')

to this:

        if read_params.color:
            bowtie_header_cmd.append('-C')
        bowtie_header_cmd += ["-x"]

Note that indentation is important.

[erikfas]

Do you mean the Tophat executable or some other "the" Tophat file? Because I was under the impression that you can't edit executables.

[Devon Ryan]

The executable is just a text file in this case. You're correct that you can't edit binary files in a text editor (well you can, but you'll just break the file).

[erikfas]

It says "Unix Executable File" for me, so not a text file. Am I looking at the wrong file?

[Devon Ryan]

Just head the file. If it doesn't look like text, then you're looking at the wrong file.

[erikfas]

Ah, I just couldn't open it because of security options in OSX. Fixed that and added the line, it works now, even with 2.2.3, so thanks for the help!

[Bharat Iyengar]

In tophat 2.0.12 the file tophay.py has this

if not params.bowtie2:
            bowtie_header_cmd += ["--sam"]

if read_params.color:
            bowtie_header_cmd.append('-C')

if params.bowtie2:
            bowtie_header_cmd += ["-x"]

Is this equivalent to what you are saying (there is no condition in your case). Does bowtie_header_cmd have to be -x all the time?

Answer 2

[Hyun Jin]

Hi guys,

I'm also trying to set up Tophat in my MAC, and I encountered the exactly same problem.

I basically followed the instructions and ran "test file" provided in TopHat getting start page (http://ccb.jhu.edu/software/tophat/tutorial.shtml). Error message as follow

[2014-06-20 22:27:46] Beginning TopHat run (v2.0.11)
-----------------------------------------------
[2014-06-20 22:27:46] Checking for Bowtie
          Bowtie version:     2.2.3.0
[2014-06-20 22:27:46] Checking for Samtools
        Samtools version:     0.1.19.0
[2014-06-20 22:27:46] Checking for Bowtie index files (genome)..
    Found both Bowtie1 and Bowtie2 indexes.
[2014-06-20 22:27:46] Checking for reference FASTA file
[2014-06-20 22:27:46] Generating SAM header for test_ref
[2014-06-20 22:27:46] Preparing reads
     left reads: min. length=75, max. length=75, 100 kept reads (0 discarded)
    right reads: min. length=75, max. length=75, 100 kept reads (0 discarded)
[2014-06-20 22:27:47] Mapping left_kept_reads to genome test_ref with Bowtie2 
    [FAILED]
Error running:
/usr/local/bin/bam2fastx --all ./tophat_out/tmp/left_kept_reads.bam|/usr/local/Cellar/bowtie2-2.2.3/bowtie2 -k 20 -D 15 -R 2 -N 0 -L 20 -i S,1,1.25 --gbar 4 --mp 6,2 --np 1 --rdg 5,3 --rfg 5,3 --score-min C,-14,0 -p 1 --sam-no-hd -x test_ref -|/usr/local/bin/fix_map_ordering --bowtie2-min-score 15 --read-mismatches 2 --read-gap-length 2 --read-edit-dist 2 --read-realign-edit-dist 3 --index-outfile ./tophat_out/tmp/left_kept_reads.mapped.bam.index --sam-header ./tophat_out/tmp/test_ref_genome.bwt.samheader.sam - ./tophat_out/tmp/left_kept_reads.mapped.bam ./tophat_out/tmp/left_kept_reads_unmapped.bam

I noticed that version of TopHat(v2.0.11) Bowtie(2.2.3) and Samtools(0.1.19.0) are identical, so I just wonder this is some sort of bug.

Please post if you find any solution, I appreciate it

hyj

[Hyun Jin]

Hi Guys,

Downgrade of bowtie2 (2.2.3 --> 2.2.2)solved the problem!

[2014-06-20 23:19:32] Beginning TopHat run (v2.0.11)
-----------------------------------------------
[2014-06-20 23:19:32] Checking for Bowtie
          Bowtie version:     2.2.2.0
[2014-06-20 23:19:32] Checking for Samtools
        Samtools version:     0.1.19.0
[2014-06-20 23:19:32] Checking for Bowtie index files (genome)..
    Found both Bowtie1 and Bowtie2 indexes.
[2014-06-20 23:19:32] Checking for reference FASTA file
[2014-06-20 23:19:32] Generating SAM header for test_ref
[2014-06-20 23:19:32] Preparing reads
     left reads: min. length=75, max. length=75, 100 kept reads (0 discarded)
    right reads: min. length=75, max. length=75, 100 kept reads (0 discarded)
[2014-06-20 23:19:32] Mapping left_kept_reads to genome test_ref with Bowtie2 
[2014-06-20 23:19:33] Mapping left_kept_reads_seg1 to genome test_ref with Bowtie2 (1/3)
[2014-06-20 23:19:33] Mapping left_kept_reads_seg2 to genome test_ref with Bowtie2 (2/3)
[2014-06-20 23:19:33] Mapping left_kept_reads_seg3 to genome test_ref with Bowtie2 (3/3)
[2014-06-20 23:19:33] Mapping right_kept_reads to genome test_ref with Bowtie2 
[2014-06-20 23:19:33] Mapping right_kept_reads_seg1 to genome test_ref with Bowtie2 (1/3)
[2014-06-20 23:19:33] Mapping right_kept_reads_seg2 to genome test_ref with Bowtie2 (2/3)
[2014-06-20 23:19:33] Mapping right_kept_reads_seg3 to genome test_ref with Bowtie2 (3/3)
[2014-06-20 23:19:33] Searching for junctions via segment mapping
[2014-06-20 23:19:34] Retrieving sequences for splices
[2014-06-20 23:19:34] Indexing splices
Building a SMALL index
[2014-06-20 23:19:34] Mapping left_kept_reads_seg1 to genome segment_juncs with Bowtie2 (1/3)
[2014-06-20 23:19:35] Mapping left_kept_reads_seg2 to genome segment_juncs with Bowtie2 (2/3)
[2014-06-20 23:19:35] Mapping left_kept_reads_seg3 to genome segment_juncs with Bowtie2 (3/3)
[2014-06-20 23:19:35] Joining segment hits
[2014-06-20 23:19:35] Mapping right_kept_reads_seg1 to genome segment_juncs with Bowtie2 (1/3)
[2014-06-20 23:19:35] Mapping right_kept_reads_seg2 to genome segment_juncs with Bowtie2 (2/3)
[2014-06-20 23:19:35] Mapping right_kept_reads_seg3 to genome segment_juncs with Bowtie2 (3/3)
[2014-06-20 23:19:35] Joining segment hits
[2014-06-20 23:19:35] Reporting output tracks
-----------------------------------------------
[2014-06-20 23:19:35] A summary of the alignment counts can be found in ./tophat_out/align_summary.txt
[2014-06-20 23:19:35] Run complete: 00:00:03 elapsed

[erikfas]

Thanks, downgrading did it!

[Bharat Iyengar]

Hi.. I am running a slightly different command.

tophat -r 350 \
    --mate-std-dev 150 \
    --no-discordant \
    --no-mixed \
    --no-coverage-search \
    --transcriptome-index ../transcriptome_merged/merged2 \
    -T \
    -M \
    -p 20 \
    -o blood \
    ../bwtindex/Dre_nuclear_2 Blood_1.fq Blood_2.fq

I am getting the same error as you guys. This did not happen when I allowed genomic mappings (i.e not used the option -T), although it was a different file. I had sporadic problems with --no-discordant before but that error comes very late in the process. I used old version of bowtie2, 2.2.3 and 2.2.2; nothing works.