GEM index fatal error

Question

[lyz10302012]

Hi,

I am trying to run GEM to create mappability file and get the following errors. I wonder if someone can give me a solution.

gem-indexer -i GRCh38_full_analysis_set_plus_decoy_hla.fa -o hg38_index
Creating sequence and location files...Fatal error: exception Failure("Command 'echo "-------gem-indexer_fasta2meta+cont" > hg38_index.log && gem-indexer_fasta2meta+cont -i GRCh38_full_analysis_set_plus_decoy_hla.fa -c dna --filter-function iupac-dna --strip-unknown-bases-threshold

Thanks,
Eric

[mvizue01]

I would very much like to know how to overcome this error as well I am trying to generate a Mappability file for GRCh38 as none is available from ENCODE.

the error that I encountered was:

Creating sequence and location files...Fatal error: exception Failure("Command 'echo "-------gem-indexer_fasta2meta+cont" > ./genome.log && gem-indexer_fasta2meta+cont -i ../WholeGenomeFasta/genome.fa -c dna --filter-function iupac-dna --strip-unknown-bases-threshold 50 --complement-size-threshold 2000000000 --compl

the error log does not contain sufficient data for a non specialist to debug this and there is precious little information on the tool.

The Binaries have been added to my PATH so access should not be an issue. Any constructive comments welcome.

Thanks,

Matthieu

[GenoMax]

This error message does not contain enough information for us to help you debug this either.

Have you looked at this past thread that may be of help: Where can I get ?or how can I make a mappability track for hg38 assembly

[mvizue01]

Genomax,

Thank-you for the reply. I have seen that link, I am trying to follow the tutorial that has been posted to this link to first index the reference before generating the mappability track. The error that I am encountering is linked to the gem-indexer command and not the gem-mappability command.

For information I am using the precompiled i3 binaries downloaded from the Github page for the project. This is being used on a Centos7 install with all resent system updates installed. For information the pypi installer option that is listed on this page does not work as the repo does not use SSH.

Do you have a better idea of what the issue is and what may be done to overcome the issue?

[Oleczkaff]

And I got the following error, while running same code:

GEM::Error (archive_builder_text.c:180,archive_builder_text_process_character) MultiFASTA parsing (genome/Homo_sapiens-GRCh38.p13/Homo_sapiens-GRCh38.p13_contigs.fa:46604068). Invalid character found ('>')

Any idea how to solve it?

Answer 1

[dineika_chandra]

I have recently run into the same error using the Linux-x86_64-core_2-20130406-045632 binary and bypassed it by doing the following:

1. Adding the working directory to the path export PATH=$PATH:$PWD
2. Running the tool on the low memory setting gem-indexer --max-memory 'force-slow-algorithm'
3. I also allocated 20 GB memory for this job

It took 2.5 days to generate the mappability wig files for hg38 on 1 node (1 task per node)

[artemd]

To everyone who is still having the same error as the original post

Fatal error: exception Failure("Command 'echo "-------gem-indexer_fasta2meta+cont

Make sure you follow the advice dineika_chandra gave and add the bin folder of the tool to the path (EVEN IF YOU RUN IT FROM THEN BIN FOLDER ITSELF)

So if you try to run the indexer using a command like this:

/groups/goroup1/ari/tools/GEM-binaries-Linux-x86_64-core_i3-20130406-045632/bin/gem-indexer  -i GRCh38.fa -o GRCh38.gem_idx

You need to run:

export PATH=/groups/goroup1/ari/tools/GEM-binaries-Linux-x86_64-core_i3-20130406-045632/bin/:$PATH

for the above command to work.

It took me far too long to understand the reason for this error, it's so frustrating there isn't any helpful details in the "fatal error" that is just thrown out.

[Ram]

Hi,

There was no reason to add this as a standalone answer. I've moved it to a comment on the appropriate post. Please be more mindful in the future.

Answer 2

[iAmphioxus]

I encountered the same error on one (running CentOS 7) of the two servers that I have tested. Also, I have tested both new (GEM-Tools static binary bundle 1.7.1 for i3) and old versions (GEM-Tools static binary bundle 1.7 for i3 and GEM-Tools static binary bundle 1.6.2 for i3) and observed the same error. The parallel test on the other server (running openSUSE 13.2) turned out to be fine. So I guess this error might be system-specific. But without a more meaningful error message, it is difficult to find out the real cause.

Answer 3

[pyjiang2]

I encountered the same problem here, and I found another thread in this post which I found useful:

Where can I get ?or how can I make a mappability track for hg38 assembly

rather than using gem-indexer, use gemtools index, which do not throw that error message to me. [still running now]

---

Well, although I get passed the index. When I run gem-mappability, it gives me an error saying

Loading index (likely to take long)...Illegal instruction (core dumped)

Now I do not know how to deal with it.

---

Updates: Maybe it is memory issue. When I allocate large memory, and run gem-indexer, and it works.