Error When Running Fastx (Fastx_Quality_Stats)

Question

[neal.platt]

When running fastx, in particular fastxqualitystats, i am generating an error that says:

fastx_quality_stats: failed to open input file 'small.rna.fastq': Value too large for defined data type

My input file is close to 10Gb. I am running this locally on my desktop (4 GB RAM). Is this a memory error? I know I can run the fastx utilities on Galaxy. I am just hoping to speed things up by doing things locally.

Any advice is greatly appreciated.

[Istvan Albert]

cut your files into smaller pieces (man split) and see if the problem persists.

[neal.platt]

I split the file and things appear to work. Looks like I will need to be running on a more powerful computer. Thanks for the help Istvan.

Answer 1

[lelle]

I just did a quick search of the fastx source code and can not find the error message you are seeing. It seems to come from the operating system. A google search of "Value too large for defined data type" shows that the messages also appears for other programs. It seems to be a problem that occurs with large files on 32bit systems.

So if you are running a 32bit operating system you might not be able to open a 10 GB file correctly.