What Raw Sequence File Formats Do You Work With?

Question

[Science_Robot]

I'd like to build a parser to load all output from various sequencer technologies into an database. There are many various file formats. I am mostly familiar with FASTQ and FASTA/QUAL produced by Illumina and 454, respectively. What raw sequence file formats do you work with?

[Pierre Lindenbaum]

You want to put some FASTQs (or whatever) in a database ? why ? what kind of data to you need to index ? What kind of information do you need to quickly find that won't be available using samtools or the Samtools API ?

[Science_Robot]

Forget about the database. I really just want to know what kind of raw sequence file formats are common.

[Science_Robot]

The reason being that I need random access.

Answer 1

[Istvan Albert]

Your the best strategy might be to convert everything to FASTQ and build your service for the FASTQ format. Each platform has utilities that convert to FASTQ.

Check out Screed as well, a simple read-only sequence database, designed for short reads

https://github.com/ctb/screed

[lh3]

+1. Screed is interesting. It would be even better if it supports gzip compression. Pretty much everything larger than 1GB is compressed on my disk.

[Science_Robot]

I'm trying to move away from flat files. Screed looks interesting. Wouldn't be hard to add compression.