Illuminaio Readidat Error In Data.Frame Meanbindata: "Arguments Imply Differing Number Of Rows"

Question

[Cyriac Kandoth]

When running illuminaio's readIDAT function on certain IDAT files, it fails with this error:

Error in data.frame(MeanBinData = c(3514.35571289062, 1911.96472167969,  : 
  arguments imply differing number of rows: 568460, 57647

This error shows up in only a quarter of ~1400 IDATs I am working with, from a specific batch. So my hunch is that there is something wrong with the IDATs themselves, rather than the readIDAT function. But I'm unfamiliar with the IDAT format specs. I can't find anything helpful online. Can someone here share any insight?

The numbers "568460, 57647" in the error message are shared across these IDATs. These are all from Illumina's HumanHap550 SNP array.

Update: The subset of IDATs that Illuminaio fails to parse are from the older HumanHap550 v1 BeadChip, while the ones that work are from HumanHap550 v3. The firmware/software that generated those older IDATs are also possibly obsolete.

[Mike Smith]

Hi Cyriac, I'm one of the illuminaio authors. Sorry for not noticing this earlier. If you'd like me to look into this further, could you provide me with an IDAT that fails and one that doesn't and I'll try to diagnose the problem.

[Cyriac Kandoth]

Thanks Mike. I'll send you a PM with a link containing a good/bad IDAT. While debugging, I discovered that the IDATs that Illuminaio fails to parse, are from an older HumanHap550 v1 BeadChip. While the IDATs that work fine, are from the newer HumanHap550 v3.

[Chris Miller]

often adding a 'sep="\t"' argument to your read function will sidestep these errors in R. (assuming the files are tab-delimited)

[Sean Davis]

The IDAT is a proprietary Illumina microarray format, similar in spirit to .CEL files, so a sep="\t" won't do the trick.

Answer 1

[Mike Smith]

I've patched illuminaio to accommodate the problematic chips. Version 0.5.5 of illuminaio should be able to cope with them.

The HumanHap550v1 data is stored in an encrypted format that I've only every seen used for expression arrays, while the v3 files are in Illumina's more common binary format. You'll see that there are quite a few more fields in the output from the older data (hence why the files are so much larger), but the 'MeanBinData', 'DevBinData' and 'NumGoodBeadsBinData' columns are those you need if you're interested in the same information from both platforms.

The error arose from the presence of an extra data field, 'IllumicodeBinData', which doesn't have as many values as the rest. This caused R to fall over when it tried to make a data.frame of all the values. If you're interested in this column you'll find it in the 'Extra' slot in the output from readIDAT().

Let me know if you have any more problems.

Answer 2

[Sean Davis]

I'd suggest isolating one of the files that is problematic and reading it in with readIDAT. Then, read in one of the non-problematic ones and compare the outputs. In particular, the ChipType should be the same for both and if it is not, something is amiss. It may be that you have a mix of array types and that perhaps one of the array types is an expression array?