How determine to read specific size from file using R language

Question

[emadelhewihy]

need help? , my problem is that i have fastq file with size 50 Gigabyte i cannot read it using R because my ram is 8 giga, so i want to read specific size from it and make it as stream to read x size until i finish operation on all the 50 giga !

[GenoMax]

You could split the original file into smaller chunks.

[emadelhewihy]

how i do that using R !? is there function to do that ?

[GenoMax]

You could use reformat.sh from BBMap suite to break the files into smaller chunks. reformat.sh will work on Windows/Unix as long as you have Java installed. Look at the "sampling" options in in-line help for reformat.sh.

[WouterDeCoster]

Not sure why your question is about R and you also tag Python but in Python you could just iterate over the file without loading it into memory first. I don't know about R though.

[emadelhewihy]

i also tagged python to know how to do that in case of i did not found that by R, so is there link to do that by python?

[WouterDeCoster]

I assume you have no Python experience? If you have, the Biopython tutorial and cookbook would be worth going through.

Answer 1

[jomo018]

You can use read.table() with nrows and skip tags.

[emadelhewihy]

the file is Fastq extention so it does not have specific schema like TSV or CSV it look like as string

[WouterDeCoster]

jomo018 just gave you the solution. No need to explain how a fastq file looks like.