Problem with fastq file

Question

[kamoltip.lao]

Dear all,

I tried to open the example RNA-seq raw reads by using the simple command like head, but I got the error as shown below

$ head ~/RNA-seq/Liu2015/SRR1272191_IDCLV_1.fastq.gz
??˲?Ʋ%8?_Qf2K1?)?/? gp+̪~??{?֣??p_ `?p ?K???&?[?΀???Z??????|;?????1??????_?Yu?fU???????խ:???????T??????????????{x????????[?X?t?\?????xV;?c???@?{<???N~????~?|?V?˩??Mo?&,X??v?q?w?k'?袰?Ύ?؍?8:[?ug??_k????뭶?7?V?q????Օ?YV???u?????^?Vz[?4???d?d?`?m????{?????Ew??vL??u?+֖?/?ʶ?VeY4?¦ٖO?x???*l?'?????q???C?=?
                                      ????f5???'m?  ?
                                                         ???d??o??d5???9X????σv8?Kaӽ+????`???|>?tG?{{>??s?<$??mz??
                                  }
??l???n?j??ț?\????+=??Z?eiSM??ö?-6v??????_??O?=lw???C?????#??lS??])??`?q?????x?a????l???n?????Z۫2o??}0??ME?Sx\?J/l?oYi??~????>؈ܠ??~??g钦?x??]?G??$w?o?????&{b?-Ά??T?eȮ?CW?NnV????T_?!???m????5?6?n??VU??t?w:<~?.??????Wz)r^i??Qta?=?M:?,_?w"?T?N;?=???ֶ?)L?z?Q?:???|?????e??????W?O????-v$.^?\?ˮ?ƕ(@?x?[???;S???    ?+᳊???ޕ¦??`??#?=qZ??o?'z??`???"?e?m????}i?2?K?{Y6??X??oL???G?~?a???m?Oq?x\?|R?Ů?q?S??,~???*펝?x????>znaӽ+???
ۙ?o???*Gp

I didn't find any problem when I used this read to run FastQC and TrimGalore.

Could you please give me a suggestion, What should I solve this problem?

Thank you so much,

Best,

Kamoltip

[wjidea]

seems like an unicode-encoding problem. what system are you using?

[ATpoint]

Output of zcat your.fastq.gz | head?

[kamoltip.lao]

I check it in the preferences of the terminal the default is UTF-8, I tried to enable all encodings but the result was the same. Thanks

[kamoltip.lao]

I ran zcat and the result was an error

$ zcat ~/RNA-seq/Liu2015/SRR1272191_IDCLV_1.fastq.gz| head
zcat: can't stat: ~/RNA-seq/Liu2015/SRR1272191_IDCLV_1.fastq.gz (~/RNA-seq/Liu2015/SRR1272191_IDCLV_1.fastq.gz.Z): No such file or directory

So, I tried to use gunzip and head it,

$ gunzip ~/RNA-seq/Liu2015/SRR1272191_IDCLV_1.fastq.gz| head
$ head ~/RNA-seq/Liu2015/SRR1272191_IDCLV_1.fastq
@SRR1272191.1.1 HWI-ST423:163:D06RFACXX:7:1101:1579:1996 length=101
NTTAAATATATGTCTTCTCCCTTTTCCTTTGGTGGAGACTTTCTTTTCAGCAGCCGGAGCAGATTTGTATTTAGTGTTTCTGAGCCGAGCAGAGCTCCTGG
+SRR1272191.1.1 HWI-ST423:163:D06RFACXX:7:1101:1579:1996 length=101
#1:==BDBF<AAAAF?AGB<GBDGEBE<CCHA<A88C1?CF<F?FDFFGC9F;?B66';A@97=)7;C:==A>@.7.7)>A:AC33;8;@###########
@SRR1272191.2.1 HWI-ST423:163:D06RFACXX:7:1101:1658:1997 length=101
NAGAACTTGAGGACTAGTCCGTTTCTCCAGCTCGTAAGCCTTGGGAAACACAGGATGCTCAGTTCCTGAGGGAATTATTTCAGCACCCTGTGAAACCAAAG
+SRR1272191.2.1 HWI-ST423:163:D06RFACXX:7:1101:1658:1997 length=101
#1=DDFFF?FFDHGGIIHIHGGCFEHHGIIDHIIBGE>DEDGGIIGIDFBFGIIDGIIIDGIE>DCGH>EEBBBB;;>D@BCCA5(,,9<:C>AC:998<<
@SRR1272191.3.1 HWI-ST423:163:D06RFACXX:7:1101:1507:1998 length=101
NCCACGTCATCAGCATCACGTACTCATCCCTGCACATCTCATGGAAGGCTGGACACCTCTTCTCACTACAAGGCTTCACCTCCTCTCCGGTGCCCTCGCAG

So, I have to gunzip every files that I want to look it, right?

Do you have any option that I can look at it without decompress?

Thanks for your kind help

[GenoMax]

So, I have to gunzip every files that I want to look it, right?

Do you have any option that I can look at it without decompress?

No you don't need to do that. Your system should have a program called zmore. You can look at compressed files directly by using zmore your_file.fq.gz (press the key q to quit out of zmore). A program called less is also smart enough to decompress the files on the fly so you could look at them by doing less your_file.fq.gz (press the key q to quit out of less).

[ATpoint]

Please put in some effort and use the code button (10101) to activate highlighting of your code snippets to improve readability. Yes, if you want to see compressed files with head you have to decompress, but things like zcat file.fq.gz | head are fine. No need to decompress the whole file. Alternatively use zless file.fastq.gz. As this comes down to a lack of background in Unix basics, I strongly recommend to spend some time on the basics of Unix command line and file formats/suffixes before you continue with your analysis.

[kamoltip.lao]

I'm sorry, this is my first post. I'll improve it next time. And thank you so much for your kind suggestion, I'm a beginner and don't have enough background in UNIX as you mentioned. I'll spend more time on the basics of Unix.

Thank you so much

[WouterDeCoster]

If the gunzip worked then also the zcat should, on a Linux system. You probably made a mistake in the path.

[ATpoint]

No need to apologize, I only intended to give recommendations. You'll see that well-formatted and well-written questions and comments have a much higher change of getting good answers :)

Answer 1

[kamoltip.lao]

Dear all,

Thank you so much for suggestions from everyone, I tried them all and I would like to inform everyone about my result.

For $ zcat test.fastq.gz | head it does not function in my case. It always shows the error. zcat: can't stat: test.fastq.gz (test.fastq.gz.Z): No such file or directory.

Anyway, $ zless test.fastq.gz and $ zmore test.fastq.gz work very well.

Thanks again for your kind help :)

[GenoMax]

Curious about what OS you are using? Looks like the zcat program you have is of non-GNU variety.

[ATpoint]

Ok, now I see. I suspect you are on a Mac, right? If so, the Mac zcat has that issue, don't ask me why. Better install the GNU coreutils, e.g. via brew install coreutils (homebrew) and then use gzcat (the g indicates the GNU zcat). That should solve the issue.

[kamoltip.lao]

You're right, I use macOS 10.14.2. After I installed GNU coreutils, I can use gzcat test.fastq.gz | head without any error. Thank you so much for kind help

Answer 2

[WouterDeCoster]

Your reads are compressed, as you can see from the .gz extension. So therefore it looks like gibberish if you use head. If you really want to see them you first have to decompress them, either on the fly as suggested by ATpoint with zcat your.fastq.gz | head or decompress the file and write the decompressed reads to disk using gunzip your.fastq.gz. The obtained uncompressed file your.fastq can be opened using head.