uploading bigwig file (converted from BED) to UCSC genome browser: Unrecognized format line 1

Question

[chrisclarkson100]

I have bed file data that was mapped with Bowtie 1 and the output of this was converted to bed files.

After having done my data analysis my lab need to upload the bed file data, in coverage format, to the UCSC genome browser i.e. the data are needed in 'bigWig' format.

So I went about converting the data from BED to bigWig as follows:

#bed to bedGraph
bedtools genomecov -i test.bed -g /storage/projects/teif/mm9_generic_data/mm9.genome -bg > test.bedGraph

head test.bedGraph
chr1    3000389 3000399 1
chr1    3000399 3000425 2
chr1    3000425 3000435 1
chr1    3000937 3000973 2

#bedGraph to bigWig
/usr/local/ucscutils/bedGraphToBigWig test.bedGraph /storage/projects/teif/mm9_generic_data/mm9.genome test.bw

#convert bigwig to wig in order to check contents
/usr/local/ucscutils/bigWigToWig test.bw test.wig

head test.wig
fixedStep chrom=chr1 start=3000390 step=1 span=1
1
1
1
1
1
1
1
1
1
1
2
2

I also did the same for nucleosomes. I don't know why but the resultant wig file was different to the above- see below:

 bedtools genomecov -i nucleosomes.bed -g /storage/projects/teif/mm9_generic_data/mm9.genome -bg > nucleosomes.bedGraph

head nucleosomes.bedGraph
chr1    3000018 3000019 1
chr1    3000019 3000034 2
chr1    3000034 3000035 3
chr1    3000035 3000044 7


  /usr/local/ucscutils/bedGraphToBigWig nucleosomes.bedGraph mm9_data/mm9.genome nucleosomes.bw
head nucleosomes.bw

/usr/local/ucscutils/bigWigToWig nucleosomes.bw nucleosomes.wig

head nucleosomes.wig
#bedGraph section chr1:3000018-3007577
chr1    3000018 3000019 1
chr1    3000019 3000034 2
chr1    3000034 3000035 3
chr1    3000035 3000044 7

However there's a problem with the submission of the bigWig files to UCSC genome browser:

Unrecognized format line 1 of file: generegulation.org/NGS/test.bw (note: chrom names are case sensitive, e.g.: correct: 'chr1', incorrect: 'Chr1', incorrect: '1')

As you can see from the content of all the above created files there is no 'Chr' in the place of 'chr'.... This error happens in both the case of both 'test.bw' and 'nucleosomes.bw'. I have googled this error and each time that someone has had this problem it does not seem to be the case that there is a problem with the capitalisation of 'chr'... See links below:

similar problem with a bigwig file: https://groups.google.com/a/soe.ucsc.edu/forum/#!topic/genome/zWYFn6yQ4p0

Does anything about my data immediately stand out as obvious such that the data is not uploadable to UCSC?

Answer 1

[ruisergioluis]

That is a problem that I faced some time ago. I consider UCSC genome browser a fundamental and central tool for data visualization that can have a strong impact on the final analysis result.

However, I realized that I spent a lot of time preparing the side files to be interpreted by genome browser, and occasionally some mistakes appear slowing down the visualization.

I started using BamInsight that I developed. It grabs the BAM file and creates the .bw and all the side files necessary to visualize your data. In a final step, it sends all the created files to the FTP server that you define at the start. It also takes into account stranded reads in cases of RNA-seq stranded datasets.

Take a look at BamInsight github page: https://github.com/rluis/BamInsight

All the information to run bamInsight is in the github page, but I send you below the command ready to use:

baminsight original mm9 [e-mail] bam_FIle.bam -FTPHOST yourftp.hostname.com 
-FTPUser ftpusername -FTPPassword ultrasecretpassword -FTPPort 21 -FTPPath  /path/to/be/allocated/baminsight/output

If you have stranded RNA data, another bamInsight mode can be used (stranded mode). Write me back in that case, and I explain how to use it. However, all the info is in github.

Take a look and test it, please. Your feedback will be important. It still in the experimental phase, but I believe it can be helpful.

Answer 2

[genecats.ucsc]

You can't use the file upload button from the custom tracks page for bigWig files. You have to upload them to an http (or ftp) accessible server somewhere, and the paste the url into the text box. You would paste something like this into the text box:

track name="name of track" type=bigWig bigDataUrl=http://path.to.my/file.bigWig

None of the "big" file formats like bigBed, bigWig, BAM, etc work with the file upload button they all must be web accessible. The reason for this is so we can perform byte range requests to the server they are hosted at, allowing us to only store the current viewing range of data instead of the whole file.

If you are already doing this I can take a closer look at your files to see what's going on.

If you have further questions about UCSC data or tools feel free to send your question to one of the below mailing lists:

• General questions: [e-mail]
• Questions involving private data: [e-mail]
• Questions involving mirror sites: [e-mail]

ChrisL from the UCSC Genome Browser

[chrisclarkson100]

Yes, I mean I am providing the link to my web site where the BiWig file is stored in the same way as you wrote. Here is the mistake that I get:

Unrecognized format line 1 of file: mywebsite/filename.bw (note: chrom names are case sensitive, e.g.: correct: 'chr1', incorrect: 'Chr1', incorrect: '1')

[chrisclarkson100]

It looks like it did not recognize it as BigWig initially. After adding "type=bigWig" the mistake disappeared, thank you!