Getting Watson Snps From Biomart

Question

[Andrea_Bio]

Hello

I was wondering if anyone could give me advice about how to get the watson snps from biomart. I have already emailed the biomart user group.

I presume the problem I am having is due to the size of the data. I tried downloading all of the watson snps. A file download started but downloaded at a rate of 1kb per second and timed out after 12 hours. I then tried requesting the data by email but still hadn't received an email within 2 days. I tried the same thing by requesting just a single chromosome and faced the same problems: nothing received by an email request and a very slow download of about 1kb per second which i gave up on after 12 hours as it seemed stuck at 10mb and obviously wasn't going to get me the data.

What am i doing wrong? I appreciate it is a large data set and a slow query with complicated joins in the table but that doesn't seem to be the problem as the data starts to download.

I have seen this query here which seems related http://biostar.stackexchange.com/questions/2356/problem-with-dbsnp-sequence-variation-download-using-biomart

I am also trying to get the watson snps via the ensembl perl api but there are 3mil and my code has only got 3k so far and I'm scared I'll have my ip blocked if i keep it running

thanks a lot

[Andrea_Bio]

Just had a reply from them. They are checking the issue. I will let you know what they say

Answer 1

[Prateek]

Try getting it directly from the source

The file is watson_snp.gff.gz

looks like this..

#gff-version 1 
chr1    JW  gt_novel    41921   41921   .   +   .   SNP BJW-   1117373;alleles G/C;ref_allele G;ref_counts 2;oth_counts 2
chr1    JW  genotype    42101   42101   .   +   .   SNP rs2691277.1;alleles T/G;ref_allele T;ref_counts 0;oth_counts 1
chr1    JW  genotype    45408   45408   .   +   .   SNP rs28396308;alleles C/T;ref_allele C;ref_counts 0;oth_counts 3
chr1    JW  gt_novel    46244   46244   .   +   .   SNP BJW-1120399;alleles T/C;ref_allele T;ref_counts 1;oth_counts 4
chr1    JW  genotype    47815   47815   .   +   .   SNP rs2691334;alleles A/C;ref_allele A;ref_counts 0;oth_counts 1

[Prateek]

#gff-version 1
chr1    JW  gt_novel    41921   41921   .   +   .   SNP BJW-1117373;alleles G/C;ref_allele G;ref_counts 2;oth_counts 2
chr1    JW  genotype    42101   42101   .   +   .   SNP rs2691277.1;alleles T/G;ref_allele T;ref_counts 0;oth_counts 1
chr1    JW  genotype    45408   45408   .   +   .   SNP rs28396308;alleles C/T;ref_allele C;ref_counts 0;oth_counts 3
chr1    JW  gt_novel    46244   46244   .   +   .   SNP BJW-1120399;alleles T/C;ref_allele T;ref_counts 1;oth_counts 4
chr1    JW  genotype    47815   47815   .   +   .   SNP rs2691334;alleles A/C;ref_allele A;ref_counts 0;oth_counts 1

[Prateek]

much better - thx for reformatting.

[Prateek]

don't forget to go through the 00readme.txt file

[Andrea_Bio]

thanks for the comment but i need the ensembl data really as I am getting their consequences at the same time as getting the data

[Andrea_Bio]

i awarded you the answer as I ended up having to get the snps this way. thanks a lot

Answer 2

[Pierre Lindenbaum]

retry or check your connection , it works fine here:

time curl  "ftp://jimwatsonsequence.cshl.edu/jimwatsonsequence/watson_snp.gff.gz" > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 27.2M  100 27.2M    0     0   114k      0  0:04:03  0:04:03 --:--:--  123k

real    4m3.558s
user    0m0.336s
sys     0m0.568s

[Andrea_Bio]

its not my connection thanks as I asked other people in different locations to try for me and they couldn't get it either.

Answer 3

[Neilfws]

If you tried this earlier today, try again. BioMart has been very unresponsive for much of the day; they might be having issues or doing maintenance.

[Andrea_Bio]

tried all week :)

[Andrea_Bio]

just had an email from them. they are looking into it for me