scatter plots with single chromosome gives IndexError

Question

[fenrir.sivar]

I'm using cnvkit from the BCBIO 1.0.9 pipeline to find a rearrangement on chromosome 8 and 21 of a sample against hg19, wgrs, 35x

I'm using the following commands to create some plots:

cnvkit.py scatter -s Sample.cns -o Sample.png Sample.cnr results in the ususal overview plot with all chromosomes in one plot

specifying a single chromosome chr1 results in a single plot of chr1 as expected

cnvkit.py scatter -s Sample.cns -c chr1 -o Sample.chr1.png Sample.cnr

cnvkit.py scatter -s Sample.cns -c chr21 -o Sample.chr21.png Sample.cnr

results in

IndexError: positional indexers are out-of-bounds error, same for all remaining chromosomes:
How can I get an overview per chromosome?


Traceback (most recent call last):
  File "/data/tools/bin/cnvkit.py", line 13, in <module>
    args.func(args)
  File "/data/tools/bcbio/anaconda/lib/python2.7/site-packages/cnvlib/commands.py", line 899, in _cmd_scatter
    **scatter_opts)
  File "/data/tools/bcbio/anaconda/lib/python2.7/site-packages/cnvlib/scatter.py", line 49, in do_scatter
    y_min, y_max, title, segment_color)
  File "/data/tools/bcbio/anaconda/lib/python2.7/site-packages/cnvlib/scatter.py", line 216, in chromosome_scatter
    show_gene, window_width)
  File "/data/tools/bcbio/anaconda/lib/python2.7/site-packages/cnvlib/scatter.py", line 341, in select_range_genes
    if cnarr else CNA([]))
  File "/data/tools/bcbio/anaconda/lib/python2.7/site-packages/skgenome/gary.py", line 369, in in_range
    return self.as_dataframe(next(results))
  File "/data/tools/bcbio/anaconda/lib/python2.7/site-packages/future/builtins/newnext.py", line 59, in newnext
    return iterator.next()
  File "/data/tools/bcbio/anaconda/lib/python2.7/site-packages/skgenome/intersect.py", line 85, in iter_ranges
    subtable = table.iloc[region_idx]
  File "/data/tools/bcbio/anaconda/lib/python2.7/site-packages/pandas/core/indexing.py", line 1478, in __getitem__
    return self._getitem_axis(maybe_callable, axis=axis)
  File "/data/tools/bcbio/anaconda/lib/python2.7/site-packages/pandas/core/indexing.py", line 2091, in _getitem_axis
    return self._get_list_axis(key, axis=axis)
  File "/data/tools/bcbio/anaconda/lib/python2.7/site-packages/pandas/core/indexing.py", line 2073, in _get_list_axis
    raise IndexError("positional indexers are out-of-bounds")
IndexError: positional indexers are out-of-bounds

Answer 1

[joseph.fass]

Huh. So a quick workaround: select a region smaller than the length of the chromosome / sequence. Chr6 is:

$ samtools view -H normal/reads.bam | grep chr6
@SQ     SN:chr6 LN:171115067

So I subtracted one from each end:

$ cnvkit.py scatter PN.cnvkit/primary.cnr -s PN.cnvkit/primary.cns -c chr6:2-171115066 -o PN.cnvkit/PN.chr6.png 
/home/ubuntu/miniconda3/envs/cnvkit-venv/lib/python3.6/site-packages/skgenome/tabio/tab.py:19: FutureWarning: read_table is deprecated, use read_csv instead, passing sep='\t'.
  dframe = pd.read_table(infile, dtype={'chromosome': 'str'})
/home/ubuntu/miniconda3/envs/cnvkit-venv/lib/python3.6/site-packages/skgenome/intersect.py:97: FutureWarning: clip_lower(threshold) is deprecated, use clip(lower=threshold) instead
  subtable.start = subtable.start.clip_lower(start_val)
/home/ubuntu/miniconda3/envs/cnvkit-venv/lib/python3.6/site-packages/skgenome/intersect.py:100: FutureWarning: clip_upper(threshold) is deprecated, use clip(upper=threshold) instead
  subtable.end = subtable.end.clip_upper(end_val)
Showing 12606 probes and 8951 selected genes in region chr6:1-171115066
Wrote PN.cnvkit/PN.chr6.png

Oddly enough, it's still saying it's starting at position 1 ... so I'm not sure whether the problem is at the left or right end.

Answer 2

[Brad Chapman]

Thanks for this report and apologies for missing this when initially posted. I was able to reproduce this problem and sent in a pull request to CNVkit that should fix the problem: https://github.com/etal/cnvkit/pull/393 Hope this small fix will work for you and comes in a reasonable time to help with your work.

Answer 3

[Eric T.]

This is now fixed in CNVkit version 0.9.6.

Answer 4

[joseph.fass]

Still seeing this in 0.9.6.dev0 ...

$ cnvkit.py scatter PN.cnvkit/primary.cnr -s PN.cnvkit/primary.cns -c chr6 -o PN.cnvkit/PN.chr6.png
/home/ubuntu/miniconda3/envs/cnvkit-venv/lib/python3.6/site-packages/skgenome/tabio/tab.py:19: FutureWarning: read_table is deprecated, use read_csv instead, passing sep='\t'.
  dframe = pd.read_table(infile, dtype={'chromosome': 'str'})
Traceback (most recent call last):
  File "/home/ubuntu/miniconda3/envs/cnvkit-venv/lib/python3.6/site-packages/pandas/core/indexing.py", line 2200, in _get_list_axis
    return self.obj._take(key, axis=axis)
  File "/home/ubuntu/miniconda3/envs/cnvkit-venv/lib/python3.6/site-packages/pandas/core/generic.py", line 3359, in _take
    verify=True)
  File "/home/ubuntu/miniconda3/envs/cnvkit-venv/lib/python3.6/site-packages/pandas/core/internals/managers.py", line 1341, in take
    indexer = maybe_convert_indices(indexer, n)
  File "/home/ubuntu/miniconda3/envs/cnvkit-venv/lib/python3.6/site-packages/pandas/core/indexing.py", line 2615, in maybe_convert_indices
    raise IndexError("indices are out-of-bounds")
IndexError: indices are out-of-bounds

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/miniconda3/envs/cnvkit-venv/bin/cnvkit.py", line 13, in <module>
    args.func(args)
  File "/home/ubuntu/miniconda3/envs/cnvkit-venv/lib/python3.6/site-packages/cnvlib/commands.py", line 914, in _cmd_scatter
    **scatter_opts)
  File "/home/ubuntu/miniconda3/envs/cnvkit-venv/lib/python3.6/site-packages/cnvlib/scatter.py", line 49, in do_scatter
    y_min, y_max, title, segment_color)
  File "/home/ubuntu/miniconda3/envs/cnvkit-venv/lib/python3.6/site-packages/cnvlib/scatter.py", line 216, in chromosome_scatter
    show_gene, window_width)
  File "/home/ubuntu/miniconda3/envs/cnvkit-venv/lib/python3.6/site-packages/cnvlib/scatter.py", line 341, in select_range_genes
    if cnarr else CNA([]))
  File "/home/ubuntu/miniconda3/envs/cnvkit-venv/lib/python3.6/site-packages/skgenome/gary.py", line 369, in in_range
    return self.as_dataframe(next(results))
  File "/home/ubuntu/miniconda3/envs/cnvkit-venv/lib/python3.6/site-packages/skgenome/intersect.py", line 92, in iter_ranges
    subtable = table.iloc[region_idx]
  File "/home/ubuntu/miniconda3/envs/cnvkit-venv/lib/python3.6/site-packages/pandas/core/indexing.py", line 1500, in __getitem__
    return self._getitem_axis(maybe_callable, axis=axis)
  File "/home/ubuntu/miniconda3/envs/cnvkit-venv/lib/python3.6/site-packages/pandas/core/indexing.py", line 2221, in _getitem_axis
    return self._get_list_axis(key, axis=axis)
  File "/home/ubuntu/miniconda3/envs/cnvkit-venv/lib/python3.6/site-packages/pandas/core/indexing.py", line 2203, in _get_list_axis
    raise IndexError("positional indexers are out-of-bounds")
IndexError: positional indexers are out-of-bounds

I installed via bioconda, not the Github repo ... could they be out of sync or something?

By the way ... the link in your comment immediately above this ... bizarrely ... goes to a home improvement site (after removing the space %20). Weird.

[Eric T.]

Thanks, I've fixed the link.

I don't know if v0.9.6 has landed on bioconda yet, but 0.9.6.dev0 is the development release prior to the stable 0.9.6 release, and the latter is the one you want. You can pip install it directly from GitHub and it should work if you've already installed the R dependencies.