RuntimeError: dictionary changed size during iteration

Question

[jaqx008]

Hello all. I ran into this error and have been trying to fix it with numerous online suggestions but non really worked. I am trying to use a python script to find siRNA overlap in my alignment file. But I keep getting the following error. some of the languages to fix it from online suggestions seem complicated and I am needing help in moving forward to get the overlap.

command:

python overlapping_reads.py --input X.sorted.bam --minquery 21 --maxquery 21 --mintarget 21 --maxtarget 21 --overlap 19 --output X.sorted.siRNA.fasta

Error:

Traceback (most recent call last): File "overlapping_reads.py", line 128, in <module> args.mintarget, args.maxtarget, args.overlap) File "overlapping_reads.py", line 46, in __init__ overlap=self.overlap) File "overlapping_reads.py", line 77, in query_positions for genomicKey in self.alignement_dic.keys(): RuntimeError: dictionary changed size during iteration

Thanks

[Bastien Hervé]

Your overlapping_reads.py code would be helpful :) Thanks

And your python version.

[jaqx008]

python version is Python 3.6.3. overlap.py code is bellow. I have tried to down grade the python but the 3.6.3 still shows in my terminal. couldn't paste the entire code because of the number of characters. this is only part of the code.

import argparse
from collections import defaultdict

import pysam

def Parser():
    the_parser = argparse.ArgumentParser()
    the_parser.add_argument(
        '--input', action="store", type=str, help="bam alignment file")
    the_parser.add_argument(
        '--minquery', type=int,
        help="Minimum readsize of query reads (nt) - must be an integer")
    the_parser.add_argument(
        '--maxquery', type=int,
        help="Maximum readsize of query reads (nt) - must be an integer")
    the_parser.add_argument(
        '--mintarget', type=int,
        help="Minimum readsize of target reads (nt) - must be an integer")
    the_parser.add_argument(
        '--maxtarget', type=int,
        help="Maximum readsize of target reads (nt) - must be an integer")
    the_parser.add_argument(
        '--overlap', type=int,
        help="Overlap analyzed (nt) - must be an integer")
    the_parser.add_argument(
        '--output', action="store", type=str,
        help="Pairable sequences")
    args = the_parser.parse_args()
    return args

[Devon Ryan]

Can you just give us a link to github or something? BTW, the function you posted isn't terribly informative for your question.

Answer 1

[Bastien Hervé]

Why downgrade your python version ? Actually that could be great if you could show your code till line 77. Or at least between line 70 and line 80

File "overlapping_reads.py", line 77, in query_positions for genomicKey in self.alignement_dic.keys(): RuntimeError: dictionary changed size during iteration

With the given error self.alignement_dic.keys() and RuntimeError: dictionary changed size during iteration

I google it and I found this : https://stackoverflow.com/questions/11941817/how-to-avoid-runtimeerror-dictionary-changed-size-during-iteration-error

In python 3 and above you need to loop over a dictionnary like this :

for key, value in dic.items():

To only loop over your keys :

for i in list(dic):

[jaqx008]

more from around line 77

coord = read.reference_start
                pol = 'F'
            dic[(chrom, coord, pol)].append(read.query_sequence)
    for key in dic:
        dic[key] = set(dic[key])
    return dic

def query_positions(self, bam_object, overlap):
    all_query_positions = defaultdict(list)
    for genomicKey in self.alignement_dic.keys():
        chrom, coord, pol = genomicKey
        if pol == 'F' and len(self.alignement_dic[(chrom,
                                                  coord+overlap-1,
                                                  'R')]) > 0:
            all_query_positions[chrom].append(coord)
    for chrom in all_query_positions:
        all_query_positions[chrom] = sorted(
            list(set(all_query_positions[chrom])))
    return all_query_positions

def pairing(self):
    F = open(self.output, 'w')
    query_range = self.query_range
    target_range = self.target_range
    overlap = self.overlap
    stringresult = []
    header_template = '>%s|coord=%s|strand %s|size=%s|nreads=%s\n%s\n'
    for chrom in sorted(self.chromosomes):
        for pos in self.all_query_positions[chrom]:
            stringbuffer = []
            uppers = self.alignement_dic[chrom, pos, 'F']
            lowers = self.alignement_dic[chrom, pos+overlap-1, 'R']
            if uppers and lowers:
                for upread in uppers:

[Bastien Hervé]

Hard without data so I can't test what i'll give you.

Modify :

for genomicKey in self.alignement_dic.keys():

By :

for genomicKey in list(self.alignement_dic):

[Bastien Hervé]

Did that solve your issue ?

[jaqx008]

I am just going to try now. Does that mean i Should edit the script and replace genomicKey in self.alignement_dic.keys(): with genomicKey in list(self.alignement_dic):? I don't understand

[Bastien Hervé]

Yes (Aaaaah Biostars... I can't just answer "yes" because my message is too short, so here some text...)

[jaqx008]

I understand now. It worked. Thanks so much. It ran and gave me the output. Thanks Bastien

[Bastien Hervé]

Nice !

Some explanation from here : https://github.com/kennethreitz/legit/issues/212

kapinga's comment :

dict.keys() returns a generator in Python 3, but in Python 2, dict.keys() returns a list.

[jaqx008]

Oh Ok. Will read it. but it sounds good for the python version