Fred's metabarcoding pipeline : Build the OTU table / keyerror in python script

Question

[jojo_ezz]

Hello, I am using Frédéric Mahé's metabarcoding pipeline at: https://github.com/frederic-mahe/swarm/wiki/Fred%27s-metabarcoding-pipeline .

Everything worked fine until I reached the last step. Unfortunately I am stuck at "Build the OTU" table step. When I launch the bash script that reads intermediary files and a python script (bash script below):

FASTA="coolproject_132_samples.fas"  
SCRIPT="OTU_contingency_table.py"
STATS="${FASTA/.fas/_1f.stats}"
SWARMS="${FASTA/.fas/_1f.swarms}"
REPRESENTATIVES="${FASTA/.fas/_1f_representatives.fas}"
UCHIME="${FASTA/.fas/_1f_representatives.uchime}"
ASSIGNMENTS="${FASTA/.fas/_1f_representatives.results}"
QUALITY="coolproject.assembled.qual"
OTU_TABLE="${FASTA/.fas/.OTU.table}"

python \
    "${SCRIPT}" \
    "${REPRESENTATIVES}" \
    "${STATS}" \
    "${SWARMS}" \
    "${UCHIME}" \
    "${QUALITY}" \
    "${ASSIGNMENTS}" \
    Smpl[0-9]*.fas > "${OTU_TABLE}"

I get in my terminal this error:

> root@Precision-Tower-7910:/home/coldsunshine/Desktop/mahe_translem#
> bash 7_otu_table.sh   Traceback (most recent call last):   File
> "OTU_contingency_table.py", line 262, in <module>
>     main()   File "OTU_contingency_table.py", line 249, in main
>     seeds, stampa)   File "OTU_contingency_table.py", line 184, in print_table
>     for amplicons in swarms[seed]: KeyError: '57ad7eeb4e072adfc4543438061d7e87c8ce7dc5'

Is anyone familiar with this kind of error when using this pipeline or anyone can help me figure out what is happening since I am no expert in python. The python script can be found on the same Internet page as the bash script at https://github.com/frederic-mahe/swarm/wiki/Fred%27s-metabarcoding-pipeline.

Please help me, I am stuck at the last step...

Answer 1

[Frédéric Mahé]

Hi, the regular expression that is used to parse swarm's output needed to be updated to reflect the fact that vsearch does not write a ";" at the end of fasta header lines anymore (">foo;size=123;" is now ">foo;size=123"). The python script at https://github.com/frederic-mahe/swarm/wiki/Fred%27s-metabarcoding-pipeline has been modified to accommodate that.

Answer 2

[jojo_ezz]

Thankfully Frédéric Mahé who developed this pipeline answered me.

The error was a missing "?" in the OTU_contingency_table.py script.

Just added a ? to the separator and the script worked perfectly:

separator = "_[0-9]+|;size=[0-9]+;| "

Need to be like this:

separator = "_[0-9]+|;size=[0-9]+;?| "