Hi guys,
Sorry I am new here and would appreciate it if someone can help me. For my research, I need to retrieve the abstract of papers with keyword Covid-19, coronavirus, ... and save the abstract along with the corresponding mesh terms.
What I have done so far I am able to retrieve the abstracts and KEYWORDS using the below piece of codes. However, later I got to know that the keywords cannot help me because for instance in the keyword I have "covid19", "Covid-19", "learning in pandemic", "learning tools for pandemic events" ,... As you see "covid19", "Covid-19" refer to one thing ---> covid-19 and "learning in pandemic", "learning tools for pandemic events" refers to the same thing---> pandemic and learning.
But as I need to retrieve a lot of data I cannot go through each keyword individually to map them to the same concept. Thats why I thought instead of retrivieing kewords I can use mesh terms (with the hope that it does the same).
But I am not able to get it from pubmed data base!!!
I tried different ways none of them was successful:
1. Here the mesh are blank
fetch = PubMedFetcher()
pmids = fetch.pmids_for_query('covid-19', retmax=10)
def retrieve():
path = '.\DeepLearning_Lesson1\pubmedfile'
while True:
for pmid in pmids:
# abstracts[pmid] = fetch.article_by_pmid(pmid).mesh
if fetch.article_by_pmid(pmid):
abstract = fetch.article_by_pmid(pmid).abstract
mesh = fetch.article_by_pmid(pmid).mesh
time.sleep(1)
t = threading.Thread(target=retrieve)
t.start()
And then I have tried this:
2.
pubmed = PubMed( email="myemail.com")
results = pubmed.query("coronavirus", max_results=1)
path = '.\DeepLearning_Lesson1\pubmedfile2'
import pandas as pd
def retrieve():
mydic = {}
for article in results:
js = article.toJSON()
jso = json.loads(js)
if jso['abstract'] != None and jso['abstract'] != '' and len(str(jso['pubmed_id'])) <= 10 and 'keywords' in jso:
if len(jso['keywords']) != 0:
with open(os.path.join(path, str(jso['pubmed_id'])+'.txt'), 'w+', encoding='utf-8') as fp:
fp.write(jso['abstract'])
mydic['pubid'] = str(jso['pubmed_id'])
mydic['keywords'] = jso['keywords']
This is the scenario that we can retrieve the keywords associated with the abstracts but as I explained above the keyword cannot help me.
3. I have also tried this, although could get the abstract but the mesh is empty or raising error if I change the code:
from Bio import Entrez
Entrez.email = "sgnbx@mail.umkc.edu" # Always tell NCBI who you are
pmids = [12654674 ,29058482 ,28991880 ,28984387,28551248]
handle = Entrez.efetch(db="pubmed", id=','.join(map(str, pmids)),
rettype="xml", retmode="text")
record = Entrez.read(handle)
abstracts = [pubmed_article['MedlineCitation']['Article']['Abstract']['AbstractText'][0]
if 'Abstract' in pubmed_article['MedlineCitation']['Article'].keys()
else pubmed_article['MedlineCitation']['Article']['ArticleTitle']
for pubmed_article in record['PubmedArticle']]
mesh = [pubmed_article['MedlineCitation']['MeshHeadingList']['MeshHeading']['DescriptorName'][0]
if 'MeshHeading' in pubmed_article['MedlineCitation']['MeshHeadingList'].keys()
else None
for pubmed_article in record['PubmedArticle']]
Any help is really appreciated.