Is there a way to add filters such as article type ("only show meta-analysis/systematic reviews") to a PubMed / Entrez Esearch API search?

Question

[kolja.o37]

Thanks for taking the time to read this. I would like to know if it is possible to filter results from the PubMed API by e.g. article type (only show meta analysis/clinical trial/etc) or use the additional filters such as species

My code:

import requests
import json
db = 'pubmed'
domain = 'https://eutils.ncbi.nlm.nih.gov/entrez/eutils'  
query = "cancer"
retmode='json'

queryLinkSearch = f'{domain}/esearch.fcgi?db={db}&retmode={retmode}&term={query}&sort=relevance'  
response = requests.get(queryLinkSearch)
pubmedJson = response.json()

print(pubmedJson)

It works and is giving me a list of IDs. What I would like to do is add a filter now, e.g. to only show meta-analyses.

I didn't see anything regarding filtering in the API docs. When I check the link of the website it adds the following filter:

https://pubmed.ncbi.nlm.nih.gov/?term=cancer**&filter=pubt.meta-analysis**

But when I try to add this filter to my query I get the error:

{'ERROR': 'Invalid filter key: pubt.meta-analysis'}}

Does somebody know if this is possible? Thank you!

Answer 1

[LauferVA]

kolja.o37 , try the Entrez and Medline functionality in the Bio package in python3.

For instance, we could write:

from Bio import Entrez
from Bio import Medline
import pandas as pd

def filterPubmedArticles(keyword, start_year, end_year, pubtype):
    handle = Entrez.esearch(db='pubmed', term=f'("{keyword}"[Title/Abstract]) AND ("{start_year}"[Date - Publication] : "{end_year}"[Date - Publication]) AND "{pubtype}"[Publication Type]', retmax=1000)
    record = Entrez.read(handle)
    handle.close()

    id_list = record['IdList']
    if not id_list:
        print("No articles found - consider broadening query terms.")
        return
    else: 
        handle = Entrez.efetch(db='pubmed', id=id_list, rettype='medline', retmode='text')
        records = Medline.parse(handle)
        articles = []

# loop through records; grab desired fields (here just title & abstract)
        for record in records:
            title = record.get('TI', '')
            abstract = record.get('AB', '')
            articles.append({'Title': title, 'Abstract': abstract})

# now use pandas to write the resulting records to a DataFrame.
        df = pd.DataFrame(articles)
        df.to_excel('review_articles.xlsx', index=False)
        print(f"{len(articles)} review articles found. Output saved to 'review_articles.xlsx'.")

# Set values for the function inputs for demonstration purposes:
keyword = 'Pangenome'
start_year = '2021'
end_year = '2023'
pubtype = 'review'

filterPubmedArticles(keyword, start_year, end_year, pubtype)

If that doesn't work, consider putting meta-analysis in the keyword section, rather than article type, etc.

Answer 2

[GenoMax]

You can use Entrezdirect with following filter types. PubMed guide offers many ways of limiting searches.

$ esearch -db pubmed -query "cancer AND systematic review [PT]" 
<ENTREZ_DIRECT>
  <Db>pubmed</Db>
  <QueryKey>1</QueryKey>
  <Count>40601</Count>
  <Step>1</Step>
</ENTREZ_DIRECT>

You can get information about the articles by using

$ esearch -db pubmed -query "cancer AND systematic review [PT]" |esummary | xtract -pattern DocumentSummary -element Title,Value 
Prevalence and severity of anxiety and depression in Chinese patients with breast cancer: a systematic review and meta-analysis.    37448492    PMC10336240 pmc-id: PMC10336240;    10.3389/fpsyt.2023.1080413
Traditional herbal medicine for anorexia in patients with cancer: a systematic review and meta-analysis of randomized controlled trials.    37441530    PMC10333490 pmc-id: PMC10333490;    10.3389/fphar.2023.1203137

$ esearch -db pubmed -query "cancer AND meta-analysis [PT]"
<ENTREZ_DIRECT>
  <Db>pubmed</Db>
  <QueryKey>1</QueryKey>
  <Count>40282</Count>
  <Step>1</Step>
</ENTREZ_DIRECT>