How to retrieve article ids list from pubmed

Question

[aqsaawan.459]

Hey, I'm doing a project which includes data mining but for that I want to build a crawler which extract pubmed articles ids using a single term like cancer or tumor? (semantics based ) I'm currently working in python. If someone is familiar with this issue kindly help me

[GenoMax]

Please use ADD COMMENT/ADD REPLY when responding to existing posts to keep threads logically organized.

New questions should be posted to a new thread.

[aqsaawan.459]

okay got it! thanks

[WouterDeCoster]

If an answer was helpful you should upvote it, if the answer resolved your question you should mark it as accepted.

[aqsaawan.459]

now I want to upload these ids to mysql database, But I do't know hoe to connect python to mysql?

[Arup Ghosh]

This tutorial will help, https://www.tutorialspoint.com/python/python_database_access.htm . Please mark the answer as accepted.

Answer 1

[Arup Ghosh]

Use Biopython.

An example, cancer as a search term.

from Bio import Entrez
Entrez.email = "[e-mail]"
handle = Entrez.esearch(db="pubmed", term="cancer", retmax="10", sort="relevance", retmode="xml")
records = Entrez.read(handle)
print(records["IdList"])

[WouterDeCoster]

I would like to suggest that you write Python3 in your examples (using print as a function) to make this code snippet valid on all python versions. Python2 will only be around for limited time, so better bite the bullet now and get used to Python3 syntax.

[Arup Ghosh]

thanks for the suggestion.

[aqsaawan.459]

What if I want urls of a specific word instead of ids?

[Arup Ghosh]

Sorry, what do you mean by "urls of specific word"? If you are talking about article URLs, use https://www.ncbi.nlm.nih.gov/pubmed/ as base URL and add the fetched PMIDs after it.

[venkatarao142152]

Hlo sir. I want to extract specific gene articles. for every gene I have number of articles so how can I extract pubmed id and abstract of the article alone. could you please suggest how to extract id and abstract of the article alone. thanking you sir

Answer 2

[Maria_Levchenko]

You can also try out Annotations API from Europe PMC. It allows you to retrieve a list of PMIDs for artciles that contain text-mined terms. Diseases (including cancer or tumor) is one of the annotation types. So for retrieving article iDs for cancer you could use the following response URL: https://www.ebi.ac.uk/europepmc/annotations_api/annotationsByEntity?entity=cancer&filter=1&format=ID_LIST&pageSize=4

[venkatarao142152]

how can I download all the abstracts belongs to the cancer articles sir.