Doctors Contact List, Doctors Email List, Physicians Database Scraping
Project Title: Doctors Contact List / Physicians Database Scraping verified from Healthgrades.com
The goal of this project is to build out a spreadsheet that lists all associations of Conditions Treated and Procedures Performed for each medical specialty.
To do data verification, we need to complete all steps below for each of the specialty links on this page http://www.healthgrades.com/specialty-directory:
Go to the New York, NY location page for the specialty. For example, if “Allergy Immunology” had been selected as the specialty, you’d see this page where locations are listed (http://www.healthgrades.com/allergy-immunology-directory). Visit the link to New York, NY on this page (see screenshot below).
On the results page, visit the first link to a doctor’s profile, for example,:
Note that we will only need to visit ONE PROFILE PAGE for each specialty. You do not need to loop through multiple profile pages for different doctors.
On the profile page, there are two links we need to follow to aggregate data from the pages that result: Procedures Performed and Conditions Treated (see image below):
Also note the specialist name for the specialty (i.e. “Allergy Immunology” specialists are called “Allergist-Immunologist”). That data can be found here:
When those links are clicked, they will open a page like this:
On this page, we need to get all the values (separated by the | character) for both “Procedures Performed” and “Conditions Treated”.
Start back at the beginning and repeat steps 1-5 again for the next specialty, until you have the list Procedures and Conditions for every specialty. Note that some specialties will not have this information, which is okay, in those cases just skip them.
The final output should look something like this:
Specialty, Specialist, Procedures Performed, Conditions Treated
For smililar work requirement feel free to email us on firstname.lastname@example.org