Web Data Extraction Services
Project Title: Web Data Extraction Services
I’m looking for a company to provide web data extraction services.
The target is a single site which must be scraped on a weekly basis. Broadly we expect the process to run a number of searches (a small number of 1000s) and follow through the result pages for each. Each result page has a well defined html structure with say 20 results on each. Each result can have one (say) half a dozen slightly different layouts with around 6 salient elements.
Each result points to a page with more detail (around another 6 items, some being extensive text others being image links and so on).
We want a job to run each week which will retrieve all the results and, where a result has changed, or is new to retrieve the data on the linked page.
I would probably want the scraper to run on AWS.
I hope that information is enough for you to provide an estimate for development costs and I look forward to your response!
For similar work requirement feel free to email us on email@example.com.