Web Crawling Online Prices of Certain Products

Project Title: Web Crawling Online Prices of Certain Products

Project Description:
Our client would like to crawl online prices of certain products in several countries. The specifications of the crawling are not finally defined yet, but we already have an approximate scope, which is:
• Crawling of online prices for spare parts and accessories related to home appliances by one company

• Ca. 500 products of interest (will be provided by us)
– Provided with material number and/or product denomination

• 8-10 countries (mainly Europe but also China, India, Turkey and USA)

• 5-7 online shops/ trade places per country (will be provided by us)

• Output of interest:
– Product denomination / description
– Manufacturer
– Online price
– Currency
– Information whether value-added tax (VAT) is included or excluded
– Indication whether price is really offered or whether it is only a “starting from…” price
– If applicable: quantity in PCs
– If applicable: delivery costs
– Link under which price was found
– Date of retrieval

Therefore, we would like to clarify first question we have regarding such crawling project. May I ask you to answer the following questions:

• Do you crawl in the following countries?
– Germany
– Netherlands
– Spain
– Poland
– Italy
– France
– Turkey
– India
– China
– USA

• Where in terms of location do you crawl from?

• Do you have offices/ branches/ employees in the countries named?

• How do you deal with geo-blocking/ geo-fencing?

• What crawling software do you use?

• How you validate and qualify the crawled data?

• Do you offer data cleansing (rule and approach according to our specification)?

• How fast can results be obtained and data be delivered, e.g. for 50 online shops?

• Are there any constraints in terms of countries that cannot be crawled? And if so, what constraints in particular?

• What is your pricing model/ service fee?

• Is there anything else we need to consider?

Please do not hesitate, if you have further questions.

For similar work requirements feel free to email us on info@webscrapingexpert.com.