Job Description

I need a person that can scrape VERY large data-sets.

web scraping, web data extraction, online directory website scraping,
data mining, screen scraping, web scraping bots and spiders, API

The first project will be to scrape and collect data from a premium
subscription data feed API (we already pay a fee for this datafeed
and we have an API key already).

We need to evaluate if you can access this API and can scrape and
store and archive the data.

we also need a person that can scrape and COMPARE and MATCH data from various data sources. So, if you scrape the same or a similiar data set from
multiple websites or data feeds... you need to be able to de-duplicate the data.

1) please tell me what software tools you use to scrape with.
2) please tell me the largest successful scraping and data warehousing projects you have completed in the past?

Skills: data-collection

Other open jobs by this client