Closed - This job posting has been filled and work has been completed.
I need an imacro scraping task that will scrape imdb.com. This information is for personal research purposes only and I do have the right to copy it for those purposes. You must have the enterprise edition of imacro, which allows you to distribute the imacro player free to me (after I hire you). Other original scraping program may be acceptable.
The task will scrape all movies for all years, starting here:
For each movie, the task will scrape all the data available for each movie. To do this, follow the links at the bottom of the page for each movie, under the heading "explore more about . . . ", the links will only be active if there is available data there, so the task will follow the available links and scrape the data behind each one. Do not follow the links under the sub-headings "external links", "related items," or "professional services". Some of the links are to multiple pages, for example I want to scrape each review (some movies have several hundred reviews).
Data will be saved as many different csv files.
Again, I am open to alternatives here. I can accept an sql database table, but I will need to be able to produce excel files from it.
Before payment, the task needs to have run on at least 100 movies (at least half of which are from the contemporary era, to assure the task is scraping all available data), and I need to approve the output. Thanks.