Less than 1 week –
Less than 10 hrs/week –
I need an expert for creating a reliable scrpay based crawler.
This Crawler should crawl and analyse ALL Pages of a given project.
for each found page within the given project different elements should be extracted and collected.
- all Links (<a>) on the Page and all data for each link like href, rel, css-class, title, anchortext
- all headings - H1- H6 + Text within <h1>...</h1>
the crawler should be very reliable in crawling a hughe amount of different websites