I need to write crawler. I am newbie programmer. So I want to learn from your code. It's main goal of this job.
I prefer go lang. Parser needs run in parallel on several CPU cores simultaneously with different proxy per process.
It seems python has limits per 1 CPU.... and works much slower than compiled go lang or jvm. However I like Python syntax.
I will chose Scala or Python solution only if I can't find Go Lang programmer here.
Go Lang has default library for html dom or similar to jquery, I will prefer it more then regular expressions. As it easier to customize.