Renewable energy data download from website into Excel file
Closed - This job posting has been filled and work has been completed.
We are a start-up working on smart and renewable energy.
This job requires the contractor to download renewable energy generation data from a market platform's website and save it in a specific format in an Excel file. The goal is to have this data for each hour of the year 2012 and 2013.
What we are ultimately trying to show with this input is that renewable energy is negatively correlated (i.e. tends to reduce) power prices.
The data is here: http://www.transparency.eex.com/en
It is downloadable from the menu on the right (arrow). One downloadable file represents the generation data for one day, as a table of 4 times 92 data points (power generation in quarter hours from 4 network operators).
For each of solar and wind, this download has to be done 365 times for 2012 plus however many days have already passed in 2013, approximately 250 days. That makes 615 downloads for solar, and 1230 for solar and wind together.
For each download, a short one-word captcha has to be entered. Once "every week", the date of the day added should be added in the final Excel file so it can be matched with the download for that day. We will do some spot-checks on whether the dates are matching the data.
In Excel, the data should be summarised into hourly sums for all columns (sum of 4 quarter hours and 4 network operators (the latter are in the columns)) and provided as one vector of around 8760 hourly values for 2012, and one vector for 2013.
The work can probably be sped up significantly if you are proficient in Visual Basic, Python or similar: you can download the Excel files for each day individually and write a little program (using the file names which contain the download time so they can be sorted) that reads the data from the individual files, manipulates it as desired, and writes it out into a summary file automatically.
The project should be delivered in two chunks. Upon successful delivery of 2012 data for $15, the successful applicant will receive another $5 for 2013 data.
We are looking forward to your offer!