We are looking for someone to do some Python development for us on a contract basis. We are based in Washington DC and will need help rewriting code to streamline our ETL processes as well as some of our twitter analysis processes.
Our first priority:
We need the some existing python code to be automated. What we need:
o Data that is being collected in our AWS S3 buckets must be moved to another location to be processed.
o Several pre-existing content and user processes are run on the data.
o The resulting files are archived.
Ideally this would be done every hour or so.
We need a scheduled process to run on one of our AWS servers. This code will check an S3 bucket for JSON files. It should discern what type of data is in the JSON file (Facebook, twitter etc) and run a script (that already exists) against that JSON file. The results of this should be stored in 2 MySQL tables.
We would also like to create a log of this process in a MySQL table and a daily status/transaction summary email.
Our goal is to build a long term relationship with a contractor with the purpose of developing/improving our social media analytics infrastructure.
We are a one of the world's premier research and strategic consulting firms. We specialize in political polling and campaign strategy, helping political candidates, parties, advocacy groups, and ballot initiatives succeed across the United States and around the globe. Here is our website http://www.gqrr.com