A mining pool server is going to consist of three pieces of software:
The Bitcoin Daemon. Good old bitcoind will be responsible for interacting with the bitcoin network by attempting to produce new blocks. bitcoind will also arrange for payments in and out of the mining pool.
The Pool Server. Bitcoind isn’t equipped to handle multiple miners connecting, so a pool server program will serve as an intermediary between the miners and bitcoind. It will keep track of the number of shares (or parts of the work) that the miners submit, so that you can work out sharing the profit. I used ‘pushpool’ because it seems to be the most widely used.
The Frontend. You want users to be able to register accounts, manage their payouts, and see the pool’s performance online. A frontend will interact will interact with the poolserver and read the database to allow your miners to interact with the pool via the web. I will be using SimpleCoin (or more specifically, a fork which I am calling SimpleBTC – more on that later), a PHP-based frontend.