The Way We Work
August 1, 2011 by Brian McDonough

When employers hire remote contractors, the same fear they face with in-house hires — Am I hiring the right candidate? — is often magnified, because hiring and managing remotely is a new skill for some. Remote work, however, offers one solution that generally isn’t available with an in-house candidate: the test hire.

We’ve been advocating the idea of “test driving” a contractor relationship for years. Here’s the nutshell:  Before you hire someone for a job that will span 100 hours, hire her to a two- or three-hour test job that lets you assess her skills, timeliness and communication style. Or test several top candidates and then hire the best for that major project. In general, you get the best return on that investment if the test job is work of actual value — something you can use, rather than merely a skills test.

But there are a lot of ways execute that idea. Since oDesk has about four contractors for every in-house employee, we thought we’d turn to our own managers for tips on how to effectively test prospective workers. After the jump, we’ll talk about web designers, software developers, project managers, product analysts and customer service reps.

How to Test a Web Designer

Shipra Kayan is an interaction designer at oDesk, and she hires web designers and user interface prototypers for long-term relationships.

Shipra Kayan, oDesk

“For those hires, I’ll usually have a project in mind. I break off a chunk of the project that I feel like the candidate should be able to do in five hours,” she says. “I give them the small task and a 10-hour limit. Anything less and you’re really not giving them a chance. I don’t do fixed-price test projects because I want to understand how fast they are at turning things around, what times of the day they work, etc.”

Kayan says she relies heavily on oDesk’s management tools during the test process.  “I stalk the work diaries, and have sometimes already made up my mind even without talking again to the candidate or seeing the end results,” she says. “If things are not going well, I contact them and stop the project immediately, paying them for the hours worked to date.”

How to Test a Project Manager

Matt Cooper, oDesk’s vice president of marketplace operations, says the key to test work is to “closely replicate the work that will be done on the main job.” He uses this technique to hire project managers:

Matt Cooper, oDesk

“A prospective project manager is given a scenario and asked to create a 30-minute presentation on how they would drive the program,” he says. After his typical interview process, which involves a detailed job posting, emails following up on promising applications, and Skype interviews, he hires his top three candidates for the presentation assignment. “In some cases, where it’s a lower-level job, I may test more. I’d say three times the number you need to hire is a reasonable ratio.”

The scenario he gives is actually the one he’s hiring for. This not only lets him identify the best candidate, but to take the best ideas from unsuccessful candidates — he paid for ‘em, after all — and incorporate them into the project.

“I usually do the test projects as a fixed price assignment, paying $100 for the presentation, setting the expectation that it should be three to four hours of work,” Cooper says.  “At the hourly equivalent, it’s equal, if not a slight premium, to what they would have earned in the full job.”

How to Test a Software Developer

Matt Cooper isn’t the only person at oDesk hiring developers, but you can’t beat his technique:

“We hire developer candidates on a real two-week project, with one of our existing developers looking over their shoulder,” Cooper says. “They work on real projects under supervision, and at the end of the two weeks, the other developers review their work and either vote them in or out.”

This has the collateral effect of building esprit de corps, making other developers — in-house and remote contractors — feel like they have a real stake in building a successful team, but mainly it gets quickly to the heart of whether a candidate is worthy.

“It’s been very effective,” Cooper says. “You don’t vote to hire someone if you think you’ll need to clean up their mess. And it has a ‘Lord of the Flies’ vibe that I find entertaining.”

How to Test a Customer Service Rep

Mike Barnett runs oDesk’s Customer Service department, and he says that a traditional, discrete test job doesn’t quite do the trick, but a well-governed probation period covers similar ground.

“We have a trainee program,” Barnett says. “Before we hire any rep we conduct an extensive interview over Skype, at least voice and ideally video.”

The customer service manager usually does the first interview, and a team leader — a senior rep out in the field managing the day-to-day workflow — handles the second interview.

Mike Barnett, oDesk

“We’re looking for the following skills: customer-service experience, strong writing and typing skills, a customer-centric mindset, empathy, strong listening skills, problem solving skills, and technical competency,” he says. “We ask them to demonstrate these abilities, or we look for them, during the interview.”

New reps are hired as trainees for 90 days. They’re limited to 20 hours per week and, after initial training, are deployed to help customers. Their team leaders are constantly monitoring and coaching them, and a group Skype chat lets everyone offer help as questions arise. After that 90 days, a trainee is either promoted to CSR Level 1, with the weekly limit bumped up to 30 hours, or the contract is ended. Barnett notes that the strong interview and training processes have ensured that he has never had to replace a trainee at the end of the trial period.

How to Test a Project Analyst

Subha Shetty, our Director of Project Management, hires contractors for a number of jobs, including product analyst, lead generation and documentation. She recently hired someone to evaluate our time-tracking tools.

Shetty notes that she starts with a simple assignment and a low hourly limit, but makes sure to give the contractor everything necessary to succeed. “That includes a list of specific tasks for the short term, a clearly agreed-upon time frame, and a consistent communication pattern.”

To pick someone for the fairly large job of evaluating our time-tracking tools, Shetty assigned a leading candidate to do a portion of the project before deciding to hire him to complete the work.

“I broke down the overall project into chunks and gave him the first chunk as a test,” she said. “It was to select the vendors to do the competitive analysis. I then judged his methodology, analytical ability and actual choice of vendors. I gave him five to 10 hours for this job. This helped me determine the quality of work and deliverable.”

Because she was satisfied with the results, Shetty increased the contractor’s weekly limit and assigned him the rest of the job. She mentions that how many candidates she tests depends on how many strong contenders she finds.  “If it’s a job for which the talent pool is more readily available, like lead generation,” she says, “I test with two to four candidates, giving them the same task and comparing their results.”

Tests for All Tasks

Every job requires a slightly — or dramatically — different test job, but there’s still no better way to see whether an apparently qualified candidate can handle the job than to observe her doing the job. If these test-hire tips aren’t enough for you, come back on Wednesday. We’ll pick the brains of oDesk marketing guru Erica Benton as she tells us the best way to use test jobs for key marketing positions, such as writers, public relations specialists and social media experts.

Meanwhile … tell us about how you test-hire candidates, or if you’re a contractor, tell us about the best experiences you’ve had with test jobs. The comments are open!

Brian McDonough

Freelance Writer

Brian McDonough has been a writer and editor for more than 15 years, and has managed teams of in-house and freelance writers for newspapers, magazines and web sites.

  • Pingback: Where to find talent for your St. Louis Real Estate Business: Give ‘em work! | Training site of Keller Williams Realty Saint Louis()

  • Pingback: Making the Leap From Freelancers to Digital Agency - oBlog()

  • Pingback: PreScouter CEO: Building A Great Remote Team Through oDesk - oBlog()

  • N Grant

    Hey Jay

    I hear you, I have tried elance in the past & the only way around that is to get them to create individual modules & not the whole system.

    “Unfortunately, many of the oDesk contractors that I

    • N Grant

      Just a quick followup to this post, I accidently said the budget was 1K, I have now been swamped, I guess people in Asia think that oz if full of rich dumb ppl.

      The really interesting thing is that some of the quotes I have got to create just 1 x company logo is more than 1K, these guys have got to be kidding.


      • The Little Penny (@thelittlepenny)

        Price should not only depend on quality but how the logo will be used. A logo that will become the corporate brand for a Fortune 500 company, therefore potentially impacting sales and marketing long-term, will cost a lot more than the logo for a small local business that doesn’t have as much impact on a large audience.

  • BenjaminJ

    The test job is very critical for evaluating on merit and skills and thus creating team of competent people who can not only work in their individual capacities but can also help with their team.

  • Pingback: Hire Your Right Hand: How to Find & Use Virtual Assistants |

  • Jay

    I like the ideas presented in this article, but found the section on “How to Test a Software Developer” to leave me with more questions than answers. For example, one problem I have found with using oDesk for software developers is the issue of having to turn over access to your code. That is what I would like to know how other people are handling? An organization’s code base is often their most valuable asset. Moreover, anyone with access to that code is often given a peak into the inner workings of business processes that are secret. Unfortunately, many of the oDesk contractors that I’ve worked with have no problem accepting a project, making promises, and then disappearing.

    • Michael S.

      Well until I read Jay’s questions I though I was alone with this problem.

      Having the source code placed with an escrow service is money well spent. It’s a lot cheaper than paying to have the code written three times as has been the case with me.

      Let me back up. I was with vWorker as an Employer for years and had successful projects, but when it came to source code, the contractors always agree immediately to hand over the code at the end of the project. Somehow the horse got before the cart and the programmer had his money and all I had was the working program.

      Of course we wouldn’t need source code if we knew for sure the programmer will be there for the life of the software.

      I look forward to hearing answer to this thread.

      • Eric Johannsen

        Use a software version control system (e.g. Subversion) and require them to check in the code, in a working state, nightly. Have your existing programming team (if you have one) review the checked-in code daily during at the start of the relationship, then do a code review for each milestone deliverable. If you don’t have programmers on staff, at least you can see that code is entering a repository that you have access to (and control). There are many very low cost hosted version control solutions out there.

  • SEO Company BD

    I think these are the best ways to select the right contractor. But unfortunate few of the employers post fixed price job or mentioned test task in the job description. And honest, I observed that they don’t check all the cover letter they got. I have submitted lot of bids with completed test task mentioned in the job description but very few times I got response. even sometimes I do “Propose Different Terms” and reduce my bid rate only to draw attention of my employer.

    Hope all the employers will follow these nice post. Btw, thanks to all.

  • Pingback: The Test Job: How oDesk Hires Contractors | Blogger Spot()

  • Pingback: The Test Job: How oDesk Hires Contractors | HandyPostings()

  • Tamara

    Editorial Note: The followup post, Test Hire 2, is actually slated for this Wednesday, August 3, 2011. (Not Tuesday, August 2, 2011, as originally stated.) Sorry for any confusion.

  • Pingback: Testul de munc?: Cum oDesk angajeaza Contractori | ro-Stire()