Good Technical Interviews Require Good Technical Interview Questions
We're hiring at Microsoft UDC and interviewing a lot of people. Which is why this recent article on Coding Horror stood out to me, on the surprising fact that many software engineers struggle when it comes to actually writing code.
I haven't actually interviewed any of the people coming in yet, so I'm not really talking about any candidates that we've looked at because I have no experience with them. It was a timely article, though, and I am familiar with the struggle of the interviewer, having done it a lot at Mozy.
It's clear from the article, if you aren't already convinced, that asking interviewees to code as a part of the selection process is an absolute must. I'd also say, as a sidenote, it's also an indication of a serious software company. If you are interviewing with a company who doesn't test your programming skills, wouldn't you wonder how many people are on your team who can't code?
Anyway, coming up with good questions can seem hard, though. Some things to avoid:
- Complex problems that require even good engineers a lot of time to solve. You probably only have about 20 minutes for this question, which probably really only leaves the candidate about five to figure out how to solve the problem, and another fifteen of implementation and discussion. "Implement an ORM framework" is probably too complex or big.
- However, slightly complex problems can be good questions, if you provide enough background to cut the question down to the time you have.
- Domain-specific problems aren't that good at figuring out if someone can generally code. An outstanding C programmer who doesn't know C++ won't know how to implement their own insertion operator, even though they might be awesome and could easily learn it if they needed to. Even a great developer who has spent his whole life optimizing 3D rendering engines for computer games may have never once connected to a database, so he might not know anything about how to write a SQL query.
- Know the problem well enough to determine if any proposed solution will work; don't shut down a candidate just because he solves the problem differently than how you would do it.
The great thing is, apparently even simple problems can be good enough to find out if someone knows how to code. I wouldn't recommend you rely solely on the simplest, but coming up with a decent set of problems isn't too tough if you give it some thought. Here's some suggestions:
- Consider the source of your technical question and how long it took you to solve it. If you dredged it up out of your past experience, it might be a pretty good question because it obviously has some practical application. If you were able to come up with a solution to the problem after thinking about it for just a few minutes, and additionally if one of your colleagues is able to do the same, the problem is not too difficult for an interview question.
- This doesn't mean your candidates will easily get it, however. Don't get discouraged. If people on your team can solve this problem but your candidate cannot, that means the candidate is not of the same caliber as people on your team. That's not a good fit.
- It IS true that some people don’t think as well under pressure as they do on their own, or that your problem might just be in an area that they aren’t familiar with. It’s good to have a set of simpler questions on hand to ask them in case they stumble on the harder one. If they don’t get one of your hard questions but they can solve a simple one, at least you know they can solve SOME coding questions.
- Practice your questions with your colleagues. This will help you with clarifying the context and purpose of your question, make sure you structure it so there's enough time to answer the question, and make sure the question isn't too hard, too easy, or unclear.
- Choose questions that help you evaluate core competencies: algorithm and class design, proper use and understanding of data structures, pointer math, modulus math, understanding of data types, string manipulation, threading and thread safety, etc. Be sure your questions are designed to help you select the kind of people you want.
- For this reason, I don't like to ask really deep language-specific questions anymore (although if a candidate claims to be an expert in a specific language, these are fair game) or API-specific questions. Even if I use C++ and WxWidgets this year, I may not next year. What I want is someone who knows how to program. It doesn't take long at all for a good programmer to learn a new language; probably much less time than it takes to manage a bad programmer out of your team.
What really surprised me when I started interviewing at Mozy was that I'd heard all this stuff before, but had a hard time believing it. That is, until I experienced it firsthand during interviews there. I remember interviewing a candidate who had several years of experience in game development. Game development! These guys are the super-elite of developers, right? So I wondered if asking him to reverse a string might be too easy for him.
It wasn't.
Not a complex string reverse. Not a unicode string. Not in some obscure language he wasn't strong in. Just a basic string reverse in C, with a null-terminated char* argument containing the string to reverse. I even gave him the flexibility to reverse it in-line or to make a copy. I was surprised when he really struggled with this.
He wasn't the only one. String reverse is not a hard function, but I was surprised how many people struggled with it. So don't be surprised if people struggle with questions you think they should be able to answer. The numbers seem incredulous, but apparently it is really hard to find a good developer. So don't give up, my son.
Interview Process Too Rigid? Ye Shall Know It By The Fruits
I don’t suppose that every company gets concerned that their hiring process is too rigid. For example, McDonald’s.
Most software companies that I know of, however, at least claim to hire only the very best, at least claim to be very selective and rigorous in their hiring process. I was fairly heavily involved in the hiring at Mozy where we had this philosophy.
When you apply a high standard to your hiring decision, I figure invariably there will be times when you wonder if you are setting your standard too high. In your attempt to select only the best, are you actually turning away good candidates that would actually do great work for you, yet failed the interview for some reason?
I know we felt this way sometimes at Mozy. We’re asking ourselves this question now also, at Microsoft’s Utah site. Not real seriously, mind you, but I think it is only natural to take a critical look at your interview process from time to time to see if it is really helping you select the people you want.
So how do you know whether your process is too strict? As my boss and I tossed this around today, I suggested that perhaps you can measure it by evaluating the team you currently have.
Suppose you have a team of eight people. Knowing what you now know about each member of your team, would you still choose to hire them? Or are there one or two in the team that you wouldn’t choose to hire again?
I suggest that if you find yourself in this situation, perhaps this is the best indicator of all about the strength of your interview process – namely, that you might need to strengthen your process a bit. You’ve got some great information on how to do this, also. What changes, if you’d had them in your interview processes back in the day, would have filtered out the employees you wish you didn’t have? Maybe those changes are some candidates for your modified interview process.
On the other hand, sometimes you might look across your team and feel like you have strength in every position, and if given the opportunity, you’d rehire every person on the team in a heartbeat. Wouldn’t this seem to indicate that your process is working? I’d argue it is.
Today you might find yourself worried that your process is so strict that you are turning good talent away. But you might have done that in your last hiring round. Yet you ended up with a solid team, strong at every position. That’s a great situation to be in, and there’s not much need to fix what seems to be working.
