Grading on a Scale – I Mean, a Curve
Miguel de Icaza tweeted this today:
Performance Review Season at Novell is easy when your entire team is made of 100% super-awesome hackers.
It's a very nice thing for Miguel to say about his team, something I have no reason to doubt. I don't know Miguel too well, but I have been in a handful of meetings with him back in my Novell days. He doesn't strike me as the type of person to be disingenuous, and regardless of how you feel about .NET on non-Windows platforms, you have to be impressed with the impact and effectiveness of Mono, so I figure his team probably is pretty awesome.
I guess when you are among the privileged at Novell, you can do things your way. I know when I was at Novell, having a team full of awesome people didn't make performance reviews any more straightforward. Novell is by no means unique in this; I can personally attest to this fact.
At most big companies I've been at, how performance is measured depends on what time of year it is. If you are measuring performance during any other time of the year, you do this by comparing a person's efforts against a published definition of their job description and level. For example, on your company's intranet somewhere there is a document that describes the expected level of performance for a mid-level software engineer. If you are a mid-level software engineer, you can compare your performance against the published standard and determine if your performance is currently on track or ahead of or lagging expectations.
This is true at any time of year except during performance review time. During performance review time, suddenly your performance is measured by how you compare to your peers. So now, if you are a mid-level software engineer, your performance is based on how your performance and contributions compare to the other mid-level software engineers in your organization.
For some reason this seems to be a pretty common way to measure employee performance, and yet it makes absolutely no sense to me. In casual conversations with peers (and, frankly, also with a number of managers who will remain unnamed here), I think I can confidently say that it makes no sense to most other people either.
Imagine you are attending college and you sign up to take a calculus class. On the first day of class, the instructor hands out a syllabus which clearly explains how you will be graded, which boils down to this: If you get 95% of all the questions right on homework and tests, you will get an A, and the scale breaks down from there. As the semester proceeds, you monitor your performance on homework and tests, studying to meet that magical 95% mark. On the last day of class, you enter confidently knowing you have a 96%-97% average which gives you a little bit of a buffer on the final. You leave the final feeling pleased with yourself, sure that you got at least 19 of the 20 questions right.
Then the grades come out, and you have a B-.
You go and ask the professor, and he says, "Well, I changed the grading from a scale to a curve. See, everyone was getting As, and we can't have that. So I graded on a curve instead. You got a B-."
This is effectively how most employee performance measurement techniques work. Your performance is graded on an objective, defined scale all year long. Then comes performance review time, and suddenly you are now being graded on a curve, not a scale. Before you knew where you stood; now, you have no idea.
A normal distribution curve makes sense when applied to a statistically average sample. The question is, does your employee population comprise a statistically average sample? In order to do that, you'd pretty much have to be willing to hire anyone who walks in the door asking for a job. You probably don't do that; you are probably very selective in who you hire and keep. You probably try to hire only the best talent you can find. You probably take the approach of preferring to sometimes pass on talented people that you aren't sure of, rather than give the benefit of doubt, and so you've probably rejected some talented candidates.
That means your employee population is not statistically average. So why is it that during performance review time you try to make this anomalous sample conform to a normal distribution curve?
I've worked for companies where they say that, organization-wide, some percentage of the population of any given job function MUST be underperforming, some percentage MUST be overperforming, and the rest MUST be performing at around average, within about a standard deviation or so. This isn't just a belief. It is enforced through policy. Essentially the policy says it is not possible to have an organization with 100 employees that are all high performers. Instead, you might mandate through policy that 10 of them must be underperforming, 10 must be overperforming, and the remaining 80 are average or near-average. Yet when you hire, you insist that you hire only the best.
This doesn't make any sense, and in my opinion it is a dangerous practice.
First off, you aren't fooling anyone, especially not the intelligent, highly-educated type of thought worker that tends to be evaluated this way. You wanted smart people, you hired smart people, and now you have to pay the consequences: You can't implement obviously flawed policy like this. They will pick up on it and it will cause them to lose faith in your organization.
Second, it encourages exactly the wrong type of behavior. Once they realize that their actual performance measurement has less to do with their evaluation against an objective metric than it does against their peers, employees will be incentivized to game the system. This may manifest itself in many ways: refusing to help teammates or to work well as a team; manipulating the assignments given to them and others to maximize their relative performance; taking opportunities to point out to you (the manager) how excellent they are in comparison to others or how rotten the others are in comparison to themselves; attempting to make others look poorly or to make themselves look good in public, e.g. meetings; trying to hire people who they know to be less competent than they are; avoiding promotions (and thus responsibility) for the sake of cherrypicking in a job function that they can excel in; etc. None of these helps your organization to become better.
Third, it encourages employees to leave. Good employees. Employees that passed your interview process. Employees that are exceeding the objective metrics, but don't rank at the right of the curve. They know they can choose a different company with lower hiring standards and at least perform at the top. If you are really hiring only the best, then this person is among the best, and they will be hard to replace.
Fourth, this system encourages sucking up. By definition, it is an entirely subjective measurement. As a manager, you are likely to be persuaded by those who have perfected the art of sucking up and give them better evaluations. You might do this without even realizing it.
So, what should you do about it?
Well, if you are empowered to do something about it, I think you should avoid doing it altogether. Evaluate your employees based on an objective scale. Period. Hire great people, expect greatness from them, and then stop doing dumb things that encourage them to not be great. Just get out of the way and revel in the greatness.
The main argument I've heard in favor of the curve is budgetary. "We budget 10% of employee salary for annual bonus. High performers get 18%-20%, low performers get 0%-2%, and the rest get from 5%-15%. The average is 10% so we meet budget." The corollary argument is that the bonuses should not be spread evenly across the team. More money for the high performers.
That's fine, but didn't you try to hire ALL high performers? If so, wouldn't you say you expect ALL of them to be in the 18%-20% range? Why not instead budget your bonus pool for the high performance amount across the board? You should still evaluate each employee's performance, but now you aren't being constrained by budget, so you can do it objectively and fairly against a published metric, the one you've been using in one-on-one's all year long. If an employee underperforms, you can still give a 2% bonus or none at all. Take the money you saved and use it next year or something.
If you aren't really empowered to do anything about it, my suggestion is to press onward anyway. Be excellent anyway. Don't let the system cause you to be non-excellent. Be a good team player. Try to surround yourself with the best people. Take the assignments you are given and do your best with them. Discuss with your manager any concerns you have about whether you are getting the opportunity to show what you've got, but move forward and do the job right even if the system doesn't reward you. I believe good things come about from good works, sooner or later. If that doesn't work, I guess you can always go work for someone else who's a bit more reasonable.
You can also blog about it.
