“Stack Ranking” Observations from My Time at Microsoft

by Joey deVilla on August 26, 2013

im a pc im a mac

“I hear you’re a free man,” the Microsoft manager said to me, clasping a hand firmly on my shoulder, in that “I’ve got your back, bro” sort of way, and not without a gleam of envy in his eyes.

The manager and I never directly worked together, but we both were on the Developer and Platform Evangelism team. To borrow a phrase from the U.S. Army, our job was to win over programmers’ and IT professionals’ “hearts and minds” and convince them of the benefits of using Microsoft tools and technologies. It was February 2011, and we were at a cocktail party at an internal conference in downtown Seattle. He’d heard about my separation.

“That’s right,” I said, taking another swig of some local microbrew and probably wincing a little. It had been a rough couple of months; she’d moved out around Christmas, and I landed in the hospital with a near-fatal case of the flu in January.

“There’s an upside,” he said, “there’s not a married man out there who hasn’t imagined what they’d do if all of a sudden they were single again, knowing what they know now.” He wasn’t the first married guy to remind of this fact, and like all of them, he was right: I was living the married guy’s combination fantasy and nightmare.

“So what are you going to do now?” he asked.

“I have been asking myself that same question for the past few weeks,” I replied. Weeks. The time since the ex-wife and I were under the same roof could still be measured in mere weeks. “I’ve decided the best thing for me to do is to emulate Tony Stark,” I said half-jokingly.

“Tony Stark…”, he said, without even the slightest glimmer of recognition on his face. “Who’s that?”

tony stark testing repulsor glove

Like religious evangelism, technical evangelism requires an understanding of the culture in which you work. I would argue that this understanding extends to knowing its fictional icons. You’d probably (and quite rightly) wonder about someone who works in advertising who hasn’t heard of Don Draper, or a NASA employee who doesn’t know who you’re talking about when you mention Captain Kirk. At the time this conversation took place, there had been two wildly successful Iron Man movies, and even Forbes had done a piece on Tony Stark in their list of top fictitious rich guys.

Continuing my chat with the manager, who was a pretty nice guy and showed genuine concern about my major life change, I discovered that he’d be completely lost if you sat him in front of Visual Studio and asked him to write a “Hello World” application for the desktop, the web, or mobile, and system setup beyond pulling a laptop out of a box, plugging in an external monitor and connecting to wifi often confounded him. And yet, he was an overseer of Microsoft’s outreach to developers and IT pros, often called to speak in front of them and make hand-waving presentations of software and tech that he neither used nor knew how to fit together.

Further conversation led me to discover that the skill that had earned him his lofty level 63 position (like World of Warcraft, Microsoft has levels; also like World of Warcraft, the real fun begins at level 60) was that he could make the scorecard green.

executive scorecard

At Microsoft, you’re only as good as your metrics. “Metrics” often translated into money. As the Stratechery article If Steve Ballmer Ran Apple puts it:

Employees were incentivized by dollars and cents in the form of bonuses and stock grants. Bonuses and stock grants were tied to a stack ranking system, that devolved your performance to a number. What was measurable mattered, particularly if it was measured in money.

The result is inevitable: Microsoft is a company filled with people motivated by measurables like salary, bonus, and job level. Anyone who isn’t would necessarily leave. Unsurprisingly, said people make choices based on measurables, whether those be consumer preferences, focus group answers, or telemetrics. The human mind is flexible, but only to a point.

Yet, if Apple’s success has proven anything, it’s that measurables aren’t the half of it. Things like design can’t be measured, nor can user experience. How do you price delight, or discount annoyance? How much is an Apple genius worth?

In the consumer market, it’s the immeasurables that matter. It’s the ability to surprise and delight, and create evangelists. It’s about creating something that developers demand access to, and that consumers implicity trust. The consumer market is about everything you can’t measure, everything Microsoft’s legion of mini-Ballmer’s can’t see and will never appreciate.

commitments

Once a year, you sit with your manager and come to an agreement on a set of commitments — a set of “deliverables” — for which either you individually, or as part of a team, are responsible for. As a developer evangelist, I had individual commitments that included doing x presentations in front of y people, discovering and publicizing a specified number of “success stories” where Microsoft development tools and platforms played a part, and hitting a target number of Azure sign-ups that could be directly attributed to me.

The aggregate commitments applied to both our own local teams as well as the developer evangelism group as a whole, and were things ranging from tech and tool adoption rates to “sat”, short for satisfaction, which as far as Microsoft was concerned, was an actual quantifiable property, derived from surveys and distilled into a single, mysterious, context-free number that seemed to bounce between 130 and 150 during my tenure.

There’s a mid-term review where you and your manager review your commitments to see how well you’re doing, and where you need to redouble your efforts. Near the conclusion of the fiscal year, which ends on June 30th at Microsoft, you write up how you well you think you met your commitments, your manager does the same, and after which the two of you have a long one-on-one meeting and compare notes. It’s at this meeting when you’ll get a very clear idea of which bucket you’re going to be put in: the 70, the 20, or the 10.

70 - 20 - 10

It’s often referred to in the tech press as “stack ranking”, but within the walls of Microsoft Canada, I’ve only heard it referred to as “70-20-10”. Those numbers refer to the three categories into which every manager at Microsoft had to place every one of his or her direct reports. These categories are:

  • “Achieve”, into which 70% of your direct reports would go. The official description of the sort of person your supposed to put into this bucket is “Demonstrates potential at minimum to broaden in one’s role or to advance one career stage or level as a leader – either as a People Manager and/or individual contributor. Past performance suggests capability of delivering consistent and significant contributions over long-term. Competencies typically are at expected levels.”
  • “Exceed”, where you’d place the top 20%. If you did particularly well that year, you’d be put into this category. The description reads: “Demonstrates potential to advance faster than average as a leader – either as a People Manager and/or individual contributor – preferably multiple levels or two career stages. Past performance suggests capability of delivering exceptional results over long-term. Competencies typically are at or above expected levels.”
  • “Underachieve”, or the bottom 10%. This category comes with a set of consequences and often a lot of self-loathing and voluntary departure from the company. “Demonstrates limited potential to advance.”

Once a year, during our team offsite, the managers would take the time to explain this system, all the while taking great pains to explain why it was a good thing, in that sort of “toxic sludge is good for you” way. There are all sorts of problems with this sort of system, two of which were pointed out quite astutely by Bruce Lynn in the 70-20-10 blog:

My concern about the application of these ‘distributions’ is two fold. First, they are statistical constructs and most executives lack [the] numeracy to properly interpret and work with them. For example, while these distributions might generally apply over large numbers, no reasonable statistician would expect them to hold identically for smaller populations. In companies, sometimes these distributions are imposed on teams of just a handful of people. This imposition violates all sorts of statistical principles.

Also, one significant variable that such simplistic systems omit is the outcome of the group. A group that has achieved great things is likely to have different distribution of these categories than one that hasn’t. A world champion team might have more than ‘20%’ of its team as ‘All Star’ talent. In fact, the very fact of the skew to high performance could explain the overall team success. In another scenario, none of the team might be standouts (ie. no one at the 20% high achiever level), but the team work exceptional leading to a whole greater than the sum of the parts.

But more important than the clumsy application of such distributions is their application at all. These breakdowns are ‘descriptive’ stats, but misguided managers use them as ‘prescriptive’ rules. Misguided executives introduce policies which force managers to classify their people into such categories. The execs like the numbers because they are a nice, repeatable, McDonalds-like process which is easy to implement and understand on a superficial level. Unfortunately, it is often misapplied in specific instances. It is sort of like expecting a day with a ‘50% chance of rain’ to rain for half the day and be dry for half the day.

The other problem with 70-20-10 is that it can pit teammates against each other (luckily, this was not the case with my group). I’ve seen it create situations like the old joke about the two campers who encounter a hungry bear in the woods. The first camper says “I’m glad I wore my running shoes!”, to which the second replies “You can’t outrun a bear!” The first camper then says “I don’t have to outrun the bear. I just have to outrun you.”

outrunning a bear

If you managed to outrun the bottom 10, you could do very well. When I worked there, annual bonuses at Microsoft ranged from 6 to 60% of your base salary, and salaries at Microsoft are pretty good (in fact, people who leave Microsoft to work elsewhere often experience “reverse sticker shock” when they discover that most places outside The Empire don’t pay as well). My last bonus put me at a point where I could’ve bought a brand new sedan with cash.

When you combine bonus in the tens of thousands of dollars and tech culture, where Asperger’s syndrome is sometimes considered to be a badge of honour, you get antisocial, every-man-for-himself behaviour. In his Vanity Fair piece on Microsoft last year, Kurt Eichenwald wrote:

At the center of the cultural problems was a management system called “stack ranking.” Every current and former Microsoft employee I interviewed—every one—cited stack ranking as the most destructive process inside of Microsoft, something that drove out untold numbers of employees. The system—also referred to as “the performance model,” “the bell curve,” or just “the employee review”—has, with certain variations over the years, worked like this: every unit was forced to declare a certain percentage of employees as top performers, then good performers, then average, then below average, then poor. …

For that reason, executives said, a lot of Microsoft superstars did everything they could to avoid working alongside other top-notch developers, out of fear that they would be hurt in the rankings. And the reviews had real-world consequences: those at the top received bonuses and promotions; those at the bottom usually received no cash or were shown the door. …

“The behavior this engenders, people do everything they can to stay out of the bottom bucket,” one Microsoft engineer said. “People responsible for features will openly sabotage other people’s efforts. One of the most valuable things I learned was to give the appearance of being courteous while withholding just enough information from colleagues to ensure they didn’t get ahead of me on the rankings.” Worse, because the reviews came every six months, employees and their supervisors—who were also ranked—focused on their short-term performance, rather than on longer efforts to innovate.

By the time I had joined Microsoft, I had been blogging for seven years and decided to turn my skills and energies towards revitalizing the Canadian Developer Connection blog. I took it from a site with three posts a month to thirty posts a month, brought the readership up from dozens a day to thousands, often getting linked on Techmeme and occasionally garnering the attention of tech writers covering Microsoft, such as Mary Jo Foley. I became Microsoft Canada’s most prolific blogger, something that annoyed a manager who once called me into his office simply for him to express this annoyance: “You think you’re fucking hot shit, don’t you?” (To which I can now reply: “Hotter than you, chump. Hotter. Than. You.”)

starbucks coffee and xbox

70-20-10 didn’t just lead people to focus on the short term to the detriment of the long-term; it also made people very risk averse. One year, my team had the collective commitment of getting 800 people to sign up for an Azure (Microsoft’s cloud computing service) account. Some people committed to bring on 20, some went for 30, and a couple said they’d bring in 50. Here was Microsoft’s version of cloud computing, something that could rewrite the book on web services, and everyone was holding back for fear of not being able to hit that number and paying for it in the annual review. It’s this institutionalized fear that turned Microsoft from a creator of trends into a company that was content to chase them. What happened to the company whose mantra was the audacious “A computer on every desk and in every home, running Microsoft software?”

I thought the metric was meaningless. As far as I was concerned, a dozen high-quality accounts that used Azure for useful software, and often, would be worth more than a hundred sign-ups. In a fit of “screw it”, I put myself down for bringing in 400.

I got praise from my manager, who then reminded me that I didn’t have a plan for bringing in those numbers. It didn’t take long to work one out, though. After a chat with a friend who worked at Redmond, it became clear to me that all I had to do was get people to open a free Azure account and stick anything on it. I got a Silverlight expert I knew to make a web app that people could install into their Azure account, and got them to open Azure accounts by either:

  • At our developer conferences, entering anyone who opened an account into a draw for an Xbox
  • Sitting down with developers one-on-one at Starbucks, walking them through the process of opening an Azure account, and buying them a coffee (on my expense account, of course) for their time

In the process, I may have convinced one or two people who’d go on to use Azure for a project. The rest of them simply set up an account, installed the web app, and never logged back in again. In the end, what I truly did was give a few hundred people a shot at winning an Xbox, and gave several dozen a case of the coffee shits. It felt like a hollow victory. But I hit my target number and got my bonus!

Dwight-from-the-office

The other effect of 70-20-10 is that I was constantly reminded by my manager to devote more effort to “my career”. By this, he didn’t mean brushing up on my tech skills, or my presentation skills, or my writing — which he did, anyway, because he was a great manager. He was talking about “managing up”, building relationships with the higher-ups and across departments with the people who could bring me from a level 61 employee to level 62…and beyond!

I understand and appreciate the value of such relationships, and I’m a pretty good schmoozer. But the level to which this sort of relationship building was encouraged within Microsoft was almost sociopathic, and the internal guidance kept pushing us to spend what I think was too much time building alliances, as if we were competitors on the island on the reality TV show Survivor. Partnerships, alliances, joint projects, and so on all had an undercurrent of “what can this person or team do for my career”?, “How can this guy help get to level 62?”, “How can so-and-so help me keep my scorecard green?”, and as Mini-Microsoft put it, “Influence if you can, scare if you must”.

I was trying to be Scott Hanselman, and but they wanted Dwight K. Schrute.

{ 7 comments… read them below or add one }

1 Gary August 26, 2013 at 4:25 pm

This is both eye-opening and reaffirming for me. Change the terminology around a bit and you just described my five years at IBM, which I left in 2003. The employee hierarchy is in the shape of a pyramid and you *must* always be actively working towards the next promotion. Saying “I’m fine where I am and I like my job” is not permitted: it will earn you a ranking of 3, the IBM equivalent of the “underachiever” label you described, aka on your way out.

But because there are fewer and fewer employees at each level in the pyramid, an element of Survivor Island or musical chairs inevitably comes into play. You either move up or you’re elbowed out. I chose to leave voluntarily before I was elbowed, mostly because I realized that I was simply not motivated enough by status ranking or money for the whole game to be at all interesting to me. Worst of all, the sort of people who are good at that game are not the sort of people that are fun to work with. The system was engineered to essentially filter out everyone but the most aggressive. It doesn’t take into account different personalities or different sorts of softer motivation, for example, people like me to enjoy creating things rather than managing them. When I saw co-workers who I considered bullies being promoted quickly, I realized that if I was to succeed at IBM I would need to change my core personality in ways that I could not reconcile with my personal values.

The hardest part of leaving was the sticker shock you mentioned, that is, the realization that smaller companies that are more fulfilling to work at just don’t have budgets anywhere near that of behemoths like IBM or Microsoft, and they pay accordingly. But on the plus side, having had large salaries and bonuses tossed my way straight out of university (it was the dot-com boom era) taught me early on that more money doesn’t equal more happiness once you pass a certain baseline. I’d rather have “enough” money and be happy than have lots of money and be miserable. I learned a lot in that environment and met some really brilliant people, but I’ve never regretted moving on.

2 Your Manager's Peer August 26, 2013 at 7:05 pm

You forgot one important aspect of the ranking process: calibration. Your manager talks about your performance in front of his peers and then together they “calibrate” how you performed relative to your peers.

This would shine a spotlight on folks who networked well, picked opportunities to “own scorecards” and exceeded them by crunching numbers and on backs of others by running “v-teams”, vs the actual do-ers and Subject Matter Experts who would do the heavy lifting. Extra calibration points who did a bang-up job on ‘launching’ something that made management look good (product, project, even if it were an internal one).

The calibration process turned into a contest in how well you schmoozed with your boss’s peers — based on criteria that are completely subjective and emphasized ability to do well in a pack vs. standing out. Folks like you, Joey, were the “outsiders” who were clearly not a team-player and too threatening b/c you actually knew the subject-matter, the audience and were liked by folks you dealt with on the outside, versus some mediocre manager’s buddy on the inside.

Missing you @ MSFT :)

3 Joey deVilla August 26, 2013 at 11:26 pm

Thanks, Manager’s Peer!

4 Someone in the MSFT Product Groups August 26, 2013 at 11:40 pm

When I was interviewing with Microsoft a couple years ago, the stack ranking system was something that gave me pause. If you read enough of Mini-Microsoft’s posts, there’s more than enough material to make you think about going elsewhere.

After going through a couple annual reviews, I can honestly say that the system hasn’t really lived up to its horrible reputation. Yes there’s anxiety around it, but at least its fairly deterministic. Good leads & skip-levels can do a really good job of explaining how the system works. YMMV, results not indicative, etc.

Prior to working at Microsoft, I spent several years working in the Canadian software industry for smaller companies. I had an annual review everyone of those years, and received some sort of merit increase/bonus for each of those years. Why? No idea. Completely arbitrary. Everyone of those years I wrote a self-assessment & future planning review. Every single one was dutifully filed with HR & never referred to again.

Microsoft is different. There is a system. Its not perfect, and neither are the people. But its still better than anything I’ve seen before. When the system works, you know where you stand, and how to improve. And yes, the money is awesome.

Pro-Tip: If you are in the Canadian software industry, qualify for a TN, and haven’t tossed a resume at Microsoft/Google/Amazon/Facebook, you owe it to yourself to give it a try. The combination of higher salary & *much* lower taxes makes it a very sweet deal indeed.

My theory for how to succeed at Microsoft: work for a team where mutual respect & interdependence is the first priority. Have an awesome relationship with your lead, and have a skilled/experienced manager/skip-level. The challenge is to find these teams.

5 Phil August 27, 2013 at 7:16 pm

Seems like the model is all sales driven and explains a lot about how innovation falls by the wayside.

6 downtown August 28, 2013 at 6:49 pm

Stack rank reporting was big at Dell too. I think we can all thank Jack Welch for bringing it to the mainstream. In his first book, he talked about firing the bottom 10% every year. Managers were thrilled for the first 2 years because they had the money to package out under performers. After 2 years, with a strong team in place, who do you cut? Some managers were subitting records of retired and even deceased employees because they had to hit the metric. And at some point, it all changed when they realized it was impacting their metrics and it became what you and the IBM comment describe.

There is a lesson in measuring the wrong things. Yet we continue to make the same mistakes. Joey is right, the brain isn’t really that flexible.

7 Rohan Jayasekera September 7, 2013 at 3:54 pm

I’m waiting to find out whether the next CEO of Microsoft abolishes stack ranking. If s/he doesn’t, I’ll assume that Microsoft will continue its decline into irrelevance.

Leave a Comment

{ 3 trackbacks }

Previous post:

Next post: