How Software is Made

It’s funny because it’s true:

'Sandra and Woo' comic: 'Richard's Guide to Software Development'

Comic from Sandra and Woo. Click to see the original.


A Fairer FizzBuzz? and Other Thoughts

In response to last week’s article, in which I wrote about interviewing programmers for a position at my startup and giving them the FizzBuzz test, a number of people wrote in to tell me that knowing how to determine whether a given number is a multiple of another number — a task made easy with the modulus operator — was esoteric knowledge.

One such complaint came from none other than Rob Conery, who among other things is the guy behind the tutorial site Tekpub and co-creator of the excellent podcast This Developer’s Life, and doer of other interesting, geeky things. He wrote:

I would fail that test. I would fail it for a number of reasons – the first of which is what *you* perceive to be a “good programmer”. I don’t want to sound negative, but I would urge you to reconsider what you’re doing. You probably sent some very good talent out the door.

Here’s my point: these are *people*, not machines! I find the attitude of “let’s see how good you are at writing this arbitrary bit of gotcha code under the gun” soooooo very Enterprisey, where arbitrary metrics can somehow convey what the worker will do for you.

I can’t tell you the number of programmers I’ve hired “on a hunch” – that didn’t know the basics of what I needed them to do. But I could see the light in their eyes, I could *feel* their need to WIN – and you know what? They were the best damn employees I ever had.

You know who were the worst? The ultra-geek engineers that could kick out FizBuzz in minutes.

I wish this mentality would shift. I just hired a person here at Tekpub who I now couldn’t imagine doing business without. You know what he was doing when I hired him? REAL ESTATE APPRAISAL. Why did I hire him? Because he was smart and motivated. He picked up *everything* I needed him to know with 2 weeks – and he did it on his own because he *wanted it*.

Anyway – sorry Joey if I’m coming off negative. I have a soft spot for these kinds of posts that translate to “OMFG Look how DUMB these kids are!”.


Hey Rob, don’t worry about coming off negative! I have no trouble with people taking issue with my ideas in the comments, especially if it’s in the service of our craft. It’s also an honour to get a comment from one of my favourite podcasters.

Modulo Math

While I think that the modulo operator falls under “basic programming knowledge”, I don’t think it’s the crucial part of FizzBuzz. I think the decision-making part — knowing when to output “Fizz”, “Buzz”, “FizzBuzz” or the number — is far more crucial. Programming as we commonly do at present boils to down these:

  1. Giving instructions in a specific sequence
  2. Repeating actions in a loop
  3. Choosing among two or more possible actions based on one or more given conditions

I picked FizzBuzz as a test because it’s about as domain-unspecific as it gets. Maybe the “determine if a number is divisible by 3 or 5” part is too mathematical, but there’s no reason I can’t eliminate it.

Would the FizzBuzz test be more fair if I said “Assume that there’s a function called multiple_of_3 that accepts a number and returns true if and only if the number is a multiple of 3, and a function called multiple_of_5 that accepts a number and returns true if and only if the number is a multiple of 5?”

Pen and Paper

Some people wrote that doing FizzBuzz with pen and paper was unfair. Would you consider the FizzBuzz test to be more fair if I let candidates use a computer?

The Observer Effect

Other commenters wrote in and said that I may have passed up some good people because watching them like a hawk with a stopwatch as they wrote down their solution might have thrown them off their game.

Allow me to clarify: I did no such thing. I gave them a spot at my desk, sat down at another desk, pulled up my iPad and read. There was no stopwatch; just a quick look at the time every now and again, mindful of the fact that we had only an hour for the whole interview.

I tried to make it as non-intimidating as I could, and I was aware that even so, it must been at least a little intimidating. Call me a hard-ass if you must, but this is a startup, and we’re int he business of writing mobile software for enterprises. The field is one giant moving target, enterprises can be rather arbitrary and fickle customers, and we’ll be competing with established players with far more resources than we have. If you can’t take a reasonable amount of pressure (and hey, sometimes there’ll be an unreasonable amount), the programming jobs I have to offer aren’t for you.

Looking at Candidates’ Past Work

I asked candidates if they could show me samples of past work, especially work from their previous or current job. Some could, but others weren’t able to for various reasons. Some projects were company-internal websites that couldn’t accessed from outside. Some couldn’t be installed on their own computer for technological or legal reasons.

I also asked candidates to show me any side projects. If you have the time to have one, they’re a great way to demonstrate what you can do, and they’re completely under your control. However, not everyone has the time or inclination to have a side project.

One of the reasons I used FizzBuzz was to get around the problem of assessing their programming skill in the absence of being able to view a candidate’s past work.

Going with Your Gut

In his comment, Rob Conery pointed out something that I believe in: depending on your situation, the “ultra-geek engineers” might not be the best people for the job. He talked about a Tekpub hire whose last job wasn’t programming, but real estate appraisal. He went with his gut and simply hired someone who was a good fit, smart and hungry.

To that, my instinctive reply is: You know who also went with their guts? Every U.S. election pundit who predicted a Romney victory in spite of Nate Silver’s silly “math”.

But seriously: the guys I hired would’ve likely have been overlooked by Shopify, Unspace and a bunch of other hot-shot startups. Like Rob’s hires, they’re smart, hungry, and a good fit — and they also passed the FizzBuzz test.

Is it Fair to Even Have a Test?

The last two companies for which I worked — Microsoft, then Shopify — are quite different in many ways, but one thing they have in common is that they issue tests to people applying for development jobs. I thought I was being fair by issuing what I consider to be an incredibly basic test, but some people have written in to tell me that it’s wrong to have any test as part of the interview. What do you think?


The Worst Possible Name for an Apple Employee

Oh please, let this be real and not just an elaborate but funny prank.

Found via Make Use Of.

I’m sure he gets beaten up by the iPhone team every day.


Further Into FizzBuzz!

As I write this, StatCounter is reporting that my recent post on giving the FizzBuzz test to developer job candidates has received over 21,000 pageviews. Without any prompting from me, my friend Reg “Raganwald” Braithwaite (we both live in Toronto and see each other from time to time) linked to it from both r/programming and Hacker News, and the discussion that first started five years ago with Imran Ghory’s blog post began anew. I thought I’d respond to questions posted in both the comments and forums here, as well as explain a few things.

A Quick Introduction

We’re a small consultancy creating two different applications:

  1. One that does cross-platform mobile device management, and
  2. another that performs a series diagnostic tests on mobile devices.

We’re currently paying the bills and getting in some market research by doing mobile strategy assessments for enterprises and organizations. We’ve also been approached about developing mobile “dashboard” applications that provide a high-level view of supply chains.

The positions we were looking to fill were:

  • A web-and-back-end developer (PHP/MySQL)
  • An iOS developer

A company like ours, at this particular stage of its evolution, often requires its people to not only perform their jobs really, really well, but to also take on a number of other roles as the situation dictates. Circumstances have already required us to — I hate to use the word, but it’s appropriate — pivot. We’re dealing with limited resources, long enterprise sales cycles, the moving-target nature of mobile development and Murphy’s Law. When it comes to hiring, we can’t afford not to be picky.

When Candidates Got FizzBuzz Wrong

That’s me: High Expectations Asian CTO.

In the comments, Vasi wrote:

I’m curious what “failures” look like. Is it complete inability to get anything done at all? Or relatively small mistakes, like ending at 99 instead of 100?

One candidate took longer than 10 minutes while working on FizzBuzz. Since we had only an hour for each interview and a number of candidates to go through, we I had to set that time limit. The candidate had decades of experience, was the one who asked “Do you think I should use a loop?”, and handed this in when time was up:

// Written in C
for (i = 1; i <= 100; i++) {

// A whole lotta stuff scratched out


I’m sorry; this is just plain unacceptable.

The most common mistake followed this pattern:

// Written in PHP
if ($i % 3 == 0) echo "Fizz<br />";
if ($i % 5 == 0) echo "Buzz<br />";
if ($i % 15 == 0) echo "FizzBuzz<br />";
echo ($i . "<br />");

I consider knowing when to use a series of if statements and when to use ifelse ifelse to be pretty fundamental, especially for a problem as simple as FizzBuzz.

I was willing to let the classic “using the assignment operator when you meant to use the equality operator” mistake slide:

if (i % 3 = 0)

I was less willing to let this mistake go unremarked:

// Written in PHP
if ($i % 3) {
  echo "Fizz<br />";
} elseif ($i % 5) {
  echo "Buzz<br />";
} elseif ($i % 15) {
  echo "FizzBuzz<br />";
} else {
  echo ($i . "<br />");

Even if the mistakes in the conditionals were fixed, there would still be a problem with the code. Please tell me you can spot it.

Modulo Math

Max Howell was the first of a number of commenters to write:

Is the modulus operator domain knowledge? I smirk at the concept. But if an interview asked me to do some bit operations with ~|& etc. it would take me longer than five minutes, and if I had to use ^ (XOR) I’d probably be screwed.

Good question. Our company makes mobile apps for business, and the candidates were made aware of that. I consider the modulus (%) operator to be one of those things that you need to know about in programming for business, because there are many cyclical things that you deal with: days of the week, displaying tables with lines in alternating colours and so on.

On the other hand, using XOR — I’m presuming that Max is referring to the handy-dandy trick for swapping the values in two variables without the use of a temporary variable — isn’t as applicable to the sort of apps we’re writing. However, if we were in the business of writing games or other sorts of apps where bit-twiddling is important, then knowing how to use XOR and its ilk would be important.

The Smartass Solution

Timothy asked what would’ve happened if someone had submitted this solution:

// Written in C#

He pointed out that not only does this solution works, it uses less CPU power than a solution involving a loop and branching.

I replied that if it were apparent to me that a candidate submitted such a solution and made it clear to me that it was submitted in the spirit of smartassery, I’d give the canddiate points for chutzpah. I’d also give the candidate points for arguing that it’s less CPU-intensive; after all, we are a mobile app company, and battery life is one of those things that our developers have to consider.

Of course, I’d then have to follow up with “Could you give me another implementation, please? Perhaps one using looping and branching?”

Why Bother Defining multiple_of_3 and multiple_of_5?

For these reasons:

  1. It’s an anti-stupid-mistake measure. I’m less likely to miswrite multiple_of_3 than (i % 3 == 0).
  2. It’s DRY.
  3. To demonstrate more than just technical competence, but also a cultural understanding of Ruby. Ruby convention favours lots of short methods. See Eloquent Ruby for details and Shopify’s ActiveMerchant library for examples.
  4. It clearly communicates my intent to the person reading the code, who just happens to be someone evaluating me for a job position.
If I’d been even more clever, I’d have named the methods fizz? and buzz? instead.

“Is This Some Kind of Trick Question?”

Mouad suggested that candidates might believe that the test is too simple and suspect that there’s some kind of trick. I assured them that it was not a trick question and should be taken at face value.

The Sample Size

E.Z. Hart asked:

Out of curiosity, how many is ‘some programmers’? I’m just wondering what the sample size is here.

We’ve promised to certain parties to keep this number vague. The best I can do is say that the total number of candidates could fit in a standard city bus.

In Which Your Humble Narrator Gets Called a D-Bag

Peter wrote:

Giving programming tests in interviews is bullying in the most pathetic passive-aggressive way possible. The programming field is full of dbags and this type of interview test is an example of what happens when said dbags are given the chore of interviewing.

D-bag? A comment as stand-out as this calls for an equally stand-out reply:

But seriously: it would’ve been a d-bag move had the purpose of the test been to reinforce my ego by trampling on candidates, to trip them up by requiring some esoteric knowledge or for some other equally malicious purpose. That’s why I used FizzBuzz; it tests basic aptitude and it’s over fairly quickly. This job is going to be stressful, and if FizzBuzz is giving you palpitations, you’re not going to like the real thing.

A far more difficult test was the one given to me when I flew down to Palo Alto this summer and interviewed at Pebble to be their developer evangelist (I didn’t get the job, but then again, no one else did). They interviewed me over two days, and on day one, they tested me by telling me to build an app — any app — for their previous smartwatch, the inPulse, just to prove that I could code. There was no spec, not much in the way of documentation, but looking at some example code and the comments in the API header files, I was able to put together a working “Magic 8-Ball” app in less than an hour. (I later turned that app into this tutorial.) I considered the test tough, but fair, because their project is a high-stakes one.

Our company doesn’t face the same level of risk as Pebble, but there’s risk involved. We’re bootstrapping the company, which means that until we get customers or funding, everyone’s salary is coming out of our pockets. I feel completely justified in taking fair measures that ensure that we’re not wasting his money.

What Else Did We Ask Candidates?

Yes, we asked them to show us projects that they worked on and saw samples of their code. Not all candidates were able to provide samples; some didn’t have code from their previous work, some of their projects were internal web apps that couldn’t be set up on a personal laptop, and some didn’t have side projects they could show me.

I also had each candidate do a quick code review with me of code written by an earlier developer for our applications. I explained what that code was supposed to do and showed them the running application in action. We looked over the code and I asked the candidates to tell me what they thought of it. While the code worked, it was a big ball of mud with plenty of room for improvement. A number of the candidates said “That looks like something I would write! It’s good!”, while others came up with a number of good suggestions for improving it.

What Purpose Did the Recruiting Company Serve?

They helped bring in a good number of candidates, and they also helped pick out people who would be a good fit for the company. They spent a fair bit of time talking to us to get a feel for the company culture and how we worked. We explained to them that we needed people who were self-starters and comfortable with the sort of risks and uncertainty that would come with the territory. We told them not to worry too much about the technical aspects of the candidate, as we’d be handling that.

The candidates who came in via the recruiter were interviewed by them first, after which those deemed to be a good match for us were then sent our way.

What About Candidates Who Were Having a “Bad Brain Day”?

I kept that in mind. That’s why we didn’t rely solely on FizzBuzz. Remember: I’ve been doing either technical evangelism or developer relations for the past ten years. I think I know a think or two about “reading” developers; in fact, at Microsoft, I took some pretty extensive training on that topic.


“We’re Just Like Apple. You Can Buy Our Stuff Now.”

These ads for Asus’ suspiciously familiar-looking ultrabooks are plastered all over Toronto’s Bloor/Yonge subway station:

For all the aping of Apple’s MacBook/AirBook designs, it still doesn’t play the “See? We’re just like Apple!” card as hard as the Dell ad recently featured in Daring Fireball:

Process Programming

FizzBuzz Still Works

The Sorting Hat

I recently interviewed some programmers for a couple of available positions at CTS, the startup crazy enough to take me as its CTO. The interviews started with me giving the candidate a quick overview of the company and the software that we’re hiring people to implement, after which it would be the candidate’s turn to tell his or her story. With the introductions out of the way — usually at the ten- or fifteen-minute mark of the hour-long session, I ran each candidate through the now-infamous “FizzBuzz” programming test:

Write a program that prints out the numbers from 1 through 100, but…

  • For numbers that are multiples of 3, print “Fizz” instead of the number.
  • For numbers that are multiples of 5, print “Buzz” instead of the number
  • For numbers that are multiples of both 3 and 5, print “FizzBuzz” instead of the number.

That’s it!

This was the “sorting hat”: the interview took different tacks depending on whether the candidate could or couldn’t pass this test.

I asked each candidate to do the exercise in the programming language of their choice, with pen and paper rather than on a computer. By requiring them to use pen and paper rather than letting them use a computer, I prevented them from simply Googling a solution. It would also force them to actually think about the solution rather than simply go through the “type in some code / run it and see what happens / type in some more code” cycle. In exchange for taking away the ability for candidates to verify their code, I would be a little more forgiving than the compiler or interpreter, letting missing semicolons or similarly small errors slide.

The candidates sat at my desk and wrote out their code while I kept an eye on the time, with the intention of invoking the Mercy Rule at the ten-minute mark. They were free to ask any questions during the session; the most common was “Is there some sort of trick in the wording of the assignment?” (there isn’t), and the most unusual was “Do you think I should use a loop?”, which nearly made me end the interview right then and there. Had any of them asked to see what some sample output would look like, I would have shown them a text file containing the first 35 lines of output from a working FizzBuzz program.

The interviews concluded last week, after which we tallied the results: 40% of the candidates passed the FizzBuzz test. Worse still, I had to invoke the Mercy Rule for one candidate who hit the ten-minute mark without a solution.

The Legend of FizzBuzz

I was surprised to find that only one of the candidates had heard of FizzBuzz. I was under the impression that that it had worked its way into the developer world’s collective consciousness since Imran Ghory wrote about the test back in January 2007:

On occasion you meet a developer who seems like a solid programmer. They know their theory, they know their language. They can have a reasonable conversation about programming. But once it comes down to actually producing code they just don’t seem to be able to do it well.

You would probably think they’re a good developer if you’ld never seen them code. This is why you have to ask people to write code for you if you really want to see how good they are. It doesn’t matter if their CV looks great or they talk a great talk. If they can’t write code well you probably don’t want them on your team.

After a fair bit of trial and error I’ve come to discover that people who struggle to code don’t just struggle on big problems, or even smallish problems (i.e. write a implementation of a linked list). They struggle with tiny problems.

So I set out to develop questions that can identify this kind of developer and came up with a class of questions I call “FizzBuzz Questions” named after a game children often play (or are made to play) in schools in the UK.

Ghory’s post caught the attention of one of the bright lights of the Ruby and Hacker News community, Reg “Raganwald” Braithwaite. Reg wrote an article titled Don’t Overthink FizzBuzz, which in turn was read by 800-pound gorilla of tech blogging, Jeff “Coding Horror” Atwood.  It prompted Atwood to ask why many people who call themselves programmers can’t program. From there, FizzBuzz took on a life of its own — just do a search on the term “FizzBuzz”; there are now lots of blog entries on the topic (including this one now).

In Dave Fayram’s recent article on FizzBuzz (where he took it to strange new heights, using monoids), he observed that FizzBuzz incorporates elements that programs typically perform:

  1. Iterating over a group of entities,
  2. accumulating data about that group,
  3. providing a sane alternative if no data is available, and
  4. producing output that is meaningful to the user.
If you can’t write a program that does all of the above in its most basic form, you can’t program. It’s that simple. Needless to say, anyone who didn’t pass FizzBuzz didn’t pass the interview.

My FizzBuzz Solution

I can’t go fairly judge other programmers with FizzBuzz without going through the test myself. I implemented it in Ruby, my go-to language for hammering out quick solutions, and in the spirit of fairness to those I interviewed, I did it with pen and paper on the first day of the interviews (and confirmed it by entering and running it):

I consider myself a half-decent programmer, and I’m pleased that I could bang out this solution on paper in five minutes. My favourite candidates from our interviews also wrote up their solutions in about the same time.

A Forty Percent Pass Rate?!

Okay, the picture above shows a 33% pass rate. Close enough.

How is it that only 40% of the candidates could write FizzBuzz? Consider that:

  • All the candidates had some kind of college- or university-level computer programming education; many had computer science certificates or degrees.
  • The candidates had varying degrees of work experience, ranging from years to decades.
  • Each candidate could point to a sizeable, completed project on which they worked.
  • All the candidates came to us pre-screened by a recruiting company.

A whopping sixty percent of the candidates made it past these “filters” and still failed FizzBuzz, and I’m not sure how. I have some guesses:

  • Moving away from programming: Some of them may have started as developers initially, but over time, their work changed. They still hold some kind of programming-related title and their work still has something to do with the applications they were hired to develop, but these days, most of their job involves tasks outside the realm of actually writing code.
  • “Copy and paste, adjust to taste”-style programming: Also known as “Coding by Googling”,  this is when a developer writes applications by searching for code that’s similar to the problem that s/he’s trying to solve, copy-and-pastes it, and then tweaks it as needed. I get the feeling that a lot of client-side JavaScript is coded this way. This is one of the reasons I insisted that the FizzBuzz exercise be done on paper rather than with a computer.
  • Overspecialization: It could be that some candidates have simply been writing the same kind of program over and over, and now do their work “on autopilot”. I’ve heard that FizzBuzz is pretty good at catching this sort of developer unaware.

If you’re interviewing prospective developers, you may want to hit them with the FizzBuzz test and make them do it on paper or on a whiteboard. The test may be “old news” — Imran wrote about it at the start of 2007; that’s twenty internet years ago! — but apparently, it still works for sorting out people who just can’t code.

Be sure to read the follow-up article, Further Into FizzBuzz!


Goodbye, Mr. Sinofsky

Goodbye, Mr. Sinofsky, and thank you for teaching me two of the most valuable lessons in office politics during my tenure at The Empire:

To be fair, Sinofsky is that combination of “smart” and “gets things done”. In his heartfelt “farewell” article, Dare Obasanjo writes that he took some products in need of fixing up and love — Windows Vista, Surface-the-big-ass-table, “a mish mash of confusing consumer synchronization products” — and turned them around into dramatically improved versions: Windows 7 and 8, Surface-the-tablet and SkyDrive.

Other takes:

Shareholders have already braced themselves for a drop in value — not the $60 or $70 that Apple’s has dropped to the low, low, low price of $540 a share after Forstall’s and Browett’s departure announcementsbut by about a buck to $27-and-change at the time of this writing.