Here’s the keynote from Day 2 of the MIX10 conference, featuring:

• Dean Hachamovitch talking about IE9, its support for HTML5 and Chakra, its new JavaScript engine,
• Scott Guthrie talking about Microsoft’s support for jQuery, with special guest jQuery creator John Resig,
• Doug Purdy on OData, a.k.a. the Open Data Protocol,
• and Bill Buxton on thinking about natural user interfaces.

Don’t have Silverlight? You can download Silverlight here or download the video
in WMV or high-quality WMV format.

This article also appears in Canadian Developer Connection.

Pictured above: The scene at MIX10 after the Day 2 Keynote.

Whether you’ve just come back home from SxSWi, are heading back home from MIX10 or expect to go to a conference sometime soon, you want to make sure that it was worth the ducats you or your company spent sending you there. It’s one thing to come back from a conference, all inspired to try out the ideas you picked up, test drive the new technologies showcased and stay in touch with the people you met, but it’s an entirely different thing to follow through.

Web Worker Daily has an article titled How to Make a Conference Pay Off that provides these tips on how to get the most out of the conference you just attended after you’ve arrived back home (be sure to read the article for expanded versions of these pointers!):

1. Review your content – the notes, business cards, literature and so on.
2. Act on the quick “now” items.
3. Schedule the “now” items that take more time.
4. Check for information posted online.
5. Complete the tasks from step 3.
6. Follow up with the people who promised to contact. Don’t underestimate the strength of weak ties!
7. Study the materials. It is why you picked them up in the first place, isn’t it?
8. Write blog posts or articles. Sometimes the best way to cement what you’ve learned is to share what you’ve learned.

Also worth checking out: A Conference Survival Guide for the Web Worker.

This article also appears in Canadian Developer Connection.

In case you missed it or weren’t able to attend, here’s the recording of the MIX10 Day 1 keynote featuring Scott Guthrie talking Silverlight and Joe Belfiore talking Windows Phone 7:

(You can also download the video in high-quality WMV format.)

This article also appears in Canadian Developer Connection.

Actually, from left to right, it’s me, Ward Bell of IdeaBlade and Microsoft’s Glenn Block, whom John Bristowe and I interviewed in the most recent Ignite Your Coding webcast. Ward and Glenn have forgotten more about building composite apps than I will ever learn. If you attended the “Building Composite Applications with WPF and Silverlight” session in my track at TechDays Canada 2009, you saw what was essentially Ward’s presentation; he’s the only reason I know anything about Prism.

This article also appears in Canadian Developer Connection.

In a couple of weeks, Microsoft’s Technical Evangelism Team will embark on the EnergizeIT 2010 Tour, a cross-country set of presentation showing what’s possible with Microsoft tools and technology. If you want to know how Visual Studio 2010, Office 2010, SharePoint 2010 and other Microsoft tools and tech can cut your costs and help your business and organization, you should catch an EnergizeIT event near you. They’re free, and they’re happening in the next few weeks!

Depending on the city, there will be different “flavours” of EnergizeIT events. I’ve written some descriptions below, and if you want the full set of details, check out the EnergizeIT page.

The “From the Client to the Cloud” Full-Day Events (Vancouver, Edmonton, Calgary, Ottawa, Toronto and Montreal)

In Canada’s six largest cities, we’ll hold our EnergizeIT From the Client to the Cloud full-day events, where we’ll cover the Microsoft-based platform in detail. And yes, even though it’s full-day, it’ll still be free!

Please note: The Toronto session is currently booked. You can register, but you’ll end up on the first-come, first served waiting list.

In the morning, we’ll talk about the big picture. We’ll show you a scenario featuring the Microsoft-based platform as seen from different points of view: the customer, the information worker, the developer and the IT professional. You’ll see our latest and greatest as well as our up-and-coming developer goodies: Silverlight, .NET 4.0 and Visual Studio 2010. We’ll show you Windows 7 and Azure in action, talk about Office 2010 and SharePoint 2010, and tell you how all of Microsoft’s stuff works together.

Just as the morning session answers the question “What’s the latest technology?”, the afternoon sessions answer the question “How do I get to the latest technology from where I am now?” These sessions, split into two tracks – one on infrastructure management and deployment, one on the development process – will cover what you can do with our tools and technology in a little more depth. They’ll show you what you need to implement what you saw in the morning session and provide a roadmap you can follow to learn more and take action.

For more details about From the Client to the Cloud events or to register (it’s free!), visit the EnergizeIT From the Client to the Cloud page.

The “Community Connection” Evening Events (Many Cities Across Canada)

We’ll also hold Community Connection events in the evening in many cities across Canada,where we’ll do the “big picture” session (the morning session) of our From the Client to the Cloudevents.

The Community Connection evening events will take place in the following cities:

• British Columbia: Vancouver, Kelowna and Victoria
• Alberta: Edmonton and Calgary
• Saskatchewan: Regina and Saskatoon
• Manitoba: Winnipeg
• Ontario: Ottawa, London, Kitchener and Mississauga
• Quebec: Montreal, Quebec City and Trois-Rivières
• Atlantic Canada: Halifax, St. John’s, Moncton and Fredericton

For more details about Community Connection or to register (it’s free!), visit the EnergizeITCommunity Connection page.

Office 2010 Installfests (Vancouver, Ottawa, Calgary, Montreal, Mississauga)

I’ve been using the beta and release candidate versions of Office 2010 in my day-to-day work for the past few months – PowerPoint for my public speaking stuff, Outlook for email, scheduling and get-things-done stuff and OneNote for my copious note-taking. We’d like you to take it for a spin!

Join us at one of our Microsoft Office 2010 Installfests and we’ll hook you up with the latest build of Office, show you some of our favourite features and demonstrate how to get the most out of our productivity suite.

For more details about the Office 2010 Installfests or to register (it’s free!), visit the EnergizeIT Office 2010 Installfest page.

Academic Sessions

We’ll also be passing through a number of colleges across Canada, talking to students about getting ready for the working world and showing them resources that they can use to fire up their careers.

This article also appears in Canadian Developer Connection.

Here’s the keynote room, about 15 minutes before the start of Scott Guthrie’s keynote. Watch this blog as well as my Twitter stream – I’m @AccordionGuy and I’m tagging my posts with #CanMIX (for Canadian MIX) – for updates!

This article also appears in Canadian Developer Connection.

I’m in Las Vegas to catch Microsoft’s MIX10 conference, where a lot of big announcements are being made. The combination of having just come from the Confoo conference in Montreal, flying to Vegas by way of Vancouver, and “springing forward” to daylight saving time has left me feeling like the bear pictured above.

More posts from Vegas and MIX10 to follow!

This article also appears in Canadian Developer Connection.

Welcome to another installment of Counting Down to Seven, a series of articles about mobile app development that I’m writing as we count down the days to MIX10, when we reveal more about the up-and-coming Windows Phone 7 Series.

For the longest time, the sessions listed under “Windows Phone” at the MIX10 conference (taking place in Las Vegas from March 15th through 17th) have had no details – just a “more details coming soon” message. That changed yesterday, and now the sessions have full names and abstracts, which I’ve listed below in chronological order.

This article also appears in Canadian Developer Connection.

If you’re wondering what’s new in Visual Studio 2010, you’re going to want to catch these sessions taking place in March. You’ll get a grand tour of all the new capabilities as well as the new MSDN offerings that come with the new Visual Studio.

The “What’s New” sessions are taking place in these cities:

City	Date	Invitation Key
Edmonton	Monday, March 1	8ACE98
Calgary	Tuesday, March 2	9FA90A
St. John’s, Newfoundland	Tuesday, March 2	C89B02
Mississauga	Thursday, March 4	5A1CB4
Quebec City	Tuesday, March 9	1C5C3C

Here’s the event schedule:

• Registration & Breakfast
  8:30 a.m. – 9:00 a.m.
• Event Opening Ceremonies
  9:00 a.m. – 9:10 a.m.
• Live technical demonstration:
  What’s new with Visual Studio Team System 2010
  9:10 a.m. – 11:00 a.m.
• Q&A
  11:00 a.m. – 11:20 a.m.
• Event close / completion of evaluation form
  11:20 a.m. – 11:30 a.m.

If you’d like to attend one of these sessions, select the Invitation Key from the city whose session you want to attend and enter it on the Registration page.

This article also appears in Canadian Developer Connection.

Whenever you do anything where money changes hands, from getting cash from the ATM to buying anything – movie tickets, groceries, a new pair of shoes or a new car to booking a flight and hotel room, chances are that there’s some Microsoft technology involved. It could be an SQL Server database, an app written in Visual Studio, a site living on IIS or Azure or a business process powered by SharePoint, and more likely than not, someone was using Office as well. From devices that fit in your pocket to cavernous data centres, the Microsoft platform helps millions of people across a broad spectrum of industries get real work done every day.

Want to know what’s possible with the Microsoft-based platform? Want to know how it all fits together? That’s what the EnergizeIT 2010 tour is for. In March and April, we’re visiting 20 cities across Canada – as far west as Victoria and as far east as St. John’s – to host free local gatherings where we show you how you can take advantage of our tools and technology to drive your business and your career.

EnergizeIT will comprise different sorts of events in different cities, all of which are listed below.

The “From the Client to the Cloud” Full-Day Events
(Vancouver, Edmonton, Calgary, Ottawa, Toronto and Montreal)

In Canada’s six largest cities, we’ll hold our EnergizeIT From the Client to the Cloud full-day events, where we’ll cover the Microsoft-based platform in detail. And yes, even though it’s full-day, it’ll still be free!

In the morning, we’ll talk about the big picture. We’ll show you a scenario featuring the Microsoft-based platform as seen from different points of view: the customer, the information worker, the developer and the IT professional. You’ll see our latest and greatest as well as our up-and-coming developer goodies: Silverlight, .NET 4.0 and Visual Studio 2010. We’ll show you Windows 7 and Azure in action, talk about Office 2010 and SharePoint 2010, and tell you how all of Microsoft’s stuff works together.

Just as the morning session answers the question “What’s the latest technology?”, the afternoon sessions answer the question “How do I get to the latest technology from where I am now?” These sessions, split into two tracks – one on infrastructure management and deployment, one on the development process – will cover what you can do with our tools and technology in a little more depth. They’ll show you what you need to implement what you saw in the morning session and provide a roadmap you can follow to learn more and take action.

For more details about From the Client to the Cloud events or to register (it’s free!), visit the EnergizeIT From the Client to the Cloud page.

The “Community Connection” Evening Events
(Many Cities Across Canada)

We’ll also hold Community Connection events in the evening in many cities across Canada, where we’ll do the “big picture” session (the morning session) of our From the Client to the Cloud events.

The Community Connection evening events will take place in the following cities:

• British Columbia: Vancouver, Kelowna and Victoria
• Alberta: Edmonton and Calgary
• Saskatchewan: Regina and Saskatoon
• Manitoba: Winnipeg
• Ontario: Ottawa, London, Kitchener and Mississauga
• Quebec: Montreal, Quebec City and Trois-Rivières
• Atlantic Canada: Halifax, St. John’s, Moncton and Fredericton

For more details about Community Connection or to register (it’s free!), visit the EnergizeIT Community Connection page.

Office 2010 Installfests
(Vancouver, Ottawa, Calgary, Montreal, Mississauga)

I’ve been using the beta and release candidate versions of Office 2010 in my day-to-day work for the past couple of months – PowerPoint for my public speaking stuff, Outlook for email, scheduling and get-things-done stuff and OneNote for my copious note-taking. We’d like you to take it for a spin!

Join us at one of our Microsoft Office 2010 Installfests and we’ll hook you up with the latest build of Office, show you some of our favourite features and demonstrate how to get the most out of our productivity suite.

For more details about the Office 2010 Installfests or to register (it’s free!), visit the EnergizeIT Office 2010 Installfest page.

Academic Sessions

We’ll also be passing through a number of colleges across Canada, talking to students about getting ready for the working world and showing them resources that they can use to fire up their careers.

This article also appears in Canadian Developer Connection.

Are you Canadian and going to the MIX10 Conference?

If you’re going to MIX10, let me know, either in the comments or via email. A number of us from Microsoft Canada will be there and we’d love to catch up with you!

Among the Canadian contingent going to Vegas are:

• Gladstone Grant, Developer and Platform Evangelism Lead
• Allan Hoffman, ISV Group Manager
• Paul Laberge, Web Platform Evangelist
• John Oxley, Director, Audience Marketing and my manager
• Mark Relph, Senior Director, Windows Ecosystem (and former Developer and Platform Evangelism Lead)
• Jamie Wakeam, ISV Architect Evangelist
• Yours Truly, Joey deVilla, Developer Evangelist and guy with accordion

Hope to see you there!

What is MIX10?

MIX10 is the 2010 edition of MIX, Microsoft’s most “right-brained” conference. Its area of focus is on the web and other technologies that aren’t the desktop, which is traditionally where Microsoft “lives”, as well as on design, usability, information architecture and user experience. Silverlight made its first appearance here, under the less-wieldy name of WPF/E (“WPF Everywhere”), as have improved versions of Internet Explorer. Expect some interesting stuff at MIX this year!

Here’s a list of the topics that will be covered at MIX10:

The Full Monty on Windows Phone Development

Glaringly absent from yesterday’s Windows Phone 7 Series announcement made at Mobile World Conference in Barcelona was the “how”: that is, how do you develop apps for Windows Phone 7?

That question will be answered at MIX10 (March 15th – 17th in Las Vegas) in a number of ways.

If you go to MIX10, you will get the following:

• Access to a track dedicated to Windows Phone 7 Series platform
• An introduction to Windows Phone 7 Series’ development platform
• Tutorials on how to work with the Windows Phone 7 Series’ development tools
• A tour of the Windows Phone Marketplace
• And last – but certainly not least — access to the Windows Phone 7 Series developer tools!

The Hallway Opportunity

I’ve always believed that one of the marks of a good conference is the hallway. By “hallway”, I’m talking about the opportunities to meet people in those times and places between and after sessions. There’s something to meeting people in person that you don’t get online; hence the often-used saying “you had to be there”. MIX promises to have good hallway, partly because of the Microsoft teams who’ll be presenting some interesting new stuff and partly because the crowd is going to be a mixed bag of developer types, designer types and the type of people who like to straddle both worlds (I like to think of myself in that category).

Go!

MIX10 takes place at the Mandalay Bay Convention Center in Las Vegas, from Monday, March 15th through Wednesday, March 17th. If you register before February 21st, you’ll get a $200 discount off the MIX10 admission fee. Do it now!

This article also appears in Canadian Developer Connection.

Here’s the fourth in my series of notes taken from keynotes at CUSEC 2010, the 2010 edition of the Canadian University Software Engineering Conference. These are from Bits of Evidence: What we actually know about software development and why we believe it’s true, a keynote given by my friend Greg Wilson, the computer science prof we all wish we had. He’s also the guy who gave me my shot at my first article for a developer magazine, a review of a couple of Ajax books in Software Development.

My notes from his keynote appear below; he’s posted his slides online.

A Little Family History

• My great-grandfather on my father’s side came from Australia
• He was sent there, along with many other criminals from the UK, to Botany Bay
• Whenever we kids did anything bad, my mother would say to my father: “This your side of the family"
• This happened until the day my sister triumphant discovered that my maternal great-grandfather was a Methodist minister who ran off with his church’s money and a 15-year-old girl from his parish
• She never brought up my father’s family history again
• It took years and poring over 70,000 sheets of microfiche to track down my great-grandfather, all to find two lines, which said he was sentenced, but not why
• These days, students get upset if it takes more than 15 seconds to find answers to last year’s exam
• Some things haven’t changed: while technology has improved, the way we develop software for it hasn’t

Scurvy

• In the Seven Years’ War, which lasted longer than seven years (1754-63), Britian lost:
  • 1,512 sailors to enemy action
  • 100,000 sailors to scurvy
• Scurvy’s a really ugly disease. You get spots on your skin, your gums puff up and go black, you bleed from your mucous membranes, and then the really bad stuff happens
• In 1747, a Scotsman named James Lind conducted what might have been the first-ever controlled medical experiment
• Pickled food keeps fresh, he reasoned, how about pickled sailors?
• Lind tried giving different groups of sailors with scurvy various acidic solutions:
  • Cider
  • Sulfuric acid
  • Vinegar
  • Sea water (this was the control)
  • Oranges
  • Barley water
• The sailors who had the oranges were the ones who recovered
• Despite Lind’s discovery, nobody paid attention until a proper Englishman repeated the experiment in 1794
• This discover probably won the Napoleonic Wars: the British navy was the deciding factor
• As a result of this discovery, British sailors planted lime trees at their ports of call and ate the fruit regularly; it’s how the term “limey” got applied to them, and later by association to British people in general

Lung Cancer

• It took a long time for medical science to figure out that controlled studies were good
• In the 1920s, there was an epidemic of lung cancer, and no one knew the cause
• There were a number of new things that had been introduced, so any of them could be blamed – was it cars? Cigarettes? Electricity?
• In the 1950s, the researchers Hill and Doll took British doctors and split them into 2 groups:
  • Smokers
  • Non-smokers
• The two discoveries to come from their research were:
  • It is unequivocal that smoking causes lung cancer
  • Many people would rather fail than change
• In response to the study, the head of the British medical association said: "What happens ‘on average’ is of no help when one is faced with a specific patient"
• The important lesson is to ask a question carefully and be willing to accept the answer, no matter how much you don’t like it

Evidence-Based Medicine

• In 1992, David Sackett of McMaster University coined the term "evidence-based medicine"
• As a result, randomized double-blind trials are accepted as the gold standard for medical research
• Cochrane.org now archives results from hundreds of medical studies conducted to that standard
• Doing this was possible before the internet, but the internet brings it to a wider audience
• You can go to Cochrane.org, look at the data and search for cause and effect

Evidence-Based Development?

• That’s well and good for medicine. How about programming?
• Consider this quote by Martin Fowler (from IEEE Software, July/August 2009):
  ”[Using domain-specific languages] leads to two primary benefits. The first and simplest, is improved programmer productivity… The second…is…communication with domain experts.”
• What just happened?
  • One of the smartest guys in our industry
  • Made 2 substantive claims
  • In an academic journal
  • Without a single citation
• (I’m not disagreeing with his claims – I just want to point out that even the best of us aren’t doing what we expect the makers of acne creams to do)
• Maybe we need to borrow from the Scottish legal system, where a jury can return one of three verdicts:
  • Innocent
  • Guilty
  • Not proven
• Another Martin Fowler line:
  ”Debate still continues about how valuable DSLs are in practice. I believe debate is hampered because not enough people know how to use DSLs effectively.”
• I think debate is hampered by low standards of proof

Hope

• The good news is that things have started to improve
• There’s been a growing emphasis on empirical studies in software engineering research since the mid-1990s
• At ICSE 2009, there were a number of papers describing new tools or practices routinely including results from some kind of test study
• Many of these studies are flawed or incomplete, but standards are constantly improving
• It’s almost impossible to write a paper on a new tool or technology without trying it out in the real world
• There’s the matter of the bias in the typical guinea pigs for these studies: undergrads who are hungry for free pizza

My Favourite Little Result

• Anchoring and Adjustment in Software Estimation, a 2005 paper by Aranda and Easterbrook
• They posed this question to programmers:
  ”How long do you think it will take to make a change to this program?”
  • The control group was also told:
    ”I have no experience estimating. We’ll wait for your calculations for an estimate.”
  • Experiment group A was told:
    ”I have no experience estimating, but I guess this will take 2 months to finish.”
  • Experiment group B was told:
    ”I have no experience estimating, but I guess this will take 20 months to finish.”
• Here were the groups’ estimates:
  • Group A, the lowball estimate: 5.1 months
  • Control group: 7.8 months
  • Group B, the highball estimate: 15.4 months
• The anchor – the “I guess it will take x months to finish” — mattered more than experience.
• It was a small hint, hint buried in the middle of the requirements, but it still had a big effect, regardless of estimation method or anything else
• Engineers give back what they think we want to hear
• Gantt charts, which are driven by these estimates, often end up being just wild-ass guesses in nice chart form
• Are agile projects similarly affected, just on a shorter and more rapid cycle?
• Do you become more percentage-accurate when estimating shorter things?
• There’s no data to back it up!

Frequently Misquoted

• You’ve probably heard this in one form or another:
  ”The best programmers are up to 28 times more productive than the worst.”
• It’s from Sackman, Erikson and Grant’s 1968 paper, Exploratory experimental studies comparing online and offline programming performance
• This quote often has the factor changed – I’ve seen 10, 40, 100, or whatever large number pops into the author’s head
• Problems with the study:
  • The study was done in 1968 and was meant to compare batch vs. interactive programming
    • Does batch programming resemble interactive programming?
  • None of the programmers had any formal training in computer programming because none existed then
    • (Although formal training isn’t always necessary – one of the best programmers I know was a rabbinical student, who said that all the arguing over the precise meaning of things is old hat to rabbis: “We’ve been doing this for much longer than you”.)
  • What definition of “productivity” were they using? How was it measured?
  • Comparing the best in any class to the worst in the same class exaggerates any effect
    • Consider comparing the best driver to the worst driver: the worst driver is dead!
  • Too small a sample size, too short an experimental period: 12 programmers for an afternoon
  • The next similar “major” study was done with 54 programmers, for “up to an hour”

So What Do We Know?

• Look at Lutz Prechelt’s work on:
  • Productivity variations between programmers
  • Effects of language
  • Effects of web programming frameworks
• Things his studies have confirmed:
  • Productivity and reliability depend on the length of program’s text, independent of language level
  • Output in text per hour is constant [Joey’s note: Corbato’s Law!]
• The take-away: Use the highest-level language you can!
  • Might not always be possible: "Platform-independent programs have platform-independent performance"
  • Might require using more/faster/better hardware to compensate
  • That’s engineering – it’s what happens when you take science and economics and put them together

• Discoveries from Boehm, McClean and Urfrig’s (1975) Some Experience with Automated Aids to the Design of Large-Scale Reliable Software:
  • Most errors are introduced during requirements analysis and design
  • The later a bug is removed, the more expensive it is to take it out
• This explains the two major schools of software development:
  • Pessimistic, big-design-up-front school: “If we tackle the hump in the error injection curve, fewer bugs get into the fixing curve”
  • Optimistic, agilista school: Lots of short iterations means the total cost of fixing bugs go down

• Who’s right? If we find out, we can build methodologies on facts rather than best-sellers\

Why This Matters

• Too many people make the "unrefuted hypothesis based on personal observation"
• Consider this conversation:
  • A: I’ve always believed that there are just fundamental differences between the sexes
  • B: What data are you basing that opinion on?
  • A: It’s more of an unfuted hypothesis based on personal observation. I have read a few studies on the topic and I found them unconvincing…
  • B: Which studies were those?
  • A: [No reply]
• Luckily, there’s a grown-up version of this conversation, and it takes place in the book Why Aren’t More Women in Science? Top Researchers Debate the Evidence (edited by Ceci and Williams)
  • It’s an informed debate on nature vs. nurture
  • It’s a grown-up conversation between:
    • People who’ve studied the subject
    • Who are intimately familiar with the work of the other people in the field with whom they are debating
    • Who are merciless in picking apart flaws in each other’s logic
  • It looks at:
    • Changes in gendered SAT-M scores over 20 years
    • Workload distribution from the mid-20s to early 40s
    • The Dweck effect
      • Have 2 groups do a novel task
      • Tell group A that success in performing the task is based on inherent aptitude
      • Tell group B that success comes from practice
      • Both groups will be primed by the suggestions and “fulfill the prophecy”
      • We send strong signals to students that programming is a skill inherent to males, which is why programming conferences are sausage parties
    • Facts, data and logic

Some Things We Know (and have proven)

Increase the problem complexity 25%, and you double the solution complexity. (Woodfield, 1979)

The two biggest causes of project failure (van Genuchten et al, 1991):

• Poor estimation
• Unstable requirements

If more than 20 – 25% of a component has to be revised, it’s better to rewrite it from scratch. (Thomas et al, 1997)

• Caveats for this one:
  • It comes from a group at Boeing
  • Applies to software for flight avionics, a class of development with stringent safety requirements
  • Haven’t seen it replicated 

Rigorous inspections can remove 60 – 90% of errors before the first test is run (Fagan, 1975)

• Study conducted at IBM
• Practical upshot: Hour for hour, the most effective way to get rid of bugs is to read the code! 

Cohen 2006: All the value in a code review comes from the first reader in the first hour

• 2 or more people reviewing isn’t economically effective
• Also, after an hour, your brain is full
• Should progress be made in small steps? Look at successful open source projects

Shouldn’t our development practices be built around these facts?

Conway’s Law – often paraphrased as “A system reflects the organizational structure that built it” — was meant to be a joke

• Turns out to be true
• Jim Herbsleb et al proved it in 1999

Nagappan et al (2007) and Bird et al (2009) got their hands on data collected during the development of Windows Vista and learned:

• Physical distance doesn’t affect post-release fault rates
• Distance in organizational chart does
• If two programmers are far apart in the org chart, their managers probably have different goals [Joey’s note: especially in companies like Microsoft where performance evaluations are metrics-driven]
• Explains why big companies have big problems
• I remember once being told by someone from a big company that "we can’t just have people running around doing the right thing."

Two Steps Forward

• Progress sometimes means saying "oops"
• We once thought that code metrics could predict post-release fault rates until El Emam et al (2001): The Confounding Effector of Size on the Validity of Object-Oriented Metrics, where it was revealed that":
  • Most metrics values increase with code size
  • If you do a double-barrelled correlation and separate the effect of lines of code from the effect of the metric, lines of code accounts for all the signal
• It’s a powerful and useful result, even if it’s disappointing
• It raises the bar on what constitutes "analysis"
• We’re generating info and better ways to tackle problems

Folk Medicine for Software

• Systematizing and synthesizing colloquial practice has been very productive in other disciplines – examples include:
  • Using science to derive new medicines from folk medicine in the Amazon
  • Practices in engineering that aren’t documented or taught in school
• There’s a whole lot of stuff that people in industry do as a matter of course that doesn’t happen in school
• If you ask people in a startup what made them a success, they’ll be wrong because they have only one point of data
• If you’re thinking about grad school, it’s this area where we’ll add value

How Do We Get There?

• One way is Beautiful Code, a book I co-edited
  • All proceeds from sales of the book go to Amnesty International
  • In its 34 chapters, each contributed by a different programmer, they explain a bit of code that they found beautiful
  • I asked Rob Pike, who contributed to the development of Unix and C, and he replied "I’ve never seen any beautiful code."
  • I asked Brian Kernighan, another guy who contributed to the development of Unix and C, and he picked a regex manager that Rob Pike wrote in C
• This has led to a series of "Beautiful" books:
  • Beautiful Architecture
  • Beautiful Data
  • Beautiful Teams
  • Beautiful Security
  • Beautiful Testing
• The next book is "the book without a name”
  • I would’ve called it Beautiful Evidence, but Edward Tufte got there first
  • The book will be about "What we know and why we think it’s true"
  • Its main point is knowledge transfer
  • I’m trying to change the debate
  • I want a better textbook, a believe that this will be a more useful textbook
• "I want your generation to be more cynical than it already is."
• I want you to apply the same standards that you would to acne medication
  • How many of you trust research paid for by big pharma on their own products?
  • I want you to have higher standards for proof
• The real reason all this matters? [Shows a slide with a picture of the world and his daughter]
  • “She is empirically the cutest kid in the world”
  • Public debate is mostly evidence-free
  • You’re not supposed to convict or release without evidence
  • I want my daughter to inherit a better world

What University is For

• [Asks the audience] Hands up if you think you know what university is for
  • One student answers “A piece of paper!”
  • Greg’s reply: "Remember when I said I wanted you to be more cynical? Don’t go that far."
• In my high school in interior BC, there was no senior math or physics course, because there was no interest
• I went to university, hungry to learn
• I later discovered that it wasn’t about the stuff you learned
• Was it supposed to teach us how to learn?
• It’s supposed to train us to ask good questions
• Undergraduates are the price universities pay to get good graduate students
• This thinking got me through the next 15 years
• Then I went back to teaching, at U of T, and I think I know what universities are for now:
  • We are trying to teach you how to take over the world.
  • You’re going to take over, whether you want to or not.
  • You’re inheriting a world we screwed up
  • You’ll be making tough decisions, without sufficient experience or information
• Thank you for listening. Good luck.

Comments from the Q&A Session

• Who’s going to an anti-proroguing rally this weekend?
  • It won’t do you any good
  • You want to make change? It’s not made at rallies, but by working from within
  • The reason you can’t teach evolution in Texas isn’t because fundamentalists held rallies, but because they ran for the school board
  • We have to do the same
  • To make change, you have to get in power and play the game
  • “Dreadlocks and a nose ring does not get you into the corridors of power”
  • The ACM and IEEE are arguing your case
    • Join them and influence them!
• Do we want to turn out engineers, who are legally liable for their work, rather than computer scientists, who are not?
  • Some of us will have professional designations and be legally liable, some of us won’t — there’s no one future

This article also appears in Canadian Developer Connection.

Here’s the third in my series of notes taken from keynotes at CUSEC 2010, the 2010 edition of the Canadian University Software Engineering Conference. These are from Beautiful Failure, a keynote given by my friend Reg “Raganwald” Braithwaite, who’s forgotten more about combinators than I will ever learn.

My notes from his keynote appear below; Reg has also published his slides online.

• I gave a talk at Stack Overflow DevDays Toronto in which I was thinking out loud about programming about programming
• I was trying to rewrite the way we program
• The language we use for coding guides the way we think about the program and the solutions
• When you write things to change your programming language, you change the way you think

Thinking About Programming About Programming

• I often get called in by clients to automate a process
• Often, during this process, they want to change the process that I’m supposed to automate
• Automating a process forces you to think about it
• The very act of thinking about how you do things helps you understand what it is you do
• The exercise of thinking it through is useful, even if it fails or you don’t end up using it
• Languages and frameworks come and go, but everything you to do fix what’s between your ears stays with you forever
• Programming languages are just a notation for the way we think

• Some people try to do things like add a "sum" method to Ruby’s Enumerable mixin
• What happen when you try [[1, 2], [3, 4], [5, 6]].sum?
• [He showed two implementations of a “sum” method:
  • One by “Alice”, which when applied to [[1, 2], [3, 4], [5, 6]], yielded 21,
  • and one by “Bob”, which when applied to [[1, 2], [3, 4], [5, 6]], yielded [1, 2, 3, 4, 5, 6]
• With “monkeypatching”, it’s possible for two different modules to implement Enumerable#sum, and then for someone else to import both modules.
  • In which case, which version of sum will get called? It depends on the load order of the module
  • But what if these were written as gems? Then there’s trouble
• To solve this sort of problem, I decided to steal extension methods from C# and add them to Ruby [Joey’s note: extension methods are a C# feature that let you add methods to an existing class without subclassing]
• It works, but what’s wrong with what I’ve done?
• My extension methods for Ruby are a hack…
  • to fix Alice and Bob’s hack…
  • of Matz’s (Yukihiro Matsumoto) hack of Smalltalk’s hack (i.e., Ruby)
• It eliminates the annoyance without solving the core problem
• Do extension methods reengineer the way we think about problems? Or do they simply deal with an annoyance?
• Do they reengineer the way we think about programs?

• Take the Single Responsibility Principle (SRP)
• When you write an extension method, you break SRP
• When you monkeypatch, you violate SRP
• Is that bad? I don’t know
  • C# breaks SRP with extension methods
  • Rails "runs roughshod over it"
  • If two popular languages break SRP, maybe SRP isn’t all that
  • What does the sum method tell us?
  • Why is this a beautiful failure?
  • Maybe we’ve gone beyond the class — Ruby is not C++ or Smalltalk
  • Hacks like this scratch an itch and suggest a flaw — what else is flawed?
• You have an advantage over me
  • I have this ball and chain of experience
  • I’ve been fucking with computers for almost 40 years
  • They way I’ve been doing things has made me a living; I’m not incented to change the way I do things
  • You’re not tied down
• So now I present a few ideas that have occurred to me — think about them!
  • I don’t have the answers

• Unit tests tell us that compilers are flawed
  • If we need them, what is wrong with our programming languages and compilers that requires us to step out of what we’re doing to implement them?
  • Why do we need to take a great language and bolt something onto the side?
• Github tells us that our existing idea of a program is flawed
  • Most people think of programs as static things
  • In Github, there is no "program" — there are branches, forks and tags
  • Languages themselves have no notion of what a version is
  • Looking at the way we actually use tools shows that there’s a disconnect between our toolsets and the way we write code
  • Are Github commits congruent to objects?
    • If you change 4 classes in a commit, there must be something they have in common, but that’s not apparent from the way we write them
• Do we manage work the way we manage code?
  • Project management seems awfully disconnected from our tool chain
  • Consider the complete disconnect between issue tracking and time tracking
  • Maybe not so important in your company, but more important for personal projects
  • Git and Lighthouse — “like two cups connected by string”
• Do we manage object versions the way we manage API versions?

• "Do not follow in the footsteps of the sages, seek what they sought."
• What I think is particularly cool and interesting is…but to me
• Think about what your heroes were trying to achieve using the tools available to you today
• An example of following blindly in the footsteps of sages:
  • In November 2002, I attended Paul Graham’s “Lightweight Languages 2” conference in Boston
  • The morning keynote was by John Armstrong, who presented Erlang, which today is considered an important language for concurrent programming
  • The afternoon keynotes was Matz, who presented Ruby, one of the most influential dynamic languages that soon after enjoyed a meteoric rise in popularity
  • Many people in the room, die-hard Lisp-heads, were shouting them down because their languages didn’t have macros [Joey’s note: Macros are a Lisp feature that smug Lisp weenies often use in the never-ending “Why my language is better than your language” argument]

Four Ugly Failure Modes and How to Avoid Them

Confusing correlation with causation

• I think it’s one of the most prevalent diseases in the business world
• Ruby is not a silver bullet
  • Was the success of many Ruby projects [such as Rails and Twitter] because of Ruby the language?
  • Or was it that smart people who could get things done were picking Ruby at a given point in time?
• Agile is not a process
  • It’s a set of values
• Here’s how many companies fail:
  • They start a little consulting company
  • They enjoy some successes, which leads to more business
  • As a result, they hire people and the company grows
  • But they can’t hire smart people faster than the work is coming in
  • So in order to hire people to meet the demand, they start hiring people who aren’t as smart
  • That’s when things go downhill
  • Who here doesn’t think this isn’t standard for any consulting company?
• Toronto Agile User Group recruiting process
  • In our field, "best practices" are cow patties
  • I’ve gone to many companies where they combine "best practices" simply by smooshing them together
  • I’ve been to many Toronto Agile User Group meetings where very few attendees work at companies that even practice agile
  • The important thing is that the people there are attending because interested in finding a better way of doing their work – those are the people you should be hiring!
• The plural of "anecdote" is not "data"
  • Greg Wilson will talk about this in his keynote later today – listen to him!
  • Problem: Talks are given by narcissists (or masochists)
  • When you read something in a blog, see something on TV or buy a book, you’re not getting a large enough sample, and the content is biased
  • Another problem is that history is written by the survivors
  • People write about really notable successes or failures

Confirmation bias

• "Most of you will be immune to this, because you’re all sensible people"
• You might fall victim to confirmation bias if you have an overly-inflated (or under-inflated) ego
• You might also fall victim to it if your worldview is too narrow
  • If you’re a Ruby developer, you probably don’t read C# blogs, and vice versa
• Seek out more representative info; not just the stuff that confirms your opinions

Local maxima

• The innovator’s dilemma
  • If you have customers, they will trap you in a local maximum
  • They’re not trying to be mean, they’re trying to give you money
  • You might end up optimizing to serve your customer base while the rest of the world (and eventually your business moves on)
• The Principle of Least Surprise is a trap!
  • Familiarity comes from doing the old things the old way
  • This doesn’t apply to just UI, but also naming variables or coding styles
  • Once in a while, you should say "Maybe this one time, we should do things differently"
• Iterative anything is a trap
  • It’s hill climbing
  • Sometimes you have to leap
  • It’s supposed to be bad to "go dark" in development for a longer period rather than go through many small iterations, but sometimes it’s the only way to make a great leap
  • You can’t climb a big mountain if you do things in small increments

"A Market for Lemons"

• What happens when you sell to people who don’t fundamentally understand what they’re buying?
• If customers don’t understand what they’re buying, they make their decisions based on easily differentiable features
• One example is buying a house, which you’re not going to do very often in your life, so most people know very little about it
  • As a result, they focus on easily differentiable features like square footage, number of rooms, and other features that can easily be picked out
  • But it’s better to focus on whether the house’s design makes it more liveable, which is harder to suss out
• Another example of this is feature checklists on the back of product boxes
• Gresham’s Law — “bad money drives out good” — applies to talent: When you have good currency and bad currency in an economy, the bad currency drives the good currency out
  • This happens in Cuba, where the good currency – black market US dollars – gets hoarded while the local currency gets spent
  • It also applies to information: people put the crappy information out, and it drives the good information down
  • It also applies to talent: headhunters, not knowing what sort of people to look for, end up grabbing the people who put the most buzzwords on their resumes
• You don’t want to be one of those buyers

At the end of the presentation, posted a slide dedicated to his late friend, Sam Roweis (1972 – 2010).

This article also appears in Canadian Developer Connection.

Here’s the second in my series of notes taken from keynotes at CUSEC 2010, the 2010 edition of the Canadian University Software Engineering Conference. These are from NSFW, a keynote given by my friend Pete Forde, partner at Unspace and one of the bright lights of Toronto’s tech scene.

My notes appear below. Pete’s posted his slides, notes and URLs online; be sure to check them out.

Introduction

“This talk is going to be adult,” began Pete. “If you can’t handle it, you should probably leave. I’ll buy you a Dasani afterwards.”

• I’m a partner at Unspace
  • It’s a company where I decided to bet the farm on Rails, a platform I like
  • We’ve done unusual things, like hold a job fair where local Ruby programmers could show off their work
  • We’ve built software projects for the UN
  • Every project we take on has a lot of heart
  • We don’t want to work with anyone who isn’t super-excited to be there
• I’m a software developer, have been for a long time
• But deep down, I want to be a designer
  • I have no formal training — I can’t draw; I can’t paint
  • I see life as a series of carefully-executed series of five year plans
  • I dropped out of high school 20 minutes before the final exam; I told the principal that I didn’t want him to take credit for future success
    • I don’t recommend this; it’s probably not repeatable, not even by me
• You – as engineering and computer science students –- are better educated than me
  • “You probably know math and stuff”
• In the past, I was a punk, and many other things
  • I’ve been a musician
  • I’ve also been a zine publisher
  • I’ve tried on a lot of things to see if the shoe fits
  • I’ve had an interesting run
• When I get to the end of 5 years of doing something, I review what I’ve done
  • I’ve had 5 years of doing software at Unspace – what now?

On Pete

• My dad’s an engineer, and as such, is a perfectionist
  • Engineers are by and large pedantic control freaks — and that’s okay, we need you to be that way!
• I’ve discovered that I’m a starter, not a finisher
• This tendency has put me at odds with my family and I used to feel really guilty about it
• Now I realize is that you need to play to your strengths — recognize that you have an instinct, and harness it!
• Is what you’re doing against the grain?
  • "There’s no time like the present to get your life on track"
  • "I could have saved myself a lot of time if I could talk to my present-day self"
• As a starter but not a finisher, I realized that I had to recruit doers, people who could take my ideas and run with them
  • Some of these projects will succeed
  • Some will fail…but they’ll fail grandly
  • I’ve got lots of things on the go
  • Am I arrogant? I see it as audacious
  • Seth Godin said: “If you’re not uncomfortable in your work as a leader, it’s almost certain you’re not reaching your potential as a leader."
• I am an introvert
  • See the article in The Atlantic, Caring for Your Introvert
  • So what am I doing onstage?
  • People who appear practiced onstage look that way because they are practiced

On Success

• Steve Jobs says: “Find what you love”
  • People confuse “successful” with “happy”
  • Are you putting your life on hold to go and make your paycheque?
  • I’m convinced that many financially successful people are unhappy and bitte
• Malcom Gladwell’s The Sure Thing
  • It paints a different picture from the one we see in the media of the entrepreneur as daring, as a “cowboy”
  • Entrepreneurs who became empire builders turned out be highly risk-averse
  • Their success comes from seeing opportunities in arbitrage and taking advantage of them
  • Consider John Paulson:
    • His hedge fund made money shorting America’s economy
    • He bought shorts on subprime mortgages
  • These men are predatory entrepreneurs in my opinion
  • Do they really need billions?
  • Maybe they don’t do it for evil – perhaps it might be for the thrill
• Don’t want to model himself after these people
  • There’s a line written by Seth Tobocman, who wrote the comic book World War 3: "You don’t have to fuck people over to survive."
  • My twist on that is "You don’t have to fuck yourself over to be successful."
• Who would I rather model myself after? Steve Jobs
  • He said: “Good business makes for good art”
• Another good bit of advice comes from Andy Warhol: “Think rich, look poor.”
• On Being an Artist
  • There used to be a harsh disciplinary division between technology and art and it’s reflected in code and art
  • Different now in the era of Rails
  • I like holding parties and inviting all sorts of people: if you put interesting people together from all walks of life, you’ve got a catalyst for change in your living room
  • The lines are blurring: we’re all artists now
• Consider these guys
  • Ronald Tackmann, escape artist
    • "He hates being free"
  • Blane Nordahl, Silver thief
    • "The Mr. Belvedere of silver theft"
  • Both really good, treated their work like artistry

On Starting Up

• How Unspace came to be
  • It started 5 years ago with 2 friends in 170 square feet of space
  • “There wasn’t enough room to lie down and make a snow angel”
  • Everything that happened in those first years was "path of least resistance"
  • We had this weird notion that Unspace would be worth nothing and function as a quasi-legal organization whose reason for being was so that we could write off tech toy purchases
• We got lucky: Two founding partners — moved on to other things
  • One of them has since moved on, regrettably, to Ashley Madison
  • Choosing partners was important decision
• Optimism springs eternal among entrepreneurs: there’s always that feeling that nothing can go wrong
• Daniel Tenier says: “Partnerships suck”
  • It’s important to make your agreements explicit
  • Don’t be afraid to discuss bad stuff
  • Write everything down
  • You can’t make it work at all costs – you need to know when to walk away
  • Try to get to the bottom of questions like "What’s your definition of success?" Of failure? What’s the sunset clause? What’s the shotgun clause?
  • If you absolutely don’t need a partner, go it yourself (I myself, since I’m not a finisher, need a partner)
  • Look up what Chris Dixon has written about founder vesting

On Products

• Most consulting companies start as product companies that were broke
• Consulting is “kind of like a drug” — it keeps the fix coming

On Customer Development

• You need to read Steven Gary Blank’s The Four Steps to the Epiphany
• The ideas in this book led to the feeling in venture circles that customer development is a good thing
• If you’re starting a company that sells things to people, read it!

Leadership

• Seth Godin says this of leadership: It’s about painting a picture of the future for other people and then leading them to it
• Back in 2004, things went terribly wrong
• I partnered with my friend Ryan, and it lasted a month
• I had “lots of partners” – it was hard to get things done
• Having a captain is good
• In addition to being a “time-and-materials” company, we also started holding events
  • We instituted Rails Pub Nite, a monthly event that created a sense on community and gets regular attendance
    • Opposite of a user group: no agenda
    • It’s the "smartest thing we’ve ever done as a company"
    • At the time, “people making a living off Ruby you could count on both hands”
    • One of the raisons d’etre of Rails Pub Nite was to create meaningful competition
    • We went so much farther ahead by giving it the generic name Rails Pub Nite as opposed to Unspace Pub Nite
    • What we wanted to do was not create a feeling of participating in a corporate social experience
    • It was successful: Rails Pub Nite’s mailing list has 450 people, and every Pub Nite gets 40 – 50 attendees, and not just Ruby programmers, but also Java, .NET and PHP

Building Your Team

• Another benefit of Rails Pub Nite is that it lets us meet all the smart people first
• We have a “non-traditional fit test”
• I feel that 8 – 14 people is perfect size for company
• I’m tired of working for small companies that grew to large companies that started to suck
• I’d rather have 3 companies with 12 people than 1 with 40 people

On Guilt

• I have no high school education — how am I building projects for the UN?
• It’s why sometimes, I feel like a fraud
• Many people have this feeling; it’s called “Impostor Syndrome”
• I feel like living embodiment of "fake it until you make it"
• Refactoring makes me feel like a fraud
• It’s the "Embarrassing Pattern": after looking over my code, it seems that I could replace a lot of it with existing stuff and patterns
• “Your entire codebase can be abstracted away”
• "I just spent a month writing 40 lines of code"
• You have to recognize that it happens

On Getting Ahead

• Read Derek Sivers’ (he’s the guy who created CDBaby and later sold it) article, There’s No Speed Limit
• He says that “the standard pace is for chumps”
• To get ahead, you have to push yourself beyond what you think your limits are
• We can do whatever we want, as fast as we want

Adventure

• Learning Giles Bowkett’s story through his RubyFringe presentation completely changed my life
  • [Joey’s note: really, you should watch the video of this presentation, even if you never ever plan to write a single line of Ruby in your life. It’s inspiring.]
• It was all about leading a life less ordinary
• In our line of work, we create things that didn’t exist before
• When someone who doesn’t know how to create things is put in charge of people who do, it’s bad
  • I believe that Giles called them "Weasel-brained muppetfuckers"
• Giles quotes Steve Jobs: “Real artists ship”
• My advice on dating websites: "Don’t make them"

On Marketing

• I’ve mentioned Seth Godin many times already
• Sometimes his books have 3 pages of insight buried in 100 pages – I supposed it’s a case of “The Devil’s in the details”
• Read The Dip, skip Tribes
• In Tribes, Godin says that people don’t believe what you tell them, sometimes believe what their friends tell them and always believe the stories they tell themselves.
• So give people stories they can tell themselves

On Ideas

• Dr. Edwin Land: Invented instant photography – Polaroid
• Polaroid was thought of as dead until enthusiasts like Florian Kaps bought the Polaroid factory and now they’re bringing Polaroid film back to the market
• I wanted to bring him to FutureRuby, but he was too busy changing the world
• Another person to watch: Kacie Kinzer, the creator of Tweenbots
• Also watch: Neil Gershenfeld
  • He builds fablabs
  • He’s building them in places so low-tech that they don’t even have electricity
  • Get involved with the MIT "Bits and Atoms" program
• Another person to read about: William Kamkwamba (see The Boy Who Harnessed the Wind)
  • [Joey’s note: Watch his presentation at TED!]

Grand Visions for the Future

• Disney wanted EPCOT to be a utopian city, a city of the future, but bureaucracy got in the way
• Jacque Fresco: 93-year-old chronic inventor — a radical revolutionary
  • He designs amazing future habitat buildings
  • He has a whole compound of bubble domes in Venus, Florida
  • See the movie Future by Design
  • He’s 93 — "You know what that implies"

On Being Happy

• Listen up!
• Once again, a Steve Jobs quote: "Your time is limited. "
• Look at Jessica Hische, who’s obsessed with type and makes beautiful things
  • She followed her passion, and now she’s an internationally sought-after type artist
  • She recently said: “The work you do while you procrastinate is probably the work you should be doing for the rest of your life."

This article also appears in Canadian Developer Connection.

This is the first of a series of notes that I took while attending CUSEC, the Canadian University Software Engineering Conference, which took place last week in Montreal. CUSEC is the biggest conference held by and for university students interested in software development. True to the Canadian techies punching well above their weight class (a great tradition started by Alexander Graham Bell), CUSEC manages to pull in big-name and up-and-coming speakers who’ve given talks that have outshined those I’ve seen an thousand-dollar-plus conferences.

The first keynote was given by Matt Knox, who has probably distributed more Scheme runtimes than anyone else in the world (and this is a larger number than you might think), which he did in the name of putting adware on millions of machines. He’s since come to his senses and seems quite contrite.

His presentation, On Weakness, is about his life on the Dark Side and the lessons he gleaned from it. It’s based on his talk, Crimes Against Humanity, Writ Small, which he gave at FutureRuby last year, but it was good to see it again, and its message is probably even more valuable to students. My notes (which I polished for comprehensibility) and photos from his session appear below:

An Evil Job

• How many of you are:
  • Technical, as opposed to business or arts students?
  • Engineering students?
  • Programmers?
  • Evil?
• That’s what this talk is about
• One way to describe one of my former jobs is doing “Windows hijinks with Scheme”
• During my time with that job, I released many scheme runtimes
• Aaron Swartz – I think it was at a Y Combinator startup camp – said this of me: "He uses Scheme for evil!"
• It was more than just Scheme – I was writing stuff that had alternately “hard” (statically-typed languages) and “soft” (dynamically-typed languages) layers
• I was in the adware business, which is like walking into a big monkey knife fight…
• …except I was using a death ray! (Scheme == death ray, C == knife)
• I started with good intentions, in the business of building spam filters
• Business wasn’t so hit, and I ran out of money
• My job search failed, but luckily, a job went looking for me
• I was so pleased with being found that I  forgot to talk salary
• I showed up for the interview and at the end, was invited to work for them
• I did terribly when it came time to discuss what I would be paid
  • I didn’t research the New York City job market and cost of living
  • I asked for $40K
  • When I saw the look of shock of the guy’s face, I thought that I had asked for too much
  • Start reducing what I asked for; luckily he stopped me
• We want you to come in an analyze our distribution chain, they said
• It turned out to be an adware company:
  • Bought people’s “digital tchochkes” or mini-apps, such as screensavers
  • They had realized that there’s no lower bound for how cheesy something can be and still be a big seller on the internet
  • They took these mini-apps and gave them away online for free, bundled with software that gives you "special offers" from time to time
• Some of these bundled apps turned out to be worms
  • So the company had me write software to remove any worms from a system and added them to the bundle
  • So now we were bundling my anti-malware along with their adware
  • I felt like "an assassin working for the mob, but killing terrorists". The mob were bad, but the terrorists were worse
  • "Awesome! I can probably keep up with Norton…it’ll be great!"
  • And for a while, the best way to eradicate worms your system was to install their adware with my anti-malware bundled with it
• Low-level coding is dangerously seductive
  • In the beginning, it’s "like getting kicked in the face over and over again by buffer overruns"
  • But then it becomes fascinating
• I wanted to do it in Scheme, but that would require embedding a Scheme interpreter
  • Such an interpreter would have to fit into a single TCP/IP packet (about 64K)
  • Scheme is great. For any superlative — “best performance”, “smallest app”, and so on – there are usually two contenders: some other language, and Scheme.
  • I managed to squeeze a Scheme interpreter down to 19K
• My success with killing the worms led to a new request: In addition to your all this malware on other machines, why not eliminate all the competitor’s adware?
  • Now I felt like “an assassin for the mob, killing other mobsters”. Not as noble.
• Then the next request came: How about keeping our software from being killed…by anything? (including Norton)
  • The only way to uninstall the adware was to use the uninstaller, which came with it
  • I initially viewed this as "a really interesting technical problem"
• All this was made possible by a couple of Windows quirks…
  • CreateRemoteThread
    • This basically says to a process: "Hey, process! Execute this code as part of yourself, and you’ll think it’s awesome."
    • This lets you have code executing even though a process isn’t running
    • You don’t even need threads – you can hook interrupts
  • Scheduler
    • You can have a process tell the scheduler that it needs to do a do-over — "I’m not done yet, I need more time", and the scheduler will grant that time
    • You can tell even Windows that a process is so important that if it fails, it needs to protect the user by presenting a blue screen
• Windows is interesting from a purely archaeological perspective
  • Consider that all strings in Windows are 16-bit unicode, which means that nulls can be embedded in strings
  • But C strings, which is what’s used in the underlying DOS, are null-terminated and therefore can’t contain nulls
  • Interesting effects when moving null-containing strings between these layers

What Drives People to Take Up Evil Jobs?

• Aftermath of my working at the adware company:
  • Company got sued for $190 billion (by Elliot Spitzer!)
  • I was the first employee at the company — everyone else was a contractor
• I left the company with these questions:
  • "Whut happen?"
  • "Is this who I am?"
• Some jobs pay lots of money, but it’s hard to transition out of them
• Will I be stuck in adware for the rest of my life?
• There are some historical precedents:
  • Albert Speer
    • A promising architect who liked soaring buildings
    • He hooked up with rising politicians with the same aesthetic sense, one of whom was Hitler
    • He started with creating buildings, but then became the Nazis’ chief logistics guy
    • Later, a leader of the U.S. Air Force said that had he been aware of Speer’s involvement as the Nazi’s chief logistics guy, he would’ve dedicated an entire wing of the Air Force exclusively to killing him
    • It’s been suggested that Speer prolonged the war by a year or two by running the German forces more efficiently
  • Manhattan Project staff
• But I didn’t want anecdotes…I wanted science!
  • There’s a scientific study of otherwise good people doing evil things: the Milgram Experiment
    • How many people would go all the way?
    • 1% of the population is psychotic – it was hypothesized that the number of people who’d go all the way would be similar
    • Instead, 70% did
    • Results replicatable with people from all walks of life
    • Women, it turned out, “went evil” in a slightly greater proportion than the men
    • "Most human evil lives here"
• Read The Black Book of Communism
• For a more mundane example of blind obedience to authority leading to evil, see "The strip search McDonald’s prank call"
  • In the prank, the prankster calls a McDonald’s, gets an employee on the line and says “I’m a police officer. We have reason to believe that there is a thief in your restaurant and we need you to take them into the back and hold them until we arrive.”
  • They provide a description vague enough so that someone in the restaurant will match it
  • Once coralled in the back, the prankster starts giving orders to torture and/or humiliate the customer, and many employees have complied
• So what does this mean?
  • The human brain has a remote root exploit in 70% of the installed base
• "With or without religion, you would have good people doing good things and evil people doing evil things. But for good people to do evil things, that takes religion." — Steven Weinberg
  • Nope. Just authority.
• There is hope: people who were subjects of the Milgram experiments turned out to be better at resisting authoritative coercion

The Power of Communication

• Math: "There are only three reasonable numbers: 0, 1 and infinity"
• When Robert Andrews Millikan did his oil drop experiments to determine the charge on an electron, he initially got the value wrong by 30 – 40%
  • People who repeated the experiment or conducted similar experiments with results close to Millikan’s erroneous number published their results
  • People who did so but got the correct value – which did not match Millikan’s value – didn;t publish, worried that they’d done something wrong, since their numbers didn’t agree with the number published by the authority on the subject
• The world pre-blogs was so different from this world
  • Very first open source project: Oxford English Dictionary
    • Done via mail
  • Ever wondered where the term "flying off the handle" comes from?
    • It’s from sword-making – until they figured out the process of making swords as one-piece, with hand-friendly stuff wrapped around the base so you could hold them, swords often flew off their handles in battle
    • It took 900 years to evolve swords to one piece
• Not everything has been solved, but it’s easier today
• Rails is such a solution
  • It’s a series of incremental improvements
  • Can you out-Rails Rails?

This article also appears in Canadian Developer Connection.

Photo by Tom Purves.

I’m boarding a Porter flight bound for Montreal, where I’ll be attending CUSEC (Canadian University Software Engineering Conference). I’ll be there from today through Saturday afternoon, watching technical presentation, flying the Microsoft banner, hosting DemoCamp and having a beer (or twelve) with my fellow conference-goers. I’ll be posting notes and photos from the presentations and other goings-on, so watch this space!

This article also appears in Canadian Developer Connection.

Microsoft was a sponsor of CUSEC last year – that’s Canadian University Software Engineering Conference, the premier conference on building software aimed specifically at students. One of the perks of sponsorship was a “corporate speaker” slot, and it was decided that the presentation should be given it to the then-new guy…namely, me.

At the time I got slotted in as the speaker, I’d barely been a Microsoft employee for two months and was still feeling my way around both the company and its technology. By the time I would stand on the podium, I would have just passed my three-month probationary period. If I was going give a talk for forty-five minutes, it would have to be something other than “what it’s like to work at The Empire”.

Luckily, I did have something to talk about: a not-quite-normal career in tech, and the lessons I picked up along the way. The end result was a presentation titled Squeezeboxes, Start-Ups and Selling Out: A Tech Evangelist’s Story (yes, it’s a bombastic title, but it’s the sort of thing you’d expect from a guy whose personal blog’s name is The Adventures of Accordion Guy in the 21st Century.)

The presentation was scheduled for the end of Day 2 (it’s a three-day conference), which is a challenge. The audience would be tired and being students, they were likely to be more focused on the big drinkfest that would take place that evening. I decided to go for “offbeat” and built my presentation around the abstract I gave to them, which was:

You’ll spend anywhere from a third to half (or more) of your waking life at work, so why not enjoy it? That’s the philosophy of Microsoft Developer Evangelist Joey deVilla, who’s had fun while paying the rent. He’ll talk about his career path, which includes coding in cafes, getting hired through your blog, learning Python at Burning Man, messy office romances, go-go dancing, leading an office coup against his manager, interviewing at a porn company and using his accordion to make a Microsoft Vice President run away in fear. There will be stories, career advice and yes, a rock and roll accordion number or two.

They recorded my session and unleashed it on the world yesterday. I share it with you below:

If you watched the video, you’ll note that I skipped a couple of stories, namely “learning Python at Burning Man”, “messy office romances”, “go-go dancing” and making a Microsoft Vice President run away in fear. I’ll save those for another presentation. (By the bye, the guy I made run away is a President now.)

I had a blast doing this presentation, and the general consensus of the attendees was that it was one of the highlights of the conference. I’m honoured that I was invited back to host DemoCamp, and look forward to chatting with everyone. See you in Montreal!

This article also appears in Canadian Developer Connection.

For the latter half of this week, I’ll be at CUSEC – the Canadian University Software Engineering Conference – the annual Montreal-based conference by and for Canadian university students interested in topics on software development and engineering. For a conference that’s aimed at students, it punches above its weight class, having hosted some big name speakers including:

• Richard M. Stallman (last year, in a presentation where I won the auction for a Free Software Foundation plush gnu and offered to pay for it with my Microsoft corporate credit card)
• David Parnas
• Kathy Sierra
• Ralph Johnson
• Kent Beck
• Alistair Cockburn
• Dave Thomas
• Tim Bray
• Zed Shaw
• Jeff Atwood

This year’s speaker list is pretty good. Among them are:

• Douglas Crockford, Senior JavaScript Architect at Yahoo!.  If you truly want to understand JavaScript, listen to this guy! When people were dismissing JavaScript as a toy language – a strange concept in these Ajax-powered days, but this really was the case – he wrote articles like JavaScript: The Wrrrld’s Most Misunderstood Programming Language and other must-read pieces, all of which live at javascript.crockford.com. He’s also the author of the book JavaScript: The Good Parts, which is required reading for web developers. I had the pleasure of meeting him and seeing him speak at the Ajax Experience conference in Boston in 2006, and he’s both a great presenter and guy to hang out with at apres-conference events.
• Greg Wilson, Assistant Professor at U of T. Greg is many things: much-sought-after provider to academic advice and support at U of T, co-editor of Beautiful Code, DemoCamp Toronto steward, and now, the guy behind the best presentation at the Stack Overflow DevDays Toronto: Bits of Evidence: What We Actually Know About Software and Why We Believe It’s True. It was the presentation so nice, he’s doing it twice – this time at CUSEC. Don’t miss this one!
• Reg Braithwaite, Superprogrammer-at-large. Whether you know him as “Reg” or “raganwald”, you know that he’s got some seriously big-ass ideas about programming. Very few people push Ruby metaprogramming to its limits the way he does. Every time I see one of his presentations, I come out a little bit smarter.
• Pete Forde, Unspace. Pete’s one of the “corporate speakers”, a designation that probably makes him feel very uncomfortable. He’s one of the guys behind the Toronto-based development shop Unspace and behind two of the best conferences I’ve ever attended, RubyFringe (2008) and FutureRuby (2009). It’s anyone’s guess as to what he’ll talk about, but it should be good, and we can only hope that he begins it with a dance number, like he did with his presentation at the Mesh 2009 conference.
• Leigh Honeywell, Symantec. Leigh has forgotten more about security than I will ever learn, and she’s also one of the founders of HacklabTO, the Toronto “hackerspace”.

I had the opportunity to speak at last year’s CUSEC and had a wonderful time both speaking and hanging out with the students. I love the conference vibe – the energy, brainpower and passion of the attendees is palpable, and it makes me optimistic for the future of tech in Canada. I’m only too glad to be able to attend this year, and I’m honoured to be invited to host their DemoCamp event, which will take place Thursday evening.

I’ll be filing reports from CUSEC, so watch this space!

This article also appears in Canadian Developer Connection.

Ottawa-based developer Colin Melia has been a big help to me with TechDays. He presented at TechDays Ottawa, helped organize Demo Night in Canada, and posted a simple Windows Azure deployment exercise that I’ve found quite helpful and useful.

I’d like to return the favour by promoting the three sessions – that’s right, three – that he submitted to MIX10 in their open call for content. They are:

1. Everything You Touch Turns to Azure
   Feel the rush of power as you learn how to wave your hands and connect directly to your throne in the heavens – OK well you may have to settle for learning about Windows Touch in WPF/Silverlight and the Windows Azure Platform.  This is the future – make sure that everything you touch can turn to Azure.

   The session shows how the building blocks of Windows Touch, WPF/Silverlight applications and the Windows Azure Platform can be brought together to create a small yet engaging end-to-end experience.  Attendees should gain insight into the benefits and design of Touch-aware applications on Windows 7 as well as the benefits of backing user experiences with the Windows Azure Platform.

2. Get a WIF of This
   Writing services that understand multiple authentication systems is cumbersome and completely yesterday. Claims-based authentication and authorisation is the way to go. We’ll take a dive into how claims work and what Windows Identity Foundation provides by exploring the key components, but more importantly by building our own identify provider, a claims-based service and a Silverlight application that makes use of it.

   WIF recently RTM’d but the identify framework it cements is one of the most overlooked components when it comes to Internet-based application design.  Attendees should leave with a sense of how to create WIF components or WIF-aware components, as well as knowledge of the necessary design considerations.

3. The Cloud and the Silver Lining
   You need a place to host your Silverlight applications as well as the WCF RIA Services and database that back them.  This session shows you not only that the Windows Azure Platform (featuring Windows Azure, SQL Azure and other services), is a great place to put them, but also how to create the connections between the pieces.

   This session digs into the mechanics of a real-world application using Silverlight and the Windows Azure Platform.  Attendees should leave knowing how to easily test against and deploy to the Azure Platform, as well as how communication takes place between the component layers. 

I’d like to see Colin speak at MIX10. He’s a good speaker, he’s chosen some interesting and relevant topics, and he’d be a Canadian presence at MIX. If you agree with me, please vote for his sessions on the MIX10 Open Call for Entries site by Friday, January 15th! (If you want to see a list of all the proposed sessions, they’re here.)

This article also appears in Canadian Developer Connection.

Two days remain for you to cast your vote for sessions at the MIX10 conference, which I wrote about in the previous article. A number of people who submitted proposals for sessions are wooing voters, and one of the best promotions is that of Jason Alderman, who put together this comic explaining why you should vote for his session, titled Guerilla User Research – Carrying Out Missions Behind Enemy Lines to Get the Insight You Need:

This lovely hand-drawn comic is a reminder for me to fire up the scanner I bought for Christmas and get back to something for which I was notorious during my days at Crazy Go Nuts University: cartooning.

This article also appears in Canadian Developer Connection.