CUSEC 2010 Keynote: Matt Knox – “On Weakness”

CUSEC 2010 "goto 10" logoThis is the first of a series of notes that I took while attending CUSEC, the Canadian University Software Engineering Conference, which took place last week in Montreal. CUSEC is the biggest conference held by and for university students interested in software development. True to the Canadian techies punching well above their weight class (a great tradition started by Alexander Graham Bell), CUSEC manages to pull in big-name and up-and-coming speakers who’ve given talks that have outshined those I’ve seen an thousand-dollar-plus conferences.

The first keynote was given by Matt Knox, who has probably distributed more Scheme runtimes than anyone else in the world (and this is a larger number than you might think), which he did in the name of putting adware on millions of machines. He’s since come to his senses and seems quite contrite.

His presentation, On Weakness, is about his life on the Dark Side and the lessons he gleaned from it. It’s based on his talk, Crimes Against Humanity, Writ Small, which he gave at FutureRuby last year, but it was good to see it again, and its message is probably even more valuable to students. My notes (which I polished for comprehensibility) and photos from his session appear below:

Matt Knox, standing at the lectern, delivering his keynote at CUSEC

An Evil Job

  • How many of you are:
    • Technical, as opposed to business or arts students?
    • Engineering students?
    • Programmers?
    • Evil?
  • That’s what this talk is about
  • One way to describe one of my former jobs is doing “Windows hijinks with Scheme”
  • During my time with that job, I released many scheme runtimes
  • Aaron Swartz – I think it was at a Y Combinator startup camp – said this of me: "He uses Scheme for evil!"
  • It was more than just Scheme – I was writing stuff that had alternately “hard” (statically-typed languages) and “soft” (dynamically-typed languages) layers
  • I was in the adware business, which is like walking into a big monkey knife fight…
  • …except I was using a death ray! (Scheme == death ray, C == knife)
  • I started with good intentions, in the business of building spam filters
  • Business wasn’t so hit, and I ran out of money
  • My job search failed, but luckily, a job went looking for me
  • I was so pleased with being found that I  forgot to talk salary
  • I showed up for the interview and at the end, was invited to work for them
  • I did terribly when it came time to discuss what I would be paid
    • I didn’t research the New York City job market and cost of living
    • I asked for $40K
    • When I saw the look of shock of the guy’s face, I thought that I had asked for too much
    • Start reducing what I asked for; luckily he stopped me
  • We want you to come in an analyze our distribution chain, they said
  • It turned out to be an adware company:
    • Bought people’s “digital tchochkes” or mini-apps, such as screensavers
    • They had realized that there’s no lower bound for how cheesy something can be and still be a big seller on the internet
    • They took these mini-apps and gave them away online for free, bundled with software that gives you "special offers" from time to time
  • Some of these bundled apps turned out to be worms
    • So the company had me write software to remove any worms from a system and added them to the bundle
    • So now we were bundling my anti-malware along with their adware
    • I felt like "an assassin working for the mob, but killing terrorists". The mob were bad, but the terrorists were worse
    • "Awesome! I can probably keep up with Norton…it’ll be great!"
    • And for a while, the best way to eradicate worms your system was to install their adware with my anti-malware bundled with it
  • Low-level coding is dangerously seductive
    • In the beginning, it’s "like getting kicked in the face over and over again by buffer overruns"
    • But then it becomes fascinating
  • I wanted to do it in Scheme, but that would require embedding a Scheme interpreter
    • Such an interpreter would have to fit into a single TCP/IP packet (about 64K)
    • Scheme is great. For any superlative — “best performance”, “smallest app”, and so on – there are usually two contenders: some other language, and Scheme.
    • I managed to squeeze a Scheme interpreter down to 19K
  • My success with killing the worms led to a new request: In addition to your all this malware on other machines, why not eliminate all the competitor’s adware?
    • Now I felt like “an assassin for the mob, killing other mobsters”. Not as noble.
  • Then the next request came: How about keeping our software from being killed…by anything? (including Norton)
    • The only way to uninstall the adware was to use the uninstaller, which came with it
    • I initially viewed this as "a really interesting technical problem"
  • All this was made possible by a couple of Windows quirks…
    • CreateRemoteThread
    • Scheduler
      • You can have a process tell the scheduler that it needs to do a do-over — "I’m not done yet, I need more time", and the scheduler will grant that time
      • You can tell even Windows that a process is so important that if it fails, it needs to protect the user by presenting a blue screen
  • Windows is interesting from a purely archaeological perspective
    • Consider that all strings in Windows are 16-bit unicode, which means that nulls can be embedded in strings
    • But C strings, which is what’s used in the underlying DOS, are null-terminated and therefore can’t contain nulls
    • Interesting effects when moving null-containing strings between these layers

What Drives People to Take Up Evil Jobs?

Matt Knox, standing at the lectern, delivering his keynote at CUSEC

  • Aftermath of my working at the adware company:
    • Company got sued for $190 billion (by Elliot Spitzer!)
    • I was the first employee at the company — everyone else was a contractor
  • I left the company with these questions:
    • "Whut happen?"
    • "Is this who I am?"
  • Some jobs pay lots of money, but it’s hard to transition out of them
  • Will I be stuck in adware for the rest of my life?
  • There are some historical precedents:
    • Albert Speer
      • A promising architect who liked soaring buildings
      • He hooked up with rising politicians with the same aesthetic sense, one of whom was Hitler
      • He started with creating buildings, but then became the Nazis’ chief logistics guy
      • Later, a leader of the U.S. Air Force said that had he been aware of Speer’s involvement as the Nazi’s chief logistics guy, he would’ve dedicated an entire wing of the Air Force exclusively to killing him
      • It’s been suggested that Speer prolonged the war by a year or two by running the German forces more efficiently
    • Manhattan Project staff
  • But I didn’t want anecdotes…I wanted science!
    • There’s a scientific study of otherwise good people doing evil things: the Milgram Experiment
      • How many people would go all the way?
      • 1% of the population is psychotic – it was hypothesized that the number of people who’d go all the way would be similar
      • Instead, 70% did
      • Results replicatable with people from all walks of life
      • Women, it turned out, “went evil” in a slightly greater proportion than the men
      • "Most human evil lives here"
  • Read The Black Book of Communism
  • For a more mundane example of blind obedience to authority leading to evil, see "The strip search McDonald’s prank call"
    • In the prank, the prankster calls a McDonald’s, gets an employee on the line and says “I’m a police officer. We have reason to believe that there is a thief in your restaurant and we need you to take them into the back and hold them until we arrive.”
    • They provide a description vague enough so that someone in the restaurant will match it
    • Once coralled in the back, the prankster starts giving orders to torture and/or humiliate the customer, and many employees have complied
  • So what does this mean?
    • The human brain has a remote root exploit in 70% of the installed base
  • "With or without religion, you would have good people doing good things and evil people doing evil things. But for good people to do evil things, that takes religion." — Steven Weinberg
    • Nope. Just authority.
  • There is hope: people who were subjects of the Milgram experiments turned out to be better at resisting authoritative coercion

The Power of Communication

Matt Knox, standing at the lectern, delivering his keynote at CUSEC

  • Math: "There are only three reasonable numbers: 0, 1 and infinity"
  • When Robert Andrews Millikan did his oil drop experiments to determine the charge on an electron, he initially got the value wrong by 30 – 40%
    • People who repeated the experiment or conducted similar experiments with results close to Millikan’s erroneous number published their results
    • People who did so but got the correct value – which did not match Millikan’s value – didn;t publish, worried that they’d done something wrong, since their numbers didn’t agree with the number published by the authority on the subject
  • The world pre-blogs was so different from this world
    • Very first open source project: Oxford English Dictionary
      • Done via mail
    • Ever wondered where the term "flying off the handle" comes from?
      • It’s from sword-making – until they figured out the process of making swords as one-piece, with hand-friendly stuff wrapped around the base so you could hold them, swords often flew off their handles in battle
      • It took 900 years to evolve swords to one piece
  • Not everything has been solved, but it’s easier today
  • Rails is such a solution
    • It’s a series of incremental improvements
    • Can you out-Rails Rails?

This article also appears in Canadian Developer Connection.


Kiss Your Open WiFi Goodbye if the RIAA Gets Their Way

“When you pirate MP3s, you’re downloading communism” posterBack in July 2003, someone who read the Wired article titled Giving Sharers Ears Without Faces wrote to our pal (and former boss) Cory Doctorow over at Boing Boing:

One issue that I have not seen addressed in the RIAA vs. P2P front relates to the potential for an unsupecting home PC user who just happens to have an open WiFi router being used by a neighbor to share files to get sued by the RIAA when their IP address shows up on the RIAA’s list. From the surveys I’ve done, there are a lot of open WiFi routers a file swapper could easily use to both serve and download files. So, is the RIAA going to have to shut down open WiFi to get its way?

A year later, Boing Boing ran an article titled Open WiFi for plausible deniability, which cover’s Micah Joel’s running of “an open WiFi network in order to give himself plausible deniability for bad acts that can be traced to his IP address”:

I’ve already composed my reply in case I receive one of these letters someday. “Dear Comcast, I am so sorry. I had no idea that copyrighted works were being downloaded via my IP address; I have a wireless router at home and it’s possible that someone may have been using my connection at the time. I will do my best to secure this notoriously vulnerable technology, but I can make no guarantee that hackers will not exploit my network in the future.” If it ever comes down to a lawsuit, who can be certain that I was the offender? And can the victim of hacking be held responsible for the hacker’s crimes? If that were the case, we’d all be liable for the Blaster worm’s denial of service attacks against Microsoft last year.

Well, we’re now a few years and two generations of 802.11 down the road, and the RIAA has finally done it. Cory writes:

The RIAA is asking a judge to rule that anyone who provides bandwidth should be responsible for all the activities of his users. This would doom open WiFi — and all other public networking efforts. But who needs anonymous speech, anyway? After all anonymity fuels irresponsible behavior, like founding the United States.

The RIAA just wants to stand up for freedom. First they convinced Russia to force licensing and 24-hour inspection of presses, now they want to eliminate anonymous speech here at home.

Record companies are quick to cite the First Amendment when someone suggests banning music with “suggestive” lyrics, but they’re not so big on free presses and anonymous speech. It’s like they love free speech, but not enough to share it with the rest of us.

It’s all part of their “rabbit hunting with Howitzers” legal strategy. It stems from the case of Debbie Foster, who was being sued by Capitol Records, a part of the RIAA cartel, for allegedly sharing copyrighted material on a P2P network. It turned out that she wasn’t the culprit; it was someone else using her account. The case was dismissed last year with a filing that gets pretty damned close to calling out the RIAA as extortionists — or at least as close as you can get outside of a TV or movie courtroom drama. Foster didn’t stop there; she filed a motion asking the court to make the RIAA compensate her for her legal fees and got that compensation in the form of a $50,000 award earlier this month.

This award creates a legal headache for the RIAA. As Listening Post puts it: “If the ruling stands, the RIAA will have to be much more careful about who it sues going forward, adjusting its scatter-shot approach to filing such lawsuits in order to avoid suing the wrong people”.

Hence the RIAA’s latest move: filing a motion for reconsideration that forces them to pay Foster’s legal fees, a key point of which is that they’d like a ruling that the owner of an ISP account is responsible for all activity on that account.

James “Smalltalk Tidbits, Industry Rants” Robertson makes a couple of interesting observations:

  • He points to an Ars Technica story that says that the RIAA, in their motion, “lay out their disagreement with the judge’s reasoning while taking time to point out that the fees awarded far exceed any damages they could have recovered should their suit have been successful”, to which he quips “What, you mean there are risks in this strategy?”
  • He points out that it’s not just the individual running an open node at home or the small cafe running an open node to get customers who are in trouble:

    …any entity that offered a net connection – Starbucks, a hotel, a municipality (etc) – would have a huge potential liability on their hands. They might well decide to just discontinue in order to not expose themselves. Yeah, there’s a world I want to live in.