Presentations Security What I’m Up To

Video of my Bsides Tampa 2024 presentation, “xz made EZ”

Here it is — the video of my presentation, xz made EZ, which covers the security incident with the xz utils utility on Unix-y systems, which I gave at BSides Tampa 2024 on April 6th:

If you’d like them, here are the Google slides from the presentation.

Questions and answers

How did I land this presentation?

The details of the xz vulnerability were made public mere days before the BSides Tampa 2024 cybersecurity conference, and on a whim, I emailed the organizers and asked if I could do a lightning talk on the topic.

They quickly got back to me and let me know that they’d had a last-minute speaker cancellation and gave me a full slot in which to do my presentation.

The moral of the story? It never hurts to ask, and it can lead to opportunities!

What’s this xz thing, anyway?

Let me answer with this slide from my presentation:

xz is short for xz Utils, a compression utility that you’ll find in Unix-y operating systems, including:

  • Linux distributions
  • macOS

It’s usually used by Unix greybeards who generally use it in combination with tar.

What happened with xz?

xz was one of those open source projects that had a vulnerability best illustrated by this xkcd comic:

xz was like that project pointed out in the comic, except that the “random person” doing the maintaining was Lass Collin, a developer based in Finland, who was experiencing burnout. As a result, xz was languishing.

In what appeared to be a stroke of good fortune, a developer who went by the handle of “Jia Tan” on GitHub came to the rescue and started submitting patches to xz.

At about the same time, there were a number of complaints about xz’s lack of apparent maintenance. In hindsight, it looks like a clever two-pronged campaign:

  1. A group of people loudly clamoring for someone else to take the reins of the xz project, and
  2. A friendly developer who swoops in at the right time, making patches to the xz project…

…all while a burned-out Lasse Collin was facing a lot of stress.

On November 30, 2022, Lasse changed the email address for xz bug reports to an alias that redirected to both his email address as well as Jia Tan’s. At that point, Jia Tan, the apparently helpful developer who appeared at just the right time, was now an official co-maintainer.

Not long after, Lasse releases his last version of xz, and soon after Jia Tan, now the sole maintainer of the project, releases their own version.

With full control of the project, Jia Tan starts making changes — all the while, carefully disguising them — that create a “back door” within the xz application.

On any system that had Jia Tan’s tainted version of xz installed, an unauthorized user with the right private key could SSH into that system with root-level access. By becoming the maintainer of a trusted application used by many Linux versions, Jia Tan managed to create a vulnerability by what could have been one of the most devastating supply-chain attacks ever.