If you’ve tried to go past the APIs like the ones OpenAI offers and learn how they work “under the hood” by trying to build your own neural network, you might find yourself hitting a wall when the material opens with equations like this:
How can you learn how neural networks — or more accurately, artificial neural networks — do what they do without a degree in math, computer science, or engineering?
There are a couple of ways:
Follow this blog. Over the next few months, I’ll cover this topic, complete with getting you up to speed on the required math. Of course, if you’re feeling impatient…
Read Tariq Rashid’s book, Make Your Own Neural Network. Written for people who aren’t math, computer science, or engineering experts, it first shows you the principles behind neural networks and then leaps from the theoretical to the practical by taking those principles and turning them into working Python code.
Along the way, both I (in this blog) and Tariq (in his book) will trick you into learning a little science, a little math, and a little Python programming. In the end, you’ll understand the diagram above!
One more thing: if you prefer your learning via video…
My poster from May, titled Every 13 years, an innovation changes computing forever, theorizes that roughly every thirteen years, a new technology appears, and it changes the way we use computers in unexpectedly large ways.
The first entry in my list was an exception because it didn’t feature just one technology, but a number of them. It was “The Mother of All Demos,” a demonstration of technologies that are part of our everyday life now, but must have seemed like pure science fiction at the time, December 9, 1968 — 55 years ago today.
If your curiosity about artificial intelligence goes beyond bookmarking those incessant “10 ChatGPT prompts you need to know” posts that are all over LinkedIn, you should set aside some time to read Douglas’ Hofstadter’sGödel, Escher, Bach: An Eternal Golden Braid and watch his new interview.
Gödel, Escher, Bach
I might never have read it, if not for Dr. David Alex Lamb’s software engineering course at Queen’s University, whose curriculum included reading a book from a predetermined list and writing a report on it. I’ll admit that I first rolled my eyes at having to write a book report, but then noticed that one of the books had both “Escher” and “Bach” in the title. I had no idea who “Gödel” was, but I figured they were in good company, so I signed up to write the report on the book I would later come to know as “GEB.”
I’ll write more about why I think the book is important later. In the meantime, you should just know that it:
Helped me get a better understanding of a lot of underlying principles of mathematics and its not-too-distant relative, computer science, especially the concepts of loops and recursion
Advanced my thinking about how art, science, math, and music are intertwined, and inspired one of my favorite sayings: “Music is math you can feel”
Gave me my favorite explanations of regular expressions and the halting problem
Taught me that even the deepest, densest subject matter can be explained with whimsy
Provided me with my first serious introduction to ideas in cognitive science and artificial intelligence
Yes, this is one of those books that many people buy, read a chapter or two, and then put on their bookshelf, never to touch it again. Do not make that mistake. This book will reward your patience and perseverance by either exposing you to some great ideas, or validate some concepts that you may have already internalized.
At the very least, if you want to understand “classical” AI — that is AI based on symbol manipulation instead of the connectionist, “algebra, calculus, and stats in a trench coat” model of modern AI — you should Gödel, Escher, Bach.
A new Hofstadter interview!
Posted a mere three days ago at the time of writing, the video above is a conversation between Douglas Hofstadter and Amy Jo Kim. It’s worth watching, not only for Hofstadter’s stories about how GEB came to be, but also for his take on current-era large language models and other generative AI as well as the fact that he’s being interviewed by game designer Amy Jo Kim. Among other things, Kim was a systems designer on the team that made the game Rock Band and worked on the in-game social systems for The Sims.
On the “pro” side — that is, the people arguing that AI research and development IS an existential threat:
Yosuha Bengio: Professor at the Department of Computer Science and Operations Research at the Université de Montréal and scientific director of the Montreal Institute for Learning Algorithms. Specializes in neural networks and deep learning. He won the Turing Award with Yann LeCun and Geoffrey Hinton for their work on machine learning.
And on the “con” side — the people who are arguing that AI research and development IS NOT an existential threat:
Melanie Mitchell: Professor at the Santa Fe Institute, who’s worked in the areas of analogical reasoning, complex systems, genetic algorithms and cellular automata. She’s the author of the book AI: A Guide for Thinking Humans, published in 2019.
Yann LeCun: Meta’s chief AI scientist and professor at New York University, best known for his work on computer vision, optical character recognition, and convolutional neural networks. He won the Turing Award with Yoshua Bengio and Geoffrey Hinton for their work on machine learning.
They asked the audience to vote for a side at the start and conclusion of the debate, and while a clear majority were on the “pro” side (that is, they believed AI poses an existential threat), the “con” side won by gaining 4% of the vote at the end:
It’s hard to tell whether the Munk Debates really want you to pay to watch the video, as they have it locked down on this page and freely available on this one, so I’m linking to this YouTube posting for as long as it remains online. Enjoy!
I’ve made three appearances on Fox 13 News Tampa this year so far. If they call on me to answer more questions or explain some aspect of artificial intelligence, I’ll gladly do so!
My most recent appearance was on June 14, whose topic was all the noise about AI possibly being an existential threat to humanity. This is the one where I reminded the audience that The Terminator was NOT a documentary:
What might the next decade of software development look like? Richard Campbell has some ideas and shares them in this talk from the 2023 edition of the NDC London conference.
Here’s the video:
I know Richard from my former life at Microsoft. He’s the host of the .NET Rocks and RunAs Radio podcasts, and long-time developer, consultant, and tech company founder, and a damn good storyteller.
The first story he tells is about “The Animal Highway,” the space between his and his neighbors’ house, which is frequented by bears. This actually made me laugh out loud, since when I last saw Richard at a backyard barbecue at his house, we had to scare away a bear cub by being noisy. He picked up a pot and barbecue tongs, I picked up my accordion, and with whoops, hollers, and random squeezebox chords, we chased it away into the woods.
One of the themes that runs through his talk is that technology has grown in leaps and bounds. Near the start of the talk, he uses the example of the Cray X-MP. In 1985, it was the world’s most powerful computer. It sold for millions of dollars and required 200kW of power, which could perform 1.9 at gigaflops (billions of floating-point operations per second). It was used to model nuclear explosions and compute spaceflight trajectories.
The iPad 2 from 2011 also performs at 1.9 gigaflops, but it sold for hundreds of dollars instead of millions, and ran on battery power instead of requiring its own power plant. As Richard summed it up: “26 years later, the most powerful computer in the world is now a device we give to children. And they play Candy Crush on it.”
English: The first transistor ever made, built by John Bardeen, William Shockley and Walter H. Brattain of Bell Labs in 1947. Original exhibited in Bell Laboratories. Creative Commons photo by Unitronic. Tap to see the source.
Near the end of the talk, Richard uses another example of the technological changes that have happened in a lifetime. The picture above shows the first transistor ever, which was made in Bell Labs in 1947.
“It’s pretty hard to look at that,” he said, pointing to the photo of that transistor, “and think ‘M1 chip’.”
M1 chip diagram.
In case you were wondering, here’s how many transistors the different variations of the M1 chip have:
Chip version
Number of transistors
M1 (original version)
16 billion
M1 Pro
33.7 billion
M1 Max
57 billion
M1 Ultra
114 billion
If you want an understanding of how we got to the current state of computing and some good ideas of where it might go, Richard’s talk is not only enlightening, but also entertaining. I listened to it on this morning’s bike ride, and you might find it good listening during your workout, chores, commute or downtime.
Here it is — the recording of my interview on the 4:00 p.m. news on FOX 13 Tampa with anchor Chris Cato, where I answered more questions about artificial intelligence:
In this quick interview, we discussed:
The “existential threat to humanity” that AI potentially poses: My take is that a lot of big-name AI people who fear that sort of thing are eccentrics who hold what AI ethicist Timnit Gebru calls the TESCREAL (Transhumanism, Extropianism, Singularitarianism, Cosmism, Rationalism, Effective Altruism, and Longtermism) mindset. They’re ignoring a lot of closer-to-home, closer-to-now issues raised by AI because they’re too busy investing in having their heads frozen for future revival and other weird ideas of the sort that people with too much money and living in their own bubble tend to have.
My favorite sound bite: “The Terminator is not a documentary.”
A.I. regulation: Any new technology that has great power for good and bad should actually be regulated, just as we do with nuclear power, pharma, cars and airplanes, and just about anything like that. A.I. is the next really big thing to change our lives — yes, it should be regulated.” There’s more to my take, but there’s only so much you can squeeze into a two-and-a-half minute segment.
Cool things AI is doing right now: I named these…
Shel Israel (who now lives in Tampa Bay) is using AI to help him with his writing as he works on his new book,
I’m using it with my writing for both humans (articles for Global Nerdy as well as the blog that pays the bills, the Auth0 Developer Blog) as well as for machines (writing code with the assistance of Studio Bot for Android Studio and Github Copilot for iOS and Python development)
Preventing unauthorized access to systems with machine learning-powered adaptive MFA, which a feature offered by Okta, where I work.
My “every 13 years” thesis: We did a quick run-through of something I wrote about a month ago — that since “The Mother of All Demos” in 1969, there’s been a paradigm-changing tech leap every 13 years, and the generative AI boom is the latest one:
Tap to view at full size.
And finally, a plug for Global Nerdy! This blog has been mentioned before in my former life in Canada, but this is the first time it’s been mentioned on American television.
I’ll close with a couple of photos that I took while there:
In the green room, waiting to go on. Tap to view at full size.
The view from the interview table, looking toward the anchor desk. Tap to view at full size.
The cameras, teleprompters, and monitors. Tap to view at full size.
Once again, I’d like to thank producer Melissa Behling, anchor Chris Cato, and the entire Fox 13 Tampa Bay studio team! It’s always a pleasure to work with them and be on their show.
