HOUSE_OVERSIGHT_016918.jpg

SCALING
Neil Gershenfeld

Neil Gershenfeld is a physicist and director of MIT’s Center for Bits and Atoms. He is the author of FAB, co-author (with Alan Gershenfeld & Joel Cutcher-Gershenfeld) of Designing Reality, and founder of the global fab lab network.

Discussions about artificial intelligence have been oddly ahistorical. They could better be described as manic-depressive; depending on how you count, we’re now in the fifth boom-bust cycle. Those swings mask the continuity in the underlying progress and the implications for where it’s headed.

The cycles have come in roughly decade-long waves. First there were mainframes, which by their very existence were going to automate away work. That ran into the reality that it was hard to write programs to do tasks that were simple for people to do. Then came expert systems, which were going to codify and then replace the knowledge of experts. These ran into difficulty in assembling that knowledge and reasoning about cases not already covered. Perceptrons sought to get around these problems by modeling how the brain learns, but they were unable to do much of anything. Multilayer perceptrons could handle test problems that had tripped up those simpler networks, but their demonstrations did poorly on unstructured, real-world problems. We’re now in the deep-learning era, which is delivering on many of the early AI promises but in a way that’s considered hard to understand, with consequences ranging from intellectual to existential threats.

Each of these stages was heralded as a revolutionary advance over the limitations of its predecessors, yet all effectively do the same thing: They make inferences from observations. How these approaches relate can be understood by how they scale—that is, how their performance depends on the difficulty of the problem they’re addressing. Both a light switch and a self-driving car must determine their operator’s intentions, but the former has just two options to choose from, whereas the latter has many more. The AI-boom phases have started with promising examples in limited domains; the bust phases came with the failure of those demonstrations to handle the complexity of less-structured, practical problems.

Less apparent is the steady progress we’ve made in mastering scaling. This progress rests on the technological distinction between linear and exponential functions—a distinction that was becoming evident at the dawn of AI but with implications for AI that weren’t appreciated until many years later.

In one of the founding documents of the study of intelligent machines, The Human Use of Human Beings, Norbert Wiener does a remarkable job of identifying many of the most significant trends to arise since he wrote it, along with noting the people responsible for them and then consistently failing to recognize why these people’s work proved to be so important. Wiener is credited with creating the field of cybernetics; I’ve never understood what that is, but what’s missing from the book is at the heart of how AI has progressed. This history matters because of the echoes of it that persist to this day.

Claude Shannon makes a cameo appearance in the book, in the context of his thoughts about the prospects for a chess-playing computer. Shannon was doing

115

HOUSE_OVERSIGHT_016918