HOUSE_OVERSIGHT_016375.jpg

some planning capacities. AlphaZero has another interesting feature: It works by playing
hundreds of millions of games against itself. As it does so, it prunes mistakes that led to
losses, and it repeats and elaborates on strategies that led to wins. Such systems, and
others involving techniques called generative adversarial networks, generate data as well
as observing data.

When you have the computational power to apply those techniques to very large
data sets or millions of email messages, Instagram images, or voice recordings, you can
solve problems that seemed very difficult before. That’s the source of much of the
excitement in computer science. But it’s worth remembering that those problems—like
recognizing that an image is a cat or a spoken word is “Siri”—are trivial for a human
toddler. One of the most interesting discoveries of computer science is that problems that
are easy for us (like identifying cats) are hard for computers—much harder than playing
chess or Go. Computers need millions of examples to categorize objects that we can
categorize with just a few. These bottom-up systems can generalize to new examples;
they can label a new image as a “cat” fairly accurately, over all. But they do so in ways
quite different from how humans generalize. Some images almost identical to a cat
image won’t be identified by us as cats at all. Others that look like a random blur will be.

Top-down Bayesian Models
The top-down approach played a big role in early AI, and in the 2000s it, too,
experienced a revival, in the form of probabilistic, or Bayesian, generative models.
The early attempts to use this approach faced two kinds of problems. First, most
patterns of evidence might in principle be explained by many different hypotheses: It’s
possible that my journal email message is genuine, it just doesn’t seem likely. Second,
where do the concepts that the generative models use come from in the first place? Plato
and Chomsky said you were born with them. But how can we explain how we learn the
latest concepts of science? Or how even young children understand about dinosaurs and
rocket ships?
Bayesian models combine generative models and hypothesis testing with
probability theory, and they address these two problems. A Bayesian model lets you
calculate just how likely it is that a particular hypothesis is true, given the data. And by
making small but systematic tweaks to the models we already have, and testing them
against the data, we can sometimes make new concepts and models from old ones. But
these advantages are offset by other problems. The Bayesian techniques can help you
choose which of two hypotheses is more likely, but there are almost always an enormous
number of possible hypotheses, and no system can efficiently consider them all. How do
you decide which hypotheses are worth testing in the first place?
Brenden Lake at NYU and colleagues have used these kinds of top-down methods
to solve another problem that’s easy for people but extremely difficult for computers:
recognizing unfamiliar handwritten characters. Look at a character on a Japanese scroll.
Even if you’ve never seen it before, you can probably tell if it’s similar to or different
from a character on another Japanese scroll. You can probably draw it and even design a
fake Japanese character based on the one you see—one that will look quite different from
a Korean or Russian character. 37

37 Brenden M. Lake, Ruslan Salakhutdinov & Joshua B. Tenenbaum, “Human-level concept learning
through probabilistic program induction,” Science, 350:6266, pp. 1332-38 (2015).

155

HOUSE_OVERSIGHT_016375