HOUSE_OVERSIGHT_016848.jpg

But taking into account that range of probability is just where the new AI excels.
The only chink in the armor of AI is that word "vast"; human possibilities, thanks to
language and the culture that it spawns, are truly Vast.¹⁰ No matter how many patterns
we may find with AI in the flood of data that has so far found its way onto the Internet,
there are Vastly more possibilities that have never been recorded there. Only a fraction
(but not a Vanishing fraction) of the world’s accumulated wisdom and design and
repartee and silliness has made it onto the Internet, but probably a better tactic for the
judge to adopt when confronting a candidate in the Turing Test is not to search for such
items but to create them anew. AI in its current manifestations is parasitic on human
intelligence. It quite indiscriminately gorges on whatever has been produced by human
creators and extracts the patterns to be found there—including some of our most
pernicious habits.¹¹ These machines do not (yet) have the goals or strategies or capacities
for self-criticism and innovation to permit them to transcend their databases by
reflectively thinking about their own thinking and their own goals. They are, as Wiener
says, helpless, not in the sense of being shackled agents or disabled agents but in the
sense of not being agents at all—not having the capacity to be "moved by reasons" (as
Kant put it) presented to them. It is important that we keep it that way, which will take
some doing.

One of the flaws in Weizenbaum’s book Computer Power and Human Reason,
something I tried in vain to convince him of in many hours of discussion, is that he could
never decide which of two theses he wanted to defend: AI is impossible! or AI is possible
but evil! He wanted to argue, with John Searle and Roger Penrose, that "Strong AI" is
impossible, but there are no good arguments for that conclusion. After all, everything we
now know suggests that, as I have put it, we are robots made of robots made of robots. . .
down to the motor proteins and their ilk, with no magical ingredients thrown in along the
way. Weizenbaum’s more important and defensible message was that we should not
strive to create Strong AI and should be extremely cautious about the AI systems that we
can create and have already created. As one might expect, the defensible thesis is a
hybrid: AI (Strong AI) is possible in principle but not desirable. The AI that’s practically
possible is not necessarily evil—unless it is mistaken for Strong AI!

The gap between today’s systems and the science-fictional systems dominating
the popular imagination is still huge, though many folks, both lay and expert, manage to
underestimate it. Let’s consider IBM’s Watson, which can stand as a worthy landmark
for our imaginations for the time being. It is the result of a very large-scale R&D process
extending over many person-centuries of intelligent design, and as George Church notes
in these pages, it uses thousands of times more energy than a human brain (a
technological limitation that, as he also notes, may be temporary). Its victory in
Jeopardy! was a genuine triumph, made possible by the formulaic restrictions of the
Jeopardy! rules, but in order for it to compete, even these rules had to be revised (one of

¹⁰ In Darwin’s Dangerous Idea, 1995, p. 109, I coined the capitalized version, Vast, meaning Very much
more than ASTronomical, and its complement, Vanishing, to replace the usual exaggerations infinite and
infinitesimal for discussions of those possibilities that are not officially infinite but nevertheless infinite for
all practical purposes.
¹¹ Aylin Caliskan-Islam, Joanna J. Bryson & Arvind Narayanan, "Semantics derived automatically from
language corpora contain human-like biases," Science, 14 April 2017, 356: 6334, pp. 183-6. DOI:
10.1126/science.aal4230.

45

HOUSE_OVERSIGHT_016848