HOUSE_OVERSIGHT_016881.jpg

Indeed, Wiener gave scientific teeth to the idea that in the workings of history,
politics, and society, ideas matter. Beliefs, ideologies, norms, laws, and customs, by
regulating the behavior of the humans who share them, can shape a society and power the
course of historical events as surely as the phenomena of physics affect the structure and
evolution of the solar system. To say that ideas—and not just weather, resources,
geography, or weaponry—can shape history is not woolly mysticism. It is a statement of
the causal powers of information instantiated in human brains and exchanged in networks
of communication and feedback. Deterministic theories of history, whether they identify
the causal engine as technological, climatological, or geographic, are belied by the causal
power of ideas. The effects of these ideas can include unpredictable lurches and
oscillations that arise from positive feedback or from miscalibrated negative feedback.

An analysis of society in terms of its propagation of ideas also gave Wiener a
guideline for social criticism. A healthy society—one that gives its members the means
to pursue life in defiance of entropy—allows information sensed and contributed by its
members to feed back and affect how the society is governed. A dysfunctional society
invokes dogma and authority to impose control from the top down. Wiener thus
described himself as “a participant in a liberal outlook,” and devoted most of the moral
and rhetorical energy in the book (both the 1950 and 1954 editions) to denouncing
communism, fascism, McCarthyism, militarism, and authoritarian religion (particularly
Catholicism and Islam) and to warning that political and scientific institutions were
becoming too hierarchical and insular.

Wiener’s book is also, here and there, an early exemplar of an increasingly
popular genre, tech prophecy. Prophecy not in the sense of mere prognostications but in
the Old Testament sense of dark warnings of catastrophic payback for the decadence of
one’s contemporaries. Wiener warned against the accelerating nuclear arms race, against
technological change that was imposed without regard to human welfare (“[W]e must
know as scientists what man’s nature is and what his built-in purposes are”), and against
what today is called the value-alignment problem: that “the machine like the djinnee,
which can learn and can make decisions on the basis of its learning, will in no way be
obliged to make such decisions as we should have made, or will be acceptable to us.” In
the darker, 1950 edition, he warned of a “threatening new Fascism dependent on the
machine à gouverner.”

Wiener’s tech prophecy harks back to the Romantic movement’s rebellion against
the “dark Satanic mills” of the Industrial Revolution, and perhaps even earlier, to the
archetypes of Prometheus, Pandora, and Faust. And today it has gone into high gear.
Jeremiahs, many of them (like Wiener) from the worlds of science and technology, have
sounded alarms about nanotechnology, genetic engineering, Big Data, and particularly
artificial intelligence. Several contributors to this volume characterize Wiener’s book as
a prescient example of tech prophecy and amplify his dire worries.

Yet the two moral themes of The Human Use of Human Beings—the liberal
defense of an open society and the dystopian dread of runaway technology—are in
tension. A society with channels of feedback that maximize human flourishing will have
mechanisms in place, and can adapt them to changing circumstances, in a way that can
domesticate technology to human purposes. There’s nothing idealistic or mystical about
this; as Wiener emphasized, ideas, norms, and institutions are themselves a form of
technology, consisting of patterns of information distributed across brains. The

78

HOUSE_OVERSIGHT_016881