The Singularity Is Near: When Humans Transcend Biology
by Ray Kurzweil
672 pages $20 Penguin
You Are Not a Gadget: A Manifesto
by Jaron Lanier
224 pages $24.95 Knopf
Simulation and Its Discontents
by Sherry Turkle
208 pages $22.95 The MIT Press.
By Roger Berkowitz.
IN “THE ODE TO MAN” from Antigone, Sophocles conjures “Man” as the wondrous being who wears out the “imperishable earth” with his ploughs. This man “overpowers the rough-maned horses with his devices” and tames the “unbending mountain bull.” He flees the “stormy darts” of winter’s frost and he escapes “needful illness.” Such a man who tames nature is a wonder, according to the Ode’s opening line:
Manifold the wonders
And nothing towers more wondrous than man.
The Greek word for “wonder” is Deinon, which connotes both greatness and horror, that which is so extraordinary as to be at once terrifying and beautiful. This is how Sophocles understands man. As an inventor and maker of his world, man can remake and master the earth. This wonder terrifyingly carries the seeds of his destruction. Man, Sophocles imagines, threatens to so fully control his own way of life that he might no longer be man. As the chorus sings: “Always overcoming all ways, man loses his way and comes to nothing.” If man so tames the earth as to free himself from uncertainty, what then is left of human being?
A new urgency has energized those who welcome and those who fear the power of man to transform his nature. While hopes of technological utopias and fears of technological dystopias may be part and parcel of the human condition itself, we are living through a moment when extraordinary technological advances are once again raising the question of what it means to be human. The problem that confronts man in the 20th and now 21st centuries, as Hannah Arendt writes, is that we face the danger that we might so fully create and make our artificial world that we endanger that quality of human life which is subject to fate, nature, and chance. To bring oneself up to date on this current version of the debate over our human, superhuman, and inhuman futures, three recent books serve as excellent guides.
I.
AMONGST THE DREAMERS OF a new age of man, none combines credibility and popularity like Ray Kurzweil. In six best-selling books and two movies, Kurzweil has honed his argument that the rapid advance of computational technology is ushering in a new humanity, one that will merge human and machine into a higher and more intelligent form of life. Not only will man attain mastery over the world with intelligence unimaginable to mere human cognition, but also humans will be able to choose their lifespans—Kurzweil himself speaks of living till the ripe age of 700. And eventually, human and machine intelligence will merge with the intelligence of the universe, forming one omniscient cosmic whole of which all of us are parts. This eventual uniting of man, machine, and nature goes by the name the Singularity, and it is a faith with many adherents who flock to its website, attend its university, and hope to live its promise.
In The Singularity is Near, first released as a book and now showing as The Singularity is Near: The Movie, Kurzweil advances his fundamental thesis: the Singularity–the event during which the nature of human life and super-intelligent machines will merge into a new and more powerful species–is near. Really near. It will happen within the next 30 years.
The Singularity, as Kurzweil defines it, is “a future period during which the pace of technological change will be so rapid, its impact so deep, that human life will be irreversibly transformed.” Transformed could also mean superseded. The point is that humans will of necessity evolve into a hybrid species. As he writes: “This book, then, is the story of the destiny of the human-machine civilization, a destiny we have come to refer to as the Singularity.”
Underlying Kurzweil’s confidence in the coming nearness of the Singularity is his conviction that “we have the ability to understand our own intelligence—to access our own source code, if you will—and then revise and expand it.” While this may seem impossible today or in the near future, Kurzweil believes that the pace of technological advance is accelerating at a still underestimated pace. As Kurzweil writes:
The list of ways computers can now exceed human capabilities is rapidly growing…. For example, computers are diagnosing electrocardiograms and medical images, flying and landing airplanes, controlling the tactical decisions of automated weapons, making credit and financial decisions, and being given responsibility for many other tasks that used to require human intelligence.
Doubters can always point to limits on artificial intelligence. Computers make mistakes. Translation programs result in comical absurdities. Robots don’t understand slang or jargon. But Kurzweil thinks these are mere temporary glitches, soon to be overcome, on the way to a new super-human humanity.
The Singularity, Kurzweil believes, is cause for celebration. While he says, at times, that the Singularity is neither utopian nor dystopian, he leaves little doubt as to his optimism:
The Singularity will allow us to transcend these limitations of our biological bodies and brains. We will gain power over our fates. Our mortality will be in our own hands. We will be able to live as long as we want (a subtly different statement from saying we will live forever). We will fully understand human thinking and will vastly extend and expand its reach. By the end of this century, the nonbiological portion of our intelligence will be trillions of trillions of times more powerful than unaided human intelligence.
In short: as computers surpass human intelligence, they will become not merely human, but super-human. And humans too, will adapt, incorporating the power of computer intelligence into their bodies and brains, powering themselves to levels of intelligence and insight that are still hardly imaginable. Together, these intelligent machines and enhanced humans will converge in the Singularity, a point at which man and his machines will together grasp man’s destiny, to fully and intelligently create his world and himself in the most intelligent and beneficial way possible. Eventually, “the entire universe will become saturated with our intelligence. This is the destiny of the universe.” This full fusion of machine-man with the cosmos is “the sweetest music, the deepest art, the most beautiful mathematics.”
There is an ambivalence, if not hostility, towards humanity in Kurzweil’s writing that recalls the double meaning of “wonder” that Sophocles invokes over two millennia ago. The Singularity cannot be experienced by living human beings. Kurzweil awaits the time when our experiences will cease to be experienced by bodily human beings; instead, “As virtual reality from within the nervous system becomes competitive with real reality in terms of resolution and believability, our experiences will increasingly take place in virtual environments.” Once the entire universe—including all matter in the universe—is transformed into a computer-saturated intelligence, all matter will be subject to intelligent modification.
The ultimate end of such an evolution of human intelligence that “is more powerful than physics” is the Singularity. Once we perfect computation, our “intelligence saturates the matter and energy in its vicinity, and it begins to expand outward at at [sic] least the speed of light.” We will then, as a civilization, overcome gravity and other cosmological forces and “engineer the universe” we want. This is the goal of Singularity.
There is a real question about whether such a “civilization” is indeed a civilization of humans. Kurzweil insists it will be. The human-machine-natural beings of the coming Singularity will, he writes, retain the essential human character of intelligent striving: “what will remain unequivocally human in such a world [is] simply this quality: ours is the species that inherently seeks to extend its physical and mental reach beyond current limitations.” As we merge with machines and with the cosmos, our civilization will “be more exemplary of what we regard as human than it is today, although our understanding of the term will move beyond its biological origins.” It is difficult for us mere humans to know what we will be in the coming decades, but Kurzweil is hopeful that we will continue to expand our intelligence, our dreams, and our desire to improve the world.
II.
AMONGST THE MANY CRITICS of Singularity, Jaron Lanier stands out. A fixture in Silicon Valley for decades and one of the founders of virtual reality, he is hardly a Luddite. Lanier even has his own utopian version of how technology can help us to be more human—what he calls “postsymbolic communication.” And yet, Lanier’s recent book, You Are Not a Gadget, is a scream of protest, a manifesto against old friends who now form what he terms a subculture of “cybernetic totalitarians” and “digital Maoists.” While many proclaim the internet age as the dawning of a new freedom, Lanier argues that current trends on the internet prefer the crowd over the individual, value the abstraction of the network over real persons, and threaten to eradicate consciousness, spirituality, and humanity from the earth.
Lanier’s attack encompasses more than just the Singularity movement, but it is centered there. Most fundamentally, he sees the Singularity and similar movements as anti-human. “The Singularity,” he writes,
… would involve people dying in the flesh and being uploaded into a computer and remaining conscious, or people simply being annihilated in an imperceptible instant before a new super-consciousness takes over the Earth. The Rapture and the Singularity share one thing in common: they can never be verified by the living.
Lanier takes on one of the central metaphors enabling the belief in the coming Singularity, the Turing test. The Turing test is supposed to tell us whether computers have attained human intelligence. It works like this: A human and a machine have a conversation over the computer. A third person, the judge, is to determine which one is the human and which the machine. If human judges cannot determine the difference between man and machine, the machine is supposedly intelligent.
Lanier responds that the Turing test does not necessarily prove the intelligence of computers, as many futurists believe. Instead, the fact that human judges mistake computers for humans may mean that we humans have “just lowered [our] own standards of intelligence to such a degree that the machine seems smart.”
“People,” Lanier continues, “degrade themselves in order to make machines seem smart all the time.” This seems shocking, and yet it is the thesis that keeps Lanier’s book moving forward.
One example of how humans dumb themselves down to worship machines is the financial crisis. The best and brightest on Wall St. put their faith in computer models and fancy derivatives that they couldn’t understand but were blessed by computer-guided risk modeling. In a section titled “What Will Money Be?”, Lanier argues that as computers advance to act without constant human intervention, humans will be less and less valuable. And digitally connected mobs that staff wikis and other technologies will devalue individual work “until all jobs are done that way.” In such a world, the “cloud lords might still be able to hold on to their thrones” but the vast majority of human beings will be superfluous. Robots and cloud economics threaten to “usher in a dark age in which everything human is devalued,” where the economic impact of technology would be a “dismal boomerang between gradual impoverishment under robot-driven capitalism and a dangerously sudden, desperate socialism.”
The merit of Lanier’s skepticism is to remind us that as we place increasing faith in computers to govern our markets, teach our children, and make political decisions, we are lowering our standards and allowing ourselves to be governed by machines, algorithms, and crowds. He decries the loss of creativity that he attributes to the web generation: books disappear, replaced by the “one book” in which the entirety of written content subsists in cloud “accessed via user interfaces that encourage mashups of fragments that obscure the context and authorship of each fragment”; music becomes bland, homogenized by digitization software that culls the ambiguities of flexible thoughts into pre-existing musical structures; and self expression is regularized and automated in formatted facebook profiles, pre-built websites, and algorithmically organized behavior. Overall, the “deep meaning of personhood is being reduced by illusions of bits.” The danger, Lanier insists, is that the focus on networks, clouds, and abstractions forgets the humanity of real people.
The question Lanier forces upon us is: What in the person is impervious to cognitive discovery? For Kurzweil, humanity is about knowing and mastering. For Lanier, humans are subject to an unknowable and mysterious consciousness that cannot be mimicked but only castrated by the immersion of humans in intelligent software and content-filled clouds. What the debate between Kurzweil and Lanier is about, then, is the very sense of the human.
III.
OFFERING NEITHER KURZWEIL’S EVANGELISM nor Lanier’s vision of the apocalypse, Sherry Turkle’s Simulation and Its Discontents delivers an entrancing and nuanced glimpse into the powerful yet intimate experiences that are transforming our world. We today, Turkle writes, “see the world through the prism of simulation.” This will be a surprise to many readers, but Turkle—an MIT professor and one of the leading sociologists of technology—makes her case persuasively through close attention to case studies of scientists and professionals. The fact of simulation, in her telling, is neither a human good nor a technological evil. It brings advantages and disadvantages and has proponents and detractors. Both sides are represented, but Turkle focuses on the dissenters. It is by focusing on those who are discontented, Turkle argues, that we can discover the “deep commitments” that undergird our increasingly simulated world.
What is simulation? It is not merely a new way to see or model or experience the world. It is, Turkle explains, “a new way of living, both a change of lens and a change of location.” But how does the simulated world work? What changes from the era before simulation? Or, as Turkle formulates the question: “What does simulation want?” Her answer is that simulation wants an immersion so deep in a technological reality that it comes to stand in for reality itself. The promise as well as the danger of simulation is that “Individuals become immersed in the beauty and coherency of simulation.” What simulation wants is such a complete immersion in a created and built artificial environment that the simulation proposes itself as a “proxy for the real.”
Simulation and Its Discontents is actually a collection of five case studies. The title essay, by Turkle, comprises the first half of the book and is followed by four shorter case studies observing scientists working on the Mars Exploration Rover (by William Clancey) and a remotely operated submarine (Stefan Helmreich), architects learning to use simulation software (Yanni A. Loukissas), and chemistry professors using simulations in the classroom (Natasha Myers).
Turkle’s essay, which grounds the book, brings together two exceptional studies of the integration of computers and simulation at MIT. In discussing the adoption of simulation by biologists, Turkle writes that “visualization and simulation underpin biology as it manipulates and reengineers life at the molecular and cellular level.” Models of life beyond what is visible and comprehensible by human intelligence mean that our understanding of life is based upon algorithms that “predict molecular interactions.” In this work, “Scientists have built a second nature within the computer through simulations that are ever more manipulable, ever more easily experimented on. Some describe the result of such virtual practices as ‘new forms of life.’” One problem with these “new forms of life,” however, is that they sometimes don’t fit with the complexity of real life.
Turkle traces two responses to the disconnect between reality and simulation. One group of scientists strives to retain a critical stance toward simulations, to remember that they are “just models.” One scientist worries that the increased use of off-the-shelf software is producing a generation of scientists who are losing control of their science to instruments that they don’t understand. She insists that despite its speed and advantages, this software deprives her students of “some fundamental experiences they need to develop” and “does not teach them how to use simulation with vigilance.”
Against scientists who view simulation with skepticism, Turkle presents another group of scientists, often younger, who “are increasingly comfortable with black-boxed simulations.” These scientists grew up with computers that they did not have to program themselves. If older scientists value transparency, younger scientists are accustomed to a new understanding of transparency. At one time, scientists understood transparency to mean the ability to access and understand the code of the instruments; now, they desire transparent applications that offer answers seamlessly without demanding any understanding from the user. Working with simulations that are transparent and thus seamless, younger scientists “are more likely than their elders to give themselves over to feeling in the grip of a new materiality.” They share an aesthetic with architects Turkle interviewed who spoke of the feeling of flying through three-dimensional buildings on their screens, images that cease to be mere models of real buildings. As simulations “engage the body” of their users, the simulations take on a material reality so that “users experienced the system as a prosthetic extension of themselves into what felt like a tangible world of screen molecules.”
Turkle worries that our attitude towards simulation has been transformed in ways still not understood. “These days, for simulation’s most sophisticated users, a critical stance is no longer about vigilance to protect simulation from error. It is about living with shadows that bring us closer to the forms beyond them.” (82) Even the experts, she writes, now struggle to recall that simulations are not real. Thus one chemistry professor Turkle interviews intentionally degrades her images to diminish their power to convince; others, of course, publicize their pretty and seemingly perfect simulations as the truth.
Amongst the many dangers that Turkle perceives is the immersive power of simulation to offer itself as the truth. Architects whom Turkle interviews repeatedly marvel at the ways that computer simulations make buildings look beautiful and complete. They find that once a design is fed into a computer and produced in 3D, it is very difficult to conceive it differently. Even if the design isn’t yet thought through or doesn’t work, they find the beautiful coherence of the simulated rendering compelling. Similarly, scientists find that they, as well as their students, are seduced by the seeming perfection of simulated reality and thus forget to mind the gap between the simulated and the real.
“The fantasy” of simulation, Turkle writes, is “visceral in nature,” and all too often “Computer precision is wrongly taken for perfection.” Even though the scientists know that “all simulations are wrong,” they have a hard time resisting their truth.
IV.
THERE IS NO DOUBT that simulations—along with computing clouds, neural implants, and digital enhancements—will change the experience of being human. But will it threaten human creativity and endanger human freedom? Is humanity, itself, under threat?
Perhaps the gravest danger that simulation and the Singularity pose is the increased capacity for human beings to lie to others and to themselves. As Lanier writes, “No one in the pre-digital cloud era had the mental capacity to lie to him- or herself in the way we routinely are able to now.” It is counterintuitive, but the “limitations of organic human memory and calculation used to put a cap on the intricacies of self-delusion.” Now entire professions rely on vast quantities of information beyond the power of any human to comprehend or to process. There are layers upon layers of abstraction between facts and what we see—and each layer of abstraction can be manipulated, culled, directed, and spun by human as well as non-human intelligences. While facts have always been mediated through language and culture, increasingly, everyday experiences are encountered through systems, technologies, and processes that are immune to human comprehension. Confronted with competing realities, facts fade in their impact and we are ever more comfortable in a web of competing ideologies.
No thinker has explored the modern disempowerment of facts as has Hannah Arendt. In The Origins of Totalitarianism, Arendt showed how the alienation of 20th century individuals elicited a desire to escape reality and a demand for consistency. Unhappy people will choose to believe radical fictions that promise consistency and purpose over the decay and disaster of real life. A lying world of consistency is, Arendt saw, more adequate to the needs of the human mind than the vagaries of reality.
Twenty years later, in Truth in Politics, Arendt expressed shock that “highly respected statesmen,” like Charles de Gaulle and Konrad Adenauer, had been able to “build their basic policies on such evident non-facts as that France belongs among the victors of the last war and hence is one of the great powers, and ‘that the barbarism of National Socialism had affected only a relatively small percentage of the country.’” She was exasperated by the way that Stalin re-wrote the history of the Soviet Revolution to exclude the accomplishments of Trotsky and others. The political activity of lying, she saw, denies facts and creates alternative realities. In denying and creating, the political liar acts to change the world, to make reality anew, so that it conforms to our needs and desires.
What Lanier, Turkle, and Arendt show is that the realm of abstractions and simulations allows for the preservation and extension of fictions in ways heretofore unimaginable. Even when facts that contradict a fiction exist, those facts can be eliminated. That is the power of simulation.
Whether or not the Singularity occurs and whatever our technological future will bring, we are confronted by the possibility of a simulated world without facts and devoid of truth. It is one thing to lie to others, but quite another to lie to oneself. Once all standards of truth and honesty fall prey to the simulacrum of simulation, humanity will finally be freed to live in a world fully of its own making. If that happens, we will come finally to understand the wonders of man, in all his ambiguity.
Roger Berkowitz is director of the Hannah Arendt Center for Ethical and Political Thinking and associate professor of political studies and human rights at Bard College. The Gift of Science: Leibniz and the Modern Legal Tradition, his first book, was recently republished in paperback. The Arendt Center’s 2010 Conference, Human Being in an Inhuman Age, will feature both Ray Kurzweil and Sherry Turkle.
More: Jaron Lanier, “The First Church of Robotics” [The New York Times, 9 August 2010].