[Note: The Age of Em (in italics) refers to Robin’s book.  The Age of Em (not in italics) refers to the hypothetical future era when whole-brain emulations become practically feasible.]

While I’m thrilled that Robin Hanson has published his first book, and have high hopes for the social science of futurism, The Age of Em greatly displeases me.  If he rigorously reasoned from his premise – whole brain emulations – I could just suspend my technological disbelief.  Unfortunately, the reasoning simply isn’t very rigorous.  When someone asks me, “Show me the greatness of Robin Hanson,” I have to look elsewhere.

Granting Robin’s technological premise, here are the book’s main weaknesses, roughly in the order they appear in the text.  Some of my criticisms arguably bolster Robin’s hopes for the future; my point, in each case, is simply that Robin is making big literally false or deeply misleading statements.

1. Robin only pretends to dodge philosophy of mind.  His words:

[M]ost who discuss ems debate their feasibility or timing, ponder their implications for the philosophies of mind or identity, or use them to set dramatic stories. Such discussants usually ask: is it conscious? Is it me? Is it possible? When will it come? How can it enrich my story?

In this book I instead seek realistic social implications–in what sort of new social world might ems live?

Steering clear of philosophy of mind seems like an excellent way to bypass deadlocked debates and break new ground on a separate set of issues.  Unfortunately, Robin doesn’t actually do this.  Instead, he tacitly accepts an extreme version of “Ems are just as human as you or me” – and builds the whole book on this assumption.  The tell-tale sign: The Age of Em says vanishingly little about the lives of biological humans during the Age of Em!  Or as Robin tells us early on:

(If you can’t see the point in envisioning the lives of your descendants, you’d best quit now, as that’s mostly all I’ve got.)

To me, and I believe almost everyone else, my “descendants” are the biological humans.  To Robin, in contrast, my “descendants” are ems.  Frankly, he seems so wedded to this philosophical (not social scientific) position that he can’t even feign agnosticism.  What would feigning look like?  Split the book evenly between discussion of the lives of biological humans during the Age of Em and ems during the Age of Em.

2. Robin exaggerates how dramatically humans have changed over time:

The problem is, the future will probably hold new kinds of people. Your descendants’ habits and attitudes are likely to differ from yours by as much as yours differ from your ancestors. If you understood just how different your ancestors were, you’d realize that you should expect your descendants to seem quite strange. Historical fiction misleads you, showing your ancestors as more modern than they were. Science fiction similarly misleads you about your descendants.

I don’t read much historical fiction, but I have read a lot of fiction from earlier eras.  And contrary to Robin, I see a largely constant human nature.  Characters in Shakespeare seem as credible to me – nay, more credible – than characters in modern fiction.  Ancient Roman historians make as much sense to me – nay, more sense – than modern historians.  The same goes, in my view, for cross-cultural research.  As the Roman poet Terence (195/185-159 BC) put it, “I am human, I consider nothing human alien to me.”

3. Robin has a bizarre definition of “marginalized”:

Just as foragers and subsistence farmers are marginalized by our industrial world, humans are not the main inhabitants of the em era. Humans instead live far from the em cities, mostly enjoying a comfortable retirement on their em-economy investments. This book mostly ignores humans, and focuses on the ems, who have very human-like experiences.

Suppose foragers and subsistence farmers owned 90% of the industrial world’s financial assets, housing, and so on.  Even if they were never CEOs or served on boards of directors, we would not call them “marginalized.”  Why?  Because though they are outnumbered, they are fabulously wealthy and ultimately in charge.  The same goes for biological humans in Robin’s scenario.  They’ll be outnumbered, and perform little “hands-on” work.  But they’ll be fabulously wealthy and ultimately in charge.  Or so it would seem – and Robin makes little effort to show otherwise.

4. Contrary to Robin’s suggestion, there’s near-zero correlation between income and conservatism.  His words:

Foragers tend to have values more like those of rich/liberal people today, while subsistence farmers tend to have values more like those of poor/conservative people today. As industry has made us richer, we have on average moved from conservative/farmer values to liberal/foragers values…

Robin could say he’s defining “conservatism” in a technical or apolitical way.  But when you’re writing for an audience, the author rightly bears the burden of highlighting non-standard usage.  On standard definitions, it’s far more accurate to say that rich people and societies are more socially liberal but more economically conservative.

5. Robin’s conclusions only sound “taboo” because he’s using language strangely.  His words:

This book violates a standard taboo, in that it assumes that our social systems will mostly fail to prevent outcomes that many find lamentable, such as robots dominating the world, sidelining ordinary humans, and eliminating human abilities to earn wages.

In plain English, Robin predicts that biological humans have a great life in The Age of Em. Robots will “dominate” us no more than rank-and-file workers “dominate” shareholders.  That’s a very comforting vision of the future, except perhaps for sci-fi fans.

6. Robin greatly overstates the difference between his “em scenario” and the “generalized AI scenario.”  How so?  The Age of Em makes numerous arguments by analogy: Since humans typically do X in situation Y, and ems are copies of humans, ems will also typically do X in situation Y.  But he also keeps telling us that only a tiny hand-picked sub-sample of humans will be copied.  The obvious question: Why wouldn’t ems largely be copies of the most “robot-like” humans – humble workaholics with minimal personal life, content to selflessly and
uncomplainingly serve their employers?  This in turn implies that most of Robin’s “detail” is roughly the opposite of what would really happen.

7. Robin’s efforts to calm readers’ fear of the future consistently backfire.  Example:

Readers of this book may find near subsistence wages to be a strange and perhaps scary prospect. So it is worth remembering that such wages in effect applied to almost all animals who ever lived, to almost all humans before a few hundred years ago, and for a billion humans still today. Historically, it is by far the usual case.

Imagine a middle-class American’s child is destined to earn a subsistence wage.  Would it make the parent feel better to hear, “No big deal, your child will face the same fate as almost every animal who ever lived, almost all humans before a few hundred years ago, and a billion humans today”?  No, even worse!

8. On his own terms, Robin greatly overstates the quality of life for ems.  His words:

Yes, “poor” ems spend a large fraction of their time working. But such ems need not suffer physical hunger, exhaustion, pain, sickness, grime, hard labor, or sudden unexpected death. Widespread use of automation makes most jobs at least modestly mentally challenging. As most ems are poor, em poverty does not inflict the same pain of low social status that it does in societies such as ours where most people are rich. Ems could be assured of very high-quality entertainment during leisure time, and of a comfortable indefinite retirement when they were no longer competitive at work.

The obvious question: Why wouldn’t ems’ creators use the threat of “physical hunger, exhaustion, pain, sickness, grime, hard labor, or sudden unexpected death” to motivate the ems?  Robin elsewhere talks about “torturing” ems, so why not?  And of course, the best way to make such threats credible is to enforce them out without mercy.

To be fair, Robin does say:

It is possible that stronger punishment involves direct pain, and this has often happened in the distant past. But the extreme rarity of this practice today suggests that pain is not very useful as a motivator for workers in advanced industrial jobs, and so is also only rarely useful for em workers.

But again, Robin misses the obvious retort: If employers tried using pain to motivate free workers, the workers would quit.  Modern systems of slave labor – see Stalinist Russia and Nazi Germany – used pain freely, because the penalty for quitting was death.  Even Stalin’s nuclear scientists feared execution if they failed to produce a nuclear bomb in a timely manner – and as you’d expect, this fear was a powerful motivator.

9.  Robin’s arguments for his single craziest claim – global GDP will double every “month, week, day, or even faster” – are astoundingly weak.  Yes, Argument #1 has superficial appeal:

Special three-dimensional (3D) printers have been created that can print about one-half of their components in about 3 days of constant use (Jones et al. 2011). If the other half could be made just as fast, a 3D printer could self-replicate in a week. If the other half of the parts for a 3D printer took ten times longer to make, then a 3D printer could self-replicate in 5 weeks.

Together, these estimates suggest that today’s manufacturing technology is capable of self-replicating on a scale of a few weeks to a few months.

In the real world, however, there are literally hundreds of bottlenecks that radically retard this kind of growth.  Politically, something as simple as zoning could do the trick.  Robin will naturally appeal to selection – the em economy will launch in whatever country has the most em-positive regulatory environment.  But the most favorable political environments on earth still have plenty of regulatory hurdles – especially for technologies that pose a threat to reigning powers.  And politics aside, we should expect bottlenecks for key natural resources, location, and so on.  As an engineer, I’m sure Robin’s heard of Murphy’s Law.  Furthermore, if ems are bad at any crucial task, biological humans have to take up the slack, in their usual sluggish meat-space way.

Robin’s Argument #2 has no appeal at all:

Another way to estimate the economic growth rate of the next era is to assume that the next era will grow faster than our industrial era by a factor similar to the factor by which our era grows faster than the farming era, or by which the farming era grew faster than the forager era. This method estimates a roughly 1 week to 1 month economic doubling time for the next era. While this is admittedly only a weak clue regarding future growth rates, we should not ignore it as it is one of the few concrete clues available.

This alleged “concrete clue” is nothing compared to the bona fide “concrete clue” that almost all fantastic claims are false.  And the idea that the global economy will start doubling on a monthly basis is fantastically fantastic.  This has to be the least Bayesian part of the book: We start with a claim with a near-zero prior probability, make a couple of flimsy arguments, and somehow end up with a high posterior probability.  I’d like to be more charitable, but I can’t.

Lest you think Robin is just speculating about economic growth in the Age of Em, here’s his punchline: “For a concrete estimate to use in the rest of this book, based on all of the above, I
choose an economic doubling time of 1 month.”  Personally, I’d be amazed if an em economy doubled the global economy’s annual growth rate.  Which would be awesome, of course.

10. Robin’s argument against the Terminator scenario is much weaker than it looks.  His words:

Because ordinary humans originally owned everything from which the em economy arose, as a group they could retain substantial wealth in the new era. Humans could own real estate, stocks, bonds, patents, etc. Thus a reasonable hope is that ordinary humans become the retirees of this new world. We don’t today kill all the retirees in our world, and then take all their stuff, in part because such actions would threaten the stability of the legal, financial, and political world on which we all rely, and in part because we have many direct social ties to retirees. Yes we humans all expect to retire today, while ems don’t expect to become human, but em retirees are vulnerable in similar ways to humans. So ems may be reluctant to expropriate or exterminate ordinary humans if ems rely on the same or closely interconnected legal, financial, and political systems as humans, and if ems retain many direct social ties to ordinary humans.

The problem: As Robin explains, in one human year, ems experience millennia.  So even if each generation of ems only has a .5% chance of expropriating humanity, the chance of expropriation per human year is around 40%.  If Robin’s general picture is correct, what the ems see as the end of age-old human dominance will happen right before the eyes of the Age of Em’s first generation of humans.  With fire and blood.

P.S. Is there a contradiction between Criticism #5 and Criticism #10?  No.  Criticism #5 says existing humans would be pleased to live in the Age of Em as Robin describes it.  Criticism #10 says existing humans would be horrified by the Age of Em as the logic of Robin’s position suggests it would unfold.

P.P.S. Fortunately, per Criticism #6, Robin’s analogies between humans and ems are largely spurious.  If ems ever came into being, they’d be heavily selected for docility and pose no serious threat to mankind even in the long-run.  So despite everything, I hope the Age of Em comes.  I just think it’s astronomically unlikely.