Here’s my reply to Robin’s reply to my criticisms of The Age of Em.  I’m the main text; he’s in first-level blockquotes; I’m in second-level blockquotes; he’s in third-level blockquotes.

1. Robin only pretends to dodge
philosophy of mind. .. He tacitly accepts an extreme version of “Ems are
just as human as you or me” – and builds the whole book on this
assumption. The tell-tale sign: The Age of Em says vanishingly little
about the lives of biological humans during the Age of Em! .. he seems
so wedded to this philosophical (not social scientific) position that he
can’t even feign agnosticism. What would feigning look like? Split the
book evenly between discussion of the lives of biological humans during
the Age of Em and ems during the Age of Em.

These are complaints about language and emphasis. On language, since
I’m constantly applying human and social sciences to ems it would have
been quite awkward to use any other than our usual language for
describing people. It is hard to imagine a readable book where words
like “people” are constantly replaced with phrases like “machines that
act like humans but do remember I’m not making any philosophical claims
here.” On emphasis, very little happens to biological humans during the
em era, so equal emphasis would mean a very short book. That conflicts
with my goal of showing just how much I can say about this scenario.

I’m not advocating awkward language.  My point is that Robin barely discusses what is, by normal standards, the most important aspect of the Age of Em: The lives of biological human beings.  My explanation: Despite his claims of agnosticism, Robin thinks biological humans will be unworthy of his interest once trillions of ems exist.  This makes sense if ems are literally conscious.  Otherwise, not.

Robin’s explanation – that “very little happens to biological humans during the em era” – is bizarre.  On his own account, biological humans become fabulously wealthy people of leisure almost overnight.  That’s a big deal in itself, with far-reaching social and political implications.  How happy will humans be?  How many kids will they have?  How will status games change?  What will happy to partisanship?  Religion?   

3. Robin has a bizarre definition of
“marginalized” .. biological humans .. They’ll be outnumbered, and
perform little “hands-on” work. But they’ll be fabulously wealthy and
ultimately in charge.” .. 5. Robin’s conclusions only sound “taboo”
because he’s using language strangely. .. Robots will “dominate” us no
more than rank-and-file workers “dominate” shareholders.

As I mentioned in a previous post, many have reacted to talks I’ve
given by complaining about humans no longer being at the center of
action, even when they understand that biological humans could for a
while own most of the em world, and thus direct how spare resources are
spent. I used words like those people use to acknowledge their concerns.

In Robin’s scenario, such concerns are silly.  Does it make sense to feel bad for GM’s majority shareholders because they don’t personally assemble cars? 

4. Contrary to Robin’s suggestion,
there’s near-zero correlation between income and conservatism. ..
“subsistence farmers tend to have values more like those of
poor/conservative people today.” .. Robin could say he’s defining
“conservatism” in a technical or apolitical way. But when you’re writing
for an audience, the author rightly bears the burden of highlighting
non-standard usage.

In that context I had just cited studies on strong correlations between the culture and wealth of nations,
and I had just explained in quite some detail the kind of
“conservatism” I meant there. It is true that I didn’t mention
explicitly there that the word “conservatism” is used in many different
ways. But the book would be a lot more tedious if every time I
introduced and used a term I explained the many other ways people have
used that term.

My complaint is not just that “conservatism” is used in many ways, and Robin picks one.  My complaint is that “conservatism” is primarily used in one way that isn’t Robin’s way, leading to confusion.  This is a symptom of the unfortunate diary-like style of The Age of Em – the fact that you have to pre-understand Hansonian thought in great detail to grasp what he’s saying. 

7. Robin’s efforts to calm readers’ fear of the future consistently backfire. Example:

Readers of this book may find near
subsistence wages to be a strange and perhaps scary prospect. So it is
worth remembering that such wages in effect applied to almost all
animals who ever lived, to almost all humans before a few hundred years
ago, and for a billion humans still today. Historically, it is by far
the usual case.

Imagine a middle-class American’s child
is destined to earn a subsistence wage. Would it make the parent feel
better to hear, “No big deal, your child will face the same fate as
almost every animal who ever lived, almost all humans before a few
hundred years ago, and a billion humans today”? No, even worse!

Saying something is “worth remembering” just means that it may change
how you think about that thing; it is not the same as saying “don’t
worry.” It isn’t my job to make readers like the age of em, but it is my
job to make sure they keep important considerations in mind.

This looks like motte-and-bailey to me.  Robin routinely tries to paint the Age of Em in a favorable light.  Here’s one memorable instance.  When you point out that his arguments are unconvincing, he protests he’s merely trying to describe the future accurately, however awful it may be.  A few days latter, he resumes his advocacy.

10. Robin’s argument against the Terminator scenario is much weaker than it looks. His words:

A reasonable hope is that ordinary humans
become the retirees of this new world. .. ems may be reluctant to
expropriate or exterminate ordinary humans if ems rely on the same or
closely interconnected legal, financial, and political systems as
humans, and if ems retain many direct social ties to ordinary humans.

The problem: As Robin explains, in one
human year, ems experience millennia. So even if each generation of ems
only has a .5% chance of expropriating humanity, the chance of
expropriation per human year is around 40%.

Bryan misreads me as trying to offer more reassurance than I can. I
was clear that even if humans survive the year or two that comprises the
age of em, I can say little about what might happen after that. “A
reasonable hope” is quite different from “don’t worry.” I would be
remiss if I didn’t at least point readers in the direction of a
reasonable hope, even if I can offer few guarantees.

Never mind “guarantees.”  My argument above implies biological humans are likely to be wiped out one year into the Age of Em as Robin describes it.  Is my argument wrong?  If so, why? 

Bryan’s last two objections, on economics, are the ones I take most seriously.

8. Robin greatly overstates the quality
of life for ems. .. Why wouldn’t ems’ creators use the threat of
`physical hunger, exhaustion, pain, sickness, grime, hard labor, or
sudden unexpected death’ to motivate the ems? Robin elsewhere talks
about `torturing’ ems, so why not?” .. Modern systems of slave labor –
see Stalinist Russia and Nazi Germany – used pain freely, because the
penalty for quitting was death.

I was careful not to claim that ems would not be slaves. I just
suggested that we didn’t have good reasons to expect most being slaves.
Most people in history haven’t been slaves, even when competition has
been strong. There are still some slaves in the world today, and they
aren’t known for being spectacularly productive workers due to frequent
use of torture and pain. Nor were Stalinist Russia and Nazi Germany
slaves known for stellar productivity. We have a larger literature on
motivating workers, and threats of pain only seem to be useful for the
most routine and physical of labor.

As I said, this large literature focuses on motivating workers who are free to quit.  If workers aren’t free to quit, terror is an effective motivator – even for complex tasks.  Again, read the history of the Soviet nuclear program.  Stalin’s top scientists worked in the shadow of death.  Since they couldn’t flee, they worked like dogs to give him nuclear weapons, and reached their goal rapidly.  Just one example, but a powerful one nonetheless.

I agree Soviet and Nazi slaves’ productivity was normally low, but the reason is simple: Their labor camps did not prioritize productivity; their main aim was crushing hated enemies, not maximizing output.

Historic slave systems did often augment negative incentives (torture, death) with positive incentives (better treatment, cash).  But there’s a simple economic story: When slave-owners have imperfect information about slaves’ productivity, high quotas lead to lots of counter-productive punishments.  Threatening to execute everyone who falls below the 90th-percentile of output, for example, requires slave-owners to kill 90% of their slaves.  Information about ems’ productivity, however, should be much more accurate, especially since most descend from a small number of exceptional humans.  These are ideal conditions for heavy use of negative incentives.

When you seek workers to be
creative, think carefully, take the initiative, or persuade and inspire
others, you mostly seek other motivations. Our best analogy for ems
should be the few hundred most productive people in the world today, and
threats of pain are not remotely what motivate them. Who thinks torture
would make them more productive overall in the long run?

If they’re not free to quit, I think exactly that. 

9. Robin’s arguments for his single
craziest claim – global GDP will double every “month, week, day, or even
faster” – are astoundingly weak. .. In the real world, however, there
are literally hundreds of bottlenecks that radically retard this kind of
growth. Politically, something as simple as zoning could do the trick.
.. the most favorable political environments on earth still have plenty
of regulatory hurdles .. we should expect bottlenecks for key natural
resources, location, and so on. .. This alleged “concrete clue” is
nothing compared to the bona fide “concrete clue” that almost all
fantastic claims are false. And the idea that the global economy will
start doubling on a monthly basis is fantastically fantastic. This has
to be the least Bayesian part of the book: We start with a claim with a
near-zero prior probability, make a couple of flimsy arguments, and
somehow end up with a high posterior probability.

One could have similarly argued that fundamental growth bottlenecks
must prevent the previous observed huge jumps in growth rates, such as
from foraging to farming, or farming to industry. And plausibly related
obstacles did prevent those eras from starting as soon as they
might have. But eventually obstacles were overcome. No doubt our current
economy tolerates many delays that would have to be cut to enable much
faster growth, and the em economy won’t start as early as it might
because of regulatory and other delays. My book is mainly about what
happens once those obstacles are overcome. Does Bryan really think such
obstacles could never be overcome?

To make global GDP double every month, you don’t have to overcome some bottlenecks.  You have to overcome an accelerating series of bottlenecks.  The bigger and faster the changes you seek, the more obstacles you meet. 

Even when doing so might
quickly allow a city or nation to dominate the world? His “near-zero
prior” seems to come not from any fundamental analysis but, from his
strong reliance on intuition;

Whatever you call it, I’m exercising common-sense skepticism.  When someone predicts huge changes, I scoff unless they have overwhelming evidence in their favor.  So should we all.

I suspect he would have similarly assigned
a very low prior to manned flight in 1850, or to space flight in 1900.

Technically, there was manned flight in 1850.  More to the point, there’s a world of difference between predicting specific technological advances, and claiming they’ll quickly and constructively revolutionize society.  I can believe there’s a 1% chance ems will emerge in a century.  That’s not crazy.  But it is crazy to think the emergence of ems will lead global GDP to start doubling on a monthly basis.  For that, a conditional probability of one-in-a-million is generous.

Robin may protest he’s simply applying standard growth theory to a novel situation.  A better description is that he’s mechanically applying standard growth theory to an unprecedented situation the model was never designed to handle.

But as I said above, I expect many others agree with his intuition,
and I thank him for saying explicitly what others only think.

Not just “many others.”  I predict at least 95% of empirical growth economists would agree with me.  Indeed, I’d be surprised if Robin could find any empirical growth economist with no prior affiliation with futurism or science fiction who’d view his conditional growth prediction as plausible.

Robin has helped me more than anyone else to internalize Bayesian thinking.  I’m flabbergasted, then, at how un-Bayesian his growth predictions are.  If “Technology X will cause global GDP to double” doesn’t deserve an extraordinarily low prior probability, what does?  And what evidence has Robin produced to justify raising that prior above the microscopic level?