Cory Doctorow: "The Coming Civil War over General-purpose Computing" | Talks at Google



>>Presenter: I first met Cory Doctorow in
1986 when he was working for Metallica and trying to anticipate the coming Napster wars. People don't know this about him, but he is
a fervent defender of the RIAA and the MPAA. This has all been just a big lie, he says,
[laughter] to make people feel that he's one of us. Recently, actually, I ran into him in Washington,
DC, and he was telling me, "You guys do the legal minimum of compliance with the DMCA. We think that you're just pirates." I said, "You've been working for the RIAA
now for 16 years. I don't see how you can possibly say that
with any sort of honesty." Then I realized who I was talking to, of course. [laughs] [laughter]
Realistically, I met Cory when he was doing sugar water. Right? For the first time. >>Cory Doctorow: [inaudible] >>Presenter: Yeah. And then Zelig-like, I noticed as he rose
as an author. I actually don't want to spend much time introducing
him. So I won't. Everyone, please welcome Cory Doctorow. [applause] >>Doctorow: Hi folks. I give– I write science fiction novels and
stories. You've got some of my short story collections
there in front of you. They all relate, in one way or another, they
kind of circle these issues. I feel like science fiction stories put the
sinew and the marrow into the argument. Before George Orwell came along, if you wanted
to talk about surveillance, you could say, "I kinda feel like it might change my behavior
if I were being watched all the time in some abstract way." And someone else might say, "Yeah, but if
we knew everything about you, we could provide services to you and we could know when bad
things were going to happen" and so on. And now we have this great word we can use
to describe what that means. You can say it's Orwellian. So there's now a lot of muscle on the bone
when you talk about this stuff. That's what I do in the fiction. But I don't want to stand here and read stories
to you, although I have a podcast where you can hear me read stories. What I'm going to do today is the argument
that the books that you're holding in your hand are the blood and sinew for, and take
it from there. The talk runs 35 minutes. And then there's time for Q&A. The one thing I want to say as a caveat to
this: I've given this talk twice now. I gave it at the Long Now Foundation and I
gave it at DEFCON, both in the last week or ten days. Both times, there was a little bit of feedback. There's a hypothetical, technical solution
I propose, and I'll tell you when I get there. I want to clarify that it is purely hypothetical
by way of example, and not a thing that I think we should do. With that said, [laughter] I'm going to get
to it. I gave this talk in late 2011– Ha. Ah! There we go. I gave this talk in late 2011 at the 28C3
in Berlin called "The Coming War on General Purpose Computation." In a nutshell, the hypothesis of that talk
was computers and the internet are everywhere, the world is increasingly made of computers
and the internet. We used to have these separate categories
of devices like washing machines, VCRs, phones, and cars, and now we just have computers in
different boxes. Cars are computers we put our bodies into. 747s are badly secured Solaris boxes connected
SCADA controllers. [laughter] Hearing aids, pacemakers, other
prostheses: computers we put in our body. That means that from now on, all of our socio-political
problems in the future are going to have a computer in the middle of them. That will beget a regulator who says, "Can't
you just make me a computer that solves the problem? Can't you make me a self-driving car that
can't be programmed to drag race? Can't you make me a bioscale 3D printer that
doesn't print out an organism that puts the human race at risk or blows Monsanto's quarterly
profits?" That is, "Can you make me a general purpose
computer that runs all the programs except for the one that pisses me off?" [laughter]
Now, we don't know how to make that computer. We don't have a theoretical model for Turing
Complete minus one. Our closest approximation to a computer that
runs every program except for the one that abets a criminal or evinces a social problem
is a computer with spyware on it when it comes out of the box. That is, a computer that watches everything
that you do all the time, so that when the moment comes, it can say, "I can't let you
do that, Dave." A computer that runs secret programs that
the user isn't supposed to even know about. If the user finds out about it, the user can't
terminate these processes, even if the user really thinks that they run contrary to their
interests, and even if the computer that they're running on belongs to the user. In other words, digital rights management. Now, digital rights management's a bad idea
for solving social problems for at least two significant reasons. The first one is that it doesn't actually
solve the problem. Breaking DRM isn't hard for bad guys. As the copyright wars have shown us, digital
rights management is a solution that ends within 24 hours. As soon as a bored Norwegian teenager encounters
the DRM, it goes away. DRM only works if the "I can't let you do
that, Dave" program remains a secret. Once the most sophisticated attacker in the
world finds out that secret and puts it on the internet, everybody else on the internet
has the secret, too. Now, the second reason is that DRM not only
has weak security, but it weakens security. In order to be secure, you need to be certain
about what software is running on your computer. You can't secure the software on your computer
if you don't know what software is running on your computer. When you design the "I can't let you do that,
Dave" facility into a computer, you create this enormous security vulnerability. You now have a program running that users
aren't even supposed to know about. If they know about it, they can't find details
of or terminate or override. When some bad guy hijacks this, they can do
things to your computer that, by design, your computer doesn't show you. You probably remember, Sony BMG put root kits
on 51– no, 6 million CDs, 51 audio CD titles, and distributed them to their customers. They stealthily installed malware. The root kit made any process or file that
was prepended with dollar sign SYS, invisible to the file manager and process manager. Immediately, malware writers started prepending
dollar sign SYS to their program files and their processes because if they ever found
themselves on a computer whose immune system had been blown by the Sony root kit, that
immune system would no longer even be able to see their process. Now, once governments solve problems with
DRM, there's this perverse incentive to make it illegal to tell people things that might
override the DRM, things like "This is how the DRM works" or "Here's a flaw in the DRM
that might allow an attacker to secretly activate the microphone or turn on the camera or grab
your keystrokes." Now after I gave this talk at 28C3, I got
a lot of feedback from various civil libertarians and other people, including some very distinguished
computer scientists. I got a very thought provoking email from
Vint Cerf after I wrote this, which really made my day. It led me to the conclusion that within the
fields of civil liberties and technology and policy, there's a kind of good guy consensus
that if you own your computer, you should be in charge of what's running on it, at least
as between you and corporations, or you and the government. That mandating what software you may or may
not run on your computer is just not a good idea if it belongs to you. Now, most computers– Let's examine, for a
minute, what it would mean, as an owner, to be able to absolutely control the software
that was running on your computer in an adversarial relationship against, not an advanced persistent
threat, but at least against script kiddies or griefers or just your garden variety deputy
dog cop who wants to screw with your computer. Most computers today are fitted with these:
TPMs, trusted platform modules, a secure code processor mounted on the motherboard. The specification for TPM is published. There's an industry body that certifies that
devices that advertise a TPM actually have a real TPM in them and not a fake TPM. To the extent that that spec is good, and
to the extent that these people are diligent in doing their jobs and sue people who list
a device as having a TPM when it doesn't, it's possible to be reasonably certain that
if you think you have a TPM, you do have a TPM, and that it faithfully implements the
spec. TPM is secure. One of the ways in which it's secure is that
it has some secrets. But it's also secure in that it's designed
to be tamper evident. If you try to extract the keys from a TPM,
it's supposed to be really obvious that something has been done to your computer. Someone takes your real TPM out and puts a
fake TPM that they 3D printed or cooked up in their hack lab or made down in Quantico
and sticks it in your computer, it's supposed to be really obvious that it's happened. There's a TPM threat model that crooks or
governments or police forces or some other adversary try to compromise your computer,
and TPM tamper evidence lets you know when that's happened. But there's another TPM threat model. It's that a piece of malicious software infects
your computer. Now, all the censors—. When that happens, all the censors that are
attached to your computer, –the mic, the camera, the accelerometer, the fingerprint
reader, the GPS, and so on–, can be switched on without your knowledge, and the data can
be cached on the or can be sent to a bad guy or both. Not only that, of course, all of the data
on your computer– your sensitive files, your stored passwords, your web history– can also
be harvested and sent to a bad guy, or harvested and cached for a later retrieval, as can all
your keystrokes. All the peripherals that are attached to your
computer can either be subtly altered, turned off, or turned on to do bad things. Today, those peripherals might be your printer,
your scanner, your SCADA controller, your MRI machine, your car, your avionic, your
3D printer. You can understand why that would be a bit
freaky, but of course, in the future, those peripherals might also include your optic
nerve, your cochlea, and the stumps of your legs. When your computer boots up, the TPM can ask
your bootloader for a signed hash of itself and verify that the signature of the hash
comes from a trusted party, someone you trust. Once you trust the bootloader to faithfully
perform its duties, you can ask it to check the signatures on the operating system, which,
once verified, can check the signatures on the programs that run on it. And so on and so on up the stack, ensuring
that you know which programs are running on your computer, and that any programs running
in secret have gotten there by leveraging a defect in the bootloader or operating system
or the other components, and not because this computer was designed to actually hide things
from you. Now, this story always reminds me of Descartes:
he starts off by saying that he can't tell what's true and what's not true, because he
doesn't know if he can trust his senses, he doesn't know if he can trust his reason. He does some mental gymnastics, which I won't
get into here, although that's generally the thing people find interesting about him, but
what's interesting to me is that once he establishes this tiny little nub of certainty, a kind
of mental gymnastic exercise that says, "Well, I can trust my reason, I can trust my senses." Then he is able to erect this stable edifice
of a worldview on it. He knows one thing to be true, and everything
else can be hung off of that one thing. He can build it up. Now, a TPM is like that. It's a nub of stable certainty: if it's there,
it can reliably inform you about your bootloader, and thus, your operating system, and thus,
the processes running on your computer. Now, you may find it weird to hear someone
like me talking warmly about TPMs. After all, these are the technologies that
make it possible to lock phones, tablets, consoles, and even some PCs so that they can't
run software of the owner's choosing. Jailbreaking usually means finding a way to
subvert a TPM. Why on earth would I want a TPM in my computer? As with everything interesting in tech and
policy, the devil is in the details. Imagine for a moment that there's two different
ways of implementing a TPM. There may be more, but imagine these two. The first one we'll call lockdown. In the lockdown world, your TPM comes with
a set of signing keys it trusts, and unless your bootloader appears in that list– is
signed by one of those signing keys, it won't run it. It won't boot, the operating system won't
run. You're just stuck there in whatever it is
that people who installed the bootloader on your computer want you to run. You can't change that. There's another mode that I'll call certainty. In the certainty mode, you tell your TPM which
signing keys you trust. The first time you turn your computer on,
you initialize it with some authentication token– whoops– like a key or a password
or some other thing. A biometric that it knows so it knows who
it belongs to. Then you, the owner, are the only person who
gets to say what it trusts. You can say, "I don't trust this person's
operating system" or "I do trust that person's operating system." "Only run operating systems that are signed
by Cononicle, EFF, ACLU and Wikileaks. [laughter] Approximately speaking, these two
modes correspond to, of course, iOS and Android. iOS only lets you run the code that's been
approved by Apple. Android lets you tick a box and say, "I'm
a grown up. Let me choose who I trust”. Critically, Android lacks an important facility:
it lacks the facility to verify that what you think you're running is what you are running. It's freedom without certainty. Now, freedom without certainty is a big deal
in a world where the computers we're discussing can see you and hear you, where we put them
in your pocket and take them into the toilet, where they sit by your bedside, where they
fly airplanes, where we put our bodies into them, and they drive our cars around, which
is why I like the idea of a TPM, provided it's implemented in the certainty mode and
not in the lockdown mode. Now, if that's not clear, think of it this
way: there's the war on general-purpose computation, and that's what happens when control freaks
in governments or companies decide that they should have the final say in what you do on
your computer. There's also– And then there's this: there's
the counter position, which is that defenders against those people are also control freaks,
but they're control freaks like me. We want to be people with the ultimate destiny
over what we install on our computers. Both sides want control, they just differ
in where the nexus of control should be. Control requires knowledge. If you want to be sure that songs that are
moved onto an iPod, stay on the iPod, and don't come off of the iPod, the iPod needs
to know that the instructions that it's getting are coming from an Apple-approved version
of iTunes, and not one pretending to be iTunes. Otherwise, you don't get the roach motel. If you want to be sure that my PVR won't record
a watch-once, video-on-demand program, or if it does record a program, that it won't
output it to anything except something that will honor whatever business rules came along
with it. You have to be sure that you know what programs
I'm running and what they do. But if I want to be sure that you aren't watching
me through my webcam, I need to know what firmware is running, and I need to know that
the little green light always comes on when my webcam switches itself on. If I want to be sure that you aren't capturing
my passwords through my keyboard using a software keylogger, I need to know that the OS isn't
lying when it says there aren't any keyloggers resident in the system. Whether you want to be free or whether you
want to enslave, you need control and you need knowledge. That's the coming war on general purpose computation. Now I want to investigate what happens if
we win it. That's the civil war over general purpose
computation. Let's stipulate that we have a victory for
the "freedom side." It means that we have computers where owners
always know what was running on them, because the computers would faithfully report the
hash and the associated signatures for any bootloaders they find, and control over what
was running on computer's ghost to you, because the computers would allow their owners to
specify who was allowed to sign their bootloaders, operating systems, and so on. There are two arguments that we can make in
favor of this victory, why this victory would be a good one. The first one is a human rights argument. If your world is made of computers, then designing
computers to override their owner's decisions has significant human rights implications. Today, there are people who worry that the
Iranian government might demand import controls, so that all the computers that come in have
UFE-style bootlocker that only boots operating systems that have lawful interception back
doors built in. You can move the spying right to the edge,
to the user of the computer, the owner of the computer. But tomorrow, it may be that I live in the
UK, and it may be that our Home Secretary says, "If the NHS gives you a cochlear implant,
it has to intercept and report all the extremist speech it hears." The human rights stuff is easy to understand. The second argument comes from property rights. The doctrine of first sale is a very important
piece of law. It says once you buy something, you own it. You should have the freedom to do anything
you want to it, even if it gores the ox of the person who sold it to you. DRM opponents like me, we love the slogan,
"You bought it, you own it." Property rights are an incredibly powerful
argument to have on your side, anywhere. But they're especially powerful to have on
your side in a nerd fight, because you can't swing a cat in Silicon Valley without hitting
someone who thinks that property rights are an important way of solving most social problems. But it's not just nerd fights. Copyfighters get really pissed off about the
term "intellectual property," because property is also a really good way to win arguments
in policy circles. Before the term "intellectual property" came
into prominence, we had other terms like "creators' monopoly". It's very hard to go to a regulator or a lawmaker
and say, "My monopoly isn't large enough", but going and saying, "My property rights
are being not respected enough or need to be expanded so that I can make sure that they're
policed adequately," that's a very powerful argument to have on your hand. That's where the civil war part comes in. Human rights and property rights both demand
that computers not be designed for remote control by corporations or governments. Owners be allowed to specify their OS and
the programs running on them to freely choose that nub of certainty in the void that allows
them to build their whole stable edifice of certainty on. Now, remember that security is relative. You are secured from attacks on your ability
to freely use your music if you can control your computing environment. But, if you can control your computing environment,
the Recording Industry Association of America is now vulnerable to attacks on their ability
to rent you music on a single-use basis. We have this notion of streaming, this consensus
hallucination that there's a difference between a stream and a download, as though there's
some means of transmitting a stream of bits to someone's computer without actually having
them download that stream of bits, like the internet is made of mirrors and speaking tubes. [laughter] We say "Stream", we mean "I think
that your receiving software doesn't have a 'save as' button." Now, if you get to choose the nub from which
the scaffold dangles, you get control and power to secure yourself against people who
attack your interests. If the Recording Industry Association of America,
or the government, or Monsanto get to choose the nub, then they get control and the power
to secure themselves against you. So we all agree that at the very least, owners
should control what runs on their computers, or I'll ask you to stipulate that. Now, what about users of computers? Users of computers don't always have the same
interests as the owners of computers. Increasingly, we will be users of computers
that we don't own. Where you come down on the conflict between
owners and users of computers, I think, is going to end up being one of the most important
both technological and moral questions of the coming decades. There's no easy answer I have, no bright line,
for when users or owners should trump one another when it comes to computers. Let's start with a position I'll call "property
maximalism": "If I own my computer, I should have the absolute right to dictate terms of
use to anyone who wants to use it. If you don't like it, find someone else's
computer to use. This one's mine. I set the rules." How would that work in practice? Well, you got some combination of an initialization
routine where you set the root of trust, tamper evidence, law, and physical control. For example, you turn on your computer for
the first time, and you initialize a good secret password, possibly signed by your private
key, and without that key, no one is allowed to change the list of trusted parties who
are allowed to sign your bootloader. We can make it against the law to subvert
this for the purpose of taking control away from the owner. That makes writing malware that hijacks your
computer extra special, super duper illegal, but it also makes stealthy DRM installation
even more illegal. We can design the TPM so that if you remove
it, or tamper with it, it's really obvious. You give it a fragile housing, so that when
it's changed out, you can tell at a glance that it's happened. Then, if you still trust physical locks, you
can put it under lock and key, too. Now, I can see a lot of benefits to this,
but there are unquestionably some downsides to giving owners absolute control over their
computers. One wedge issue is probably going to be a
self-driving car. There's a lot of these around already. They come out of places like this and other
places. It's easy to understand, on the one hand,
why self-driving cars would be insanely great. We are terrible drivers. Cars totally kill the shit out of us. [laughter] They are the number 1 cause of
death in America for people aged 5-34. I saw my friend, Katherine [indistinct], last
night. She pointed out that it's also the number
1 way for humans to kill other humans. If you kill another person in your life, you're
almost certainly going to do it with a car. I've been hit by a car. I've also cracked up a car. I'm willing to stipulate that humans have
no business driving. It's also easy to understand how we might
be nervous about the prospect of people homebrewing their own self-driving car firmware. On the one hand, we do want the sourcecode
for these cars to be open, public, and subject to scrutiny so that defects can be discovered,
so that hidden features that may act against their owners' or users' interests can be quickly
found out. You'd want to know it if there's a kill switch
built in. You'd want to know it if your car secretly
drives you past more McDonalds when the kids are in the back seat. It's going to be plausible, I think, to say,
"Cars are safer if they have a locked bootloader and if that bootloader only runs a firmware
that's been signed by the Department of Motor Vehicles or by the FTC”. But now we're back to you, whether you get
to decide whether your computer is running the software that you want it to run. Now, there are two problems with this solution,
the solution of giving the state a veto over your self-driving car. The first one is that it won't work. As the copyright wars have shown us, firmware
locks aren't effective against dedicated attackers. People who want to sow mayhem with custom
firmware will be able to. We need a security model that doesn't believe
that all the other cars on the road are going to be well-behaved. If that's our security model, then we are
all dead meat. Self-driving cars must be conservative in
their approach to their own conduct, and liberal in conduct they expect from others, a venerable
principle familiar to people who work in computers, and also, the advice that you got on your
first day of driver's ed. And it remains good advice today. Now, the second problem with this is that
it invites some pretty sticky parallels. Do you remember the information superhighway? Now, if we can justify securing physical roads
by demanding that the state or a state-like entity gets to certify the firmware on our
cars, how would we articulate a policy explaining why the devices on our equally vital virtual
roads, our information superhighways, shouldn't also be locked with comparable firmware locks
for PCs, phones, tablets, and other devices? After all, we have a general-purpose network
now. That means that MRIs, space-ships, and air-traffic
control systems share the information superhighway with Game Boys, Arduino-linked fart machines,
and the dodgy voyeur cams sold by spammers from the Pearl River Delta. In addition to that, you're going to have
more wedge issues. You'll have things like avionics and power-station
automation. These are a lot trickier. If the FAA mandates a certain firmware for
a 747, it's probably going to want those 747s designed so that the FAA and the FAA alone
gets to choose what runs on it. Just as the Nuclear Regulatory Commission
is going to want the final say on the firmware for a reactor pile. This may be a problem for the same reason
that a ban on modifying firmware in self-driving cars is. Once you start saying it's the place of government
to sign and certify firmware on computers that they don't own, it invites people to
find other computers that they should send the firmware for. But on the other hand, cars and nukes exist
in a completely different regulatory framework to most of the other computers we use. Or rather, planes and nukes. Remember, a 747 is just a Solaris box. A nuke is just a specialized computer as well,
with a particularly exotic housing. It may be that since these things already
exist in this regulatory regime where they have no-notice inspection and so on, that
adding signed firmware locks is not going to be something that invites comparisons to
all the other computers in the world. But there's a bigger problem with owner control. What about people who use computers, but don't
own them? This is not a group of people that the IT
industry as a whole has a lot of sympathy for. We spent an enormous amount of energy as a
group, devoting ourselves to stopping non-users– or non-owners from harming owners. Users can do things like inadvertently break
the computers they're using, they download menu-bars, they type random shit they find
on the Internet into terminals, they plug malware-infected USB sticks into their computers,
they disable the firewalls, they install plugins or add repositories or add certificates to
their machine's root of trust, they punch holes in the network perimeter by accident,
and they accidentally cross-connect networks that are absolutely, positively not supposed
to be cross-connected. We also try to stop users from doing deliberately
bad things, like installing keyloggers and spyware to attack future users, misappropriating
secrets, snooping on network traffic, deliberately breaking their machines, deliberately punching
holes in the network perimeter, deliberately disabling their firewalls, deliberately interconnecting
networks that are supposed to remain secret– separate, rather. There's a kind of symmetry here. DRM and its cousins are deployed by people
who believe that you can't and shouldn't be trusted to run the computer you that you want
on your own computer. IT systems are deployed by computer owners
who believe that computer users can't and shouldn't be trusted to set policy on the
computers that they use. Now, as a former systems administrator and
a former CIO, I'm not going to pretend that users aren't a terrible challenge. But I think that there are good reasons to
treat users as having rights to set policy on the computers that they don't own. Let's start with the business case, because
I think that's the easy one to make. When we demand freedom for owners, we do so
for lots of reasons, but one of them is the possibility that programmers won't have anticipated
all the contingencies that their code might run up against. There may be a day where the code says no
and the owner needs to say yes. Owners sometimes possess local situational
awareness that can't be captured in nested "if-then" statements, no matter how deeply
you nest them. This is where communism and libertarianism
both converge. This guy, Hayek, thought that expertise was
very diffuse, and that you were more likely to find the situational awareness necessary
for good decision making very close to the decision itself. Devolution gave you better results than centralization. And then there was this guy, Marx, who believed
in the legitimacy of workers' claims over their working environment, saying that the
contribution of labor was just as important as the contribution of capital, and demanding
that workers be treated as the rightful "owners" of their workplace, with the power to set
policy. For totally opposite reasons, they both believed
that the people at the coal-face should have the first cut at running the operation. The death of mainframes was attended by an
awful lot of Sturm und Drang and hand-wringing and concern over users and what they were
going to do to the enterprise. In those days, users were even more constrained
than they are today. They could only see the screens the mainframe
let them see, and only undertake the operations the mainframe was programmed to let them undertake. When the PC and Visicalc and Lotus 1-2-3 appeared,
employees risked being fired by bringing these machines into their offices, or bringing home
office data to use with these machines. They did this because they had a computing
need that couldn't be met within the constraints set by their employer and its IT department,
and because they didn't think that the legitimacy of their request would be recognized. The standard response to a request from an
employee to do something that the IT department doesn't like is one or more of: "A regulatory
compliance prohibits you doing the thing that you think will help you do your job better"
or "If you do your job that way, we won't know if you're doing it right" or "You only
think you want to do your job that way" or "It's impossible to make a computer that works
the way that you think it does" or "Corporate policy prohibits you doing it." Now, these may be true, although sometimes
they aren't. And even when they are, they're the kind of
"soft truths" that we pay bright young things millions in VC money to try to falsify, while
if you're a middle-aged admin assistant, you merely get written up by HR for doing the
same thing. The personal computer arrived in the enterprise
through the back door, over the objections of the IT department, without the knowledge
of management, at the risk of censure and termination. It made the companies that fought it trillions. The reason that giving workers more powerful,
more flexible tools was good for firms is that people are generally smart, and they
generally want to do their jobs, and because they know stuff that their bosses don't know. As an owner, you don't want the devices you
buy locked, because you might want to do something the designer didn't anticipate. And employees don't want the devices that
they use all day locked, because they might want to do something that their bosses didn't
anticipate. This is the soul of Hayekism: that we're smarter
at the edge than we are in the middle. The business world pays a lot of lip service
to Hayek's 1940s ideas about free markets. But when it comes to freedom within the companies
they run, they're stuck a good 50 years earlier, mired in the ideology of Frederick Winslow
Taylor and his notions of "scientific management": The idea that workers are just particularly
unreliable kinds of machines whose movements and actions should be scripted and constrained
by all-knowing management consultants, who would work with the equally wise company bosses
to find the one true way to do their jobs. In other words, the exact same ideology that
let Toyota cream all three of Detroit's big automakers during the 1980s. Letting enterprise users do the stuff that
they think will allow them to make more money for their employers often results in making
more money for their employers. For the record, scientific management is about
as scientific as trepanation and Myers-Briggs tests. [laughter]
The business case for user rights is a good one, but I really wanted to just get it out
of the way so we could dig into the real meat of the argument: the human rights case. This may seem a little weird on its face,
but bear with me. This is a guy named Hugh Herr, and I saw him
give a talk earlier this year. He's the Director of the Biomechatronics lab
at The MIT Media Lab. You may have seen him do a TED talk. There's a bunch of them on YouTube. It's electrifying to see him give these talks. You should go and watch one after this. He starts out with a bunch of slides of cool
prostheses his lab has cooked up. There's legs and feet, and hands and arms,
and even this awesome thing that if you have untreatable clinical depression, they stick
your head in a magnet, and the magnet suppresses activity in the parts of your brains that
are overreacting, and people with untreatable clinical depression become treatable. It changes their lives and brings them from
the brink of suicide back into a happy place. Then he shows this slide of him, and he's
climbing up a mountain. You can see he's clinging to the mountain
like a gecko. He's super buff. He clearly knows what he's about. And he doesn't have any legs, he just has
these awesome mountain climbing prostheses. Now, he's been standing at a podium like this. In fact, he does it wearing a little lav mic
or an ear mic, and he walks up and down while he's giving the talk. Then he stops and he says, "Oh yeah, didn't
I mention? I'm robot from the knee down. I lost my legs to frostbite. These are my legs." Then he does this cool thing. He runs up and down the stage, jumping up
and down like a mountain goat. It's the coolest thing you've ever seen. When I saw him give this talk, the first person
who asked a question stuck their hand up and said, "So what do those cost?" He named a price that would buy you a brownstone
in Manhattan or a nice terraced Victorian in Zone One. A pretty penny. The second question that was asked was, "Whose
going to be able to afford these?" And he said, "Well, of course, everybody. If it's a choice between owning legs and owning
a house, you'll take the 40 year mortgage on your legs." Which is by way of asking you to consider
the possibility that there are going to be people, potentially a lot of people, potentially
you someday– remember, we are only temporarily able-bodied– who are "users" of computers
that they don't own, where those computers are going to be parts of their bodies. I think that most of the tech world should
be able to understand why you, as the owner of your cochlear implant, should be legally
allowed to choose the firmware that runs on it. After all, when you own a device that is surgically
implanted in your skull, it makes a lot of sense that you have the freedom to change
software vendors. Maybe the company that made your implant had
the best algorithm for signal processing at the time that they were stuck in your head,
but what if a competitor patents a superior algorithm next year? Should you be doomed to inferior hearing for
the rest of your life or the 20 year span of the patent, whichever comes first? This is a problem that can't be overcome merely
by escrowing the code of important embedded systems. That might help you if the company goes bust. It also can't be helped by code publication,
the thing you would want anyway for your cochlear implant, just to make sure that it was good
code. This is a problem that you can only overcome
by having the unambiguous right to change the software, even if the company that made
your implant requires you not to. So that helps owners. But what about users? Consider the following scenario: you are a
minor child and you have deeply religious parents who pay for your cochlear implants. They ask for the software that makes it impossible
for you to hear blasphemy. You are broke, and a commercial payday loan
company wants to sell you ad-supported implants that listen in on your conversations and insert
contextual ads that trigger discussions about the brands you love. Or your government is willing to install cochlear
implants, but they want to archive everything you hear and review it without your knowledge
or consent. It sounds far-fetched, but remember, the Canadian
border agency, just a few months ago, had to be slapped down from its plan to put hidden
microphones through the entirety of all of the country's airports, so they could listen
in on and record all the conversations taking place in every airport in real time and later. Will the Iranian government, will the Chinese
government, will other repressive governments take advantage of this if they get the chance? Speaking of Iran and China, there are plenty
of human rights activists who believe that boot-locking will be the start of a human
rights disaster. It's no secret that there are high-tech companies
who have been happy to build "lawful intercept" back-doors into their equipment to allow for
warrantless, secret access to their communications. These backdoors are now standard, so even
if your country doesn't want the capability, it's still there. In Greece, for example, there is no lawful
interception requirement, but of course, all the telecoms equipment they buy is made for
jurisdictions in which there is. They just don't turn on lawful intercept. During the 2004/5 Olympics bid, someone, we
don't know who, broke into the Greek telecom switches, turned on the lawful intercept capability,
listened in on the conversations at the highest levels in government, turned it off again,
and walked away. It's only because they didn't erase the logs
that we know about it. Surveillance in the middle of the network
is nowhere near as interesting as surveillance at the edge of the network. As the ghosts of Misters Hayek and Marx will
tell you, there's a lot of interesting stuff happening at the coal-face that never makes
it back to the central office. Even so-called "democratic" governments know
this. This is why, for example, last year, the government
of Bavaria started illegally installing the "Bundestrojaner", or the state-trojan, on
people's computers, when they were of interest, something that allowed them to access cameras,
microphones, hard drives, and so on. And of course, it was very badly written,
so it allowed anyone else to do that, too. Once you were infected, you were infected
for everybody. It's a safe bet that the totalitarian governments
will happily take advantage of boot-locking and move surveillance right into the box. You may not import a computer into Iran unless
you limit its trust-model so that it only boots up lawful intercept operating systems. Now, assume that we get an owner-controls
model, wherein the first person to use the machine gets to initialize its root of trust. You still get the problem, because in Iran,
every computer that comes into the country is first opened by the customs authority,
who installs a root of trust that's run by the government. Because it's tamper-evident, even if you figure
out how to override it, the next time a snitch or a policeman looks at your computer, they
can tell that you've been up to something naughty and locking the government out of
your computer. Of course, repressive states aren't the only
people who like this. There are four major customers for the existing
complexive censorware, spyware, and lockware. There's repressive governments, there's large
corporations, there's schools, and helicopter parents. That is to say, the technical needs of protective
parents, school systems, and enterprises are convergent with the governments of Syria and
China. I don't mean that they have the same ideological
grounds, but they have awfully similar technological means to attain their ends. We are very forgiving of any institution that
pursues those ends, provided that they're doing so in order to protect either shareholders
or children. For example, you may remember that there was
widespread indignation, from all sides, when it was revealed that employers were asking
prospective employees to turn over their Facebook login credentials. Employers argued that they needed to be able
to review your list of friends, what you said to them, and what you did with them, in order
to make sure that you didn't have any skeletons in your closet that would compromise your
ability to work for them. Facebook logins were fast on their way to
becoming the workplace urine test of the 21st century. A means of ensuring that your private life
didn't have any unsavory secrets lurking in it, secrets that might compromise your work
life. Now, the country wasn't buying this. From Senate hearings to op-eds, the country
rose up against this practice. But no one seemed to mind that many employers
routinely insert their own intermediate keys into their employees' devices– their phones,
their tablets, and their computers– that allows them to spy on their employees' Internet
traffic, even when it's "secure", with a little lock showing in their browser. This gives your employer access to all the
sensitive sites you access while you're on the job, from your union's message board to
your bank website to your Gmail to your HMO or private repository managed by your doctor's
office to Facebook. Now, there's a wide consensus that this is
okay because the laptop, the phone, and the tablet that your employer issues to you are
not your property. They are company property. And yet, the reason that employers give us
these mobile devices is because there is no longer any meaningful distinction between
home and work, between personal life and professional life. Corporate sociologists who study the way that
we use our devices have found consistently that employees are not capable of maintaining
strict boundaries between "work" and "personal" accounts and their devices. And of course, in America, we have the land
of the 55+ hour work week, where few professionals take any meaningful vacation time, and when
they do get away for a day or two, they bring their Blackberry along. Even in the old, predigital, traditional workplace,
we recognized that workers had human rights. We didn't put cameras in the toilets to curtail
employee theft. If your spouse came by the office on your
lunch break and the two of you went into the parking lot so that she or he could tell you
that the doctor said the cancer was terminal, you would be rightfully furious to discover
that your employer had been listening in on the conversation with a hidden mic and watching
through a hidden camera. But if you take your laptop on your lunch
break and access Facebook and discover that your spouse has left you a message saying
that the cancer is terminal, you're supposed to be okay with that because the laptop is
your employer's property. There are plenty of instances in which not
just peons, but important and powerful people, not kids and corporate employees, are going
to find themselves users of computers that they don't own. Every car-rental agency would love to be able
to lo-jack the car they rent to you. Remember, cars are just computers you put
your body into. They'd also like to log all the places you've
been for "marketing" purposes and analytics. And there's lots of money to be made in finagling
the way your GPS roots you around to make sure that you drive past certain billboards. But in general, the poorer and the younger
you are, the more likely you are to be a tenant farmer in some feudal lord's computational
land. The more likely it'll be that your legs will
cease to walk if you get behind on payments on them. That means that any thug who buys your debts
from a payday lender could literally — and legally — threaten to take away your legs
(or your eyes, or your ears, or your arms, or your insulin, or your pacemaker) if you
don't come up with the next payment. Before, I discussed how an owner override
might work. You have some kind of combination of physical
access-control and tamper-evidence, designed to give owners of computers the power to know
and control what bootloader and OS was running on their machine. How will user-override work? I think an effective user-override has to
leave the underlying computer and its programs intact, so that when the owner takes it back,
she can be sure that it was in the state she believed it was in when she handed it over. In other words, we need to protect users from
owners and owners from users, as well as users from other users. Here's one model for that. This is the hypothetical. I'm not suggesting we do this, I'm suggesting
it by way of example. Imagine that there is a bootloader that can
reliably and accurately report on the kernels and OSs it finds on your computer. This is a prerequisite for all the scenarios
we've discussed: the one in which the state controls your computer, the one in which the
owner controls your computer, and the one in which users may be able to control their
computers some of the time. Now, give the bootloader the power to suspend
any running operating system to disk, encrypting all its thread and parking them, and the power
to select another operating system from the network or an external drive. So I walk into an Internet cafe, and there's
an OS running that I can verify. It has a lawful interception back-door for
the police, it stores all my keystrokes. It stores all my files, all my screens in
an encrypted blob that the state can decrypt. Now I'm an attorney, or a doctor, or a corporate
executive, or just a human being who doesn't want all of his communications being available
to anyone who can bribe a cop. So I do some kind of three-finger salute on
my keyboard. It drops into a minimal bootloader shell,
and I can give the net-address of an alternative operating system, or insert a thumbdrive. Now the cafe owner's operating system gets
parked. I can't see inside it. But the bootloader can assure me that it's
dormant and not spying on me as my operating system fires up. When it's done, all my working files are trashed,
and the bootloader confirms it. Not just because this keeps the computer's
owner from spying on me, but it keeps me from spying on the computer's owner. Now, there will be technological means of
subverting this. You could make a thing that looks like the
bootloader but isn't the bootloader. But there is a world of difference between
starting from a design spec that aims to protect users from owners and vice-versa, and one
that says that users should always be subservient to owners. Now, human rights and property rights often
come into conflict with one another. For example, landlords aren't allowed to enter
your hotel without adequate notice– or your home without adequate notice. In many places, the hotelier can't throw you
out if you keep paying for your room, even if you overstay your reservation. Repo men can't take away your car without
serving you a notice and giving you the opportunity to dispute it. When these laws are streamlined, we get all
kinds of bad effects. Robo-signers taking away people's houses even
though they've paid their mortgage or don't even have a mortgage. The potential for abuse in a world where everything
is made of computers is, of course, much greater. Your car might drive itself to the repo yard. Or your high-rise apartment building may switch
off its elevators and its climate systems, stranding thousands of people until a disputed
license payment is settled. Now this has already happened with a parking
garage. Back in 2006, there was a 314-car Robotic
Parking model RPS1000 garage in Hoboken, New Jersey, whose owners believed that they were
up to date on their software license payments and whose vendor disagreed. So the vendor shut off the garage and took
314 cars hostage. The owner said that they were paid up, but
they paid again because what the hell else were they going to do? Now what will you do when your dispute with
a vendor means that you can go blind, or deaf, or lose the ability to walk, or become suicidally
depressed? The negotiating leverage that accrues to owners
over users in this scenario is total and terrifying. Users will be strongly incentivized to settle
quickly, rather than face the dreadful penalties that could be visited on them in the event
of a dispute. And when the owner of the device is the state
or a state-sized corporate actor, the potential for human rights abuses skyrockets. Now, this is not to say that owner override
is an unmitigated evil. There are lots of reasons why you might not
want users to override their computers. Think of a smart meter. Smart meters need to be able to turn down
your building's temperature by a couple of degrees, otherwise we have to keep using dirty
coal because it's the power source that we can raise and lower on demand. Now, that works best if users can't override
the meter on their wall. But what happens if there's a big freeze,
and a griefer or a crook or a government turns off your heat? What happens if the HVAC in your house is
cranked to 110 degrees during a heat-wave and you can't override it? Once we create a design norm of devices that
users can't override, how far does that end up creeping? Especially risky would be the use of owner
override to offer payday loan-style services to vulnerable people. If you can't afford artificial eyes for your
blind kid, we'll subsidize them, but you have to let us redirect your kid's visual focus
to sponsored toys and sugar-snacks when you go to the grocery store. But foreclosing on owner override probably
means that there will be poor people who won't get offers that they would get otherwise. I can lease you something, even if you're
a bad credit risk, if I know I can repossess it handily. But if your legs can decide to walk away to
the repo depot without your consent, you will be totally screwed the day that muggers, rapists,
griefers, and the secret police figure out how to hijack that facility. It gets even more complicated, of course,
because you're the user of many systems that you aren't– in the most transitory of ways:
the subway turnstile, the elevator, the blood-pressure cuff at the doctor's office, public buses
and airplanes. It's going to be hard to figure out how to
create "user overrides" that aren't nonsensical, although we can start by saying that "users"
are someone who are the sole user of a device for a meaningful amount of time, although
we'd then have to define "meaningful." This is not a problem I know how to solve. Unlike the War on General Purpose Computers,
the Civil War over computers seems to present a series of conundra without any obvious solutions,
at least, obvious to me. Which is why I'm talking about them to you. These problems are a long way off, and of
course, they'll only arise if we win the war on general purpose computers first. But come victory day, when we start planning
the constitutional congress for the new world, where regulating computers is acknowledged
as the wrong way to solve problems, let's not paper over this division between property
rights and human rights. This is the sort of division that, while it
festers, puts the most vulnerable people in our society in harm's way. Agreeing to disagree on this one is not good
enough. We need to start thinking now about the principles
we'll apply when the day comes. Because if we don't start now, it may be too
late. Thank you. [applause] So I've got some time for questions now. You don't have to ask me questions about this. You can ask me questions about books and stuff. >>Male #1: I have a question. So one thing usually people don't talk about
is why we should not allow people to inspect what we're doing. If you're not doing anything wrong, why do
you care? I've only seen an argument to this once, when
people said, basically, it violates a fundamental expectation of humans to be individuals rather
than part of a collective ant hive or ant colony of some sort. Now I wonder if you could speak to that. >>Doctorow: Yeah. I mean, there's– I think that that argument
starts by presupposing that everything private is secret, and everything secret is private. We say, "Oh, well, it's not a secret what
you're doing, so why do you need to keep it private?" But I can make a pretty good guess about what
you do when you go to the toilet. I'm pretty sure I knew what your parents did
to get you here. But it takes a pretty special kind of person
to want to do that in public. There are behaviors, and not nefarious ones–
In fact, some of the most important ones, the ones that, you know, are the origin of
all life and the reason you don't explode in a shower of poo, that we habitually do
in private, and that aren't the same when you have to do them in public, particularly
if you're coerced to doing them in public. The modern concept of privacy is pretty new,
but there are elements of our privacy that are quite old: the privacy of thought, the
privacy to make mistakes. I mean, remember this notion that if you want
to double your success rate, you triple your failure rate. It's very hard if you have to make all your
mistakes in public. You may– Does anyone here work on Blogger? I mean, before Blogger was a really big deal,
when it was a little deal, it was running on an NT box that Ev found somewhere. It went down all the time. And no one cared, because he wasn't in the
public eye. Now, you guys can't afford to experiment with
the Blogger backend the way Ev could. Ev could refactor his code altogether, take
it offline for two days, and then put it back on again. He was able to innovate really, really fast,
in a way that you guys can't, because you don't have privacy in what you do. What you're doing is public. If you've ever watched a kid play, and play
in a way that's sort of pushing at their boundaries, they do this thing where if they don't know
you're watching, they make a lot of mistakes, and they just keep pushing through them. But if they catch you secretly watching them
while they make mistakes, they put the thing away and they walk away from it. It just kills their play. As a father, it's the thing that breaks my
heart when I do it, because it's very tempting to look over at your kids when they're doing
something awesome and intense. But then, you humiliate them and you embarrass
them. So there's something about us that wants to
have vulnerable moments not take place in public, that wants to choose the moment of
disclosure. I think that that's– that doesn't change
just because we have Facebook, or just because we can track user behavior with 1×1 pixel
gifs. You know? Yeah? >>Male #2: You've published a lot under the
Creative Commons license. So I was curious, from the point of view of
someone who's incredibly cynical and just wants to make a living writing things, would
you advise it? Will it catch on? >>Doctorow: So if you want to make a living
writing things, I would advise you to stop trying, because [laughter] that's a bit like
saying, "I want to make a living buying lottery tickets." It's like–. That sounds like a great plan if you can find
the winning lottery tickets, but if you don't have a plan B for earning a living, you have
the wrong career. Writing is a very, very high-risk entrepreneurial
venture that almost everybody who tries it fails at. Some fraction of the people who try it succeed
using Creative Commons, and some fraction succeed without using Creative Commons, but
they're rounding errors against all the people who try to earn a living with writing. So for me, the reason to use Creative Commons
is not just commercial, although I think in my case, it enhances my commercial fortunes,
because people who get the book for free then go on to buy the book. That may not be true of everyone. There isn't a kind of global theory about
this. But there's two other dimensions to it. The first one is moral, and then the second
one is artistic. The moral case is that I copy all day long,
you copy, everybody copies. If it wasn't for mix-tapes, I would have been
a virgin until my mid twenties. You know. Copying is what we do. If I were 17 years old today, I would have
a giant hard drive and it would have a copy of everything. So I–. To pretend that when I copy, it's like part
of a legitimate, artistic adventure that allows you to assemble your influences and recall
them as you need them, but when you copy, you're just a thief, that's just dumb, right? And moreover, it leads to all these crazy
consequences, where we're talking about three strikes rules in New Zealand, for example,
where they're saying if you– I know that's an Australia shirt, but very close to Australia,
where they're saying– >>Male #2: We don't think very much of the
Kiwis. >>Doctorow: I understand. It's like Canadians and Americans. But they, you know, they're saying if you
are accused of three acts of copyright infringement, we take away your internet access and all
the stuff that comes with it, and that's partly being driven by people who say, "Well, if
you copy my stuff, it hurts my fortune." Being able to give my stuff away means that
I'm not part of the rubric for Draconian network policy. But then there's a third dimension, which
is the artistic dimension. It's the 21st century, and if you're making
art that you don't intend to have copied by people who like it, you're not making contemporary
art, because the realpolitik is if someone likes it, they'll just copy it. Right? I mean, we put DRM on ebooks. This is crazy. It's like they've never heard of typists. [laughter] It's like there have– It's like
they don't know that we live in the moment with the largest number of skilled typists
in the entire history of the world, you know? This is an amazing–. My grandmother was like a 75 word per minute
administrative assistant, and she was like a circus freak. Today, she's not even in the top quintile. Everybody can type. If you're making art that you don't intend
to have copied, you're not making contemporary art. That's cool. I mean, if you want to be the blacksmith at
Pioneer Village or reenact the Civil War, that's awesome. Go follow your weird. But I'm a science fiction writer, so I'm supposed
to make at least contemporary work, if not futuristic work. It gives me great satisfaction to allow it
to be copied. >>Male #3: So, a question about the average
user. So if somebody is sitting out watching this
broadcast on YouTube, and they think, "I want to be a part of the solution", it doesn't
feel like there's a great avenue for them to express the need for digital freedom other
than disobedience. Do you have any suggestions as to what people
might do? >>Doctorow: Well, the first talk, the 28C3
talk which is on YouTube, ends with a pretty good, compact call to action for people of
all stripes. If you're a hacker, get involved in things
like Free Software Foundation and policy stuff that the Free Software Foundation does, or
get involved with the Electronic Frontier Foundation. If you're a lawyer, join the Cooperating Attorneys
list for EFF and get involved in other groups like FSF and so on.org, or Nets Politique. Bits of freedom all over the world: we have
them. If you're an artist, use Creative Commons. And so on. I actually think that there's a lot of venues. Today, we have, between Defective By Design
and Fight for the Future, who led the SOPA/PIPA fight. We have so many different groups that are
doing really exciting things that need people to do everything from send an email to their
congressman at the right moment to design logos and packaging and write copy and put
the word out and blog and give talks to their school. If you're a student, you can join Students
for a Free Culture. That's all great. I think we don't have that answer for the
civil war thing yet. We don't. And we kind of—. And my point about the civil war is that we'll
get to the civil war pretty quickly after we win the war. We get to the thing where as soon as you give
owners total dominion over their computers, you immediately get to the moment where users
can't trivially change what's on their computers. I would be really nice if, as we sit here
advocating for owners having total control over their computers, that we start thinking
about when users should be able to change that. >>Male #4: So you gave some examples comparing
and contrasting physical devices versus electronic goods, and devices that are used by many users
and may be only one user. And you said, "Well, some of these policies,
there's analogies between physical and electronic that you should use, and some of them don't
really make sense. There's a discord." Are you saying that, overall, we should go
by case-by-case in order to examine: does this make sense? >>Doctorow: No. I mean, I hope that's not what we end up having
to do. I mean, that was the point at the end and
the beginning. I don't know how to solve this one. And it would suck if, basically, you solved
this with nested if-then loops. It would be really nice if we had a nice,
generalizable case that we could say, you know, "If it's not a nuclear power plant,
anything goes"? Or something else. Right? I mean, if every single thing that is Turing
Complete has a different set of rules for when users and owners get to control it, that's
going to be a really big rule book. So I would love to have a better one. I don't know what that is yet. Maybe, after I've given this talk for another
year or two, between all the feedback I get from people like you, I'll be able to propose
a solution. >>Male #5: It was interesting that you talked
about having a user override mode where you could change what the employer was doing and
come back. I don't know if you followed much of it, but
[beep] Chrome OS does that now. >>Doctorow: Yeah, I just heard that at the
weekend that there's a user override. And it's funny, because the first email that
got me thinking about this was Vint Cerf saying, "Why shouldn't Google be able to choose what
software runs on my Chromebook if they bought it?" >>Male #5: And we don't, and I work on that. I'll be happy to talk to you about that. And for the rest of you guys, I'm about two
months away from having a way that you can put your own keys on it, so you can sign your
own images and boot your own stuff. Right now, you have to turn the security off. But we're doing that, and I would love to
do more. >>Doctorow: I think that's a really cool model. I'm done. Of course, the really challenging thing is
going to be computers that don't have interfaces, like your legs. >>Male #6: So this is a little tangential,
but we are reaching the point where a lot of third parties can maintain public databases
of public information about you. So, like the American Credit Report is one
of the original settlers. So is there any legal theory that would give
you rights over the database? >>Doctorow: Is there a legal theory that gives
you rights over that database? I don't know, but I like the fact that you
said rights instead of property rights, because I think that we have started– We have the
best of intentions sometimes. We created property rights and facts, or property-like
rights and facts, about you that don't make any sense. Like, you know, the Well, which is very old,
and now endangered. Salon just put it up for sale, this online
conferencing system. It's motto is "Yoyow", "You own your own words." And that sounds like a really cool idea, but
it has all these weird fraud things, like if we're in a conversation and I quote something
you said, do you get to tell me to not quote it? I mean, this is a contract, not fair use. This is what our contract says. And the European data norms are starting to
move towards ownership of your personal information. But what does it mean to own your phone number? You know? Does that mean that if your phone number happens
to contain the first seven digits of pi that other people can be enjoined from writing
pi? And it's funny because we do actually have
ways of expressing value about things that aren't property that we may be able to bring
in here. We talked about interests a lot. My daughter is not my property, she's pretty
important to me. And if you kidnap her, the charge isn't theft. But we can acknowledge that my daughter has
an interest in herself, that I have an interest in her, that my wife has an interest in her,
that her grandparents have an interest in her, that the state has an interest in her,
that her friends have an interest in her. That's what it means to be a person in a society. We need to start, I think, talking about information
that way. It's crazy, I think, to talk about things
like phone numbers or your address. This is where I think, you know, even though
I'm a privacy advocate, I think the Germans were crazy about saying you own the likeness
of the front of your house. I mean, that's just dumb to me. 'Cause it means that, like, as you move through
time and space with prostheses that record the world, you can't record your neighbor's
house. You can't record your kid's first run down
the street on her bicycle without the training wheels because she rides past your neighbor's
house and they own the likeness of their house. Right? That's just dumb. We need to be able to express–. And property is a bad organizing metaphor
for a thing that a million people own. Right? You end up with, like, shareholder corporations
or, you know, there's this whole Spider Robinson aphorism when 700 people share an apple, no
one benefits, especially the apple. You know. [laughter] That's true of physical, rivalrous
property, but non-rivalrous information doesn't have that characteristic. We still may want to give exclusive access
or semi-exclusive access to certain parties. Like, the image of your colonoscopy may be
something between you and your doctor. But to call it your property is the wrong
thing. It doesn't organize well that way. I don't know what does, but I know what doesn't
work. >>Male #6: No, but the point I'm making is
that, let's say ten years from now, somebody could run a background check on your without
your opting into it? >>Doctorow: They can already do that. I mean, they already can. >>Male #6: Okay. >>Doctorow: Yeah, I mean, it would be– So
Lessig talks about four ways of organizing– of regulation. He talks about law code markets and norms. So we don't have a lot of code to help people
protect their privacy. Like, when you fire up your laptop, it doesn't–
So here's an example. If you had a browser that, every time you
turned it on, loaded the– checked to see whether it was being asked to load the Google
Analytics JavaScript, and suppressed it, but implemented all the features, all the libraries
locally so that pages didn't break. That would be code out of the box that treated–
that defaulted to treating privacy as though it's valuable. And so now, if Google wants to get your private
information from you, information about where you are on the internet, they have to offer
something of value to you that is inextricably linked, because that is extricably linked,
right? You can– We can, in fact, conceptually understand
how you divide it. So, like, if my Android phone, when I installed
an app like my daughter's Connect the Dots app, it said, "In order to use this app, you
need to tell us where you are all the time." If it let me say, "Tell this programmer where
I am all the time, but make it up", then the program would actually have to devise an offer
where where I am was actually a piece of using it. So I use another Android app all the time
called Hailo, for hailing black cabs in London, which are a pain in the ass to get when it's
raining and so on. And Hailo knows where I am all the time, and
so Hailo has an offer where if everybody else couldn't get my location trivially just by,
like, getting me to download a Connect the Dots app for my kid, Hailo would be sitting
on a giant asset. And you'd have real privacy markets. Like right now, we have this idea that we
know what your privacy is worth, and it's worth nothing because you trade it for zero. But you don't have the option of not trading
it for zero. So you could imagine–. One way that you could stop people from being
able to do background checks on you really trivially is if all the devices that you have
didn't hemorrhage information about you all the time, as though it had no worth. >>Male #7: Okay, change of gears to a fluffy,
lighthearted thought experiment. >>Doctorow: Sure. >>[Male #8]: Since you mention the Norwegian
script kiddies, imagine the Marcus Yallow Memorial Pentathlon. Which countries take gold, silver, bronze
there? >>Doctorow: Oh, wow. I'd like to think– I mean, without being
ideological about their governments and just thinking about their track records, I would
think that you'd get the four brick countries plus Israel, probably. Brazil, Russia, India, China, Israel. Without endorsing or condemning any of those
governments. >>Male #8: All right. Thanks for coming. It seems like you're looking for sort of an
overall rule for what people can and can't do with their devices. But I'm afraid it's going to end up just a
whole pile of special cases, kinda the way it is now. Like, if you look at your car, what hardware
and software you can do, some things are legal, some things violate smog laws, some things
violate safety laws, some things will be charged with criminal negligence if something goes
wrong, some things you'll get sued if something goes wrong. >>Doctorow: Are you saying that if you modify
the firmware, or if you modify the firmware and then something bad happens? >>Male #8: Well, you know, with the smog law,
just modifying the firmware, I believe, is illegal. >>Doctorow: Is that right? I didn't know that. >>Male #8]: Well, I think in California. You know, there's other things you can do
that– If I decide to reprogram my brake system and I crash into something, I'm likely to
get either sued or go to jail. But other parts of the system, you know, it's
probably okay for me to redo– if I want to put in a new engine, but in new shock absorbers. So it's– I think these issues apply to both
the hardware and the software and firmware. >>Doctorow: So, it makes a certain amount
of sense to me. I think you're describing after the fact,
largely, modulo this question about whether changing the way your car is smogged gets
you in trouble or doesn't. I think, mostly, you're describing after the
fact stuff. So in the same way that if I program my nuclear
power plant so that it melts down, I'm held liable for having written bad code. Or, if I program my software to find radio
so it turns into a spark app generator and blows all the RF in my region, again, I have
done something bad and I'm punished for that, but it's not against the law to write my own
software to find radio code. You can check code in and out of GNU radio
on GitHub without breaking the law. If you use that code in a way that ends up
breaking something, that may be illegal. And that's kinda what I'm talking about here. It may be that users take control of their
legs to run up to someone and kick them in the face. And I don't think that– I think that writing
code that lets you take over your legs is good. I think that, having taking over your legs,
to kick someone in the face is bad. And I think that we can punish the one without
punishing the other. >>Male #8: Okay, that seems fair. >>Doctorow: Yeah. >>Male #9: You touched earlier on the concept
of an illegal number, which is something that I've thought about a lot, because all information
can be a number. Which raises the question, which I think is
central to all of this, of where you draw the line. >>Doctorow: Sure. >>Male #9: And obviously, to anyone who has
studied any kind of mathematics, there is no line in numbers. So the question is: why is there a line in
anything? >>Doctorow: Right. >>Male #9: In essence, why do you believe
that this problem is at all soluble? >>Doctorow: [inhales] [laughter] Let me find
my illegal number here. There we go. Yeah. That's a really good question. You're right. Everything can be encoded as a number. I mean, now we're getting into girdle and
incompleteness and whether numbers are special. >>Male #9: That was just an example, though,
of why it seems a priori that this is likely, technically, not soluble. So the question is: why? >>Doctorow: So I think it's soluble in time
scales. It's not soluble in infinity. So, for example, we may say that–. So today, we have a bunch of rules about locking
and unlocking that are largely governed by the copyright office, because the relevant
law is the DMCA and anti circumvention rules. And so you may have heard that in the triannual
review, it was made legal to unlock phones and tablets, iPhones and tablets, and also
to unlock phones so that they can switch carriers. That's a thing that works for now. It won't work against more robust bootlockers
and it doesn't help certain classes of users. But, I mean, I don't think we pass technology
laws that are supposed to last through the ages. I think we pass technology laws that are supposed
to last, we hope, through the half life of the technology. You're right that there will come a time when
the rainbow table of all numbers exists, and all possible decodings for them exist. I mean, it may occupy all the hydrogen atoms
in several parallel universes to ours. But it's at least within the realm of contemplation. But that doesn't mean that between now and
then we should try not to have any rules about how numbers are used. You know. I mean, you might say, "Well, the specifications
necessary to get your AR15 to go full auto can be expressed as a series of numbers." That's a bad example, because you don't suppress
the rule about the specification. But, like, the 3D meshes. That currently exists as a 3D mesh. I think that was uploaded to Thingiverse. I think they've taken it down. That converts AR15s from semi to full. I may not agree with regulating that, but
I wouldn't disagree with it on the grounds that it's impossible. Right? Like, that it's impossible to say– telling
people– giving people AR15 automatic modification kits can't be made to work, or shouldn't be
made to work, because those numbers might also be poetry or something. Like, it just seems like, although that's
true, it's true in some sense that the law can safely ignore for a while. Does that sound right? It's hard. >>Male #9: It's good enough that I don't want
to take up any more time following up. Thank you. >>Doctorow: Okay. Thanks. [applause] [whistling]

33 Comments

  1. copy paste said:

    Thanks Overlord for giving me what I crave.

    June 27, 2019
    Reply
  2. Denise Ward said:

    Here we go again! Men running things without getting other perspectives (from women for example) I mean it's not like men have taken humanity to any great heights. Sure we have computers, whoopee, I love them too, but there are still millions starving and homeless and unable to access health care or even the internet. But none of the "big guys" care about that. They still strive to explore space even though their fellow humans are suffering in massive numbers. You'd think they'd want to lift them first, use these extra untapped abilities, and make the human race soar into space and computer technology. But no, it's all just a drive for more paranoia – spying and manipulating. How to use computers to stop "the bad guys"? Well here's a novel suggestion – how about making everything on computers open and freely available? None of you have given that any thought, it hasn't even entered your heads! And so now men are going to start another war zone, in cyberspace. Oh goodie, more paranoia. So give it a thought – what if all information were open to the public, how much easier would life be then? What would be the unintended consequences, etc? So much energy is used to maintain privacy but there is always someone now in the digital age who can breach privacy walls. So what if we all just understood that nothing is private, that if we wanted to we could find out anything we wanted to? What kind of a world would that create? I think a good one. It would also keep psychopaths at bay as living in the open is something they shun. The idea of living in the open would then of course, put pressure on the idea of copyrights and property rights, which in the digital age, make no sense to continue with.

    June 27, 2019
    Reply
  3. Dawid Cz said:

    so basically all day

    June 27, 2019
    Reply
  4. Can’t Relate said:

    Banana purée 2.0

    June 27, 2019
    Reply
  5. Kevin Sterns said:

    His description of a hostile insecure product that runs secret software and spies on the user is a brilliant description of Windows 10… posted 3 years before Windows 10 was released.

    June 27, 2019
    Reply
  6. Kirstine Termansen said:

    Hands 8ff

    June 27, 2019
    Reply
  7. Kirstine Termansen said:

    Go play with you own creative and talent management 📒 me seed this us smsll
    Message hacker notes hmmmmmm

    June 27, 2019
    Reply
  8. Kirstine Termansen said:

    Go play with China 🇨🇳

    June 27, 2019
    Reply
  9. Kirstine Termansen said:

    Missing Children interpol and missing Children FBI Alex and dad return the Child
    Data spionage einterteinment and design is still illegal under federal international law USA

    Plus phadofil net is the most important to solve

    June 27, 2019
    Reply
  10. Kirstine Termansen said:

    Usa government trashed by danish wire fraud and authority

    June 27, 2019
    Reply
  11. Kirstine Termansen said:

    Crap industry spying is industry based greed
    Nomatter media or 📺 company and block job, mails, voice messages reply

    Sex abuse used as subprespre,

    June 27, 2019
    Reply
  12. Re Trend said:

    i want to see RED

    June 27, 2019
    Reply
  13. AOX OS for Netjobs said:

    You don´t need civil war. Design a Racoh Box! Peaceful Salutations.

    June 27, 2019
    Reply
  14. atonewiththedust said:

    Is this video safe for the goyim eyes?

    June 27, 2019
    Reply
  15. aAAWMdN8 said:

    Funny how this is a talk at Google, when Google is on the wrong side of the coming war, being one of the supporters of the new W3C DRM standard.

    June 27, 2019
    Reply
  16. bmcgmusic said:

    gumdrop cuckold

    June 27, 2019
    Reply
  17. AlyTamale said:

    too many indians at google

    June 27, 2019
    Reply
  18. Zack Bennett said:

    The quickest way to shut down the "Well I don't have anything to hide" argument is to ask them to pull down their pants and underwear. I didn't think of that tactic, but I've used it successfully to illustrate the privacy point to several ignorants.

    June 27, 2019
    Reply
  19. Matt L said:

    God damn!

    June 27, 2019
    Reply
  20. Colin B said:

    sigh
    ok google

    June 27, 2019
    Reply
  21. Harry LGG Gardner said:

    27:50
    'and then there was this gay, … this guy Marxc'

    June 27, 2019
    Reply
  22. Khai E said:

    here cuz of Hyde

    June 27, 2019
    Reply
  23. blah said:

    Vanilla Puff 9.1

    June 27, 2019
    Reply
  24. hellrazor117 said:

    what a nerd

    probably gonna marry a computer

    June 27, 2019
    Reply
  25. dude said:

    red mode activated

    June 27, 2019
    Reply
  26. Jam2Evos said:

    okay Google

    June 27, 2019
    Reply
  27. Slavoj Milošević said:

    cummie cookies

    June 27, 2019
    Reply
  28. Reginald Skarr said:

    What the hell's with these pictures on the slideshow?

    June 27, 2019
    Reply
  29. VicRules666 said:

    OK Google, ok Google, ok Google…

    June 27, 2019
    Reply
  30. BubbaRuff said:

    sam hyde

    June 27, 2019
    Reply
  31. Joe Miller said:

    SET A FUCKING TIMER

    June 27, 2019
    Reply
  32. Sieg Nalling said:

    Low battery.

    June 27, 2019
    Reply
  33. c-shadow said:

    candy corn

    June 27, 2019
    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *