One tricky thing about lengthy podcasts is that you cover a dozen issues, but when you give the episode a title you only get to tell people about one. With Christine Peterson’s interview I went with a computer security angle, which turned out to not be that viral a topic. But people who listened to the episode kept telling me how much they loved it. So I’m going to try publishing the interview in pieces, each focussed on a single theme we covered.
Christine Peterson co-founded the Foresight Institute in the 90s. In the lightly edited transcript below we talk about a community she was part of in her youth, whose idealistic ambition bears some similarity to effective altruism today. We also cover a controversy from that time about whether nanotechnology would change the world or was impossible. Finally we think about what lessons we can learn from that whole era.
If you subscribe to our podcast, you can listen at leisure on your phone, speed up the conversation if you like, and get notified about future episodes. You can do so by searching ‘80,000 Hours’ wherever you get your podcasts (RSS, SoundCloud, iTunes, Stitcher).
Table of Contents
The community that dreamed of space settlement and atomic factories
Robert Wiblin: Tell us a bit about how you ended up where you are today.
Christine Peterson: Wow. When I was growing up, I wasn’t … I can’t say I was particularly altruistic. I wasn’t raised to be especially altruistic. My parents didn’t spend a huge amount of time doing charitable work, but when I went away to college, I met a new kind of folks, people who were focused on very ambitious goals. At that time, the primary focus among young people was environmental, as it often is today as well. That was the overriding concern among young people at that point, and so we were all searching for answers. How can we solve the environmental problems facing the Earth today?
Robert Wiblin: How did you first get pulled into efforts to try to make a really big difference to the world? What were you doing when you were in your twenties or thirties?
Christine Peterson: The first altruistic effort that really got my attention was, oddly, perhaps, space settlement. The listeners may say, “What? How is that your number one? How is this altruistic, and number two, how does this fit in with this environmental concern?” At the time, back then, the concern was running out of resources and overpopulation. Those were the two main issues. We weren’t really looking so much at climate change back then, so we were thinking about, how can we support larger numbers of people? At that time, the population was growing. It wasn’t slowing as it seems to be now, and also where can we find more resources? We’re looking for, where can people live? Where are their resources? The Earth is by definition limited, and remember, this was not that long after the big space program of the US, so we were all very aware. We’d grown up watching tremendous numbers of space launches, and men walking on the moon for the first time. It was very exciting, so we were very aware of space, and space resources.
It was starting to become known that the asteroids had tremendous amounts of resources. We were starting to learn what’s out there, and realized wow, there actually are resources out there, especially immense amounts obviously of solar power, more than you could ever use, 24 hours a day up there, of course, and continuous. We thought, wow. There’s energy. There’s resources. You could actually live in space, and this at the time was a relatively new idea. Prior to that, only in science fiction was that explored. It wasn’t taken seriously, but increasingly this was seen as an actual option, and I think it is a real option. It will happen someday, so we young, idealistic people were saying, “Hey, let’s do space settlement as another way to deal with environmental issues.” We didn’t pretend it solved all the problems, but it would clearly help relieve the overpopulation burden. It would make a lot more resources available to the human species, without having to continually take them out of the Earth.
The idea was that it would lift the burden of human civilization off our fragile biosphere, and at the same time, as we all know, right now we have all our eggs in one basket here in Earth. There are existential risks that could occur that would actually wipe out all life on Earth, and so colonizing space is another way to deal with that. It has an existential risk benefit as well.
Robert Wiblin: This is the Elon Musk SpaceX strategy, trying to go to Mars.
Christine Peterson: It is, although this, because it preceded Elon, we were looking more at initially the moon, but also free standing space settlements, perhaps at the L5 point. Some of your older listeners may remember the L5 Society, which was a very idealistic, very young organization dedicated to building space settlements, free standing, where you create the gravity by rotating the settlement.
Robert Wiblin: Right, so I don’t know that much about the history of this movement. What became of it?
Christine Peterson: I think the basic ideas are still there. Obviously people like Elon Musk are carrying forward the concepts, and I think free standing space settlements will happen eventually when the economic time is right. I think we were too early, but I think the basic concepts we developed were right. I think they will happen eventually. The activists from that movement largely moved on to nanotechnology, which is what I did myself, when we realized wow, this is another way to address the huge environmental challenges that we want to take on.
Robert Wiblin: Is that because they thought it was a more promising technology, they could see more of a path towards nanotechnology?
Christine Peterson: I would say it was … We all understood that the space settlement vision, although technically feasible, is extremely expensive to start. Once you get it going, yes, then you can mine the asteroids, and there’s tremendous value there, but the up-front costs are immense, so … The US at this point, the space program was kind of faltering, and we could see, wow, this is not taking off as we had hoped, as fast as we wished, but nanotechnology is based on the science of chemistry, and that’s a small science. The investments compared to space are more manageable, so we became a little more practical, which is kind of typical of … You get your-
Robert Wiblin: People as they get older, and-
Christine Peterson: It’s true. You say, “All right, let’s … Now we really want to get something done that is actually gonna work.” I think Foresight attracted a lot of these former, super-idealistic young people who were starting to, instead of being in their twenties, now they’re in their late twenties, they’re in their early thirties, and they’re looking for, all right, how can we get more leverage to help our environmental problems?
Robert Wiblin: In the last few years, you’ve encountered the effective altruism movement. What do you make of it? Is it similar to the groups you were involved with when you were in your twenties?
Christine Peterson: Absolutely, and it’s the same kind of folks, extremely intelligent people, quite idealistic, quite ambitious, which I think is appropriate. I think every, when you have a new generation of extremely intelligent, very idealistic people, you want them to be ambitious. You want them to take on the hardest problems in the world, and that’s what effective altruism is doing.
The nanotechnology controversy
Robert Wiblin: Yeah. How did the Foresight Institute get off the ground?
Christine Peterson: What happened was I was at MIT as an undergrad, and one of my friends who actually is also, I don’t know directly or indirectly in the effective altruist movement, because he works at Future of Humanity Institute, which is one of the EA groups, Eric Drexler was also an undergrad at that time. We were both interested in space, and then he was the one who had the insights, the original insights that, wow, atomically precise manufacturing, what we called nanotechnology back then, is technically possible.
Robert Wiblin: Do you want to explain what autonomically precise manufacturing is?
Christine Peterson: Sure.
Robert Wiblin: A lot of people have misconceptions about what nanotechnology refers to.
Christine Peterson: The term is used in a lot of different ways. The particular type of nanotechnology that excited us as undergrads at MIT at the time was saying, if you look at biological systems, and at that time some of the early work was being done seeing how biological systems build things with DNA, and RNA, and proteins, all that. We were realizing, wow, this is not unique to life. You could build artificial systems that could do something very like this, but even better. You could build products both small products and eventually large products with every atom in a designed location. Obviously you have to follow the rules of chemistry. There’s no way around that, but as long as you stay within them, then you could construct pretty much whatever you wanted with atomic precision, being inspired by what we see, that’s how it’s done in nature. That was the fundamental insight that he had, and we were both very young at the time.
When you hear revolutionary things as a very young person, you’re not that surprised, because you don’t have a baseline to compare it with, so when I heard these insights from him, I thought, “Okay. Sure. Why not?” I knew enough chemistry at that stage to say, “Well, this doesn’t seem to violate the laws of chemistry, which is critical.” That’s the first thing you check. Is this physically possible, and if it is physically possible, then you have to say, “All right. How would we get there? How long is this going to take, and how expensive is it going to be?”
Robert Wiblin: You could have gone into a lab in academia or industry and tried to develop atomically precise manufacturing, but instead, you made a nonprofit that’s focused on the risks, and rewards, and so on. Why do that?
Christine Peterson: We could see that this technology, when it reached its full extent, would have tremendously revolutionary consequences for society. Some of them would be, many of them would be positive. There’s the huge environmental benefits, huge medical benefits, and then of course there’s always the problem of military use. Military offensive use, so we felt gee, rather than going to the lab and do this ourselves prior to that, we really should try to get the word out about both the positive and the negative results of this world that’s coming, and that was the decision that we made, that all right, we will open up, open up this information to the world so that we aren’t alone in this.
Robert Wiblin: Quite a lot of people are skeptical that you can do a lot of preparatory work to make sure that new technologies are used well and don’t cause harms, and I guess to some extent it’s been a little while now, and atomically precise manufacturing isn’t here, and it doesn’t seem like it’s exactly around the corner. One could say that some of the early work there might have been wasted, or perhaps it was premature. What do you think of those criticisms?
Christine Peterson: It’s a great debate to have. I wish I knew the answer, and to go back to something we talked about before, this is one of those cases where you’re doing a one-off, unique activity, and because there is no way to run any kind of a control, we will never be sure. We’ll never know. We can guess. We can speculate whether it was a good thing or not to do that, but it’s literally impossible to know. It’s just speculation. All we can have is kind of a gut feel, and say, “Well, I think it was better that we did this, that we didn’t,” or we can say, “Well I wish we had just gone in the lab and done it.” I don’t know how to make that evaluation.
Robert Wiblin: Yeah. Who funds a startup nonprofit focused on making a technology that doesn’t exist yet safe?
Christine Peterson: We realize that they were powerful ideas, and that if for example, a book could be written that conveyed them in a persuasive way, we felt that it would start a movement, and that was true. The book was written. I didn’t write the book. I helped comment on it, but my role was more of an earn to give situation. I spent about five years, and the only job I’ve ever had that wasn’t altruistic, and just to make some money, I did my activism in my spare time, and the money that I was making went into making sure this book happened. That was my earn to give phase.
Robert Wiblin: What’s the book’s name?
Christine Peterson: The name of the book is Engines of Creation. It is still in print. It is still inspirational. I try to read it every now and then, because it is still a super inspirational book.
Robert Wiblin: That’s by Drexler, right?
Christine Peterson: That’s right.
Robert Wiblin: Let’s dive deeper into the nanotechnology question, so in effective altruism, there’s a lot of interest in this issue of revolutionary technologies, and how they could transform society, but nanotechnology hasn’t gotten so much attention, perhaps because people just don’t think that it’s going to … They think that AI’s going to come sooner, or that perhaps biotechnologies are going to come sooner. Should we be more focused on it?
Christine Peterson: I think people who think that AI is going to come sooner and biotech is going to come sooner, I would agree with that. I think that is probably true. We were having debates 20 or 30 years ago, which would come first, nanotech or AI? Back then it really wasn’t clear, and of course today it’s not 100% clear, but I think most people at this point are betting AI will be first. That’s part of the reason why foresight is starting to ramp up our AI work. We are making the same observation that everyone else, saying, “Wow. This is moving fast. So much money is piling in. It’s a worldwide effort.” It looks like this means that nanotechnology will still come but it will probably arrive in a world with AI, and that’s a different looking space.
Robert Wiblin: What kinds of scenarios would we be worried about if atomically precise manufacturing turned out to be a lot easier to create and perhaps we could actually develop it in 10 or 15 years? What are the risks?
Christine Peterson: The primary downside would be deliberate abuse. In the early days, we were looking at accident scenarios, and those are still conceivable, but I think in terms of likelihood of problems, most people would say, no, the real issue is deliberate abuse. For example, smart weapons, very smart, very targeted weapons.
Robert Wiblin: How would you target atomically precise manufacturing machines? Wouldn’t they just tend to spread out of control, and blow back on whoever tried to use them?
Christine Peterson: I would say that to some extent, this is a software issue. These devices would need to be controlled with software, and as we all know, if you look at hardware systems and software systems, the software ones are much harder to understand. They’re hard to … It’s hard to get software to do what you really want, so to the extent that any type of machine goes haywire and comes back, and bites its originators, if software’s involved, often software is the issue. In fact, software security, computer security, is a huge, huge issue. Foresight is taking, as far as I can tell, I haven’t heard of another organization that is taking it as seriously as we are. That is in part because first of all, it’s an immediate problem. It’s happening right now. The risks are very high. The vulnerability is high, and it does affect how AGI will play out in the future.
Robert Wiblin: All right. We’ll come back to that one. Nanotechnology might be a lower probability risk, but it’s also more neglected. How many people in the world do you think are working on risks from nanotechnology?
Christine Peterson: Oh, very few at this point. There are very few. Actually there are very few people working on biological and chemical risks compared to the magnitude of those problems also. There just aren’t that many people working on risk of that, risks of those types.
How do you make a community work? Should we be trying to colonise space now?
Robert Wiblin: The Foresight Institute was kind of at the bleeding edge of this nanotechnology issue, and this question of revolutionary technologies, and I guess effective altruism is now similarly kind of a young movement with a bunch of new ideas. I’m curious to know, what kind of challenges did you have early on, and might that be similar to some of the challenges that we might face in the future?
Christine Peterson: Yes. I think there are some similarities. I would say that any early movement is going to attract a wide variety of folks, many of whom are extremely competent and have great social skills, some of whom are technically competent, maybe not such great social skills. On the fringe you’ll sometimes get some folks who are enthusiastic but it’s very hard to figure out how they can contribute in any way, so the challenge for a young movement, and in the early days of a movement, you want to welcome everyone. We’re all very enthusiastic. We’re excited. We’re small. We want everyone to come in. We want everyone to participate, but then you realize, gosh, while the vast majority of folks who are coming in are useful and helpful people, some of them, although they want to help, either there’s some issue, either they don’t know what they don’t know, for example, they think their technical skills are better than they actually are, or they have some serious social skill issues, or even personality disorders.
You just get everyone. It’s like a cross section of the whole population, so the challenge for a new movement is to reconcile the goal of getting everyone involved, and making everyone feel welcome, which we would love to be able to do with the fact that not everyone has the same skillset. Some folks are even challenging to work with at all. How can we still allow them to contribute, and to feel part of the group, without slowing everybody else down?
Robert Wiblin: How did you work around that problem? Did you manage to do a good job of it?
Christine Peterson: I think we did. I think first you have to admit this, which is a stage in a young movement when you go from saying, “Everybody is equally welcome and can perform equally well in any task,” and realizing okay, that’s just not right. Let’s start to figure out, as people come to us, figure out, all right, what are their real skills? Sometimes the person, himself or herself, knows that, and sometimes they don’t, and then how can we direct them into a role in the organization that is the highest use of their time? Sometimes there are folks where really the best use of their time for the movement is in an earning to give role and they can be made welcome at open events, where their contributions are appreciated, and they’re given those warm fuzzies we all need, but we don’t necessarily put them in a full-time role at the organization.
Robert Wiblin: Let’s talk a bit more about space settlement. Do you still think that that’s an interesting priority? What do you think of Elon Musk’s strategy with SpaceX?
Christine Peterson: I think it will happen eventually, and I think it’s something that should happen for existential risk reasons. I think it’s also something that should happen for environmental reasons, so I think it will happen. I’m still in favor of it. When you consider Elon’s goal for Mars, other folks are more interested in the moon, and then there’s the L5 crowd that still advocate for freestanding space settlements. Those are the three options that, those are basically technical and economic choices. It’s not something … It shouldn’t be … The decision shouldn’t be made based on emotion, or what’s the sexiest goal. It should be based on what is economically and technically the most feasible, so Elon thinks Mars is it. There are other people who feel it’s the other way, but one thing about Elon is, he does get some stuff done, so you have to give him credit for that. He may succeed at his goal just because he’s pushing so hard, and he has some money to throw at it.
Robert Wiblin: I feel like Elon gets more done in a month than I might hope to accomplish in a lifetime. That said, I’ve been something of a critic of this idea of space colonization as a way of dealing with existential risk. The idea is to kind of back up humanity on Mars, so if there’s ever a disaster on Earth, or at least this is one reason that you might do it, go to Mars so that you have a second copy and potentially they can come back and recolonize Earth if humanity is in deep trouble or even extinct. Yeah. I think if that was your goal, wouldn’t it be a lot cheaper just to stick people under a mountain, in a mine shaft, or dig very deep, and create a very, a bunker that’s extremely well-stocked, and people could live for a long time, or put people under the sea, or in Antarctica? Those places are all very difficult to have free-standing independent colonies, but they’re still a lot easier than Mars, I would think.
Christine Peterson: I think you make a good point, and I think for some existential risk scenarios, that would be the way to go. I think longer term, we don’t really know when a very, very large rock is going to hit the Earth and really mess it up completely, so you can … There are existential risk scenarios where you really, and then you can say, go beyond that and say, “Well, there are some existential risk scenarios where even being on Mars isn’t good enough.” In the very long-term, you want to keep going. You want to just get out of the solar system entirely, but that is again a very long-term goal. Yes, I would agree with you that those other scenarios you described are ways to address some existential risk scenarios, but I should also mention that if there’s anyone who thinks that colonizing Mars is a way to deal with the AI risk to humanity, I would say I don’t see that as an answer. I don’t think if there were a Mars colony, and there were a problem with AI on the Earth, I don’t think Mars would be independent of that.
Robert Wiblin: Yeah. My impression is Elon now agrees with that.
Christine Peterson: That’s good, because I … A lot of folks had thought he was seeing it the other way, so it’s good to hear that that has been clarified.