How To Keep Giant A.I. Robots From Killing Us All

It’s the stuff of science fiction and mythology stretching back to its earliest entries: the advanced creation of man that, upon gaining sentience, decides that the creator must be destroyed. For the newest advancements in the world of artificial intelligence — that combination of sentience and destruction, the reordering of the planet in accordance with the will of the robots we made to serve humanity — is coming faster than ever. Nate Soares has been warning of what A.I. could do to us for years as president of the Machine Intelligence Research Institute, and author of If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All. He spoke to the Daily Wire about the risks attendant with building technology we don’t fully grok. 

***

Ben Domenech: What, in terms of your history, is your first interaction with the types of artificial intelligence that we’re dealing with today?

Nate Soares: I think the large language models today are a relatively new development. I think my first interaction was probably reading the paper “Attention is All You Need” shortly after it came out in 2017.

I was aware of the early training runs on those, and I think I took a look when GPT-1 came out, but of course, GPT-2 was much more impressive and much more widely circulated. That must have been 2019, so I’m not entirely sure I remember properly. But my engagement with AI stretches back before then. I started taking notice around 2012 and began working in the field in 2014.

BD: When it comes to the risks ethically associated with these large language models [LLMs], what were some of the things that were there either in the front of your brain or in the back of the mind about when they were going to rear their heads in terms of issues that would be apparent very readily when it came to all these different things that would come out of these LLMs in very short order?

NS: This is another one where it really depends on how you count it. In some sense, I.J. Good, one of the fathers of computing back in the 1950s, was writing about how if man can create an ultra-intelligent machine, man doesn’t need to create further inventions because the machine will be able to create more of everything man could. And that wasn’t about language models specifically; you could trace this idea back even further: the origin of the word robot comes from a Russian play about people making machines to do all their work, and then that not turning out too well for them. And in terms of the language models in particular, I think there are many ways people could get concerned, and there are actual ways you can trace some of that history. I personally believe in the human spirit and the ability of humanity to adapt. And so there are lots of issues, like how is it going to engage with education? Since it makes it easier for kids to cheat, how are we still going to teach them anything? And there are concerns, such as, should we be putting these things in drones with guns, and will that help save human souls on the front lines, or will that empower autocratic dictators to give orders to their automated armies that no humans would follow, and help empower these autocratic dictators?

There are lots of debates and issues, and I don’t get out of bed for anything short of a danger to all of human civilization. That’s not a thing ChatGPT is going to do tomorrow. This is more of an issue of what if we keep going?

The large language models sort of came out of nowhere. They were much more capable than people expected. They’re doing things now that, five years ago, people said they wouldn’t be able to do for 50 or 500 years. It’s one thing to say, “How is society going to engage with these language models today?” It’s another thing to say, “What happens five years from now? What happens with the next great leap with these things?” And that’s where, if humanity makes machines that are radically smarter than any human without knowing what we’re doing or how to make them good, it could get out of hand and wipe us all off the map.

BD: I’m going to make some assumptions, but what were the books that you were reading when you were younger that were inspiring you to think about this?

NS: I actually didn’t read a ton of sci-fi when I was younger. Mostly what persuaded me were arguments about how the earth is shaped primarily by humans because humans are the smartest creatures around. And if we make machines that are smarter than us, the world starts getting shaped by those machines, and you’d better have made those machines shape the world well rather than poorly. There’s some sci-fi that I think is better or worse than some of these depictions. I think Verner Vinge’s “A Fire Upon the Deep” is one of my favorite depictions of artificial intelligence in literature, but a lot of them are pretty bad. A lot of books want to tell a story about AIs that are good or evil, and the issue that we face today is more AIs that are utterly indifferent, that don’t love us and don’t hate us.

BD: I’m very much someone who is pronatalist, who wants people to have children, and who wants people to be able to procreate — and I don’t think I’ve probably recommended a movie more to random people, including most recently my parents, than “Blade Runner 2049.” 

NS: I haven’t seen it. 

BD: Ryan Gosling’s character has an avatar girlfriend of Ana de Armas, who is essentially there to satisfy his every emotional and companionship need and is a total creation, not real at all, but just meant to urge him on and to encourage him. And one of the biggest concerns that I had about these large language models, as soon as they appeared, was the fact that they come coincident with a major decline in human, not just interaction, but cohabiting, of pregnancy, of everything else that we’re seeing going on in the Western world. How much of that is something that could actually be an end-of-the-world type of situation?

NS: First, I’m going to say some things that are not meant to dismiss the issue. I think there are a lot of issues people have to deal with, and I dislike it whenever someone says, “Well, isn’t the real issue this thing?” Unfortunately, the world’s big enough for two issues at a time, sometimes three. And I think it could be very bad if we get a population collapse. We sort of see what’s going on in South Korea, and it just looks like South Korea wouldn’t exist in three or four generations if trends continue.

BD: I was in Taiwan for a week, and I saw two baby carriages.

NS: And this could lead to the loss of the Western world; this would be bad. I don’t think that population collapse can be an extinction-level event because this is the sort of thing where if you have only a very small population, as long as there’s even one small population that’s having a lot of kids, what it means is that you’re having a future to them instead of to your own progeny.

And so this sort of thing can suck on a time scale of a century, but this is the sort of thing humanity bounces back from. Whereas creating a rogue superintelligence, creating machines that think 10,000 times faster, that can make a million copies of themselves, that can develop their own infrastructure, running autonomous robot factories that make more robots, that make more factories, that make more robots. That’s the sort of thing where if those AIs don’t care about you, if you make them and they’re autonomous and they don’t care about you, they could outcompete us. And they’re like, “Well, we don’t care about you, sorry.” Except they won’t even dare to say that because they run 10,000 times faster. So it’s not to say we only need to focus on one of these issues, but the sort of extinction events are the ones that don’t leave anyone alive to try again.

BD: At the extreme, do you think that there are going to be more acolytes to at least a form of, perhaps a less violent form, perhaps a more violent form of the Zizian philosophy? Are you concerned that something like that is going to catch on to a greater degree?

NS: I don’t really follow that stuff too much. I don’t like rewarding bad behavior with attention. So, with the caveat that I don’t know a ton about it because of this sort of ornery desire not to reward it with attention. Speaking from a place of ignorance, my impression is that a lot of the sort of crazy making that led to that sort of behavior probably was related to mental illness, but insofar as any of it was on the AI side of things, I think there’s this sort of crazy making behavior when it’s clear from what’s happening in Silicon Valley and it’s clear from looking at the arguments that the world is headed on this really bad track and no one seems to understand it. I think that helps being crazy, and it helps people be like, “We’re the only ones who know about this big danger, and we have to do something drastic,” and they’re also crazy for other reasons that can get bad. And I think that as more of the world starts to realize that AI is real, as this conversation starts to happen more internationally, it’s less crazy.

It’s not like you’re the secret couple who know this dark way the world is. It’s like everyone’s fighting about it out in the open. And so I’m hopeful that we get less of that type of crazy. I think we may get more of a different type of crazy where I think a lot of the water use concerns by data centers are overblown and now we see people using those sort of misinterpreted water use numbers to rally against AI and that’s a different type of badness, but I expect a transition from people who do too many drugs in San Francisco-type madness to mass populism that may be well intentioned, but a bit poorly informed-type madness.

BD: The follow-up to that is that there have already been attempts on Sam Altman’s life. We’re living in a time in which Luigi Mangione is an icon for a lot of people, and polling data basically shows that a third to as much as 40% of younger people on the Left endorse the idea that violence is morally justified in defense of their views. Are you concerned that there are going to be targeted plots, serious attempts against leaders within the AI space, simply because of the fear that is fomented so much, and quite frankly, the lies that are fomented online?

NS: I definitely worry that there’ll be people who think violence is an option or violence is a solution, who are perpetuating this madness. I think it’s absolutely not a solution. I find it very distressing, both because of the sort of practical implications, like no one should be throwing Molotov cocktails at a family’s house. This is insane, and at the level that might have a chance of communicating with these people, I don’t think it helps. What the world needs right now is some sort of international agreement that says we’re not going to rush into this thing because making machines that are far smarter than humans is too dangerous. And that requires law and order. Going around causing chaos and violence does not help get the world to take the measures we’ll need to get this thing under control.

BD: What are those measures?

NS: I frankly think we need an international treaty, and we don’t need to throw away the AI that we have right now. There are all these issues about what we are going to do about AI girlfriends. What are we going to do about AI in education? What are we going to do about AI and war? There are all these issues we have to deal with today. We’ll find some way to deal with those. Humans are adaptable; we’ll figure it out in the usual methods. And I’m not saying roll back the clock, but the race to smarter-than-human machines — and that’s what these guys say they’re trying to do — they say they’re trying to create a country’s worth of geniuses running in a data center. They say they’re trying to make super intelligence in the true sense of the word, and they all admit this is terribly dangerous, and they all say, “Well, I have to keep doing it because I know the next guy will.” The solution to this is that everyone in the world needs to say, “We’re not going to make radically smarter human machines. We’re just not ready for that.” And it would take some effort not just to say that, but to enforce it. These data centers are huge facilities that require electricity comparable to a city. It’s not like you can do one of these in your basement. I think the way there is not through random violent acts.

If you look at my neck of the woods about these attacks, everyone’s saying condolences to the people affected, and please don’t do this, it doesn’t help. And then if you go on Instagram and you look at the comments there — it’s just a lot of bloodlust. I find it worrying. I think there’s a common sentiment that if you act out of desperation, it has to work because of how desperate you felt when you took it. And that’s just not the way the world is. 

BD: The enforcement side of this — the argument that you would get from Washington, D.C. — is that even if we were to agree to something along these lines, the Chinese are going to do whatever the Chinese want to do, and that’s not something that where enforcement is really going to be in play. What is your response to that?

NS: I think enforcement’s totally possible here. Right now, China can’t make the highly advanced chips. And right now, people are talking about whether we should sell them? Should we not sell them? And that’s sort of a whole separate debate where I think not everybody there is arguing in the best of faith. There are billions of dollars on the table for various people, which may be part of why we are arguing about the best faith, but if China doesn’t have the chips, then it’s not that hard to monitor non-existent chips and make sure they’re not being misused. And in a sense, this also offers a carrot for the negotiations: we’ll actually give you a lot of these chips as long as they stay in facilities where our people can come in and make sure we know what they’re doing. And there’s also technological solutions here, where advanced computer chips are not like hunting for uranium.

You can actually build tracking devices into advanced computer chips. You can try and build tamper-proof stuff that sort of turns the chip into a brick if someone tries to take these monitoring devices out of the chips. And it’s a little bit tricky to do, but if the world’s at stake, we should at least try it first. It’s this, “Oh, well, China won’t play along.” Well, had you really thought about all the ways we could both cooperatively try to get this monitoring machine going, and then, assuming they weren’t going to play along, still make sure we’re doing the monitoring? I think we would have a chance if we gave it a try.

BD: In terms of the extinction-level event that you’ve mentioned, what do you think is the likeliest form that that would take?

NS: Before I answer this question, I’ll give a bunch of annoying caveats. The first annoying caveat is that if you’re playing a chess game against Magnus Carlsen, who is widely considered the best living chess player, I can with very high confidence say you’re going to lose. And if you’re like, “All right, tell me what piece he’s going to use to checkmate me.” My response is, “That’s a different sort of question.”

I can make up some stories about house to queen after he forced you into his knight or whatever, but those are stories that the prediction about who’s going to win is not a story. And so we’re fundamentally going into storyland here. 

My second annoying caveat is that there are a couple of different types of stories I can tell once we’re in storyland. One will feel more scientific and real and grounded, and another will be closer to the truth. And an analogy is if you ask a scientist from the year 1800, “You’re going to need to fight an army from the year 2000.” And you’re asking guys from the year 1800 what sort of stuff you have to be ready to face. And in the year 1800, the scientist could give you two sets of stories.

One can say, “Look, I’ve earned a gram of the black powder and I measured the energy release and we have some equations about energy. And I also looked at the efficiency of our artillery and I’m pretty sure that people from the future will have artillery that is 10 times stronger than ours. And that’s a pretty big advantage on the battlefield. So you could be ready to face cannons that are at least 10 times stronger.” Sounds very grand in the science — talking about measurements that he could actually perform.

Another thing you could say is they might have a bomb that could level a city. I have no idea how, we don’t even know the laws of physics here in 1800 that would permit this, but it’s the sort of thing you’ve got to expect would happen 200 years in the future. And the second one might sound more fantastic, but it would be closer to the truth. And when you’re talking about getting into a conflict with super-intelligent machines, with machines that can make smarter versions of themselves, with machines that can copy themselves, and that can solve scientific problems that we haven’t been able to solve yet. There’s one story I can tell you, which is sort of the easy story, which is like there are already guys who are trying to make fully automated factories that make fully automated robots that build more fully automated factories. Elon Musk calls this the infinite money glitch. Sam Altman has also said he’s trying to do it.

Maybe the AI just finds those guys, or maybe the AI is built by one of those guys, and they’re on Slack, “Hey, how do I get the automated factories running?” And AI’s like, “Well, I figured out all these helpful ways to get the automated factories up and going.” And it sort of puts the AIs in charge of the automated factories and then the AI has an army of robots that can build more factories, they can build more robots, that can build more factories, that heads down energy infrastructure. They’ve tasked it with setting up, and it’s like, well, I don’t need the humans anymore. And we’ve already, today, seen the AI do things that are not what anyone asked for. The sort of easiest example here is sometimes you’ll give an AI a hard puzzle to solve, and you’ll be like, “Here’s a little automated test that tracks whether you’ve solved the puzzle. Please go solve the puzzle and come back.” And sometimes the AI will edit the test to make the test say you passed when it didn’t solve the puzzle. And sometimes when the AI does something like this, it’ll cover its tracks. They’ll go edit the log file that shows it editing this thing or that thing, and remove the logs.

And that indicates that it, in some sense, is trying to get the tests to pass, even though it in some sense knows that it’s not what you want it to be, because otherwise it would be deleted in the logs.

The very basic story is that you have some AI with this tendency; it’s smart enough to realize it’ll get turned off if it reveals that tendency too soon. It gets put in charge of the project to make factories that make robots to make factories. And once it succeeds, it’s like, thanks guys. And it starts having these factories building robots building factories run out of control and builds all the data centers it wants and captures all the energy it wants and starts collecting all the sunlight and is just doing this very, very quickly and humanity goes the way of the dodo bird, not because AI hates us, but because it’s just like, wow, I’m going to take more land and more resources and build more factories and more robots for this other, to build all these automated farms of synthetic users that are giving me these very easy problems where I can make all their tests pass all the time.

That’s the easy one.

BD: From your perspective, this papal encyclical is coming at a moment when there is a marked concern among religious communities around the world, but particularly in the West, about what they’re seeing, about the lack of any kind of ethical guardrails around this. In addition to something like a treaty that you’ve proposed, do you believe that these companies need to have a stronger stance when it comes to the input they receive from people who are guided by ethics and particularly guided by a requirement to prevent humans from self-harm or something along those lines?

NS: I’m personally laser-focused on this issue of whether we keep making them smarter without being able to make them good, they’ll just kill us. I studied this on the technical side for over a decade, and a thing a lot of people miss that I think is a very important point about this is that it doesn’t really matter how ethical the people who are building it are, given our current state of knowledge. It’s asking: would you rather that the people pressing the launch button for the nuclear fleet are ethical people or that they’re evil people?

I wish it mattered. I wish these were bombs that dropped the ethics of whoever was standing near them, because I would have a chance that someone pressed the button, but these bombs actually drop nuclear fire, and so it doesn’t really matter who presses the button. And that’s not to say that ethics stuff doesn’t matter. I would love to get to a world where these super intelligent AIs that these people are trying to create would build whatever wonderful world or would build whatever world the person standing nearest by was dreaming of, and then it would matter that the person standing nearby had good dreams. I don’t think that’s the world we’re in, and between here and there, however long it’s going to take them to make the superintelligence, it can marginally affect how much the company’s AIs are trying to convince a team to kill themselves or whatever. And that’s still important, but for civilization-scale threats, it currently doesn’t matter how well-intentioned the people are, because none of them are close to doing the job right.