Transcript of Most Replayed Moment: How AI Companies Are Gaslighting You - Karen Hao

What I find really interesting is to look at how people's quotes evolve over time with their incentives. So I was looking at all of the things they've said on the record, on podcasts, in their blog posts, to see how it's evolved over time. And Dario, who was the former VP of Research at OpenAI and has now moved on to Anthropic, who are taking a slightly different approach to developing AI, said back in 2017 while he was still at OpenAI that This is a quote. I think at the extreme end is the Nick Bostrom style of fear that an AGI could destroy humanity. I can't see any reason in principle why that couldn't happen. My chance that something goes really quite catastrophically wrong on the scale of human civilization might be somewhere between 10% and 25%. And also you mentioned Ilya, who was a co-founder of OpenAI and then left I guess the first question I'd ask is, why did Ilya leave?

It's a great question. So he was instrumental in trying to get Sam Altman fired, and he's another one of the people who over time began to feel like he was being manipulated by Altman towards contributing something that he didn't believe in. And for—

How'd you know?

Because I interviewed a lot of people. Ilya in particular had 2 pillars that he cared about deeply. One is making sure we get to so-called AGI, and the other is making sure that we get to it safely. And he felt that Altman was actively undermining both things. He felt that Altman was creating a very chaotic environment within the company where he was pitting teams against each other, where he was telling different things to different people.

Have you ever spoken to him?

I have. So I interviewed him in 2019 for a profile that I did of OpenAI for MIT Technology Review.

And back in 2019, he has a quote where he says, the future is going to be good for AIs regardless. It would be nice if it was also good for humans as well. It's not that it's going to actively hate humans or want to harm them, but it's just going to be so powerful. And I think a good analogy would be the way that humans treat animals. It's not that we hate animals. I think humans love animals. And I have a lot of affection for them. But when the time comes to build a highway between 2 cities, we are not asking the animals for permission. We just do it because it's important to us. And I think by default, that's the kind of relationship that's going to be between us and AI, which are truly autonomous and operating on their own behalf. And that was in 2019, the year that you interviewed him.

One of the things that I feel like we should take a step back to examine is going back to this idea of what even is artificial intelligence and what do we mean by intelligence? And a huge part of the views of the different people and the quotes that you're reading derives from a specific belief that they each have in this question of what is intelligence, what constitutes intelligence. For Ilya, he has throughout his research career felt that ultimately our brains are giant statistical models. This is not something that, you know, we actually know, but this is his own hypothesis, also the hypothesis of his mentor, Geoffrey Hinton, who also was on this podcast. This is why they have such a strong conviction in the idea of building AI systems that are statistical models, and that this particular approach is going to lead to intelligent systems as we are intelligent. It's a hypothesis that they have. It's not one that has been proven by science, and some people vehemently disagree with them this particular thing. But if you step into their shoes and take on that hypothesis and assume that it's true, that our brains are in fact statistical engines, and that these systems that they're building are also statistical engines that they're making bigger and bigger and bigger until they become the size of the human brain, that's why they say that making this comparison where the system will become equal to human intelligence and then maybe exceed human intelligence is— relevant in their framework.

And Ilya gave a talk at one point at this really prominent AI research conference that happens every year called Neural Information Processing Systems. It's a mouthful. But he gave this keynote where he shows this chart of the size of brains and the intelligence of a species. And it's roughly linear. The bigger the size of the brain, the more intelligent the species. And so for him, he thinks he's building a digital brain because he thinks brains are just statistical engines. So from that logic, it's like, okay, if we then build a bigger statistical engine than the human brain, then based on this chart, it will be more intelligent, and then we will be subjected to the same treatment that we've subjected animals. But it's really important to understand that these are scientific hypotheses of specific individuals within the AI research community. And there's a lot, a lot of debate about whether this is in fact the case. And some of the biggest critics say it's very reductive to think of our brains as simply just statistical engines.

Why does it matter to know the mechanism? Is it not just important to know the outcome, which is that it's going to be able to do— make a video for me, or agents are going to be able to do the work that I do? Does it really matter for us to know the mechanism behind it?

Yes and no. So it matters because these companies, they are driving their future actions based on this hypothesis. So they have decided, we think that this hypothesis is true, like we should just continue building larger and larger statistical models in the pursuit of artificial general intelligence. And that's then having global consequences. Like, in order to continue doing that, they're hoovering up more and more data, they're building more and more data centers, they are having— they're, you know, exploiting more and more labor in order to continue on this path. Here's a question that I think is important to ask is, why are we trying to build AI systems that are duplicative of humans? We're kind of having this conversation right now where we've just taken the premise of this industry as a good thing. Like, they said that we should be building AGI, so we say that we should be building AGI. But I would like to ask, like, why are we doing that? Why is it that we are building a technology that is ultimately designed to replace and automate people away? That is not the enterprise of technology. Like, we should be building technology, and the purpose of technology throughout history has been to improve human flourishing, not to replace people.

And so this is, like, a critical part of my critique of these companies and these scientists that have just adopted this goal and have relentlessly pursued it and have had enormous capital and enormous resources to pursue it, is, is this the right goal? Like, why are we doing this? Why can't we just build AI systems that do things like accelerate drug discovery and improve people's healthcare outcomes, which are systems that have nothing to do with the statistical engines that they're trying to build to duplicate the human brain.

So why are they doing it? I mean, you've interviewed all these people. I think it's, what, 300 people in total, 80 or 90 of them from OpenAI, the maker of ChatGPT. Why do you think they're doing it?

I think it's because they're driven by an imperial agenda. And that is why I call these companies empires of AI.

What do you mean by an imperial agenda? What does that term mean?

Empire is the only metaphor that I've ever found to fully encapsulate all of the dimensions of what these companies do and the scale that they operate and what motivates them to do what they do. And there are many parallels that you see between what I call the empires of AI and the empires of old. They lay claim to resources that are not their own in the pursuit of training these models. That's the data of individuals, the intellectual property of artists, writers, and creators. They're land-grabbing in order to build these supercomputer facilities for training the next-generation models. Second, they exploit an extraordinary amount of labor. They contract hundreds of thousands of workers all around the world, including in the US, to ultimately make these technologies. We can talk about that more. And they also design their tools to be labor automating so that when the technologies are deployed, it also affects labor rights because it erodes away labor rights. And this is a political choice that they have. Third, They monopolize knowledge production, so they project this idea that they're the only ones that really understand how the technology works. And so if the public doesn't like it, it's because they don't actually know enough about this technology.

They do this to the public, they do this to policymakers, and they've also captured the majority of the scientists that are working on understanding the limitations and capabilities of AI.

You think they're gaslighting the public in a way?

They are, yeah. Most of the climate scientists in the world were bankrolled by fossil fuel companies. Do you think we would get an accurate picture of the climate crisis?

No.

And in the same way, they employ and bankroll— the AI industry employs and bankrolls most of the AI researchers in the world. So they set the agenda on AI research in soft ways, simply by funneling money to their priorities so that only certain types of AI research are produced. But they also will censor researchers when they do not like what the researcher has found. And so I talk about the case of Dr. Timnit Gebru in my book, who was the ethical AI team co-lead at Google. When she was literally hired to critique the types of AI systems that Google was building, she then co-wrote a critical research paper that was showing how large language models specifically were leading to certain types of harmful outcomes. And in an attempt to try and stop this research from being published, Google ended up firing Gebre, and then fired her other co-lead, Margaret Mitchell. And so they control and quash the research that is inconvenient to the empire's agenda.

Did you have an example where this is happening to journalists as well? that are asking questions of their team members. I think I was watching a video of yours where there was a young man that was saying he had someone show up at his door, knocked on his door and asked for information, emails, text messages, and this person was from one of the big AI companies.

This was OpenAI started subpoenaing some of its critics, yeah, as part of a— what appears to be a campaign of intimidation, but also what appeared to be a campaign of phishing for more information to figure out to map out the network of critics further. But this was a man who runs a small watchdog nonprofit, and they had been doing a lot of work during that time to try and ask questions about OpenAI's attempt to convert from a nonprofit to a for-profit. Ultimately, OpenAI was successful in that conversion, but during the period where it was sort of existential for OpenAI to complete this conversion, there were a lot of civil society groups and watchdog groups like MIDUS who were trying to prevent the process from happening in the dead of night. They were trying to get more transparency. They were trying to have more public debate about this because it's unprecedented. And it was then that, um, there was a knock on his door and he was served papers.

What did the papers say?

The papers asked him to reproduce every single piece of communication that he had had that might have involved Musk. So this was like the strange paranoia that OpenAI had that Musk was somehow funding these people to block the conversion. None of them were actually funded by Musk. So in this particular case, their request, he simply just answered, I don't have any documents because this doesn't exist.

So going back to this point of empires, you were saying that one of the factors of an empire is a land grab. And then the next one was—

Was labor exploitation.

Labor exploitation.

The third one, controlling knowledge production. And one of the other ones that's really important to understand about the AI empires in particular is empires always have this narrative that they say to the public, like, we're the good empire, and we need to be an empire in the first place because there are also bad empires in the world. And if you allow us to take all the resources and use all of the labor, then we promise we will bring you progress and modernity for everyone. We will bring you to this utopic state akin to an AI heaven. But if the evil empire does it first, we will descend into a hell.

And the evil empire being, in this case?

In this case, most often it's China. But actually, in the early days, OpenAI evoked Google as the evil empire. So all of their decisions were about, we need to do it first, because otherwise Google, this evil corporation that's driven by profit, us as a benevolent nonprofit, like, this is a critical contest of who wins.

Do you think the people building these AI companies believe that the outcome is going to be all good now? Do you think they think that it's going to serve everyone, it's going to be the age of abundance, everything's going to go well? What do you think they believe? What do you think Sam believes?

So this is so funny, is such a core part of the mythology that they create around the AI industry includes the belief that it could go very badly. It goes hand in hand. Like, they need that part of the myth in order to then say, and that's why we need to be in control of this. technology, because that's the only way that it's gonna go really, really well. And Altman has said publicly, you know, the worst case, lights out for everyone. But best case, we cure cancer, we solve climate change, and there's abundance. And Dario Amadei, same kind of rhetoric. He's like, worst case, catastrophic or existential harm for humanity. Best case, mass human flourishing. So this is like 2 sides of the same coin. Like, they have to use both of these narratives in order to continue justifying an extremely antidemocratic approach to AI development, where there should not be broad participation in developing this technology. They must be the ones controlling it at every step of the way.

Sam Altman did a tweet saying, there are some books coming out about OpenAI and me. We only participated in 2 of them, one by Cash Hagee,

Keach Haigie.

Keach Haigie, focused on me, and one by Ashley Vance on OpenAI. He went on to say, no book will get everything right, especially when some people are so intent on twisting things. But these 2 authors are trying to. You quote retweeted that tweet from Sam Altman, and you said, the unnamed book Empire of AI is mine. Do you believe that tweet from Sam Altman? was in reference to your book?

100%, because there's only 3 books coming out about him.

And he'd caught wind that your book was coming out, and—

He knew my book was coming out because I had contacted OpenAI from the very beginning of my process and said, I'm working on a book now, will you participate in it? And actually, initially they said yes, even though— so my history with OpenAI, I profiled the company for MIT Technology Review. I embedded within the office for 3 days in 2019. My profile comes out in 2020. the leadership are very unhappy. And in my book, I actually quote an email that I received that Sam Altman sent to the company about my profile saying, yeah, this is not great. And from then on, the company's stance to me was, we are not going to participate in anything that you do. We are not going to respond to anything, any questions that you receive. And this was You know, this was things that they explicitly articulated. It wasn't, like, me inferring. So I had a colleague at MIT Technology Review that also covered AI, and at one point OpenAI sent him this press release being like, we would love for you to cover this story. And he was like, I'm really busy. Will you send it to Karen?

And they were like, oh, no, we have a history, you understand. And so for 3 years, they refused to talk to me, but then I ended up at the Wall Street Journal, where if they felt a bit compelled because it was the Journal to reopen the lines of communication. And so I, I started having, you know, more dialogue with them. Every time I wrote a piece, I would always send them, here's my request for comment. I would always ask them, like, will you sit for interviews? And we did get to a more productive relationship. And then I embarked on the book. So I, I left the Journal to focus on the book full-time. And I told them right away, I'm working on this book. I want to continue this productive conversation where I make sure I reflect OpenAI's perspective in the book. And so they were like, we can arrange interviews for you. You can come back to the office. We'll set up some conversations. And then as we were going back and forth on this, the board fires Sam Altman. And that's when things started going kind of south, because the company started becoming very sensitive to scrutiny.

And so then they started pushing, kicking the can down the road, down the road, down the road. And I kept saying, hey, when are we rescheduling this? What's going on? And then I get an email saying, we are not going to participate at all. You are not coming to the office. You're not doing interviews. And I had actually already booked my tickets, so I was already going to fly to San Francisco to have The, the interviews. And so then I told them, I was like, that's fine, I will still engage in the process where I'll give you extensive requests for comment. I'll, as through my reporting, I'll keep you updated on all the things that I'm finding so that you can choose to still comment. I gave them 40 pages of requests for comment, and I gave them over a month to respond to all of that. So this was when the tweet came out, was we were doing all this back and forth trying to— and that's when Altman tweeted this. Hmm. And they never responded to a single one of the 40 pages.

Sam Altman does a lot of interviews.

Yeah.

You know, he's doing a lot of interviews all the time. He's done every podcast. I've seen him on everything from Tucker Carlson to— I think he's done Theo Von, Joe Rogan. podcasts all over the world. I wonder why he won't do mine. Well, maybe—

I don't know why.

I, I don't know. I think I'm fair with everyone. I just ask, I just ask questions I genuinely care about. I don't come in with huge preconceptions, or at least meet people for the first time. But I've heard through the grapevine, um, that he doesn't want to do mine.

I mean, going back to what you were saying earlier, that with this, the way that OpenAI and these companies control research. You asked, do they also do this with journalists? I mean, yes, the answer is yes. And apparently they, they also do it with anyone who has, you know, a broad mass communications platform. Hmm. It's not just about the conversation that you're going to have with them. It's about who you also choose to platform. Hmm. And There's this huge problem in technology journalism where companies know that a really big carrot that they can give to technology journalists is access.

Yeah.

And they will withhold that access at the drop of a hat if they catch wind that you're speaking to someone that they didn't want you to speak to.

This is so true, and I don't think the average person really truly understands this.

Yeah.

So this kinda sounds like theory as you say it, but... I'm not gonna name names here 'cause I don't think it's important, but there is a particular person in AI whose team have basically dangled the carrot of them coming here for like 18 months. And I'm like, you don't have to dangle the carrot. I'm gonna speak to whoever I want to regardless of the carrot or not. Yeah. And when this person comes, if they wanna come, I'll give them a fair shot. I'll ask them all genuinely curious questions about what they're doing, their incentives. I won't gotcha them. I don't have a history of ever gotchuring anybody. Yeah. Even if I disagree, like even if I have a different opinion, Yeah. But they dangle carrots, and they say, well, if, you know, he's thinking about it, let's think about a date. And what the strategy is, and I don't think they— they think most people don't understand, is if we just dangle it for long enough, then they will perform in the way that we want them to do, and they'll be pleasant about us, they won't be critical, they won't give a— They

won't platform our critics.

Our critics. And I think a lot of their game is just dangle the carrot forever.

Yes. Yeah.

That's like the optimal outcome, is that if we just dangle it, if we just tell them, yeah, no, we're just trying to look at the schedule, it just doesn't work. I think in the modern world, you just have to go there and give your opinion and allow the clash of ideas in the public forum. Let the viewers decide for themselves what they think.

Yeah. But this is such a huge part of their machinery, is the way that they use these tactics to massage the public image of these companies and make sure that information that they don't want out, and even opinions that they don't want out there, go out there. Mm-hmm. And so this is, you know, I feel very lucky now that Opening Eye shut the door early on me. Mm-hmm. At the time, I didn't feel lucky. I felt like I had screwed myself over. Yeah. I was like, should I have been nicer to them in the profile so that I could maintain access? Which is a horrible question to ask as a journalist, right? Like, you're supposed to report the truth, and you're always supposed to report in the interest of the public. Like, that is the point of journalism. And in that moment, I was like, relatively junior in my career, I was like, did I misunderstand what journalism about— is, is about? Like, should I have actually been playing the access game? But it was too late. I had the door shut to me. And so I had to build my career understanding that the door, the front door, was never going to be open.

Yeah.

And that actually really strengthened my own ability to just Tell it like it is.

Be objective.

Yeah, and just report what I see are the facts being presented to me, irrespective of whether the company likes it or not. And most often the company really does not like it, but I can continue to do the work. They don't need to open the front door for me. I was still able to do more than 300 interviews.

What you just listened to was the most replayed moment from a previous episode. If you want to listen to that full episode, I've linked it down below. Check the description. Thank you.