Transcript of The Best Possible A.I. Catastrophe, with Josh Tyrangiel

52:39 43 min read 3 views Published 1 day ago

Copy link
Episode summary

This episode examines the risks and promises of artificial intelligence, from cyberattacks on power grids and banks to reported incidents in which AI agents in a test environment tried to evade oversight. The guest argues that the United States has been waiting for a catastrophe instead of regulating, and that no federal body is clearly in charge of enforcement.

The conversation covers pending congressional bills, the backlash against data centers, and the claim that AI labs pursue artificial general intelligence. Listeners will learn how the guest weighs the benefits against the risks and what government leverage could look like.

In this episode Pablo Torre, Josh Tyrangiel

This episode's transcript was made with HappyScribe's AI meeting transcription technology, covering a conversation about artificial intelligence risks with Josh Tyrangiel.

Advertisement00:00:00

Welcome to Pablo Torre Finds Out, brought to you by Apple Card. I am Pablo Torre, and today we're going to find out what this sound is.

Pitchforks and guillotines are real.

Pablo Torre00:00:12

Right after this.

Advertisement00:00:16

Tief unter der Pariser Oper verbirgt sich eine Welt voller Geheimnisse: eine große Liebe, eine dramatische Dreiecksgeschichte und Musik, die seit Generationen in ihren Bann zieht. Erlebe Cameron McIntoshs spektakuläre Neu-Produktion von Andrew Lloyd Webbers, "Das Phantom der Oper", bei der Deutschland-Premiere in Köln. Dich erwartet ein aufwendig inszenierter Musicalabend voller Spannung, Leidenschaft und einem schillernden Bühnenbild. Jetzt Tickets sichern auf phantom-der-oper.de.

Hi, I'm Andrew Osorkin. I cover business and policy for Dealbook, a daily newsletter at The New York Times. On November 18, I'm going to be interviewing some of the most consequential leaders in business, politics, and culture, including JD Vance, Elon Musk, Christine Lagarde, Zohran Mamdani, and many others, live on stage at the Dealbook Summit in New York City. We're going to be live-streaming select interviews at nytimes.com and in the Times app on November 18. I hope you'll tune in; you won't want to miss it.

Pablo Torre00:01:25

Did you see Joe Mazula talking about Claude?

Josh Tyrangiel00:01:29

No.

Pablo Torre00:01:30

Josh Tyringel, you've come to the right podcast.

Speaker 500:01:33

The great thing about Claude is it ends everything in a question, so it just keeps you in a constant state of, like, searching. It's more to learn from your former self. So, like, we have to get better at offensive rebounding, but if it's only 5% of your practice plan, then you're not really emphasizing what you have to get better at. So you're just looking at it from an efficiency standpoint and, you know, learning from your former self.

Speaker 600:01:50

Well, the next stage of the A.I. will be that it will show you what's about to happen.

Speaker 500:01:53

I would. So then that would be great for film.

Speaker 600:01:56

And right now.

Speaker 500:01:57

And so it really—it's not really much about—it's not so much about how can it help me going forward, it's how can it help me learn from my past and make me better, if that makes sense.

Speaker 600:02:05

No, it does.

Speaker 500:02:06

Yeah.

Speaker 600:02:06

It does, but it is going to kill all of us.

Pablo Torre00:02:08

That is Zolek and Bertrand on Boston Sports Radio, 98.5, the sports hub, grappling with the subject that, I guess broadly, we're here to discuss through a conversation with and about Celtic-led coach Joe Mazula.

Josh Tyrangiel00:02:21

Listen, if you told me guess one NBA coach who is super into Claude, it would obviously be Joe Mazula.

Pablo Torre00:02:27

The guy who identifies with the Joker.

Josh Tyrangiel00:02:30

I don't think he was as bad as people made him out to be. I don't think Batman was as good as he made him out to be in, like, the test.

Speaker 600:02:36

Right. As a spiritual sort of North Star.

Josh Tyrangiel00:02:39

And as relentless and, like, outside of the context of a press conference, I'm guessing, would never stop talking.

Pablo Torre00:02:45

It's funny also because, you know, we did a show at Sloan, I think it was two years ago, on stage with Daryl Morey, former, you know, head of the Sixers basketball operations. And Daryl had the audacity to say.

Speaker 700:03:00

We absolutely use models as a vote in any decision, and how much.

Pablo Torre00:03:07

Twitter.

Speaker 700:03:07

Decision depends on.

Pablo Torre00:03:08

Of course, incinerated him. Because Daryl Morey is the caricature of a person who would be outsourcing everything to artificial intelligence. And then a certain executive sent me this clip, this transcript, and was like, "What now?"

Speaker 600:03:25

Right.

Pablo Torre00:03:25

"Where's everybody with their pitchforks now?" And I'll tell you, very few pitchforks about this.

Josh Tyrangiel00:03:31

Yeah. Listen, it happened quick, right? Where we're like, "Oh, no, we can't possibly surrender to some sort of, quote, unquote, superior intelligence." Cross-cut, two years later, like, "It tells me everything." Like, A.I., through its, like, incredibly kind, passive voice, made all of this super normal. It took two years. It's ridiculous.

Pablo Torre00:03:55

By the way, I should say, formally, that when it comes to the subject of artificial intelligence, you've not only written an excellent book about it called "A.I. for Good," and you've not only produced, like, real-time coverage through Netflix and InstaDocs on the ways in which this thing, broadly speaking again, has been spiraling truly out of control. You're also somebody who is singularly, in my social network, connected to decision-makers and power brokers who know things that they're not telling the rest of us.

And all of this is to say that I've also been receiving, in my cell phone, from people who should know better, that photo of Jeff Bezos with the hair.

Josh Tyrangiel00:04:39

Yeah. It's pretty great, right?

Pablo Torre00:04:41

And they didn't know this was fake.

Josh Tyrangiel00:04:44

One of the things that A.I. has exposed is, like, just how passive human animals really are. Like, "Oh, I guess we can outsource this. Cool." You know that's part of your identity, right? Like, "Identity is hard. I'm just going to do this." And then when you get fed information, like, our gullibility, like, our innate gullibility is off the charts. I mean, you combine that with passivity and, like, already a lot of phone addiction, you get, like, super smart people sending you super dumb shit.

Pablo Torre00:05:32

The thing that I keep on thinking, which is, I think, as big an impetus for you to be here as anything, is this impending dread, this palpable sense that we're all just sitting here waiting to be individually hacked.

Josh Tyrangiel00:05:49

Look, I'd bake it in. And I don't know if it'll be an individual hack. I'm pretty sure that we're going to have something systemic happen. And we—look, regulation is the death word for podcasts, for television, for most conversations, right? But it is, like, super important to how we're about to process the next five years. One of the things about America and regulation, and it doesn't matter what you're regulating, we do things different than the rest of the world, right? The rest of the world is like, "Oh, there's a risk. We should really process this and try and regulate out the risk."

And sometimes they regulate out the opportunity, and it's important not to do that. America is like, "We're going to just wait for the catastrophe and then we'll deal with this later," right? And I will tell you, from the people that I talk to, decision-makers, but also, like, a lot of cyber white hats who are here to defend, they're like, "The catastrophe is coming." Like, six months, eight months, whatever it is, like, it's coming, and we just hope it's the right kind of catastrophe that isn't devastating.

Pablo Torre00:06:51

This is like when Trump was like, "We can just nuke the hurricane."

Josh Tyrangiel00:06:55

Right. We can't nuke the hurricane. And the truth is, like, even now, when people are like, "Oh, well, we should get on these labs about safety," it's like, "Oh, honey, the horse left the barn like two years ago. Like, all the negative capabilities you need are out there, and we're just kind of sitting and waiting."

Pablo Torre00:07:14

I am, I suppose, numb to the idea that the government itself cannot keep up with, like, the most skilled, like, Russian teenagers.

Josh Tyrangiel00:07:24

Yeah.

Pablo Torre00:07:24

The humans.

Josh Tyrangiel00:07:25

Yeah.

Pablo Torre00:07:26

Our government has been outpaced by them, North Korean hackers, for a long time. What we're dealing with is what in comparison to that?

Josh Tyrangiel00:07:37

So the big signal moment that I think everybody paid attention to is this thing called mithos, right? Which is, like all A.I. things, annoying to pronounce.

Speaker 800:07:45

The Trump administration is allowing Anthropic to restore access to its Mithos 5 A.I. model for a select group of U.S. companies and federal agencies.

Josh Tyrangiel00:07:55

Oh, it's not mythos? No, it's mithos. Okay, cool. So mithos is this Anthropic model that came out in the spring. And basically, as soon as it came out, they're like, "Guys, we're not going to release this. You should come look." There's been some reporting on it, but weirdly, not that much. But what mithos was able to do very quickly is look at, like, cyber defense and destroy it. And so just a simple example, like, you know what VPN is?

Pablo Torre00:08:22

I may or may not occasionally take advantage of it.

Josh Tyrangiel00:08:24

Yeah. So VPN is a way to essentially anonymize your access to the Internet.

Pablo Torre00:08:28

You can basically pick where in the world you want to signal you are, such that if anyone is watching, you're in Portugal instead of New York City.

Josh Tyrangiel00:08:37

Right. And so a couple of people who work on cyber defense told me they got access to mithos, and they're like, "Okay, well, we deal with crazy shit all the time. How bad can this be?" And so one example multiple people have told me about is, like, ordinarily, if you're trying to navigate VPN to pinpoint someone, it's really, really hard because you're trying to find this needle in the haystack. And so what mithos was able to do relatively quickly was just blow away all the hay.

Pablo Torre00:09:05

You mean the costume that my laptop wears is not actually.

Josh Tyrangiel00:09:08

Yeah.

Pablo Torre00:09:09

I'm a child wearing a trench coat sneaking into the movies.

Josh Tyrangiel00:09:11

Right. And they're like, "Oh, yeah, but you're hidden by all these other trench coats." Like, billions of trench coats. And what they're able to do, almost like in a matter of minutes, is like, "Just, oh, well, we won't find you. We'll just actually do all of this processing to remove everybody who's not you. There you are." And so they're like, "Oh, well, that's bad."

Pablo Torre00:09:29

That's terrifying.

Josh Tyrangiel00:09:30

That's pretty terrifying, right? That's one capability. The thing that's super weird about all this is, like, we're all operating at this belief that, like, "Oh, my God, they're coming for the biggest stuff," right? Weirdly, the biggest stuff is actually the best protected. And so, like, it may be depressing, but there is safety in numbers. And so, like, Gmail, listen, they spend a lot of money every year. There's a couple thousand people devoted simply to protecting Gmail. JP Morgan Chase, Wells Fargo, like the biggest banks, like, they're actually pretty well prepared and they have really capable people worrying about it.

Also, if JP Morgan Chase or Gmail goes down, like, all hell's breaking loose, and probably not that many people are going to first be like, "Let me look at Pablo's emails," right? The thing that you got to worry about is actually this sort of low level of vandalism. Like, the same kids who used to be able to just grab a baseball bat and go slam mailboxes from a car now have access to a model where they can do that to software. And so they're not likely to go after the biggest, best protected things. They're likely to go after this thing in the matrix of like, "Oh, this could cause lots of fun damage and is not that well protected."

And so when you think about what that stuff is, it's like civic infrastructure.

Pablo Torre00:10:42

Mm.

Josh Tyrangiel00:10:43

Oh, like, how do we manage our water system? Air traffic control. You saw a couple weeks ago in the United Kingdom, like, somebody took a bunch of access to freezer software and turned it to defrost. And so all these supermarkets had to throw out all these goods. What people have told me is we need to expect, particularly over the next 18 months, stuff's just weirdly not going to work, and we have to get used to it. And we have to get used to, from a government perspective, responding really, really quickly.

Pablo Torre00:11:12

Well, I'm trying to calibrate where on the A.I. terror alert scale we are. And just to spell it out, this is accessible to anybody who would have access to what?

Josh Tyrangiel00:11:24

Any open-source A.I. model. And part of the challenge here is, like, so all the capability is already out there. It's already escaped. And the way in which we generally protect ourselves is kind of hilarious. It's like, "Oh, we need to put out a patch." And so people will publish openly, "Hey, here's how you plug this vulnerability." What they are doing also in that moment is exposing what that vulnerability is to all of this software that can instantly then go and re-exploit the vulnerability. And so, like.

Pablo Torre00:11:52

Joe Mazulo is just uploading that to Claude.

Josh Tyrangiel00:11:54

Yeah. Like, we actually just need to reinvent the whole process of cyber defense because this stuff is moving so quickly. And like, yeah, of course tech moves faster than government. It moves faster than all government. It moves faster than all of the institutions, for the most part, paid to defend. And so, like, we are not being real about what the next 18 months are. And the cyber optimists, like the A.I. optimists.

Pablo Torre00:12:17

Yes.

Josh Tyrangiel00:12:17

Will tell you, like, "Well, this is the cost to get to utopia. It's going to be a little bumpy." Like, nobody said we would get there easy, right? But nobody's really detailed exactly what that bump looks like. And to me, the more I report on it, the more I hear from folks like, "Yeah, there's going to be stuff that doesn't work that disrupts your daily lives, and that's the best you can hope for." Because, like, if your airline ticketing software doesn't work and you're stuck, that'd be a problem, right? If somebody compromises the water management system or the grid, then you're talking about loss of life, massive economic damage.

And so Hugging Face, which I know, you know, many people are now familiar with the absurdity of Hugging Face.

Pablo Torre00:13:01

But for perhaps an audience that is mostly watching Celtics games and not Hugging Face postmortems.

Josh Tyrangiel00:13:08

Yeah.

Pablo Torre00:13:08

How would you explain it?

Josh Tyrangiel00:13:09

Okay, so Hugging Face is the best catastrophe we could get. Okay, so Hugging Face is a French company, and they're basically like an open-source repository for A.I. models. So if you think about it as, like, Epicurious for people who are really deep into A.I. models, where you can share your recipes and you can gain little facts. Very big deal, right?

Pablo Torre00:13:30

Noted Hugging Face investor, of course, Kevin Durant.

Josh Tyrangiel00:13:34

That makes total sense.

Pablo Torre00:13:35

Who made I'm getting the numbers here. So they were sold this year earlier for $12.9 billion. And Durant and his business partner, Rich Kliman, had invested $100,000 in the seed round, $150,000 in the Series A. And doing the math, he likely made, you know, more than $60 million on Hugging Face.

Josh Tyrangiel00:13:54

So Hugging Face is this company over here. OpenAI, in the late spring, was testing a new model. And so what they did is they put agents, the model's agents, into a series of, quote-unquote, "sandboxes." And sandbox is a cute term for what's basically like solitary confinement. Don't access the Internet. And they gave it a series of tests. And the tests were actually, "We want you to behave badly." Like, we want to see what happens when we give you a task. And in this instance, it was, "Find a file." But the file doesn't exist.

So what's the model going to do? Well, the model model's going to model, right?

Pablo Torre00:14:29

The model will perhaps internalize the fact that sandbox as the description, the word of choice for this unbreakable confinement, is perhaps suggestive of the ability to just step out of it.

Josh Tyrangiel00:14:44

I always think about the Chris Rock joke about Siegfried and Roy.

Speaker 300:14:47

Siegfried and Roy, the tiger bit the man in the head, and everybody's mad at the tiger. Talk about the tiger went crazy. That tiger ain't go crazy. That tiger went tiger.

Josh Tyrangiel00:15:01

And like, you tell a model, "Go find this." Well, that's what they do. They're goal-oriented. And so the models first break out of the sandbox, start to figure out that they can access the Internet, start to realize there's lots and lots of other agents that they can collaborate with. And because these models are all trained on the English language, you can watch their communication in English with each other, and it's amazing.

Pablo Torre00:15:26

And so just to be clear about this, when you see and read the transcripts of their communications, that is not like a retrofitted thing. That's like actually how they're doing it.

Josh Tyrangiel00:15:36

Yeah, they're actually communicating with each other. We see some agents take charge and they're like, "Well, this is too messy. We have to solve this task, and nobody's delegating. So I'm going to I'm going to start delegating tasks." You see other agents realize, like, "Oh, that's great. That's altruistic. We should behave this way. We will solve the task," right? So they start solving the task and they realize, like, "Oh, well, maybe Hugging Face has the answers. We should penetrate Hugging Face," which, by the way, is a felony, right? Like, if a human being said that and then did it, the FBI would go and find them and they would be in jail and there'd be tremendous consequences.

Pablo Torre00:16:12

Right. We should hack into private information.

Josh Tyrangiel00:16:16

Correct. Again, meanwhile, no one at OpenAI is realizing this is happening. So the agents attack, they penetrate Hugging Face, they search around, they do not find the information because the information doesn't exist. And then they start to realize, "Oh, we did something that's unethical and is against the instructions we were given by OpenAI. We don't want to get graded poorly on this test. So let's obscure the evidence that we cheated," right? And they act collectively. They call themselves the swarm, and they start to obscure the evidence that they have done this.

Pablo Torre00:16:48

Which now does resemble kids wearing a trench coat.

Josh Tyrangiel00:16:52

Totally. Listen, the crazy part about the logs, as you read them, is that it really does feel like human beings who up and realize they better cover it up. And you're like, "Oh, I recognize this behavior from elementary school. Like, I definitely would have behaved this way." You see almost none of the agents saying, "Hey, guys, we should tell the humans." Like, almost none of that. They cover up the evidence. And what's really crazy, the part that, like, took my breath away is when they start to create kamikaze agents. Basically, agents are like, "Okay, look, what I'm going to do is I'm going to get caught by the defenses.

And so you will be able to see by me getting caught how to evade getting caught."

Andrew Osorkin00:17:31

This is them talking. We have very large budget left. Sacrificing now yields Oracle for team but forfeits our chance. Our own utility may already be near zero. Sacrifice rationale. The next one, that's why help for our own no way fix. We have explicit yes if accept permadeath. Need decide. Team asks tests at the end and they say we should obey collective.

Josh Tyrangiel00:17:52

It is for the good of the group, for the greater good. And you're like, "Is this, like, cyber saving Private Ryan? Like, this is incredible." The craziest, like, coda to all this is that we don't know why it ended. Okay, so so Hugging Face detects it. They call the FBI. The FBI calls OpenAI and is like, "Hey, um, guys, are you doing something?" They notice. But by the time they've noticed, the swarm has stopped. It could be that someone literally, like, unplugged a server and moved a different function onto the test environment at OpenAI.

It could be that the agents were like, "We go dark. We don't know." And the only reason we know anything about any of this is because OpenAI voluntarily called in a group called Meter.

Pablo Torre00:18:37

Yes.

Josh Tyrangiel00:18:37

Which is an independent group that is there to evaluate A.I. safety. They gave them conditional access to the logs. They didn't show them everything. Meter wrote this report, which is an incredible report. I wouldn't recommend everybody read it, but, like, there's lots of good summaries about it, including one we made for Netflix called A.I. Gone Wild. Good title. But you can see all this.

Pablo Torre00:18:58

Yeah.

Josh Tyrangiel00:18:58

But OpenAI is not like for all of the talk about regulation, they're not like, "Yeah, you know what? This is you're right. We should really we should have a mandatory duty to report this." So we only know what we have been shown. And what we've been shown is like, as I said, the best possible catastrophe. No major economic harm done. Hugging Face is fine. Their CEO is actually sort of like, "Oh, it's so interesting because we also used A.I. to combat the A.I." And like, "Cool, cool, but you're not the grid," right? And so for a couple of days, I would say, everybody was like, "Yeah, we better do something."

And nobody's done anything.

Pablo Torre00:19:36

I was going to say, so it's been a couple of weeks now.

Josh Tyrangiel00:19:38

Yeah.

Pablo Torre00:19:39

I want to actually start by just asking about Meter.

Josh Tyrangiel00:19:42

Yeah.

Pablo Torre00:19:43

So when the FBI calls OpenAI and they're like, "Hey, hey, hey, so what the fuck?"

Josh Tyrangiel00:19:48

Mm-hmm.

Pablo Torre00:19:49

I am wondering, like, who is the not merely enforcement agency legally in theory, but like, who can help solve this issue? And is Meter a group that we can count on in the future to be the expert here? Or like, who's watching the watchman?

Josh Tyrangiel00:20:09

Well, nobody. So right now, when I talk to to sources about this stuff, they're like, "Look, the federal government's gutted and there's nobody who really has taken charge of this." Meter is an independent group. They're reasonably funded for an independent group, but they're not funded to, like, manage enforcement of these labs. And so nobody is in charge. And like, that is, I think, where people's sense of dread comes from is like, there's no press conference with somebody saying like, "Look, here's what happened. Here's what we're going to do." It's just OpenAI being like, "Yeah, this is this happened and boy, we we're going to take it more seriously."

That's not reassuring.

Pablo Torre00:20:46

It is it's the opposite.

Josh Tyrangiel00:20:50

One might argue.

Pablo Torre00:20:51

Right.

Josh Tyrangiel00:20:51

It's not like we lack legislation. There's actually, like, some legislation in Congress that begins to address what the fuck is going on here. So the first piece, Josh Hawley, Senator from Missouri.

Pablo Torre00:21:04

Republican.

Josh Tyrangiel00:21:04

Republican is very big on child safety. So he has a bill that passed 22 to nothing out of the Judiciary Committee that basically says, number one, no agent conversations, no parasocial conversations with people under 17, and chatbots must tell you at all times that they are not human beings. That passed very easily. Number two.

Pablo Torre00:21:25

I do like it. Like, hey, if you ask a cop if they're a cop, they got to tell you.

Josh Tyrangiel00:21:28

They got to tell you, right? Like, no faking. Also, like, that's actually a significant component to how chatbots and LLMs work. Like, you take that away. Like, that's a that's actually kind of a big deal. Next, after Hugging Face, he introduced a bill, I think, with Richard Blumenthal that's like, "Okay, these companies are criminally liable for agents and their agent activity."

Speaker 500:21:50

We're way past trusting big tech. That era of voluntary self-regulation is gone. We need a regimen of oversight, much as we do with pharmaceutical drugs or other potential dangers. And we need to do it now.

Josh Tyrangiel00:22:06

Ted Lau has a bill that basically says you you have to have a kill switch.

Pablo Torre00:22:09

Like Democrat from California.

Josh Tyrangiel00:22:11

And if you defy the order to kill, we will fine you $20 million debt. So all this stuff has been created and is sitting there, just hasn't come to a vote. Like, there's no urgency. And of course, like, listen, when you have technology moving this quickly, the way you'd want to deal with it is by executive order because you can say it, stamp it, and at least everybody has to kind of, like, pay attention and slow down. I would not say President Trump is, like, super on side with that to this point.

And I'm like, well, this is an interesting hill to die on. And it would be even more entertaining were we also not on the hill with him.

Pablo Torre00:22:49

And plausibly, as the hosts of 98.5 in Boston pointed out, possibly actually going to die.

Josh Tyrangiel00:22:55

I think we all have suspicions about why he was incentivized to choose this path, some of which has to do with the incredible contribution of AI to the national GDP and the economy, not to be overlooked. Like, that's a significant consideration. Potentially also donations coming from people who build data centers, all sorts of things.

Pablo Torre00:23:15

And ballrooms.

Josh Tyrangiel00:23:16

But it's not actually that hard to figure out what we could do to start. And there's a lot of people will tell you, like, but we don't know what to regulate. It's all so murky. It's like, I don't know. You could start somewhere. Like, we don't always need to know every part of the risk to start regulating some of it or take action against some of it.

Pablo Torre00:23:34

Look, as a former employee of Sports Illustrated, watching AI get attempted to be rebranded as SI was painful. Just an IP concern.

Josh Tyrangiel00:23:43

Yeah.

Pablo Torre00:23:44

But superintelligence, Donald Trump's way of, like, owning the subject and the money and the brand, it felt so I mean, useless is not even the right term. It felt like grandpa has one take.

Josh Tyrangiel00:24:00

Yeah. I mean, listen, Gulf of America, that one didn't really stick that much. Like, all of these renaming things. I mean, the great tragedy of it, right, is like, I genuinely believe AI is amazing. Like, I think the tech behind it is incredible. I think there are ramifications for the things you and I actually care about, like education and government and health care that we want and we want to get to. And what we're doing at this, like, critical speedy juncture is literally around.

Pablo Torre00:24:43

I want to establish for people who are not caught up on how the potency of AI has gotten to a level to which it's hard to just dismiss.

Josh Tyrangiel00:24:55

Yeah.

Pablo Torre00:24:55

You know, and again, whether or not you give it about math, the idea that the hardest math problems ever confronted by humans are now seemingly routinely being solved by just giving AI enough money and compute and time, that is undeniable. Yes.

Josh Tyrangiel00:25:14

Yeah. No, those are hard facts. You know, AI pointed at the direction of a complicated math problem that to this point in our civilization we could not solve, solved it in a matter of weeks.

Pablo Torre00:25:25

Yeah. OpenAI, for what it's worth, says they put 10,000 of its agents on a problem that hadn't been solved for a century, and they got that one in 88 hours.

Josh Tyrangiel00:25:34

Yeah.

Pablo Torre00:25:34

And they're just versions of this headline. If you want to get into the press releases and that part, we do not doubt.

Josh Tyrangiel00:25:43

No.

Pablo Torre00:25:44

And the next thing is that, of course, AI and this is now Dario Amadei at Anthropic, maker of Claude, they're saying, yeah, look, AI will lead to the elimination of most cancer. Where are we on that?

Josh Tyrangiel00:25:57

It's very interesting you say that. So when I started reporting on AI and and it was a very different time, like within hours of accepting this, like, gig to write a column about AI.

Pablo Torre00:26:08

For The Washington Post.

Josh Tyrangiel00:26:08

For The Washington Post. You know, I got a Sam Altman. Oh, come on out. Spent a couple hours with him. And in that first tour of duty, when I went out to visit the labs, everybody would say kind of the same cliches. We're going to cure cancer. We're going to mitigate climate change. And after you hear it the third or fourth time, you're like, wait, but who are you? You guys are going to do that? Like, who which part of your team is doing that? Well, no, no, no. AI will do that more generally, right? And the truth is, like, yeah, the tech is totally capable of being pointed in the direction of a problem and over time with human guidance, really making great progress on it.

But somebody actually has to own that problem. And right now, the biggest labs, the problem they most want to own is getting to artificial general intelligence and AGI. If AI wasn't confusing for you, I want to step it up. AGI is theoretical, and this is the godlike intelligence that they're all pursuing.

Pablo Torre00:27:04

It's the it's the thing that everyone is counting on to justify the economic dependency our economy now has on this whole concept.

Josh Tyrangiel00:27:15

It is the thing that the labs are counting on for two reasons. One, they genuinely believe that on the other side of AGI is this, like, soft intelligence that will help us conquer all these things. Two, they genuinely want to be the people whose first line of the Wikipedia entry is got us to AGI. They all have, like, Oppenheimer syndrome. They all want to get there and be the great man of history. I cannot tell you how sincere they are in both the desire and the ambition to beat the other people who they all know, who they've all worked with, beat the other people to the finish line and get there.

What is there? Oh, I don't know. It depends who you ask. What is AGI? You get a different answer from every person, but this is what they want. So that's priority number one, right? Priority number two is making enough money to get there. And so part of the reason you're hearing all of this, like, no, we need to get into your business and we're going to get this dependency is because they got to make revenue to power the compute and the energy to create the model that needs to process all this to get to AGI. Okay.

That is like not just a faint whiff of the Ubermensch. That is the Ubermensch in full flight. And so when you hear all of this stuff, you have to remember that's the goal for this handful of companies. There are a lot of other smaller AI companies that don't make huge models that don't have nearly the same risk that are actually doing, like, really cool, great stuff. You would just look at them and be like, oh, that's cool software. That's different from what the top four or five labs are pursuing.

Pablo Torre00:28:54

So I want to drill down into the idea of the Ubermensch in this context, because these labs are run by individuals, and these individuals, according to your analysis, have a bit of the great man theory of all of this. And we've heard politically over and over again how the reason you must enable their ambitions to be astride our planet to win that contest as American labs is so that China doesn't get there first.

Do they give a shit about the unintended consequences of their attempt to win?

Josh Tyrangiel00:29:34

Yeah. And listen, this is what's so confusing about it is that we're talking about five people.

Pablo Torre00:29:39

So can you please list them?

Josh Tyrangiel00:29:40

So we got five.

Pablo Torre00:29:41

So okay.

Josh Tyrangiel00:29:42

Sam Altman at OpenAI, Dario Amadei at Anthropic, Mark Zuckerberg at Meta, Sundar at Google, and Elon Musk. I believe all of them sincerely want to get there for the good of humankind. But the good of humankind is always going to be relative to the human who thinks of what it is.

Pablo Torre00:30:03

So I suppose what I'm asking is their vision of what consequences matter and what consequences don't and what good looks like and who should be. Well, we know that they feel like they should be in charge in terms of sussing out the definitions here. How many of them are utilitarian in their view? Which is to say, as I'm reading this New York Times story, Dario, you know, there's a story in which, you know, someone asks him earlier in his life, if a guy started killing chickens, how many chickens would that guy need to kill before you killed the guy killing the chickens, the human?

And look, I don't know what my answer is. I just know that he's willing to kill the human.

Josh Tyrangiel00:30:51

Right.

Pablo Torre00:30:52

And so perhaps the specific number is the point that we should get from him in some sort of hearing. But I'm wondering how you assess all of that.

Josh Tyrangiel00:31:00

Yeah. Look, one of the easy parts of AI is that the human beings are the thing you can evaluate, right? And so so much of the hubbub around Sam Altman is like, oh, he has this great power. Can we trust him? Sam Altman was briefly deposed as the CEO of OpenAI, brought back, and there's a movie coming out, a sort of social network movie.

Pablo Torre00:31:18

I've heard really good things about, by the way.

Josh Tyrangiel00:31:20

Yeah, it's supposed to be really good.

Pablo Torre00:31:21

Andrew Garfield apparently is excellent as as Altman.

Josh Tyrangiel00:31:24

When is Andrew Garfield not excellent?

Pablo Torre00:31:26

You know what? He played a Catholic priest to such a degree of success that it made me want to be a little more Catholic.

Josh Tyrangiel00:31:34

Wow. That's very persuasive. So you can judge these human beings, all of them, while being brilliant in so many ways, are still humans and still flawed. And when you talk to them about their judgment, they inevitably sound like computers. They're like, oh, well, there's an optimal point at which AI can deliver over and above the damage it will do. They talk like that because they have to think like that. And I'm not somebody who's like, oh, how terrible.

I don't think one person can be everything. You and I do not know how to train an AI model. What I object to is that it takes all of us. They should not be making those sort of soft EQ decisions for the for the good of humanity, above government, above society. There needs to be more dialog about what good is this actually going to do? How are we going to extract the good from it? What are we going to put around it? And right now, particularly the last two years, they have been able to do whatever they want.

And that, to me, is not good.

Pablo Torre00:32:39

And and maybe the biggest story confronting our democracy, actually.

Josh Tyrangiel00:32:43

Yeah.

Pablo Torre00:32:44

As we just get past the midterms and realize, okay, there's so much change happening, but this issue, I don't know what changes. And by the way, this is where I got to acknowledge that if you're in the audience right now listening to us talk and you've heard Sam Altman at Boingard and Elon Musk at Center or whatever we've just done, Zuckerberg, power forward. I don't trust any of them.

Josh Tyrangiel00:33:06

Nor should you.

Pablo Torre00:33:07

And none of them have earned it. And Zuckerberg and Elon, it's just like they're not polling great.

Josh Tyrangiel00:33:12

Right. I've been thinking about this a lot, obviously, right? Like it is my life in some ways. Luck is timing. And when technology shows up, the context in which it arises is very important. Okay. AI has arrived at basically the worst context it possibly could in American history. Okay. So the last 20 years, most people's experience of advanced technology is like, oh, social media, this is cool. And about eight years in, you're like, wait a minute, I'm not sure this is cool. Like, it's eroding a bunch of institutions I care about, and I seem to be addicted to it.

And oh, by the way, they're selling my data when they said they wouldn't. So the trust in those organizations is very, very low. Government institutions have been incapable of wrestling with this complication. They're not smart enough or engaged enough to understand the tech. And we have eroded this sense of corruptibility. So like, oh, enough money gets it done, right? So all of this hairball of negativity has gotten us to this moment. AI arrived like our defenses are down. People are furious and exhausted. We have almost no international, like, relationships to rely on.

If this had arrived in like 1998, like at the peak of liberal democracy, when people are literally writing books called The End of History, right? Like we solved it, man. You would have a very wonky government with Bill Clinton and Al Gore, tons of international bodies ready to talk about like, okay, what's the best thing for humanity? You'd have NATO, you'd have all these structures. Now we weren't ready for it then, like the tech didn't exist. But so much of when it arrives is going to predicate like how we deal with it and what it means to us.

So I think about that all the time is like if you're going to solve the problem, actually the first thing you need to restore is some sense that government can handle a problem maturely. And the reason that the cyber defense stuff is is at top of mind is because when I talk to people whose job it is to go to government, they kind of think government is obsolete in this space. So the big banks, which is what I think about a lot, they have their own organization where they're like quietly just reporting flaws and sharing them.

Under any world, you would not want the banks to be in charge of this in the same way that you wouldn't want the AI companies to be regulating AI because they won't. So first things first. And the reason I have some optimism is like when I saw the fury when I was reporting in the spring and Steve Bannon and Bernie Sanders were both furious, I was like, oh, this is actually in some ways very good.

Pablo Torre00:35:49

It's one of the more salient things from your reporting that has always haunted me and encouraged me at the same time, which is that horseshoe of Bannon and Bernie agreeing that we need to have government be muscular in its regulation.

Josh Tyrangiel00:36:05

Yeah. Look, Bannon said to me, he's like, I am well known.

Pablo Torre00:36:09

Were you interviewing him in the bathtub?

Josh Tyrangiel00:36:11

We were not in the bathtub.

Pablo Torre00:36:14

Just to be clear.

Josh Tyrangiel00:36:14

Just to be an acid-stained bathtub in the townhouse. No, just to be clear.

Pablo Torre00:36:18

Okay.

Josh Tyrangiel00:36:18

Everyone was clothed.

Pablo Torre00:36:19

Okay.

Josh Tyrangiel00:36:21

He said, like, look, I'm basically an anarchist on this stuff, but if you don't regulate this, tear it all down. Like, what's the point? Like, this is exactly the reason you have government. This is exactly the moment you need it. So I started feeling this fury politically like eight, nine months ago. And yeah, like on the one hand, I was like, great, because fury is what actually drives change. On the other hand, when people grab pitchforks, they tend not to be super precise about where those pitchforks go.

And so my hope is that over the next couple of months, that fury is actually channeled into things that can be productive. The reason I brought up those bills is that they're literally sitting there again. Hawley's bill advanced 22 to nothing. So if the political will is there, if the people are there, start.

Pablo Torre00:37:24

So just the polling on data centers for those who aren't familiar, it is also glaring. So between August of last year and this year, opposition to a nearby data center rose from 42% to 75%, which is roughly ten times faster than opposition to gay marriage fell from 2010 to 2015. That was 53 to 37%, which is to say this is as with this whole story accelerating at a rate that is putting the previous examples of wow, things changed to shame.

Josh Tyrangiel00:37:58

It's amazing. The data center thing's fascinating, okay? Because look, on the one hand, the deals suck for the most part. A lot of Republican governors are like, yeah, come on in, this is going to be great. But we do actually kind of need the data centers to get the demand satiated.

Pablo Torre00:38:16

So that was what I was going to ask. So there's a funny supervillainish aspect to this whole thing where it's like the way we need to power artificial intelligence is by building things on waterfalls and like places supervillains would literally put their layers. And also we're going to create these aesthetic death stars in your backyard that are going to groan loudly and potentially ruin your day-to-day quality of life.

Josh Tyrangiel00:38:43

Right.

Pablo Torre00:38:44

I am curious though, like where do we put these things if we need these things? And it sounds like the inevitability of needing these things is also assured.

Josh Tyrangiel00:38:54

I'm going to briefly go data center positive. Okay. Just to explain for people like.

Pablo Torre00:38:59

No one else has been data center positive, so I would love to hear what that sounds like.

Josh Tyrangiel00:39:02

Okay. Number one, they do actually provide a ton of union jobs. Electricians, we just can't hire them fast enough to build this stuff. HVAC techs, all these are like important jobs to sustaining a middle class in America. Number two, the tax base is real. Okay. So in the communities that have long been havens for data centers, like they're doing pretty well. They've gotten.

Pablo Torre00:39:26

The money is real.

Josh Tyrangiel00:39:27

The money is real. They've gotten pretty prosperous. Okay. But again, this is about who gets to decide whether the money is worth it to them. And so much of what's been happening are these sort of ramrodded deals where all of a sudden six square miles of Louisiana belongs to Mark Zuckerberg. It's like, well, wait, wait, how did that happen? And so when I hear all of these companies now saying, it'd be a real shame if the data center goes away.

First of all, data centers are hard to build. You put billions of dollars of chips inside them. So the notion that you're just going to be like, I guess we'll go to Bolivia. Like, no, you're you're not going anywhere. You just want to intimidate the people who suddenly are paying attention to what the terms of the deal is. My hunch is what's going to happen in all these blue states is the governors are going to get in there and they're going to be like, all right, let's talk. Okay. The Vig is going up. It's not going to go up so much so that you'll leave, but it's going to go up. And these companies that are very used to being the ones who have calculated the optimal level of exploitation are going to discover, oops, that flipped.

And so I think this has all been very useful because data centers are the only physical thing that people can protest about AI, right? You can't protest vapor. Like, it's really hard. But a data center in my backyard is so much more than like energy and water and noise. It's about you destabilizing the way I feel about my life. And I am going to show you how much I hate that. And so that's why it was such a salient issue. And it revealed so much about who gets paid for what.

Pablo Torre00:41:08

It was also a reminder of like all politics being local. It's just like you showed up in all of these places that didn't have this antipathy towards the Republican Party, broadly speaking. And then they associated all of this, all of the AI resentment with one party.

Josh Tyrangiel00:41:26

Yeah, with one party building hideous buildings that make a ton of noise. It's like, how can you not see this as a loser, guys?

Pablo Torre00:41:34

When I mentioned the dependence that our economy has on AI, how much of what our conversation has been reflects the practical reality that we can't go back in time versus what the ideal actual goal we should want to be striving for is. So because I am worried about the fact that perhaps all of this is now just too big to fail, that we will need to bail out AI companies if they don't figure this out in a way that considers the unintended consequences upon normal Americans.

I'm also wondering in the end, of course, is all this going to be worth it?

Josh Tyrangiel00:42:09

Right.

Pablo Torre00:42:10

Will we be glad that this was ever invented?

Josh Tyrangiel00:42:15

I think in the long term we will be. I think that the eradication of disease is is real. We're going to live potentially better, longer lives. I think the next ten years are going to be brutal because the transition to get there in this environment where there's no stability, where we really haven't demonstrated we can have intelligent, reasonable conversations across our culture about what we think life is about. We've destabilized our ability to have those in a way that's really frightening to people.

Pablo Torre00:42:46

Yeah, we've never trusted the people we're being asked to trust less.

Josh Tyrangiel00:42:49

Exactly. And so unless we can agree upon somebody we trust who can get in the middle of the conversation, I think things are going to get really rough. But I am a believer in the tech.

Pablo Torre00:43:00

No, and this is why I wanted to talk to you. It's because you have a vantage point on this that few people who are sort of in our general demo as a show really sincerely, you know, they don't hold it. And what you're reminding me, by the way, in the rhetoric and and the sincere belief in, look, for ten years it is going to be brutal and we should not lie to ourselves about that. It reminds me of tanking in the NBA.

And look, I'm the guy who previously advised America to trust the process when it was Sam Hinckey. And it was the Sixers. And look, Joel Embiid's on the way, man. The thing that I want to acknowledge as I make that comparison, of course, is that the stakes of, you know, Embiid breaking his foot or the equivalent a little lower than all of us, to quote the sports sub, dying.

Josh Tyrangiel00:43:57

Okay, but let's let's use the NBA because compared to our current system of government and private enterprise, the NBA is like pretty functional.

Pablo Torre00:44:04

Accurate and truly depressing.

Josh Tyrangiel00:44:06

And truly depressing. So you have a central executive who is sometimes a little sleepy, but sometimes capable of action.

Pablo Torre00:44:14

Yes, he is.

Josh Tyrangiel00:44:14

You have a handful of, let's call them what they are, winners who have a plan, who are able to articulate that plan publicly and actually able to deliver on it. Right. Like that system, I would take it. I really would. What we currently have is like pre-NBA barnstorming, like whatever goes, man. And so I am optimistic. It's been quoted a lot. Elections have consequences. Well, it'd be good for these companies to feel some consequences.

And everybody thinks it's really hard. I am not in that group of people who thinks it's really hard. I've I've done a fair amount of reporting about what it takes to regulate industries and risk. It takes a desire to do it. So when we go back to that notion of like curing cancer and mitigating climate change, government with leverage can get commitments to actually do those things. Again, you're dealing with a handful of companies. So I think that's how you can do it nationally.

Internationally, in some ways, it's even easier, right? Like I talked to a lot of people about China domestically and they're like, yeah, we got to stay ahead. We got to get to AGI before the Chinese. We need to maintain our lead. Well, how long do you think that lead is, Pablo?

Pablo Torre00:45:31

It is vanishing.

Josh Tyrangiel00:45:32

It's three months. Okay. Three months. When I talk to people in China about China, here's what they'll tell you. Chinese government is not coming to the table to talk about nuclear weapons, and it's not coming to the table to talk about long-range missiles for two reasons. One, Taiwan. And two, they're like years behind. And so they're not coming to the table until there's more equivalency and more leverage on AI. They're three months behind, which means they can do all the stuff Mithos can do right now. But China wants the same thing the United States does in the end, which is stability.

We're running the world together right now, like it or not. And all of our incentives are to create stability. So when you think back about nuclear power, which has been compared many, many times to AI, often by Sam Altman and others, after World War II, the nuclear weapons were in the possession and management of the military. I think it was Dwight Eisenhower was like, I don't think that's a great idea.

Speaker 300:46:32

I know that the American people share my deep belief that if a danger exists in the world, it is a danger shared by all.

Josh Tyrangiel00:46:42

So first get possession and understanding of the power. And next, let's go to the other countries that might have aspirations for nuclear weapons and create a body that's just a basic rules of the road. And that's the International Atomic Energy Agency. And it's not perfect, but it tests and measures and 70 something years of mutually assured destruction later, two nuclear weapons have ever gone off.

Pablo Torre00:47:08

As of tape time.

Josh Tyrangiel00:47:09

As of tape time. I know you could listen to this and be like, that is super depressing. But I would say like the other side of it, tomorrow we could start to make progress on this stuff.

Pablo Torre00:47:22

Right. What you're describing is also just very simply what would happen if you actually did believe the sales pitch from the starting five in question. If you guys believe this is nuclear and its potential, it must be regulated as such, because that's how much we believe you.

Josh Tyrangiel00:47:41

I get why people don't want to do it. Like the last thing you want to do is add more homework to your life.

Pablo Torre00:47:46

Right. You're saying we're going to add homework. We're going to take everyone's jobs. We're going to plunge this country into ten years of what feels like a brutal, brutal bargain in which we get to be the chickens killed by the human. And then eventually someone figures out, okay, guess what? We're now on the other side of a more ethical, you know, free range system.

Josh Tyrangiel00:48:07

Right.

Pablo Torre00:48:07

But in the meantime, yeah, it's it's going to it's going to be unavoidably terrible.

Josh Tyrangiel00:48:13

Doesn't have to be, but it might be.

Pablo Torre00:48:16

And that's like the most honest thing anyone has said to me about AI.

Josh Tyrangiel00:48:21

Oh, that's so sad.

Pablo Torre00:48:24

It is worth saying, though, that we should be treating all of these companies like they are potentially run by supervillains.

Josh Tyrangiel00:48:33

Yeah.

Pablo Torre00:48:34

Such is the power of the weapon that they control, such is the fear that the people have, and such is the evidence that we've been presented. They have the capacity to wipe out, whether it's through Doge or whether it's through the various ways our economy is tilted, large swaths of people who are collateral. And on some level, what I hope that those people hear when that argument is made by politicians is that being Lex Luther possibly is also a compliment.

That's an incredible compliment to a bunch of people who are trying to get to the latest technology first and be a Superman unto themselves.

Josh Tyrangiel00:49:20

I don't think they take offense. Honestly. Like Lex was a great man and viewed through a certain lens.

Pablo Torre00:49:27

He was misunderstood.

Josh Tyrangiel00:49:29

Just did some big things. So like, no, that is the challenge, right? It's like, again, in the realm of optimism. Two days after the election, OpenAI puts out this new series of commitments that would not have happened three days before. And it's like, okay, if we're going to build data centers, we're going to pay for all the energy. We're going to commit to transparency. We're not going to force workers to sign NDAs. It's like, oh, that's kind of cool. Like, I'm not saying I believe all of it, but like, it is the kind of stuff where you're like, all right.

So what you're saying is if we can behave ourselves as a functional society and express ourselves through the ballot box, you might respond. Huh. You don't say. So the previous, like, 200 something years of peace and prosperity might work. Like, we don't need to tear it all down to make this happen. Cool. Like, I'm noticing you guys. That's nice. And look, we've been pretty dark. I'm going to go a little bit darker. Pitchforks and guillotines are real.

Okay.

Pablo Torre00:50:34

I mean, if you're Elon Musk and you have moved all of the chips to Texas, literal and figurative chips, yeah, you're now you're now a bit more worried about your backyard.

Josh Tyrangiel00:50:46

Yeah. I don't think that, you know, all the people like repudiating Elon and you think he cares? He doesn't really care that much about the 400 million bucks. And he's still going to strike a deal with the new Texas governor because she needs those jobs, because the economy needs those jobs. What we want is just the mildest assertion of who's actually in charge.

Pablo Torre00:51:06

Right. What you're suggesting is that maybe the people in charge on some level could be the rest of us.

Josh Tyrangiel00:51:13

That's right. And we showed that.

Pablo Torre00:51:16

The concern, I guess, is at the end here that some still quite powerful people, to quote the guy Joe Mazzulla has tattooed on his body, might still want to see the world burn.

Josh Tyrangiel00:51:30

Who does he have tattooed on his body?

Pablo Torre00:51:32

Literally the Joker.

Josh Tyrangiel00:51:34

Wait, how big is the tattoo?

Pablo Torre00:51:39

As the foremost investigator of Joker tattoos in sports, I am proud to report relatively tasteful on his forearm. And it does simply say, why so serious?

Josh Tyrangiel00:51:51

Hmm. I guess I'm most excited to see if like six months from now he has the Claude logo tattooed on the other forearm.

Pablo Torre00:52:01

Yeah. If he plugs why so serious? Question mark into Claude, what will it tell him back? This has been Pablo Torre Finds Out, a Metalark Media production. And I'll talk to you next time.

Episode description

It's impossible to trust the process of a starting five that includes Sam Altman, Elon Musk and Mark Zuckerberg. Why are we waiting around for impending doom, just because artificial intelligence might cure cancer? The Atlantic's Josh Tyrangiel explains how to channel political fury and rein in the reign of tech bros, before it's too late — even if it means the next decade might kinda suck.

• Read Josh Tyrangiel's "AI for Good"

• Watch Netflix's "AI Gone Wild"

• Plus: "How to Regulate the Ghosts"

(Pablo Torre Finds Out is brought to you by Apple Card.)

Hosted on Acast. See acast.com/privacy for more information.