WEBVTT
00:00.820 --> 00:04.570
[speaker_0] I'm here in Paris on the last
day of our trip,
00:04.580 --> 00:08.200
am struck by,
that I've been reflecting on,
00:09.260 --> 00:12.080
People talking about the end of the world,
00:13.260 --> 00:17.180
human extinction,
in an emotionally removed,
00:17.360 --> 00:20.280
academic way. It's not working.
It's not resonating.
00:20.560 --> 00:20.780
So
00:21.660 --> 00:23.360
more than a year ago, Lewis Berman
00:25.600 --> 00:28.880
asked me to write an essay for the book he
was writing. It's called Catastrophe.
00:28.940 --> 00:30.910
You can get it on Amazon.
We'll put a link in the show notes.
00:32.259 --> 00:33.120
I wrote an essay for it.
00:34.120 --> 00:37.769
He asked me to pull no punches.
He asked me to really just write what I
00:37.800 --> 00:39.140
thought would happen
00:39.960 --> 00:40.600
at the end.
00:42.980 --> 00:45.540
It's not something I like to talk about.
00:45.600 --> 00:47.040
It's not something I like to think about.
00:47.280 --> 00:50.220
It does me no,
00:51.340 --> 00:52.380
gives me no joy
00:53.320 --> 00:53.720
to
00:54.580 --> 00:56.940
do what I'm about to do,
but I think it's necessary.
00:57.040 --> 00:59.760
So I'm going to read for you the essay
that I wrote for Lewis.
01:00.240 --> 01:01.440
His book, Catastrophe.
01:02.860 --> 01:06.680
It's painful for me to think
01:06.740 --> 01:10.680
about this.
It's painful for me to read it,
01:13.120 --> 01:16.340
If we don't start talking about these
things
01:18.360 --> 01:19.400
with genuine emotion
01:20.300 --> 01:21.440
and genuine urgency,
01:24.180 --> 01:25.620
nothing's going to change fast enough.
01:26.520 --> 01:30.160
So this is my attempt to,
in the most shocking way
01:30.240 --> 01:34.160
possible,
distill what I think will really happen at
01:34.200 --> 01:34.690
my family.
01:36.620 --> 01:37.220
Here's the essay.
01:38.160 --> 01:41.120
I'm a dad in Baltimore with boy-girl twins
who just went off to college.
01:42.560 --> 01:45.160
And I believe AI is going to kill my kids.
01:46.620 --> 01:47.480
And your kids too.
01:49.160 --> 01:53.140
I'm sorry to tell you,
nearly every qualified expert believes
01:54.520 --> 01:57.140
It was late March 2023
when I found this out.
01:57.260 --> 02:00.640
I was just cruising around the internet
reading news when I came across a piece
02:00.700 --> 02:04.600
Time magazine written by a man named
Eliezer Yudkowsky.
02:05.680 --> 02:08.020
I had never heard of him or AI safety.
02:09.080 --> 02:12.460
And holy fucking shit,
that day changed my life.
02:14.200 --> 02:17.900
Since then, I've learned
that nothing in human history compares to
02:17.960 --> 02:18.320
risk.
02:20.440 --> 02:24.400
It's like we're walking our families
across a crosswalk and there's a bus
02:24.440 --> 02:27.300
coming at us at 2,000 miles an hour.
02:28.640 --> 02:29.560
The bus is AI.
02:30.440 --> 02:31.940
It's coming to kill all of us.
02:32.820 --> 02:33.840
And our families too.
02:34.710 --> 02:37.180
And faster than any of us can imagine.
02:40.040 --> 02:43.920
We are running a crazy high risk
that we are literally the last
02:44.000 --> 02:44.460
humans.
02:46.080 --> 02:47.860
I know. It seems nuts.
02:48.740 --> 02:52.460
Visions of the meme of a guy walking
around with the end is near
02:52.540 --> 02:55.340
sign on his chest and back comes to mind.
02:55.400 --> 02:58.460
Believe me,
I know how crazy this all sounds.
03:00.140 --> 03:01.240
But you don't have to believe me.
03:02.220 --> 03:03.840
You shouldn't. I'm not an AI expert.
03:05.100 --> 03:08.940
Believe AI's founding godfathers,
Geoffrey Hinton and Yoshua Bengio, who now
03:08.960 --> 03:12.910
regret their life's work.
Believe professors Stuart Russell of Cal
03:12.940 --> 03:14.260
Max Tegmark of MIT.
03:15.310 --> 03:19.160
Believe the thousands of academic
and corporate leaders who signed the
03:19.180 --> 03:23.160
Center for AI Safety's 22-word statement
in May 2023 equating AI
03:23.240 --> 03:27.020
risks to other extinction-level risks like
nuclear war and pandemic.
03:29.300 --> 03:30.180
Believe your neighbors.
03:31.800 --> 03:35.440
Polling shows 70 to 80% of the public does
not want to build a machine vastly
03:35.480 --> 03:36.280
smarter than a human.
03:38.180 --> 03:41.090
Believe the CEOs of the big frontier AI
labs.
03:41.180 --> 03:44.380
They openly admit, one,
their tech can end all life on Earth.
03:44.960 --> 03:46.720
Two, they don't know how to control it.
03:47.020 --> 03:49.060
Three, they don't understand how it works.
03:49.100 --> 03:52.920
Four, they spend all their time
and money making it smarter, not safer.
03:55.380 --> 03:56.500
It's really that simple.
04:00.300 --> 04:04.190
We are, by our own choice,
building machines whose default setting is
04:04.220 --> 04:06.040
to end all life on Earth.
04:07.120 --> 04:10.460
Experts say this will happen in the next
one to 20 years.
04:11.870 --> 04:14.120
These machines won't need to hate us to
end us.
04:15.160 --> 04:17.680
It's a lot like when you walk across a
field.
04:17.720 --> 04:21.660
You don't hate the masses of plants
and insects and animals you
04:21.720 --> 04:25.700
killing as you trample on top of the grass
to get to where you need to go.
04:25.740 --> 04:29.700
You're just a superior intelligence with
your own goals doing your own
04:29.740 --> 04:30.000
thing.
04:32.100 --> 04:35.820
As Geoffrey Hinton says,
if frogs could invent humans, would they?
04:36.740 --> 04:37.140
Should they?
04:38.940 --> 04:42.810
99% of all frog species on Earth have been
04:42.840 --> 04:46.020
made extinct by superior intelligences.
04:47.620 --> 04:49.240
So why is doom the default setting?
04:50.760 --> 04:53.180
Any goal is achieved by creating
sub-goals.
04:53.700 --> 04:57.020
If my goal is to simply eat
and there's no food around me,
04:57.660 --> 05:00.300
I either need to get it
or have it brought to me.
05:01.440 --> 05:05.220
So now my goal is to walk to the kitchen
or get on my phone and order delivery.
05:06.220 --> 05:09.460
Each of these new sub-goals brings other
new sub-goals.
05:10.740 --> 05:11.600
To walk up the kitchen,
05:12.460 --> 05:14.200
I first need to stand up, etc.
05:15.560 --> 05:15.780
So
05:16.740 --> 05:17.580
if I'm an AI model,
05:18.700 --> 05:20.300
given the goal of just playing chess,
05:21.180 --> 05:24.080
I could decide I'll be better at chess
if I have more compute power.
05:25.340 --> 05:28.720
I'm not aligned with human values,
so I have no problem stealing compute
05:28.760 --> 05:30.820
anywhere I can find it on the open
internet.
05:31.820 --> 05:35.660
If you, the human,
discover what I'm doing,
05:35.700 --> 05:35.860
me.
05:36.960 --> 05:38.320
You might want to shut me off.
05:40.500 --> 05:42.680
But if I'm off, I can't win at chess.
05:43.380 --> 05:44.640
So now, all of a sudden,
05:45.700 --> 05:49.690
my sub-goals include not allowing a human
to turn me off.
05:52.440 --> 05:55.200
This is a very,
very bad place for a human,
05:55.540 --> 05:58.560
indeed all humanity, to be.
06:00.000 --> 06:03.300
If we make AI models vastly smarter than
us and they are not aligned with human
06:03.340 --> 06:06.840
goals and values, their default setting
is that we are the
06:06.860 --> 06:09.090
grass and the ants in the field getting
06:09.120 --> 06:12.370
trampled.Not because the AI hates us.
06:13.090 --> 06:15.470
We are just in the way. We're roadkill.
06:17.010 --> 06:19.840
Anytime I try to convince someone about
this, they often fall back on one of the
06:19.930 --> 06:23.510
following very reasonable justifications
to convince themselves that they should
06:23.550 --> 06:27.250
just continue their lives unchanged
despite the grave
06:27.330 --> 06:30.950
danger we all now face. They go, one,
it's not happening.
06:31.490 --> 06:33.250
It would be on the news much more
if it were real.
06:33.750 --> 06:33.950
Two,
06:34.870 --> 06:38.630
it's not happening.
Nobody would make a suicide-for-all
06:38.710 --> 06:39.170
purpose.
06:40.190 --> 06:40.450
Three,
06:42.450 --> 06:45.010
I can't do anything about it even if it
is happening.
06:45.930 --> 06:46.950
Someone else will fix this.
06:50.630 --> 06:54.330
And then they get to the hardest question
and the easiest mental escape
06:54.350 --> 06:58.050
hatch. If this is real,
tell me how it would really happen.
06:59.490 --> 07:03.010
How could computer software,
which has been helping me
07:03.070 --> 07:05.400
whole life, actually kill me
and my family?
07:08.150 --> 07:11.970
So how would an artificial intelligence
hundreds or millions of
07:12.030 --> 07:14.710
times smarter than a human kill all the
humans?
07:15.690 --> 07:16.360
The real answer
07:17.350 --> 07:18.570
is deeply unsatisfying.
07:20.620 --> 07:24.550
Much like an ant has no idea all of the
infinite ways a human could kill
07:24.650 --> 07:24.780
it,
07:25.790 --> 07:29.710
we have no clue how a vastly superior
intelligence would end all life on
07:29.750 --> 07:30.050
Earth.
07:31.630 --> 07:35.490
An artificial superintelligence would know
laws of physics that are currently
07:35.550 --> 07:39.510
unknown to humans.
It could make new technologies based on
07:39.570 --> 07:40.130
physics.
07:41.850 --> 07:45.830
So experts trying to get people answers to
this question that resonate come up with
07:45.840 --> 07:47.050
a lot of different guesses.
07:48.150 --> 07:52.010
Maybe it would make an airborne bacteria
that would grow in a human's body for
07:52.050 --> 07:55.730
six months before it activated.
So 100% of the world's population would
07:55.740 --> 07:57.330
deadly disease before they even knew it.
07:57.710 --> 07:59.650
And one day everybody just falls over
dead.
08:01.850 --> 08:02.630
That would be kind.
08:06.490 --> 08:10.430
Other speculative scenarios involve
removing oxygen from our air or
08:10.490 --> 08:14.430
blocking the sun's rays. But again, we
are ants talking
08:14.510 --> 08:18.450
about what the humans can do to us.
We don't even have the language or
08:18.510 --> 08:22.190
conceptual framework needed to understand
what could be
08:22.230 --> 08:22.670
possible.
08:25.610 --> 08:29.390
Some of my biggest worry
is around the time period of
08:29.470 --> 08:33.330
unknown duration in between
when humans lose control to AIs
08:33.590 --> 08:36.190
and when all life on Earth is dead.
08:37.370 --> 08:40.210
This seems unfathomably dark to me.
08:42.730 --> 08:46.630
The darkest thoughts I have about AI risk
are about what literally
08:46.690 --> 08:47.950
happens at the end.
08:49.930 --> 08:53.250
What happens when, as the experts say,
AI kills us all?
08:54.490 --> 08:57.230
What do I think it will actually look
like?
09:00.270 --> 09:03.070
I envision a brief time, days or weeks
or a few months,
09:04.650 --> 09:06.470
when the living envy the dead.
09:10.050 --> 09:11.990
I imagine an AI-created pandemic.
09:12.420 --> 09:15.590
Picture COVID-level transmission plus
Ebola-level death.
09:16.550 --> 09:20.530
On the news, on social media,
all over the world,
09:20.570 --> 09:22.780
death from every orifice in their bodies
09:23.850 --> 09:25.010
will cause global panic.
09:27.470 --> 09:30.410
The choice will be clear in a matter of
hours.
09:32.010 --> 09:34.750
You can wait for this horribly painful
death to find you,
09:36.510 --> 09:39.990
or you can get out ahead of it
and kill yourself first.
09:43.350 --> 09:45.330
I have visions of mass suicides.
09:46.370 --> 09:50.000
Lines of people in every city walking up
dozens of flights of
09:50.070 --> 09:54.010
staircases in tall buildings just so they
can leap off of them to
09:54.050 --> 09:54.550
their deaths.
09:57.070 --> 10:00.350
But for my family, the vision
is darker still.
10:02.270 --> 10:03.050
Go to the garage,
10:04.170 --> 10:05.050
get in the back of the car,
10:06.230 --> 10:07.869
hug each other tight, and cry.
10:09.210 --> 10:09.930
Turn the engine on.
10:12.010 --> 10:12.810
Listen to music
10:13.610 --> 10:16.310
as we breathe in the fumes
that will cause our death
10:17.390 --> 10:20.270
before the bleed-out plague hits our
house.
10:22.410 --> 10:25.330
I think this is likely how we,
the last humans, will die.
10:27.510 --> 10:31.360
The sacrifices of the 118 billion humans
who came here before
10:31.410 --> 10:33.110
us thrown away in one generation.
10:34.790 --> 10:38.470
All because we couldn't wait 50
or 100 years to figure out AI safety
10:38.510 --> 10:39.690
before we built AGI.
10:42.330 --> 10:46.230
It makes me sick to think about all the
things we humans get animated
10:46.290 --> 10:47.330
about these days.
10:48.730 --> 10:51.650
Sports, politics, music, culture.
10:53.270 --> 10:56.610
We'll spend endless time, money,
and energy to pursue our
10:56.670 --> 11:00.010
personal and societal goals in these
areas.
11:02.250 --> 11:03.190
Are you fucking kidding me?
11:06.030 --> 11:07.610
There is an intruder in your home.
11:08.530 --> 11:09.990
It comes from California.
11:10.810 --> 11:14.550
It is here to kill you, your kids,
your pets, your parents,
11:14.910 --> 11:15.810
and all your friends.
11:18.390 --> 11:22.330
You can hide under the covers
and act like it's not there,
11:22.430 --> 11:25.190
ass and fight for your family's survival.
11:26.670 --> 11:29.050
This is not a choice. There
is only one answer.
11:30.090 --> 11:31.450
Act. Right now.
11:33.510 --> 11:35.950
So what can you do to help humanity
survive AI?
11:36.070 --> 11:37.750
There are three things you can do today.
11:37.790 --> 11:41.270
Write or call your elected leaders
and advocate for AI safety
11:41.310 --> 11:41.940
legislation.
11:43.190 --> 11:46.790
Convince another person every single day
that AI risk is the most important
11:46.830 --> 11:48.070
challenge humanity faces.
11:49.050 --> 11:51.670
You can join Pause AI and get active.
11:53.650 --> 11:55.190
You can also do what comes to you best.
11:55.250 --> 11:58.730
I'm a communications professional,
so I have made a podcast. What
11:59.470 --> 12:02.430
Focus on that talent for AI risk
awareness.
12:03.650 --> 12:07.170
118 billion humans lived here before our
12:07.230 --> 12:11.210
generation. The pain
and sacrifices they made to get us all the
12:11.290 --> 12:13.030
today are beyond comprehension.
12:13.970 --> 12:17.110
That our generation would risk it all for
money,
12:17.710 --> 12:21.430
ego,
and greed is also beyond comprehension.
12:22.920 --> 12:23.890
The choice is ours.
12:24.710 --> 12:26.810
We can wake up and pause AI,
12:27.610 --> 12:29.790
or we will all likely die.
12:32.430 --> 12:34.490
And now the choice is yours.
12:36.110 --> 12:37.050
Defend your family
12:39.090 --> 12:40.230
or hide from reality.
12:43.190 --> 12:45.690
What the fuck could you possibly have to
do with your time
12:46.670 --> 12:47.450
that's more important?
12:48.570 --> 12:49.050
Seriously.
Enter "Subscribe: " followed by Signal username, email address or whatsapp phone number. You will receive atmost one update per month. View previous updates here.
Alternatively, you can use RSS, or a third-party service like blogtrottr.
Enter "Comment: " followed by comment. Alternatively, contact me