[speaker_0] I'm here in Paris on the last day of our trip,
am struck by, that I've been reflecting on,
People talking about the end of the world,
human extinction, in an emotionally removed,
academic way. It's not working. It's not resonating.
So
more than a year ago, Lewis Berman
asked me to write an essay for the book he was writing. It's called Catastrophe.
You can get it on Amazon. We'll put a link in the show notes.
I wrote an essay for it.
He asked me to pull no punches. He asked me to really just write what I
thought would happen
at the end.
It's not something I like to talk about.
It's not something I like to think about.
It does me no,
gives me no joy
to
do what I'm about to do, but I think it's necessary.
So I'm going to read for you the essay that I wrote for Lewis.
His book, Catastrophe.
It's painful for me to think
about this. It's painful for me to read it,
If we don't start talking about these things
with genuine emotion
and genuine urgency,
nothing's going to change fast enough.
So this is my attempt to, in the most shocking way
possible, distill what I think will really happen at
my family.
Here's the essay.
I'm a dad in Baltimore with boy-girl twins who just went off to college.
And I believe AI is going to kill my kids.
And your kids too.
I'm sorry to tell you, nearly every qualified expert believes
It was late March 2023 when I found this out.
I was just cruising around the internet reading news when I came across a piece
Time magazine written by a man named Eliezer Yudkowsky.
I had never heard of him or AI safety.
And holy fucking shit, that day changed my life.
Since then, I've learned that nothing in human history compares to
risk.
It's like we're walking our families across a crosswalk and there's a bus
coming at us at 2,000 miles an hour.
The bus is AI.
It's coming to kill all of us.
And our families too.
And faster than any of us can imagine.
We are running a crazy high risk that we are literally the last
humans.
I know. It seems nuts.
Visions of the meme of a guy walking around with the end is near
sign on his chest and back comes to mind.
Believe me, I know how crazy this all sounds.
But you don't have to believe me.
You shouldn't. I'm not an AI expert.
Believe AI's founding godfathers, Geoffrey Hinton and Yoshua Bengio, who now
regret their life's work. Believe professors Stuart Russell of Cal
Max Tegmark of MIT.
Believe the thousands of academic and corporate leaders who signed the
Center for AI Safety's 22-word statement in May 2023 equating AI
risks to other extinction-level risks like nuclear war and pandemic.
Believe your neighbors.
Polling shows 70 to 80% of the public does not want to build a machine vastly
smarter than a human.
Believe the CEOs of the big frontier AI labs.
They openly admit, one, their tech can end all life on Earth.
Two, they don't know how to control it.
Three, they don't understand how it works.
Four, they spend all their time and money making it smarter, not safer.
It's really that simple.
We are, by our own choice, building machines whose default setting is
to end all life on Earth.
Experts say this will happen in the next one to 20 years.
These machines won't need to hate us to end us.
It's a lot like when you walk across a field.
You don't hate the masses of plants and insects and animals you
killing as you trample on top of the grass to get to where you need to go.
You're just a superior intelligence with your own goals doing your own
thing.
As Geoffrey Hinton says, if frogs could invent humans, would they?
Should they?
99% of all frog species on Earth have been
made extinct by superior intelligences.
So why is doom the default setting?
Any goal is achieved by creating sub-goals.
If my goal is to simply eat and there's no food around me,
I either need to get it or have it brought to me.
So now my goal is to walk to the kitchen or get on my phone and order delivery.
Each of these new sub-goals brings other new sub-goals.
To walk up the kitchen,
I first need to stand up, etc.
So
if I'm an AI model,
given the goal of just playing chess,
I could decide I'll be better at chess if I have more compute power.
I'm not aligned with human values, so I have no problem stealing compute
anywhere I can find it on the open internet.
If you, the human, discover what I'm doing,
me.
You might want to shut me off.
But if I'm off, I can't win at chess.
So now, all of a sudden,
my sub-goals include not allowing a human to turn me off.
This is a very, very bad place for a human,
indeed all humanity, to be.
If we make AI models vastly smarter than us and they are not aligned with human
goals and values, their default setting is that we are the
grass and the ants in the field getting
trampled.Not because the AI hates us.
We are just in the way. We're roadkill.
Anytime I try to convince someone about this, they often fall back on one of the
following very reasonable justifications to convince themselves that they should
just continue their lives unchanged despite the grave
danger we all now face. They go, one, it's not happening.
It would be on the news much more if it were real.
Two,
it's not happening. Nobody would make a suicide-for-all
purpose.
Three,
I can't do anything about it even if it is happening.
Someone else will fix this.
And then they get to the hardest question and the easiest mental escape
hatch. If this is real, tell me how it would really happen.
How could computer software, which has been helping me
whole life, actually kill me and my family?
So how would an artificial intelligence hundreds or millions of
times smarter than a human kill all the humans?
The real answer
is deeply unsatisfying.
Much like an ant has no idea all of the infinite ways a human could kill
it,
we have no clue how a vastly superior intelligence would end all life on
Earth.
An artificial superintelligence would know laws of physics that are currently
unknown to humans. It could make new technologies based on
physics.
So experts trying to get people answers to this question that resonate come up with
a lot of different guesses.
Maybe it would make an airborne bacteria that would grow in a human's body for
six months before it activated. So 100% of the world's population would
deadly disease before they even knew it.
And one day everybody just falls over dead.
That would be kind.
Other speculative scenarios involve removing oxygen from our air or
blocking the sun's rays. But again, we are ants talking
about what the humans can do to us. We don't even have the language or
conceptual framework needed to understand what could be
possible.
Some of my biggest worry is around the time period of
unknown duration in between when humans lose control to AIs
and when all life on Earth is dead.
This seems unfathomably dark to me.
The darkest thoughts I have about AI risk are about what literally
happens at the end.
What happens when, as the experts say, AI kills us all?
What do I think it will actually look like?
I envision a brief time, days or weeks or a few months,
when the living envy the dead.
I imagine an AI-created pandemic.
Picture COVID-level transmission plus Ebola-level death.
On the news, on social media, all over the world,
death from every orifice in their bodies
will cause global panic.
The choice will be clear in a matter of hours.
You can wait for this horribly painful death to find you,
or you can get out ahead of it and kill yourself first.
I have visions of mass suicides.
Lines of people in every city walking up dozens of flights of
staircases in tall buildings just so they can leap off of them to
their deaths.
But for my family, the vision is darker still.
Go to the garage,
get in the back of the car,
hug each other tight, and cry.
Turn the engine on.
Listen to music
as we breathe in the fumes that will cause our death
before the bleed-out plague hits our house.
I think this is likely how we, the last humans, will die.
The sacrifices of the 118 billion humans who came here before
us thrown away in one generation.
All because we couldn't wait 50 or 100 years to figure out AI safety
before we built AGI.
It makes me sick to think about all the things we humans get animated
about these days.
Sports, politics, music, culture.
We'll spend endless time, money, and energy to pursue our
personal and societal goals in these areas.
Are you fucking kidding me?
There is an intruder in your home.
It comes from California.
It is here to kill you, your kids, your pets, your parents,
and all your friends.
You can hide under the covers and act like it's not there,
ass and fight for your family's survival.
This is not a choice. There is only one answer.
Act. Right now.
So what can you do to help humanity survive AI?
There are three things you can do today.
Write or call your elected leaders and advocate for AI safety
legislation.
Convince another person every single day that AI risk is the most important
challenge humanity faces.
You can join Pause AI and get active.
You can also do what comes to you best.
I'm a communications professional, so I have made a podcast. What
Focus on that talent for AI risk awareness.
118 billion humans lived here before our
generation. The pain and sacrifices they made to get us all the
today are beyond comprehension.
That our generation would risk it all for money,
ego, and greed is also beyond comprehension.
The choice is ours.
We can wake up and pause AI,
or we will all likely die.
And now the choice is yours.
Defend your family
or hide from reality.
What the fuck could you possibly have to do with your time
that's more important?
Seriously.