WEBVTT
00:01.620 --> 00:06.500
But I think the development of full artificial
intelligence could spell the
00:06.539 --> 00:08.100
end of the human race.
00:11.859 --> 00:16.680
Once humans develop artificial intelligence, it
would take off on
00:16.699 --> 00:20.719
its own and redesign itself at an ever increasing
rate.
00:23.100 --> 00:27.859
In between five and 20 years from now, there's a
good chance, a
00:27.899 --> 00:30.839
50% chance, we'll get AI smarter than us.
00:32.680 --> 00:34.059
I think we're
00:34.899 --> 00:39.740
moving into a period when, for the first time
ever, we may have
00:39.759 --> 00:41.339
things more intelligent than us.
00:41.340 --> 00:43.879
You believe they can understand?
00:43.919 --> 00:44.339
Yes.
00:44.360 --> 00:47.120
You believe they are intelligent?
00:47.279 --> 00:47.659
Yes.
00:47.899 --> 00:51.959
You believe these systems have experiences of
their own
00:52.699 --> 00:55.419
and can make decisions based on those experiences?
00:55.860 --> 00:57.599
In the same sense as people do, yes.
00:57.599 --> 00:57.779
Are
00:58.599 --> 00:59.359
they conscious?
00:59.939 --> 01:02.639
I think they probably don't have much
self-awareness at present.
01:02.999 --> 01:04.799
So in that sense, I don't think they're conscious.
01:04.819 --> 01:07.859
Will they have self-awareness, consciousness?
01:08.079 --> 01:08.179
Oh, yes.
01:08.179 --> 01:08.579
Yes?
01:08.739 --> 01:10.099
Oh, yes, I think they will in time.
01:10.299 --> 01:15.219
And so human beings will be the second most
intelligent beings on the
01:15.279 --> 01:15.739
planet?
01:15.739 --> 01:16.939
Yeah.
01:17.499 --> 01:19.859
You believe that ChatGPT-4
01:20.639 --> 01:21.659
understands?
01:21.659 --> 01:22.319
I believe it
01:22.999 --> 01:24.539
definitely understands, yes.
01:24.539 --> 01:25.979
And in five years time?
01:25.999 --> 01:29.939
I think in five years time, it may well be able to
reason better than us.
01:30.159 --> 01:32.819
And we can't afford to get it wrong with these
things.
01:32.859 --> 01:34.759
Can't afford to get it wrong, why?
01:35.339 --> 01:36.759
Well, because they might take over.
01:37.759 --> 01:39.219
Take over from humanity?
01:39.399 --> 01:40.579
Yes, that's a possibility.
01:40.579 --> 01:41.059
Why would they want to?
01:41.059 --> 01:42.219
I'm not saying it will happen.
01:43.239 --> 01:47.339
If we could stop them ever wanting to, that would
be great, but it's not clear we can
01:47.359 --> 01:48.619
stop them ever wanting to.
01:50.419 --> 01:54.039
So the question is, what's going to happen when
we've created beings that are more
01:54.059 --> 01:54.999
intelligent than us?
01:56.099 --> 01:56.719
And
01:57.959 --> 02:01.039
we don't know what's going to happen. We've never
been in that situation before.
02:01.679 --> 02:06.399
Anybody who says it's all going to be fine is
crazy, and anybody who says they inevitably
02:06.419 --> 02:09.339
going to take over, they're crazy too. We really
don't know.
02:09.739 --> 02:13.819
But because we really don't know, it will make a
lot of sense to do a lot of basic
02:13.859 --> 02:18.719
research now on whether we can stay in control of
things that we create that are
02:18.759 --> 02:19.879
more intelligent than us.
02:20.419 --> 02:23.199
What are the implications
02:24.019 --> 02:28.999
of these systems autonomously writing their own
computer code and executing
02:29.199 --> 02:30.379
their own computer code?
02:30.379 --> 02:32.479
That's a serious worry, right?
02:33.119 --> 02:36.539
So one of the ways in which these systems might
escape control
02:37.259 --> 02:40.979
is by writing their own computer code to modify
themselves,
02:42.099 --> 02:45.079
and that's something we need to seriously worry
about.
02:45.079 --> 02:50.019
What do you say to someone who might argue, "If
the systems become malevolent,
02:50.119 --> 02:51.559
just turn them off?"
02:51.559 --> 02:54.739
They will be able to manipulate people, right?
02:55.099 --> 02:59.959
And these will be very good at convincing people
because they'll have learned from all
02:59.979 --> 03:03.959
the novels that were ever written, all the books
by Machiavelli,
03:04.939 --> 03:09.619
all the political connivances, they'll know all
that stuff, they'll know how to do it.
03:13.199 --> 03:17.499
We thought AI would happen in decades or
centuries,
03:18.339 --> 03:18.539
but it
03:19.419 --> 03:24.159
might be just in a few years. And I saw how it
could go wrong because we
03:24.159 --> 03:26.619
didn't and we still don't have ways to make sure
03:28.239 --> 03:31.479
this technology eventually doesn't turn against
us.
03:32.259 --> 03:36.979
So two months later, I'm a leading signatory of
the pause
03:37.019 --> 03:41.619
letter where we and 30,000 other people asked the
AI
03:41.679 --> 03:46.059
labs to wait six months before building the next
version.
03:47.439 --> 03:49.039
As you can guess, nobody paused.
03:49.939 --> 03:50.339
Then,
03:51.579 --> 03:55.939
with the same people and the leading executives of
the AI labs,
03:56.419 --> 04:00.279
I sign a statement, and the statement goes,
04:00.279 --> 04:03.659
"Mitigating the risk of extinction
04:04.299 --> 04:06.939
from AI should be a global priority."
04:08.739 --> 04:12.159
I then testify in front of the US Senate a- about
those risks.
04:12.359 --> 04:17.299
I travel the world to talk about it. Okay, so this
was a controlled experiment.
04:18.099 --> 04:22.499
What is it going to be in a few years when these
systems are much more powerful?
04:22.719 --> 04:27.579
There's already studies showing that they can
learn to avoid showing
04:27.619 --> 04:31.379
their deceptive plans in these chain of thoughts
that we can monitor.
04:32.619 --> 04:36.999
When they'll be more powerful, they would not just
copy themselves on one other computer
04:36.999 --> 04:38.239
and start that program.
04:38.979 --> 04:42.759
They would copy themselves over hundreds or
thousands of computers over the internet.
04:44.279 --> 04:48.499
But if they really want to make sure we would
never shut them down, they would have an
04:48.519 --> 04:51.319
incentive to get rid of us.
04:52.899 --> 04:53.279
So
04:54.179 --> 04:58.799
I know I'm asking you to make a giant leap into a
future that looks so
04:58.899 --> 05:00.579
different from where we are now,
05:01.939 --> 05:04.459
but it might be just a few years away or a decade
away.
05:04.899 --> 05:09.279
So we are on a trajectory to build machines that
are smarter and smarter,
05:10.279 --> 05:13.939
and one day, it's very plausible that they will be
smarter than us,
05:14.599 --> 05:14.959
and then
05:15.619 --> 05:17.279
they will have their own agency,
05:19.419 --> 05:20.419
their own goals
05:22.019 --> 05:25.219
which may not be aligned with ours.
05:26.659 --> 05:28.539
What happens to us then?
05:30.339 --> 05:30.939
Poof,
05:32.619 --> 05:33.599
we are
05:34.259 --> 05:36.399
blindly driving into a fog
05:37.359 --> 05:42.199
despite the warnings of scientists like myself
that this trajectory could
05:42.259 --> 05:44.379
lead to loss of control.
05:45.419 --> 05:50.179
Beside me in the car are my children, my grandson,
05:50.459 --> 05:51.359
my loved ones.
05:53.379 --> 05:54.999
Who is beside you
05:55.639 --> 05:56.259
in the car?
05:58.179 --> 06:00.899
Who is in your care for the future?
06:00.899 --> 06:02.099
... regulation.
06:02.119 --> 06:07.019
And- and- and just wrap it up, you know, in very
clear terms, what is
06:07.039 --> 06:08.839
the worst case scenario that you're worried about?
06:10.410 --> 06:14.969
Well, the worst case scenario is, uh, human
extinction, uh,
06:15.389 --> 06:19.910
eh, a lot of, uh, uh, the CEOs and leading AI
researchers, including
06:19.929 --> 06:21.209
myself, Geoff Hinton,
06:21.810 --> 06:24.989
um, have signed a declaration saying that,
06:25.589 --> 06:30.009
uh, making sure we mitigate that existential risk
should be a priority.
06:30.709 --> 06:35.370
Oh, yeah. Yes, in fact, I have reduced my
expectations for when we will
06:35.449 --> 06:38.329
get to AGI or whatever human level.
06:39.110 --> 06:40.609
Uh, I used to think it would be
06:41.229 --> 06:44.929
5 to 20 years, and now I think it could be, you
know, 2 to 10 years.
06:44.929 --> 06:46.549
Two years? Wow.
06:47.449 --> 06:47.469
Well-
06:47.469 --> 06:52.430
I mean, two would be the lower, look, the, the
smallest but, but, uh, um, it could be
06:52.489 --> 06:55.690
five, it could be 10, uh, really, we should plan
06:56.350 --> 06:58.190
considering all these possibilities-
06:58.190 --> 06:58.210
Yeah.
06:58.210 --> 07:01.769
... and in particular the shorter one just in case
'cause that could be catastrophic.
07:01.769 --> 07:06.369
And the world is ready for a potential AGI, as in
human level
07:06.749 --> 07:07.769
AI intelligence,
07:08.949 --> 07:09.589
within two years?
07:09.589 --> 07:10.049
Not at all.
07:10.049 --> 07:10.709
All right.
07:10.709 --> 07:11.389
Not at all.
07:13.649 --> 07:17.209
These are his words, these are exact quotes from
him,
07:17.209 --> 07:20.429
"Barring cataclysms,
07:21.029 --> 07:25.669
I consider the development of intelligent machines
a near-term inevitability.
07:27.769 --> 07:28.589
Rather quickly,
07:29.409 --> 07:31.309
they would displace us from existence.
07:33.029 --> 07:35.029
I'm not as alarmed as many
07:36.109 --> 07:38.769
since I consider these future machines our
progeny,
07:39.509 --> 07:44.269
mind children built in our image and likeness,
ourselves in more potent form."
07:45.149 --> 07:45.969
Number three,
07:48.609 --> 07:51.489
this will not stop with human-level abilities.
07:51.509 --> 07:54.709
We will, we will move beyond human-level
abilities.
07:57.609 --> 08:01.729
We might, uh, you might imagine we could s- we
could stop but, but I think realistically
08:01.749 --> 08:05.869
we can't because there are many of us and point
number one, there's no consensus.
08:06.349 --> 08:10.529
So some people will do it and it will go to the
limit.
08:11.909 --> 08:16.849
So, and then finally over time power and resources
will flow to
08:16.869 --> 08:20.669
the most intelligent beings, it doesn't really
matter if they're human or intelligent,
08:21.169 --> 08:22.229
power and resources
08:22.869 --> 08:24.589
will flow to the most intelligent.
08:25.229 --> 08:25.729
So, so
08:26.429 --> 08:27.449
this what we just said,
08:28.649 --> 08:33.409
we have the transition, we have the ascent of
humanity, succession to
08:33.469 --> 08:34.909
AI is sort of inevitable.
08:37.329 --> 08:39.529
So that's my second main message,
08:40.509 --> 08:43.609
AI, the inevitable next step in the development of
universe
08:44.349 --> 08:48.869
and we should embrace it with courage and pride,
also with humility
08:49.569 --> 08:51.569
(sighs) and a sense of adventure.
08:54.849 --> 08:59.769
And so as a result of that, I, I, and you can find
it on my blog from about
08:59.789 --> 09:04.709
2009 I think is the first time I publicly talked
about it, I predicted a 50% chance
09:04.729 --> 09:06.969
of, um, AGI by 2028.
09:07.669 --> 09:07.889
How did-
09:07.889 --> 09:09.609
And I still, I still believe that today.
09:09.609 --> 09:11.349
But, so I think that
09:12.069 --> 09:16.609
very, very powerful, very intelligent, um,
artificial intelligence is coming.
09:17.009 --> 09:19.489
Um, I think theers, this is very, very likely.
09:20.049 --> 09:23.409
It, I don't think it's coming today, I don't think
it's coming next year or the year
09:23.429 --> 09:26.209
after, it's probably a little bit further out than
that.
09:26.209 --> 09:26.469
What, in 2028? You-
09:26.469 --> 09:27.289
And I don't think...
09:27.969 --> 09:32.809
2028, that's a 50% chance so, you know, if it
doesn't happen in 2028 I'm not gonna be
09:32.849 --> 09:33.709
surprised, obviously.
09:33.709 --> 09:35.489
But that, but you, I think you also,
09:36.229 --> 09:39.969
you're not like an irredeemable optimist only,
you, you see a potential-
09:40.009 --> 09:40.029
Yeah.
09:40.029 --> 09:42.149
... for it to go very badly in a different
direction.
09:42.429 --> 09:45.129
Talk about what that pathway could look like.
09:45.129 --> 09:49.789
Well, yeah. I want to explain, I think what's, I,
I don't believe the people who are
09:49.869 --> 09:53.769
sure that it's gonna go very well and I don't
believe the people who are sure that it's
09:53.789 --> 09:58.189
gonna go very, very badly because what we're
talking about is an incredibly
09:58.289 --> 10:02.969
profound transition. It's like the arrival of
human intelligence in the
10:02.989 --> 10:07.269
world. This is another intelligence arriving in
the world and so it is an
10:07.269 --> 10:12.029
incredibly deep transition and we do not fully
understand all the
10:12.029 --> 10:13.929
implications and consequences of this.
10:14.929 --> 10:19.229
But there is, you know, many things we don't
understand about the future so I have to
10:19.269 --> 10:24.209
accept that there is a possibility that things m-
may go badly because
10:24.329 --> 10:28.309
I don't know what's gonna happen. I, I can't know
that about the future and such a big
10:28.369 --> 10:28.689
change
10:29.349 --> 10:34.329
and even if the probability of something going bad
is quite small, we
10:34.389 --> 10:36.449
should take this extremely seriously.
10:36.469 --> 10:36.849
But then-
10:36.969 --> 10:41.909
Yeah. If I had a magic wand and I could slow
things down, I would, I would use
10:41.909 --> 10:46.449
that magic wand but I don't. There are dozens of
company- well, there's probably 10
10:46.489 --> 10:50.489
companies in the world now that can develop the
most cutting edge, uh, models,
10:51.109 --> 10:55.369
uh, including, I think, some national, um,
intelligence services who have secret
10:55.409 --> 10:55.969
projects.
10:56.989 --> 11:01.909
How, how do you stop this process? I don't see any
way, realistic plan that
11:01.949 --> 11:03.529
I've heard of, of stopping this process.
11:03.529 --> 11:06.969
And so you have i- i- it's interesting 'cause you
have these, uh, blog posts that you
11:06.989 --> 11:08.809
wrote when you started DeepMind-
11:09.009 --> 11:09.109
Yeah.
11:09.109 --> 11:11.769
... um, you know, back in 2008 where you talk
about,
11:12.409 --> 11:15.849
um, th- the motivation was, uh, to accelerate
safety.
11:15.909 --> 11:16.329
Mm-hmm.
11:16.369 --> 11:20.569
O- on net what do you think the impact of DeepMind
has been on safety versus
11:20.589 --> 11:21.449
capabilities?
11:21.469 --> 11:22.929
Ooh.
11:24.209 --> 11:24.909
Interesting.
11:26.409 --> 11:27.689
(sighs)
11:30.049 --> 11:32.349
I don't know. It's hard to d- hard to judge
actually.
11:32.889 --> 11:36.509
Reasoning process and I think we're now at that
first part, I think we can start training
11:36.549 --> 11:40.649
models now with the scale of the data is beyond
what a human can experience in a
11:40.649 --> 11:43.029
lifetime. So I think this is the first unlocking
step
11:44.169 --> 11:48.709
and so, yeah, I think there's a 50% chance that
I'm saying by
11:48.869 --> 11:52.889
2028. Now it, it's just a 50% chance. I mean, I'm,
I'm sure what's gonna happen is we're
11:52.889 --> 11:56.369
gonna get to, you know, 2029 and someone's gonna
say, "Oh Shane, you were wrong." It's
11:56.409 --> 11:58.149
like, "Come on, it's a 50% chance."
11:58.789 --> 11:59.069
(laughs) So-
12:05.849 --> 12:10.129
One of my motivations in creating OpenAI
12:10.489 --> 12:12.889
was, in addition to developing this technology-
12:13.725 --> 12:18.586
... was also to address the questions that are
posed by AGI, the difficult questions,
12:18.586 --> 12:20.385
the concerns that we raised.
12:21.305 --> 12:25.845
In addition to working with governments and
helping them understand what is coming and
12:25.865 --> 12:30.746
prepare for it, we also doing a lot of research on
addressing the technological
12:30.845 --> 12:34.525
side of things so that the AI will never want to
go rogue.
12:35.265 --> 12:37.526
And this is something which I'm working on as
well.
12:38.685 --> 12:42.606
But still, if you, if you extrapolate this kind of
progress, we'll be in a very, very
12:42.705 --> 12:43.585
different place
12:44.425 --> 12:46.125
in five years, let alone 10 years.
12:47.505 --> 12:49.285
It doesn't seem implausible.
12:49.905 --> 12:53.425
It doesn't seem at all implausible that people
have computers,
12:54.626 --> 12:57.965
data centers that are much smarter than people.
12:58.205 --> 13:02.505
And by smarter, I don't mean just have more memory
or have more knowledge, but I also ha-
13:02.565 --> 13:05.385
mean have deeper insight into
13:06.185 --> 13:09.706
the same subjects that we people are studying and
looking into.
13:10.045 --> 13:13.666
It means learn even faster than people.
13:14.705 --> 13:16.405
Like, what could such AIs do?
13:17.166 --> 13:22.145
I don't know. Certainly, if such an AI were the
basis of some artificial life, it would
13:22.185 --> 13:22.485
be...
13:23.145 --> 13:27.245
Well, how do you even think about it? If you have
some very powerful data center that's
13:27.245 --> 13:28.125
also alive
13:28.805 --> 13:30.946
in a sense, that's what we are talking about.
13:31.426 --> 13:35.026
And when I imagine this world, I w- I, my, my
reaction is, "Gosh, this is very
13:35.045 --> 13:37.565
unpredictable what's gonna happen." Very
unpredictable.
13:38.105 --> 13:41.266
But the bare minimum, there is a bare minimum
which we can articulate,
13:42.965 --> 13:43.325
that
13:44.805 --> 13:45.405
if such
13:46.265 --> 13:50.485
super... If, if such very, very intelligent, super
intelligent data centers are built,
13:50.485 --> 13:51.245
being built at all,
13:52.065 --> 13:56.846
we want those data centers to hold warm and
positive feelings towards
13:56.985 --> 13:58.686
people, towards humanity.
13:59.725 --> 14:01.265
Because those... This is going to be
14:02.265 --> 14:03.185
non-human life,
14:04.105 --> 14:07.185
in a sense. Potentially, it could be, potentially
be that.
14:08.105 --> 14:11.285
And so I would want that any instance of such
14:13.105 --> 14:15.266
super intelligence to have warm feelings towards
humanity.
14:15.545 --> 14:17.025
And so this is what we are doing with
14:17.785 --> 14:22.066
the Superalignment Project. Yes, the, the, the
friendship piece, I think is optional, but
14:22.185 --> 14:24.325
I do think that we want a very pro-social AI.
14:25.265 --> 14:27.265
I think it's... I think it's possible.
14:27.265 --> 14:29.005
I don't think it's guaranteed, but I think it's
possible.
14:29.005 --> 14:29.706
If
14:36.025 --> 14:41.585
you
14:41.625 --> 14:46.345
extrapolate the curves that we've had so far,
right? If, if you say, "Well, I don't know.
14:46.345 --> 14:50.805
We're starting to get to, like, PhD level, and,
and last year we were at undergraduate
14:50.845 --> 14:54.865
level, and the year before, we were at, like, the
level of a high school student." Again,
14:54.886 --> 14:57.966
you can, you can quibble with at what tasks and
for what.
14:58.345 --> 15:00.706
We're still missing modalities, but those are
being added.
15:00.706 --> 15:03.826
Like, computer use was added, like, image
generation has been added.
15:03.985 --> 15:08.085
If you just kind of, like, eyeball the rate at
which these capabilities are increasing,
15:08.865 --> 15:12.825
it does make you think that we'll get there by
2026 or 2027.
15:13.045 --> 15:16.105
I think there's still worlds where it doesn't
happen in, in a 100 years.
15:16.505 --> 15:19.705
Those wor- the number of those worlds is rapidly
decreasing.
15:19.885 --> 15:24.645
We are rapidly running out of truly convincing
blockers, truly compelling
15:24.645 --> 15:26.245
reasons why this will not happen in...
15:27.405 --> 15:31.945
Yes. I mean, I think this is... Yeah. This is,
this is one of the core things that, you
15:31.965 --> 15:36.425
know, whether it's the, the bio risks from models
that, you know, I've kind of stated in
15:36.425 --> 15:41.245
testimony, you know, are likely to come in two to
three years, or the risks
15:41.285 --> 15:45.165
from truly autonomous models, which I think are
more than that, but might not be a whole,
15:45.205 --> 15:46.885
whole lot more than that.
15:47.185 --> 15:47.385
Yeah.
15:47.885 --> 15:52.825
I would, again, emphasize the testing and auditing
regime for all the risks ranging
15:52.845 --> 15:57.785
from, you know, those, those we f- we face today,
like misinformation came up, to
15:57.785 --> 16:02.245
the biological risks that I'm worried about in two
or three years, to the, uh, you know,
16:02.305 --> 16:06.225
risks of autonomous replication that are some
unspecified period after.
16:06.985 --> 16:10.425
Um, but, but I often put it in a simple way, which
is, one, the systems are getting much
16:10.465 --> 16:11.065
more powerful.
16:11.685 --> 16:12.185
Two,
16:12.885 --> 16:17.545
we... There, there's obviously not much of a
barrier to getting the systems to
16:17.625 --> 16:19.405
act autonomously in the world, right?
16:19.405 --> 16:24.105
People have taken GPT-4, for instance, and turned
it into AutoGPT.
16:24.105 --> 16:28.985
There was even a WormGPT, which is, you know, s-
supposed to act as a computer worm.
16:29.305 --> 16:33.525
So, you know, powerful smart systems that can take
action.
16:33.545 --> 16:34.685
So, you know, kind of
16:35.565 --> 16:38.125
very long leash of su- of human supervision.
16:38.485 --> 16:41.785
And because of the way they're trained, they're
not easy to control.
16:41.785 --> 16:46.285
I mean, we all saw Bing in Sydney. So you put
those three things together, and, you know,
16:46.285 --> 16:47.265
there's at least,
16:47.985 --> 16:52.405
uh, you know, I think some chance that as the
systems get more and more powerful,
16:53.265 --> 16:57.905
you know, they're, they're gonna do things that we
don't want them to do, and it may be,
16:57.965 --> 17:01.745
it may be difficult to fully control them, to
fully rein them humans.
17:01.945 --> 17:05.045
And there's a further thing of, like, can it get
past humans?
17:05.305 --> 17:10.225
Can it in- can it outperform the sum total of
human, say, you know,
17:10.305 --> 17:13.645
scientific or engineering output? That's, that's
like a...
17:13.645 --> 17:15.285
You know, that's, that's another point.
17:15.585 --> 17:20.145
That point could be, you know, could be like a
year away because the model gets...
17:20.185 --> 17:23.645
Is better at making itself smarter and smarter, or
it could be many years away.
17:23.905 --> 17:27.225
And then there's this further point of like, okay,
you know, could the model, like, you
17:27.245 --> 17:31.065
know, like, explore the universe and send out a
bunch of, like, you know, von Neumann
17:31.105 --> 17:34.665
probes, and, you know, build Dyson spheres around
the sun, and, you know,
17:35.605 --> 17:40.565
calculate the meaning of life is 42 or whatever.
Um, you know, that's, that's like a...
17:40.565 --> 17:44.265
That's, that's like a further point that also
raises questions about, you know, what's
17:44.625 --> 17:48.425
practical in an engineering sense in, in all of
these kind of weird, weird things.
17:48.425 --> 17:50.445
So that's, that's like another further point.
17:50.785 --> 17:54.725
It's possible all of these points are pretty
compressed together because there's like a
17:54.745 --> 17:57.285
feedback loop, but it's possible they're very far
away from...
17:57.285 --> 18:02.225
You know, I think I've, I think I've often said
that, you know, my, my chance that
18:02.325 --> 18:07.293
something goes...... you know, really quite
catastrophically wrong on the scale
18:07.333 --> 18:09.053
of, of, you know, human civilization.
18:09.053 --> 18:13.493
You know, it might be somewhere between 10 and 25%
when you put
18:13.514 --> 18:17.474
together the risk of something going wrong with
the model itself
18:18.073 --> 18:22.893
with, you know, something going wrong with human,
you know, people or organizations or
18:22.974 --> 18:27.793
nation states misusing the model or, or it kind of
inducing conflict
18:27.853 --> 18:32.453
among them or, or just some way in which kind of
society can't, can't handle it at the
18:32.493 --> 18:36.353
end of it. And if you were to pause for an
indefinite length of time, then you raise
18:36.373 --> 18:38.793
these questions like, "Well, h- how do you really
get everyone to stop?
18:38.793 --> 18:40.154
There's an international system here.
18:40.353 --> 18:43.774
There's dictators who wanna use this stuff to take
over the world." I mean, you know,
18:43.793 --> 18:47.693
people use that as an excuse, but it's also true.
Uh, quite, quite what to do.
18:47.734 --> 18:52.433
I think our concern was that with the rate at
which the technology was progressing,
18:52.853 --> 18:57.373
a kind of big, loud public release might
accelerate things so
18:57.493 --> 19:01.793
fast that the ecosystem might not, might not, um,
know how to handle it.
19:01.793 --> 19:06.493
And, you know, I didn't want our kind of first act
on the public stage, you know, after
19:06.613 --> 19:07.793
we'd said e- you know, after we
19:08.553 --> 19:13.313
put so much effort into bein', bein', bein', uh,
responsible to, to accelerate
19:13.373 --> 19:15.493
things, to accelerate things so greatly.
19:15.973 --> 19:19.353
Um, I, you know, certainly as, you know, as soon,
as soon as the other models were out
19:19.353 --> 19:22.693
and kind of the gun had been fired, then we
started putting these things out.
19:22.693 --> 19:26.213
We're like, "Okay. All right. You know, n- now,
now there's definitely a market in this.
19:26.213 --> 19:30.153
People, people, people, people know about it, and
so, you know, we should, we should, we
19:30.213 --> 19:33.893
should, we should, we should get out ahead." And,
you know, indeed, we've, we've managed
19:33.993 --> 19:37.793
to, you know, put ourselves among, you know, among
the top two or three players in this
19:37.813 --> 19:42.013
space. With... and, you know, it, it, y- y- it, it
really created an environment where
19:42.013 --> 19:44.353
things were, you know, racing forward very
quickly.
19:44.353 --> 19:47.413
And look, I, I love technology as much as the next
person.
19:47.413 --> 19:51.253
There was something like, you know, super exciting
about the whole, you know, "Mak- make
19:51.313 --> 19:53.413
them dance. Oh, we're responding with something."
You know?
19:53.413 --> 19:57.153
I mean, I can, I can get just as excited about
this as everyone, but given the rate at
19:57.153 --> 20:01.253
which the technology is progressing, um, you know,
there was a worrying aspect about this
20:01.273 --> 20:05.593
as well. Um, and so i- in this case, I'm at least,
at least on balance glad that, you
20:05.613 --> 20:07.693
know, we, we weren't the ones who fired that
starting gun.
20:07.733 --> 20:08.313
Yeah, got it.
20:08.313 --> 20:08.613
...
20:11.473 --> 20:12.873
about this or we're about that.
20:13.073 --> 20:18.013
And what was the thought at the time of, "Hey, we
should go do something
20:18.173 --> 20:21.593
on our own"? W- was that y- your thought? Was it a
group of people?
20:21.753 --> 20:24.093
Yeah, it was, it was definitely the thoughts of a
group of people.
20:24.153 --> 20:29.113
So there were seven co-founders who left, um, and
then I, I think in total we got
20:29.173 --> 20:33.753
four- 14 or 15 people from OpenAI, which was about
10% of the, the size of the
20:33.793 --> 20:37.913
organization at the time. Um, it w- you know, it
was, it's funny to look back on those
20:37.953 --> 20:42.773
days because, you know, in those days, we were the
language model part of OpenAI.
20:42.833 --> 20:47.673
Like, we, w- you know, we along with a couple
people who stayed, uh, you know, were,
20:47.693 --> 20:52.013
were those who had developed and scaled the, the
language models.
20:52.013 --> 20:54.933
There were many different other things going on at
OpenAI, right?
20:54.933 --> 20:59.013
There was, you know, there was a robotics project,
a theorem proving project, projects to
20:59.033 --> 21:01.533
play video games. Some of those still exist,
21:02.193 --> 21:06.613
um, but, uh, you know, we felt that we were this
kind of coherent group of people.
21:06.913 --> 21:08.753
We had this view about,
21:09.373 --> 21:13.213
you know, language models and scaling, which to be
fair I think the organization
21:13.293 --> 21:18.193
supported, but then we also had this view about,
you know, we, we need to, we need
21:18.193 --> 21:23.033
to make these models safe in a certain way, and,
you know, we need to do them
21:23.033 --> 21:27.953
within an organization where we can really believe
that these principles are incorporated
21:28.013 --> 21:28.593
top to bottom.
21:28.593 --> 21:33.393
What's, what's something foundational, uh, I guess
not to use a cute
21:33.433 --> 21:33.673
term-
21:33.673 --> 21:33.713
(laughs)
21:33.713 --> 21:38.433
... but something foundational that you believe at
Anthropic that you would draw in
21:38.453 --> 21:39.753
distinction to
21:40.393 --> 21:41.533
others in this space?
21:41.533 --> 21:44.633
Yeah. I would, I would say a couple things.
21:44.833 --> 21:49.773
Um, so, you know, one is just this idea that we
should be building in
21:50.273 --> 21:52.253
safety from the beginning with models.
21:52.253 --> 21:57.233
And, and in particular, safety is itself a task,
like, you know, is
21:57.253 --> 22:00.013
this thing, the model's doing right or wrong?
We're, we're, we're...
22:00.013 --> 22:03.913
You know, right or wrong could be something as
prosaic as, is the model telling you how
22:03.913 --> 22:08.873
to hotwire a car or as, you know, scary and
sophisticated as, is the model gonna
22:08.893 --> 22:13.013
help me build, y- build a bio weapon or is it
gonna, you know, take over the world and
22:13.053 --> 22:16.293
make swarms of nanobots or, you know, whatever
futuristic thing?
22:16.713 --> 22:20.933
Figuring out whether the model's gonna do that and
the behavior of the model is itself an
22:20.973 --> 22:23.373
intellectual task of the kind that models do.
22:23.813 --> 22:24.253
And so
22:25.293 --> 22:29.493
the problem and the solution to the problem are
mixed together in this way.
22:32.573 --> 22:35.053
You know, it's quite a dangerous,
22:35.953 --> 22:37.173
quite a dangerous technology,
22:37.953 --> 22:39.153
um, and I, I,
22:39.853 --> 22:40.953
I fear I may have
22:41.673 --> 22:43.653
done some things to accelerate it, which is...
22:44.833 --> 22:45.293
I don't know.
22:46.833 --> 22:47.113
So
22:48.053 --> 22:51.633
I mean, I guess, y- you know, part of what I've
been fighting and maybe what has slowed
22:51.633 --> 22:53.953
me down somewhat is that I'm a, I- I'm a little...
22:53.973 --> 22:55.893
I don't wanna, I don't wanna make Terminator real.
22:55.973 --> 22:56.213
(laughs)
22:56.213 --> 22:56.773
(laughs)
22:56.773 --> 22:57.933
You know? So
22:58.713 --> 23:00.393
I- I've been sort of, I guess,
23:01.113 --> 23:06.053
at least until recent years, dragging my feet on,
on AI and ro- and
23:06.093 --> 23:07.113
humanoid robotics.
23:08.213 --> 23:11.573
And then I sort of come to the realiz- realization
it's, it's happening whether I do it
23:11.593 --> 23:12.353
or not, so
23:13.393 --> 23:17.813
you got really two choices. Particip- y- you can
either be a spectator or a participant.
23:18.673 --> 23:21.953
And so, like, well, I guess I'd rather be a, a
participant than a spectator.
23:23.053 --> 23:27.293
Um, so now it's, you know, pedal to the metal on
humanoid robots and, um,
23:28.793 --> 23:29.993
digital superintelligence.
23:32.973 --> 23:35.793
Like, I think, I think we're quite close to
digital superintelligence.
23:37.013 --> 23:38.053
I think it may happen
23:39.013 --> 23:42.053
this year, and if it doesn't happen this year,
next year for sure.
23:42.693 --> 23:44.633
A digital superintelligence defined as
23:45.533 --> 23:50.353
smarter than any human at anything. Um, I, I guess
I sort of agree with Geoff
23:50.353 --> 23:53.393
Hinton that maybe it's a 10 to 20% chance of
annihilation.
23:53.893 --> 23:57.733
Uh, but look on the bright side, that's 80 to 90%
probability of a great outcome.
23:57.733 --> 24:01.213
The other conversation we've had, and
24:01.913 --> 24:06.733
you came out in the same side as Geoffrey Hinton
in this, was
24:07.353 --> 24:12.101
80% of probability it's gonna be awesome.20%
probability we're
24:12.101 --> 24:12.642
screwed.
24:12.881 --> 24:12.901
Yes.
24:12.901 --> 24:14.621
Are you still- are you still on those odds?
24:14.621 --> 24:19.341
Yeah. Yeah, I mean, like I say, it's- it's- it's-
24:19.521 --> 24:23.581
it's most likely gonna be great, and there's-
there's some chance, which could be 10 to
24:23.621 --> 24:24.642
20% that,
24:25.261 --> 24:27.441
um, it goes bad. Um,
24:28.521 --> 24:30.101
the chance is not zero that it goes bad,
24:31.341 --> 24:34.181
but overall, at one point, one could say that the
cup is 80% full,
24:34.962 --> 24:37.441
uh, is one positive way to look at it. Maybe 90%.
24:39.401 --> 24:44.301
That's not even a question for me, whether we're
gonna go take a swing at building
24:44.321 --> 24:47.181
the next thing. I'm just incapable of not doing
that.
24:47.461 --> 24:50.722
There's a- I would guess that, like, sometime in
the next 12 to
24:51.261 --> 24:56.141
we'll reach the point where, like, most of the
code that's going towards
24:56.141 --> 25:00.281
these efforts is written by AI. And I don't- as-
as this goes on.
25:00.341 --> 25:03.201
This isn't like that one to two year thing of what
happens when you have a-
25:03.941 --> 25:05.822
a- like a super powerful software engineer.
25:06.222 --> 25:06.461
But
25:07.241 --> 25:09.681
I think over time, you know, if- if you...
25:09.681 --> 25:13.201
Like, everyone is gonna have these superhuman
tools, um, to be able to create a ton of
25:13.201 --> 25:15.441
different stuff. I think you're gonna get this
incredible diversity.
25:15.901 --> 25:17.701
Um, part of it is gonna be solving,
25:18.641 --> 25:23.541
like, things that we hold up as, like, these,
like, hard problems, like
25:23.801 --> 25:26.061
solving diseases or, like,
25:27.161 --> 25:32.081
solving different things around science or, um, or
just, like, different technology that
25:32.101 --> 25:33.581
makes our lives better. But
25:34.441 --> 25:36.961
I would guess that a lot of it is going to end up
being
25:37.821 --> 25:41.681
kind of cultural and social pursuit and enter-
Basically, okay, there's security holes in
25:41.701 --> 25:46.241
so many different things, and if you could travel
back in time a year or two
25:46.281 --> 25:47.521
years, right? It's like,
25:48.241 --> 25:51.361
that's not AI. It's like you just... Let's say you
just have like one year or two years
25:51.421 --> 25:55.381
more knowledge of the security holes, you could
pretty much hack into, like, any system,
25:55.541 --> 25:59.261
right? So like, you know, I stay up at night more
worrying, well, what if, like,
25:59.981 --> 26:04.341
some actor that... Whatever. It's like from
wherever you sit, there's gonna be some actor
26:04.381 --> 26:07.761
who you don't trust. If they're the ones who have,
like, the super strong AI, whether
26:07.781 --> 26:12.321
it's some, like, other government that we- that
does sort of like an opponent of- of- of
26:12.341 --> 26:16.601
our country or some company that you don't trust
or whate- whatever it is, um,
26:17.721 --> 26:17.961
like,
26:18.721 --> 26:20.361
I think that that's potentially
26:21.341 --> 26:26.181
a much bigger risk. As in- I- I do think that
there- there is some
26:26.241 --> 26:27.961
set of people who are afraid of like,
26:28.781 --> 26:32.421
you know, it really just kind of spins and goes
from being like somewhat intelligent to
26:32.441 --> 26:35.341
extremely intelligent overnight. And I just think
that there's all these physical
26:35.361 --> 26:38.301
constraints that make that, so that that's
unlikely to happen.
26:38.801 --> 26:42.061
Um, I- I- I just don't, I don't really see that-
that playing out.
26:42.641 --> 26:46.201
I know there- there's all this science fiction
about, okay, you create intelligence and
26:46.621 --> 26:51.421
now it like starts taking on all these human-like
behaviors and- and things like that.
26:51.421 --> 26:55.261
But I actually think that the current incarnation
of all this stuff at least kind of
26:55.281 --> 26:58.981
feels like it's going in a direction where
intelligence can be pretty separated from
26:59.041 --> 27:03.201
consciousness and agency and things like that,
that, um,
27:03.921 --> 27:06.721
I think just makes it a super valuable tool, if I
had to guess.
27:07.101 --> 27:11.941
It's- it's really interesting to me that you
basically agree with the premise that there
27:11.961 --> 27:15.461
will be an intelligence explosion and something
like super intelligence on either end.
27:15.721 --> 27:17.861
But then if that's the case, tell me if I'm
misunderstanding you.
27:17.901 --> 27:21.281
If that's the case, why even bother with personal
assistants and whatever?
27:21.281 --> 27:24.501
Why not just get- get to super human intelligence
first and then deal with everything
27:24.501 --> 27:24.541
else later?
27:24.541 --> 27:27.401
Well, I think that that's just one aspect of the
flywheel.
27:28.021 --> 27:28.381
Right? So
27:29.281 --> 27:34.121
part of what- what I generally disagree with on
the fast takeoff thing is it takes time
27:34.161 --> 27:36.041
to build out physical infrastructure.
27:36.041 --> 27:36.221
Mm-hmm.
27:36.381 --> 27:41.221
Right? So if you want to build like a gigawatt
cluster of compute,
27:42.021 --> 27:44.081
that just is gonna take some time.
27:47.021 --> 27:49.441
(music)
27:51.281 --> 27:55.561
Stargate, put that name down in your books,
because I think you're going to hear a lot
27:55.621 --> 27:59.881
about it in the future. A new American company
that will invest $500
27:59.901 --> 28:03.021
billion at least in AI infrastructure.
28:03.681 --> 28:05.781
The data centers are actually under construction.
28:05.901 --> 28:08.141
The first of them are under construction in Texas.
28:08.141 --> 28:11.041
The Abilene location, which is our first location.
28:11.801 --> 28:15.561
Abilene is a town on the western central plains of
Texas.
28:15.921 --> 28:18.881
We kind of scratch and fight for everything good
that comes our way.
28:18.881 --> 28:23.181
I had people reaching out and they said, "When is
the President going to come see you?"
28:23.481 --> 28:26.841
And I said, "Well, he hasn't texted me yet, so..."
(laughs)
28:26.841 --> 28:29.181
I'm thrilled we get to do this in the United
States of America.
28:29.461 --> 28:32.081
I think this will be the most important project of
this era.
28:32.081 --> 28:36.941
Last month I came to, um, celebrate your winning
28:37.321 --> 28:41.461
and promised that, uh, we will invest $100
billion.
28:41.881 --> 28:46.241
And you told me, "Oh, Masa, go for 200." (laughs)
28:47.581 --> 28:49.241
Now I came back with 500.
28:49.301 --> 28:53.521
Explain the math to us. How does it all add up to
needing $500 billion?
28:53.521 --> 28:57.721
Well, that covers the capacity we think we need
for the next few years, given our growth
28:57.741 --> 29:01.621
projections. The- the interesting question is, if
we had- if we knew how to get a
29:01.621 --> 29:03.141
trillion dollars right now, which we don't,
29:03.781 --> 29:06.961
would we be able to deploy that profitably in the
next few years?
29:08.481 --> 29:12.281
And I'm not sure about that, but I feel confident
we can, like, make 500 billion of value
29:12.301 --> 29:12.601
back.
29:13.081 --> 29:17.521
I was in the real estate business. These
buildings, these are big, beautiful buildings,
29:17.521 --> 29:18.941
they're going to employ a lot of people.
29:18.941 --> 29:23.141
How do you personally grapple with the pace of it
all?
29:23.261 --> 29:25.641
It does feel like it's going very fast.
29:25.861 --> 29:29.881
Certainly think if I could like transport myself
back three years ago, it would seem like
29:30.161 --> 29:31.581
unimaginable progress.
29:31.581 --> 29:36.381
The money piece, I think so long as AI is
delivering
29:36.501 --> 29:37.921
value to society,
29:38.541 --> 29:42.961
the money will be there to build the
infrastructure. I have no concerns around that.
29:43.441 --> 29:46.801
Um, even if it's not all 100% accounted for today,
29:47.401 --> 29:51.261
that money is there, and there's a lot of money
that wants to invest in this category.
29:51.941 --> 29:55.941
What we're undergoing right now is sort of the
largest infrastructure build in human
29:55.961 --> 30:00.401
history, right? This is like where we built the
interstate high- highway system.
30:00.861 --> 30:05.721
You know, that enabled massive interstate commerce
and transportation and catalyzed this
30:05.761 --> 30:07.441
massive boom in the US economy.
30:08.481 --> 30:12.861
What's happening in AI infrastructure is we're
building the superhighway system that's
30:12.861 --> 30:16.741
going to enable modern intelligence systems,
that's going to create all sorts of new job
30:16.821 --> 30:18.401
categories that don't even exist today.
30:18.461 --> 30:18.661
Mm-hmm.
30:18.661 --> 30:21.281
It's going to create new ways of doing business,
it's going to create new ways of
30:21.301 --> 30:24.541
invention, um, and it's going to really drive
humanity forward.
30:24.541 --> 30:28.781
I don't think the world has really had the
humanoid robots moment yet.
30:29.161 --> 30:31.341
Like the first day, you're like walking down the
street
30:32.061 --> 30:35.241
and there's like seven robots that walk past you
doing things or whatever, it's going to
30:35.361 --> 30:36.361
feel very sci-fi.
30:37.201 --> 30:40.421
And I don't think that's very far away from like a
visceral, like, oh man, this is going
30:40.421 --> 30:41.961
to do a lot of things that people used to do.
30:41.981 --> 30:46.461
So- so yeah, it's coming, and we- we have always
tried to just be super honest about what
30:46.481 --> 30:50.141
we think the impact may be, realizing that we'll
be wrong on a lot of details.
30:50.141 --> 30:53.261
It doesn't exist.
30:53.261 --> 30:57.461
How confident are you that AI's promise will
outweigh the perils?
30:57.461 --> 31:01.501
I'm no- I'm not confident, but the...
31:02.081 --> 31:04.521
You know, that's- that's what we need to get
oriented around.
31:04.561 --> 31:08.421
That's why I was surprised AI wasn't really an
issue in the last election.
31:08.501 --> 31:10.341
It's definitely got to be shaped.
31:10.341 --> 31:13.201
Hearing you say you're not confident is kind of
scary.
31:13.201 --> 31:14.221
People should
31:15.021 --> 31:19.681
think through, how is AI going to reshape, uh, our
lives?
31:19.761 --> 31:24.621
It is- it is a profound, uh, change agent.
31:24.641 --> 31:28.741
And, you know, I don't think it should just be the
people who know the technology,
31:28.761 --> 31:32.201
because there's a lot of choices about how it gets
used. It will...
Enter "Subscribe: " followed by Signal username, email address or whatsapp phone number. You will receive atmost one update per month. View previous updates here.
Alternatively, you can use RSS, or a third-party service like blogtrottr.
Enter "Comment: " followed by comment. Alternatively, contact me