But I think the development of full artificial intelligence could spell the
end of the human race.
Once humans develop artificial intelligence, it would take off on
its own and redesign itself at an ever increasing rate.
In between five and 20 years from now, there's a good chance, a
50% chance, we'll get AI smarter than us.
I think we're
moving into a period when, for the first time ever, we may have
things more intelligent than us.
You believe they can understand?
Yes.
You believe they are intelligent?
Yes.
You believe these systems have experiences of their own
and can make decisions based on those experiences?
In the same sense as people do, yes.
Are
they conscious?
I think they probably don't have much self-awareness at present.
So in that sense, I don't think they're conscious.
Will they have self-awareness, consciousness?
Oh, yes.
Yes?
Oh, yes, I think they will in time.
And so human beings will be the second most intelligent beings on the
planet?
Yeah.
You believe that ChatGPT-4
understands?
I believe it
definitely understands, yes.
And in five years time?
I think in five years time, it may well be able to reason better than us.
And we can't afford to get it wrong with these things.
Can't afford to get it wrong, why?
Well, because they might take over.
Take over from humanity?
Yes, that's a possibility.
Why would they want to?
I'm not saying it will happen.
If we could stop them ever wanting to, that would be great, but it's not clear we can
stop them ever wanting to.
So the question is, what's going to happen when we've created beings that are more
intelligent than us?
And
we don't know what's going to happen. We've never been in that situation before.
Anybody who says it's all going to be fine is crazy, and anybody who says they inevitably
going to take over, they're crazy too. We really don't know.
But because we really don't know, it will make a lot of sense to do a lot of basic
research now on whether we can stay in control of things that we create that are
more intelligent than us.
What are the implications
of these systems autonomously writing their own computer code and executing
their own computer code?
That's a serious worry, right?
So one of the ways in which these systems might escape control
is by writing their own computer code to modify themselves,
and that's something we need to seriously worry about.
What do you say to someone who might argue, "If the systems become malevolent,
just turn them off?"
They will be able to manipulate people, right?
And these will be very good at convincing people because they'll have learned from all
the novels that were ever written, all the books by Machiavelli,
all the political connivances, they'll know all that stuff, they'll know how to do it.
We thought AI would happen in decades or centuries,
but it
might be just in a few years. And I saw how it could go wrong because we
didn't and we still don't have ways to make sure
this technology eventually doesn't turn against us.
So two months later, I'm a leading signatory of the pause
letter where we and 30,000 other people asked the AI
labs to wait six months before building the next version.
As you can guess, nobody paused.
Then,
with the same people and the leading executives of the AI labs,
I sign a statement, and the statement goes,
"Mitigating the risk of extinction
from AI should be a global priority."
I then testify in front of the US Senate a- about those risks.
I travel the world to talk about it. Okay, so this was a controlled experiment.
What is it going to be in a few years when these systems are much more powerful?
There's already studies showing that they can learn to avoid showing
their deceptive plans in these chain of thoughts that we can monitor.
When they'll be more powerful, they would not just copy themselves on one other computer
and start that program.
They would copy themselves over hundreds or thousands of computers over the internet.
But if they really want to make sure we would never shut them down, they would have an
incentive to get rid of us.
So
I know I'm asking you to make a giant leap into a future that looks so
different from where we are now,
but it might be just a few years away or a decade away.
So we are on a trajectory to build machines that are smarter and smarter,
and one day, it's very plausible that they will be smarter than us,
and then
they will have their own agency,
their own goals
which may not be aligned with ours.
What happens to us then?
Poof,
we are
blindly driving into a fog
despite the warnings of scientists like myself that this trajectory could
lead to loss of control.
Beside me in the car are my children, my grandson,
my loved ones.
Who is beside you
in the car?
Who is in your care for the future?
... regulation.
And- and- and just wrap it up, you know, in very clear terms, what is
the worst case scenario that you're worried about?
Well, the worst case scenario is, uh, human extinction, uh,
eh, a lot of, uh, uh, the CEOs and leading AI researchers, including
myself, Geoff Hinton,
um, have signed a declaration saying that,
uh, making sure we mitigate that existential risk should be a priority.
Oh, yeah. Yes, in fact, I have reduced my expectations for when we will
get to AGI or whatever human level.
Uh, I used to think it would be
5 to 20 years, and now I think it could be, you know, 2 to 10 years.
Two years? Wow.
Well-
I mean, two would be the lower, look, the, the smallest but, but, uh, um, it could be
five, it could be 10, uh, really, we should plan
considering all these possibilities-
Yeah.
... and in particular the shorter one just in case 'cause that could be catastrophic.
And the world is ready for a potential AGI, as in human level
AI intelligence,
within two years?
Not at all.
All right.
Not at all.
These are his words, these are exact quotes from him,
"Barring cataclysms,
I consider the development of intelligent machines a near-term inevitability.
Rather quickly,
they would displace us from existence.
I'm not as alarmed as many
since I consider these future machines our progeny,
mind children built in our image and likeness, ourselves in more potent form."
Number three,
this will not stop with human-level abilities.
We will, we will move beyond human-level abilities.
We might, uh, you might imagine we could s- we could stop but, but I think realistically
we can't because there are many of us and point number one, there's no consensus.
So some people will do it and it will go to the limit.
So, and then finally over time power and resources will flow to
the most intelligent beings, it doesn't really matter if they're human or intelligent,
power and resources
will flow to the most intelligent.
So, so
this what we just said,
we have the transition, we have the ascent of humanity, succession to
AI is sort of inevitable.
So that's my second main message,
AI, the inevitable next step in the development of universe
and we should embrace it with courage and pride, also with humility
(sighs) and a sense of adventure.
And so as a result of that, I, I, and you can find it on my blog from about
2009 I think is the first time I publicly talked about it, I predicted a 50% chance
of, um, AGI by 2028.
How did-
And I still, I still believe that today.
But, so I think that
very, very powerful, very intelligent, um, artificial intelligence is coming.
Um, I think theers, this is very, very likely.
It, I don't think it's coming today, I don't think it's coming next year or the year
after, it's probably a little bit further out than that.
What, in 2028? You-
And I don't think...
2028, that's a 50% chance so, you know, if it doesn't happen in 2028 I'm not gonna be
surprised, obviously.
But that, but you, I think you also,
you're not like an irredeemable optimist only, you, you see a potential-
Yeah.
... for it to go very badly in a different direction.
Talk about what that pathway could look like.
Well, yeah. I want to explain, I think what's, I, I don't believe the people who are
sure that it's gonna go very well and I don't believe the people who are sure that it's
gonna go very, very badly because what we're talking about is an incredibly
profound transition. It's like the arrival of human intelligence in the
world. This is another intelligence arriving in the world and so it is an
incredibly deep transition and we do not fully understand all the
implications and consequences of this.
But there is, you know, many things we don't understand about the future so I have to
accept that there is a possibility that things m- may go badly because
I don't know what's gonna happen. I, I can't know that about the future and such a big
change
and even if the probability of something going bad is quite small, we
should take this extremely seriously.
But then-
Yeah. If I had a magic wand and I could slow things down, I would, I would use
that magic wand but I don't. There are dozens of company- well, there's probably 10
companies in the world now that can develop the most cutting edge, uh, models,
uh, including, I think, some national, um, intelligence services who have secret
projects.
How, how do you stop this process? I don't see any way, realistic plan that
I've heard of, of stopping this process.
And so you have i- i- it's interesting 'cause you have these, uh, blog posts that you
wrote when you started DeepMind-
Yeah.
... um, you know, back in 2008 where you talk about,
um, th- the motivation was, uh, to accelerate safety.
Mm-hmm.
O- on net what do you think the impact of DeepMind has been on safety versus
capabilities?
Ooh.
Interesting.
(sighs)
I don't know. It's hard to d- hard to judge actually.
Reasoning process and I think we're now at that first part, I think we can start training
models now with the scale of the data is beyond what a human can experience in a
lifetime. So I think this is the first unlocking step
and so, yeah, I think there's a 50% chance that I'm saying by
2028. Now it, it's just a 50% chance. I mean, I'm, I'm sure what's gonna happen is we're
gonna get to, you know, 2029 and someone's gonna say, "Oh Shane, you were wrong." It's
like, "Come on, it's a 50% chance."
(laughs) So-
One of my motivations in creating OpenAI
was, in addition to developing this technology-
... was also to address the questions that are posed by AGI, the difficult questions,
the concerns that we raised.
In addition to working with governments and helping them understand what is coming and
prepare for it, we also doing a lot of research on addressing the technological
side of things so that the AI will never want to go rogue.
And this is something which I'm working on as well.
But still, if you, if you extrapolate this kind of progress, we'll be in a very, very
different place
in five years, let alone 10 years.
It doesn't seem implausible.
It doesn't seem at all implausible that people have computers,
data centers that are much smarter than people.
And by smarter, I don't mean just have more memory or have more knowledge, but I also ha-
mean have deeper insight into
the same subjects that we people are studying and looking into.
It means learn even faster than people.
Like, what could such AIs do?
I don't know. Certainly, if such an AI were the basis of some artificial life, it would
be...
Well, how do you even think about it? If you have some very powerful data center that's
also alive
in a sense, that's what we are talking about.
And when I imagine this world, I w- I, my, my reaction is, "Gosh, this is very
unpredictable what's gonna happen." Very unpredictable.
But the bare minimum, there is a bare minimum which we can articulate,
that
if such
super... If, if such very, very intelligent, super intelligent data centers are built,
being built at all,
we want those data centers to hold warm and positive feelings towards
people, towards humanity.
Because those... This is going to be
non-human life,
in a sense. Potentially, it could be, potentially be that.
And so I would want that any instance of such
super intelligence to have warm feelings towards humanity.
And so this is what we are doing with
the Superalignment Project. Yes, the, the, the friendship piece, I think is optional, but
I do think that we want a very pro-social AI.
I think it's... I think it's possible.
I don't think it's guaranteed, but I think it's possible.
If
you
extrapolate the curves that we've had so far, right? If, if you say, "Well, I don't know.
We're starting to get to, like, PhD level, and, and last year we were at undergraduate
level, and the year before, we were at, like, the level of a high school student." Again,
you can, you can quibble with at what tasks and for what.
We're still missing modalities, but those are being added.
Like, computer use was added, like, image generation has been added.
If you just kind of, like, eyeball the rate at which these capabilities are increasing,
it does make you think that we'll get there by 2026 or 2027.
I think there's still worlds where it doesn't happen in, in a 100 years.
Those wor- the number of those worlds is rapidly decreasing.
We are rapidly running out of truly convincing blockers, truly compelling
reasons why this will not happen in...
Yes. I mean, I think this is... Yeah. This is, this is one of the core things that, you
know, whether it's the, the bio risks from models that, you know, I've kind of stated in
testimony, you know, are likely to come in two to three years, or the risks
from truly autonomous models, which I think are more than that, but might not be a whole,
whole lot more than that.
Yeah.
I would, again, emphasize the testing and auditing regime for all the risks ranging
from, you know, those, those we f- we face today, like misinformation came up, to
the biological risks that I'm worried about in two or three years, to the, uh, you know,
risks of autonomous replication that are some unspecified period after.
Um, but, but I often put it in a simple way, which is, one, the systems are getting much
more powerful.
Two,
we... There, there's obviously not much of a barrier to getting the systems to
act autonomously in the world, right?
People have taken GPT-4, for instance, and turned it into AutoGPT.
There was even a WormGPT, which is, you know, s- supposed to act as a computer worm.
So, you know, powerful smart systems that can take action.
So, you know, kind of
very long leash of su- of human supervision.
And because of the way they're trained, they're not easy to control.
I mean, we all saw Bing in Sydney. So you put those three things together, and, you know,
there's at least,
uh, you know, I think some chance that as the systems get more and more powerful,
you know, they're, they're gonna do things that we don't want them to do, and it may be,
it may be difficult to fully control them, to fully rein them humans.
And there's a further thing of, like, can it get past humans?
Can it in- can it outperform the sum total of human, say, you know,
scientific or engineering output? That's, that's like a...
You know, that's, that's another point.
That point could be, you know, could be like a year away because the model gets...
Is better at making itself smarter and smarter, or it could be many years away.
And then there's this further point of like, okay, you know, could the model, like, you
know, like, explore the universe and send out a bunch of, like, you know, von Neumann
probes, and, you know, build Dyson spheres around the sun, and, you know,
calculate the meaning of life is 42 or whatever. Um, you know, that's, that's like a...
That's, that's like a further point that also raises questions about, you know, what's
practical in an engineering sense in, in all of these kind of weird, weird things.
So that's, that's like another further point.
It's possible all of these points are pretty compressed together because there's like a
feedback loop, but it's possible they're very far away from...
You know, I think I've, I think I've often said that, you know, my, my chance that
something goes...... you know, really quite catastrophically wrong on the scale
of, of, you know, human civilization.
You know, it might be somewhere between 10 and 25% when you put
together the risk of something going wrong with the model itself
with, you know, something going wrong with human, you know, people or organizations or
nation states misusing the model or, or it kind of inducing conflict
among them or, or just some way in which kind of society can't, can't handle it at the
end of it. And if you were to pause for an indefinite length of time, then you raise
these questions like, "Well, h- how do you really get everyone to stop?
There's an international system here.
There's dictators who wanna use this stuff to take over the world." I mean, you know,
people use that as an excuse, but it's also true. Uh, quite, quite what to do.
I think our concern was that with the rate at which the technology was progressing,
a kind of big, loud public release might accelerate things so
fast that the ecosystem might not, might not, um, know how to handle it.
And, you know, I didn't want our kind of first act on the public stage, you know, after
we'd said e- you know, after we
put so much effort into bein', bein', bein', uh, responsible to, to accelerate
things, to accelerate things so greatly.
Um, I, you know, certainly as, you know, as soon, as soon as the other models were out
and kind of the gun had been fired, then we started putting these things out.
We're like, "Okay. All right. You know, n- now, now there's definitely a market in this.
People, people, people, people know about it, and so, you know, we should, we should, we
should, we should, we should get out ahead." And, you know, indeed, we've, we've managed
to, you know, put ourselves among, you know, among the top two or three players in this
space. With... and, you know, it, it, y- y- it, it really created an environment where
things were, you know, racing forward very quickly.
And look, I, I love technology as much as the next person.
There was something like, you know, super exciting about the whole, you know, "Mak- make
them dance. Oh, we're responding with something." You know?
I mean, I can, I can get just as excited about this as everyone, but given the rate at
which the technology is progressing, um, you know, there was a worrying aspect about this
as well. Um, and so i- in this case, I'm at least, at least on balance glad that, you
know, we, we weren't the ones who fired that starting gun.
Yeah, got it.
...
about this or we're about that.
And what was the thought at the time of, "Hey, we should go do something
on our own"? W- was that y- your thought? Was it a group of people?
Yeah, it was, it was definitely the thoughts of a group of people.
So there were seven co-founders who left, um, and then I, I think in total we got
four- 14 or 15 people from OpenAI, which was about 10% of the, the size of the
organization at the time. Um, it w- you know, it was, it's funny to look back on those
days because, you know, in those days, we were the language model part of OpenAI.
Like, we, w- you know, we along with a couple people who stayed, uh, you know, were,
were those who had developed and scaled the, the language models.
There were many different other things going on at OpenAI, right?
There was, you know, there was a robotics project, a theorem proving project, projects to
play video games. Some of those still exist,
um, but, uh, you know, we felt that we were this kind of coherent group of people.
We had this view about,
you know, language models and scaling, which to be fair I think the organization
supported, but then we also had this view about, you know, we, we need to, we need
to make these models safe in a certain way, and, you know, we need to do them
within an organization where we can really believe that these principles are incorporated
top to bottom.
What's, what's something foundational, uh, I guess not to use a cute
term-
(laughs)
... but something foundational that you believe at Anthropic that you would draw in
distinction to
others in this space?
Yeah. I would, I would say a couple things.
Um, so, you know, one is just this idea that we should be building in
safety from the beginning with models.
And, and in particular, safety is itself a task, like, you know, is
this thing, the model's doing right or wrong? We're, we're, we're...
You know, right or wrong could be something as prosaic as, is the model telling you how
to hotwire a car or as, you know, scary and sophisticated as, is the model gonna
help me build, y- build a bio weapon or is it gonna, you know, take over the world and
make swarms of nanobots or, you know, whatever futuristic thing?
Figuring out whether the model's gonna do that and the behavior of the model is itself an
intellectual task of the kind that models do.
And so
the problem and the solution to the problem are mixed together in this way.
You know, it's quite a dangerous,
quite a dangerous technology,
um, and I, I,
I fear I may have
done some things to accelerate it, which is...
I don't know.
So
I mean, I guess, y- you know, part of what I've been fighting and maybe what has slowed
me down somewhat is that I'm a, I- I'm a little...
I don't wanna, I don't wanna make Terminator real.
(laughs)
You know? So
I- I've been sort of, I guess,
at least until recent years, dragging my feet on, on AI and ro- and
humanoid robotics.
And then I sort of come to the realiz- realization it's, it's happening whether I do it
or not, so
you got really two choices. Particip- y- you can either be a spectator or a participant.
And so, like, well, I guess I'd rather be a, a participant than a spectator.
Um, so now it's, you know, pedal to the metal on humanoid robots and, um,
digital superintelligence.
Like, I think, I think we're quite close to digital superintelligence.
I think it may happen
this year, and if it doesn't happen this year, next year for sure.
A digital superintelligence defined as
smarter than any human at anything. Um, I, I guess I sort of agree with Geoff
Hinton that maybe it's a 10 to 20% chance of annihilation.
Uh, but look on the bright side, that's 80 to 90% probability of a great outcome.
The other conversation we've had, and
you came out in the same side as Geoffrey Hinton in this, was
80% of probability it's gonna be awesome.20% probability we're
screwed.
Yes.
Are you still- are you still on those odds?
Yeah. Yeah, I mean, like I say, it's- it's- it's-
it's most likely gonna be great, and there's- there's some chance, which could be 10 to
20% that,
um, it goes bad. Um,
the chance is not zero that it goes bad,
but overall, at one point, one could say that the cup is 80% full,
uh, is one positive way to look at it. Maybe 90%.
That's not even a question for me, whether we're gonna go take a swing at building
the next thing. I'm just incapable of not doing that.
There's a- I would guess that, like, sometime in the next 12 to
we'll reach the point where, like, most of the code that's going towards
these efforts is written by AI. And I don't- as- as this goes on.
This isn't like that one to two year thing of what happens when you have a-
a- like a super powerful software engineer.
But
I think over time, you know, if- if you...
Like, everyone is gonna have these superhuman tools, um, to be able to create a ton of
different stuff. I think you're gonna get this incredible diversity.
Um, part of it is gonna be solving,
like, things that we hold up as, like, these, like, hard problems, like
solving diseases or, like,
solving different things around science or, um, or just, like, different technology that
makes our lives better. But
I would guess that a lot of it is going to end up being
kind of cultural and social pursuit and enter- Basically, okay, there's security holes in
so many different things, and if you could travel back in time a year or two
years, right? It's like,
that's not AI. It's like you just... Let's say you just have like one year or two years
more knowledge of the security holes, you could pretty much hack into, like, any system,
right? So like, you know, I stay up at night more worrying, well, what if, like,
some actor that... Whatever. It's like from wherever you sit, there's gonna be some actor
who you don't trust. If they're the ones who have, like, the super strong AI, whether
it's some, like, other government that we- that does sort of like an opponent of- of- of
our country or some company that you don't trust or whate- whatever it is, um,
like,
I think that that's potentially
a much bigger risk. As in- I- I do think that there- there is some
set of people who are afraid of like,
you know, it really just kind of spins and goes from being like somewhat intelligent to
extremely intelligent overnight. And I just think that there's all these physical
constraints that make that, so that that's unlikely to happen.
Um, I- I- I just don't, I don't really see that- that playing out.
I know there- there's all this science fiction about, okay, you create intelligence and
now it like starts taking on all these human-like behaviors and- and things like that.
But I actually think that the current incarnation of all this stuff at least kind of
feels like it's going in a direction where intelligence can be pretty separated from
consciousness and agency and things like that, that, um,
I think just makes it a super valuable tool, if I had to guess.
It's- it's really interesting to me that you basically agree with the premise that there
will be an intelligence explosion and something like super intelligence on either end.
But then if that's the case, tell me if I'm misunderstanding you.
If that's the case, why even bother with personal assistants and whatever?
Why not just get- get to super human intelligence first and then deal with everything
else later?
Well, I think that that's just one aspect of the flywheel.
Right? So
part of what- what I generally disagree with on the fast takeoff thing is it takes time
to build out physical infrastructure.
Mm-hmm.
Right? So if you want to build like a gigawatt cluster of compute,
that just is gonna take some time.
(music)
Stargate, put that name down in your books, because I think you're going to hear a lot
about it in the future. A new American company that will invest $500
billion at least in AI infrastructure.
The data centers are actually under construction.
The first of them are under construction in Texas.
The Abilene location, which is our first location.
Abilene is a town on the western central plains of Texas.
We kind of scratch and fight for everything good that comes our way.
I had people reaching out and they said, "When is the President going to come see you?"
And I said, "Well, he hasn't texted me yet, so..." (laughs)
I'm thrilled we get to do this in the United States of America.
I think this will be the most important project of this era.
Last month I came to, um, celebrate your winning
and promised that, uh, we will invest $100 billion.
And you told me, "Oh, Masa, go for 200." (laughs)
Now I came back with 500.
Explain the math to us. How does it all add up to needing $500 billion?
Well, that covers the capacity we think we need for the next few years, given our growth
projections. The- the interesting question is, if we had- if we knew how to get a
trillion dollars right now, which we don't,
would we be able to deploy that profitably in the next few years?
And I'm not sure about that, but I feel confident we can, like, make 500 billion of value
back.
I was in the real estate business. These buildings, these are big, beautiful buildings,
they're going to employ a lot of people.
How do you personally grapple with the pace of it all?
It does feel like it's going very fast.
Certainly think if I could like transport myself back three years ago, it would seem like
unimaginable progress.
The money piece, I think so long as AI is delivering
value to society,
the money will be there to build the infrastructure. I have no concerns around that.
Um, even if it's not all 100% accounted for today,
that money is there, and there's a lot of money that wants to invest in this category.
What we're undergoing right now is sort of the largest infrastructure build in human
history, right? This is like where we built the interstate high- highway system.
You know, that enabled massive interstate commerce and transportation and catalyzed this
massive boom in the US economy.
What's happening in AI infrastructure is we're building the superhighway system that's
going to enable modern intelligence systems, that's going to create all sorts of new job
categories that don't even exist today.
Mm-hmm.
It's going to create new ways of doing business, it's going to create new ways of
invention, um, and it's going to really drive humanity forward.
I don't think the world has really had the humanoid robots moment yet.
Like the first day, you're like walking down the street
and there's like seven robots that walk past you doing things or whatever, it's going to
feel very sci-fi.
And I don't think that's very far away from like a visceral, like, oh man, this is going
to do a lot of things that people used to do.
So- so yeah, it's coming, and we- we have always tried to just be super honest about what
we think the impact may be, realizing that we'll be wrong on a lot of details.
It doesn't exist.
How confident are you that AI's promise will outweigh the perils?
I'm no- I'm not confident, but the...
You know, that's- that's what we need to get oriented around.
That's why I was surprised AI wasn't really an issue in the last election.
It's definitely got to be shaped.
Hearing you say you're not confident is kind of scary.
People should
think through, how is AI going to reshape, uh, our lives?
It is- it is a profound, uh, change agent.
And, you know, I don't think it should just be the people who know the technology,
because there's a lot of choices about how it gets used. It will...