Yes. Um, I sympathize with those who, um, are devoting their lives
to, um, alignment issues and trying to build AIs aligned
with, uh, humans.
Uh,
in the long run, however,
I think there is no way of, uh, stopping, um, all kinds of,
um, AIs from having all kinds of goals that have very
little to do with, uh, humans.
When I was a boy, I, um,
I was thinking about what to do in my life.
And then I thought the most
exciting thing is to solve the
riddles of the universe, and- and that means you have to become a physicist.
However, then I realized
that there's something even grander.
You can try to build
a machine
that isn't really a machine any longer, that learns to become a much better physicist
than I could ever hope to be.
There's lots of material out there that you can use to build robots and
self-replicating robot factories, and all this stuff.
And they're going to do that, and they will be scientists, um, and curious, and they
will explore what they can do....
physics, you need
matter and energy to compute and to build more robots and infrastructure and more
AI civilization and A- AI ecologies consisting of trillions of
different types of AIs. And- and so it seems inconceivable to- to
me that this thing is not going to expand some...
So,
there's plenty of time
to conquer the entire universe
and to fill it with intelligence and senders and receivers such that, um,
AIs can travel the way they are traveling in our laps
today, which is by radio from sender to receiver....
in the ascent of humanity, I think succession to AI is
inevitable. Maybe this- these will be two machines or maybe it will be humans augmented
by machines.
As we understand ourselves, it's inevitable that we will become stronger and stronger.
And- and we will continue changing as we have
throughout
history and pre-history.
I think this much is obvious,
but this is still very human-centric.
We're saying what's, we're saying what's happening to us, you know? It's us.
Why be so human-centric?
What about th- the universe? What is the universe thinking as all this happens?
Uh, what do- w- what would be the universe's perspective?
...
but I- I want to touch on... So for you, there is an inevitable consciousness, uh, to
this system, which is why in part you feel like a lot of successors would be worthy
because they would be, by definition, if that complex necessarily awake.
Let me know if that's safe to nutshell.
Trying to decide how I feel about the notion of a worthy
successor.
Hmm.
Even that, even that notion involves the entitlement that- that it
has-
Yeah.
... grown to be worthy of us.
Hm.
Well, one big thing will happen will be
augmentations.
Mm-hmm.
If we understand how our minds work, then we ought to be able to add, you know, a better
memory.
Uh,
I- I want to, I want to be augmented. I want to-
(laughs)
... I want to think better.
I want to, uh,
you know, there are many advantages to digital substrate over the biological one.
Um, so we could be better.
I would like to, uh,
augment my mind so that I can understand more, that m- my memory was better and I could
calculate more possibilities and I could experience
more sensors-
Yeah, yeah, yeah, yeah, yeah.
... like seeing, seeing the dark and
have a, have a tail. (laughs)
(laughs)
Yeah.
More arms.
Yeah.
Um, yeah. Uh, we want all those things.
Obviously all of the things you might see in science fiction.
Uh, the ability to explore into the, into the universe
and- and the ability to control physical matter
as
generally.
Yep.
So- so if you decentralize th- then the assumption is there would not be one,
one possibility. There would be multiple possibilities, and they would be competing and
then that one would lose out. But as much as the paperclip maximizer could lose out and
all these...
Possibly any one of these highly centralized dictatorial,
uh, arrangements would lose out to,
um,
a more, uh, decentralized one.
The-
the doomers, the doomers are just saying, "Oh, somehow there's gonna be, it's going to
be... I- i- that it's bad in the same way that nuclear weapons are bad." That they...
And that's just, they're just blinded by that metaphor,
um,
by thinking that- that the AI will be out to kill them.
Um,
that's just, it's just silly. And I- I don't, I don't think...
Well, they n- the, the doomers
don't actually give coherent reasons for what they bel- what they believe.
Uh, and so it's hard to argue with them.
Uh, so
maybe it's fair just to hope that they're, they're biased and blind. I don't...
...
at this point.
No, I do have a prediction.
Yeah.
The prediction is one chance in four.
In?
Let's see.
Of what?
By 2030.