... of superintelligence by 2030.
Say that again? Sorry, sorry.
Property of superintelligence by 2030.
You can just give me-
Some percent.
Hm?
Some percent. Some one percent.
Oh, one percent. Okay.
Wait, what's the question? Ready?
Property-
ASI by 20- 2030.
Okay.
Superintelligence by 2030.
Okay.
What?
Maybe we can do this later, but what is your quick, uh, definition of ASI?
Uh, an AI that is better than the best humans at all tasks, you know, it's like-
Well, that's AGI, no? I thought, like-
No, much-
... like significantly better.
Yeah.
Like, to the point that it's kind of hard.
Okay. Uh-
ASI is sum total of humanity.
That's what ASI is.
Better than the experts in the field?
So it's 100 percentile-
Better than the experts, and better than the best humans at every task.
So it's 100 percentile AGI and can be high enough.
But there's no guarantee of generalizable intelligence.
But, but, like-
Uh-
Raghav, Raghav-
... 500.
Guys.
(clapping)
Yeah, yeah. So, yeah. I'm saying better than best humans at every task.
So-
Okay. Then I'm not sure. Like, I think that's where there are, like, a lot of things I'm,
I disagree with.
(laughing) We should have probably had...
Okay, so-
So, I think you can-
Agreed on the definition. (laughs)
... have this conversation later.
Yeah. This should have been done in the first place.
Maybe I... Sorry.
Wait, wait, wait.
Maybe if I do a quick clarification and maybe we can agree to this.
Can you all just-
Yeah, sure.
With this promise.
Like, in my head, AGI would be like, like, uh, pick the smartest person in the class,
pick the dumbest person in the class. Uh, like, I'm just saying, like, an average.
And the intellect difference between them, uh, is like, okay, an AGI, I expect the AGI to
be that much smarter or slightly smarter than them. Right?
Like, not that much, much smarter.
Than the best human.
Than, yeah.
In that job.
So- so, like, how much, however, whatever the difference between-
The median human.
... the dumbest- dumbest guy in the class and the smartest guy in the class-
Yeah.
... you, like, add another data of that. That's how much I'm expecting AGI to like...
Like, that- that in my head is like AGI.
No, but that, the second data is from the mean-
Yeah.
... human or smartest-
This person-
... human or the dumbest human?
I'm saying, like, take the smartest also.
I think it's, like, they're in the same ballpark.
Where- whereas-
But-
... what- what I'm saying about ASI is what- what...
In my head is a difference of intellect between me and my dog. Right?
Like- like, there's like, there's so much difference that it's not like the smartest dog
or the dumbest dog or the median dog or it's just means it's just, like-
Yeah.
... in the same gap it doesn't matter how smart the dog is.
Okay. So then I will refine more specifically.
(laughs)
Yeah.
AI that is better than best humans at all tasks.
This includes a physicist, like an AI physicist that's better than the best physicists on
Earth, an AI mathematician that's-
But, like, just slightly better.
Uh, slightly better, also works.
Just, like, just... Okay, okay, fine.
Then, okay-
They're the same, right?
What's the property-
That would be a, like, massive disadvantage.
(laughs)
Okay, what's the property by 2030 if you had to give one.
Uh, like,
like, a good chunk. Like, probably like 25%. Or like... yeah, so like, like 25%.
(laughs)
Then there's no debate. (laughs)
(laughs)
You got it.
But that's, but then I disagree with all the threat models here.
I don't think a slightly smarter scientist is actually, like going to kill us.
Oh.
Like I, like- I'm not that much of a...
I'm more worried about like the-
Yeah.
... the dog, like me and my D-... like me, like my dog trying to, like, plot to con-
Oh, but-
... I'm like, "Ha ha he."
But now the probability, what is the probability of RSI?
Okay, I think now I need to... Because we seem to be agreeing so much, I need to present
an entirely different argument which I've not presented till now.
Yes.
Which is,
once you have s-... like, again, my belief, and I have certainly no argument for it.
Once you have a slightly superhuman AI, you go from there to, like,
unimaginably superhuman AI kind of civilization, like, very quickly.
Oh.
And, uh, reasoning for that, and I don't know if we actually can discuss this, but, um-
With RSI in the-
RSI, intelligent explosion.
Yeah, yeah. Like, basically it can parallelize things.
So, like, right now when you're calling even, like, AI model actually, like, there are
millions of people calling the AI model at the same time, so it's very parallelized.
Yeah.
But, like, each instance is kind of isolated.
You know, you call one model, it gives you an answer, then it forgets the answer.
Like, it's just stored on some hard disk somewhere.
Yeah.
Like, that output is not being preserved.
However, if you have something that's actually smarter than the best humans, given to you
has memory, has the things you've thought of before, it can work on those and it can-
That isn't, isn't that like saying if I give, like, a million dogs enough time they'll be
able to come up with, like, cancer treatment?
No, but they are, they are already pretty good, right?
Then you need to, like, also parallelize smartness.
And-
No, they're... Okay, sorry.
Yeah, yeah, yeah.
Let's let them-
If this one's good, then we'll put a timer, yeah.
Yeah. I'm starting-
Give him a time.
... with the assumption of, okay, let's say in physics we have an AI physicist that's
better than Einstein, maybe only slightly better than Einstein.
This is the starting assumption. We go from slightly smarter than Einstein to
unimaginably smarter than Einstein very soon, and then we have the argument you provide.
And that is, I mean first of all, you have the memory thing. If you have su-...
uh, AI that is slightly smarter than Einstein,
and it can keep referring back to all the things it has already thought, and it can do
this over 22 years, that leads to a lot of improvements.
Then there's like a serial speed up where you actually compare the numbers and you see
how long does it take, you know, let's say a human physicist to write, you know, one page
of interesting text. How long does it take, you know, an AI today to write one page of
interesting text? There's like a 10 to 100 speed up.
So, like, what took you maybe 10, like 10 minutes or 100 minutes to write here, the AI is
writing that in one minute.
And if that continues then you do end up with, you know,
what, uh, actual Einstein would have achieved in one year, you know, this AI-
Yeah.
No, sorry. What an actual Einstein would have achieved in 100 years, you know, this AI is
doing it in one minute.
Yeah.
There's that serial speed up. And then there's the parallel speed up on top of all of
this, which is you don't just have one AI that's smarter than Einstein, you have maybe
like 100,000 Einsteins who are running in parallel
because, you know, again, there's that much APs are available on earth here to start
running one AI, you're running hundreds of thousands of copies of that thing.
So the actual comparison, like the moment you have, like one AI slightly smarter than
Einstein, like my brain immediately jumps to 100,000 AIs smarter
than Einstein running at like 10 to 100 times the speed of, uh, Einstein.
Okay.