WEBVTT
00:01.600 --> 00:04.880
As of today, I still think there is a
00:05.080 --> 00:08.580
25% chance we will get superintelligence
by 2030.
00:09.620 --> 00:13.480
There's at least a 10% chance literally
every person on Earth
00:13.520 --> 00:17.440
will die by 2030. There's at least
00:17.500 --> 00:21.080
a 10% chance we will get a permanent
dictatorship.
00:21.340 --> 00:25.300
A handful of people will run the entire
world for centuries
00:25.380 --> 00:27.740
together, with no ability to, you know,
overthrow them.
00:28.300 --> 00:30.520
At least 10% chance of this by 2030.
00:31.560 --> 00:35.060
The only way all these numbers change
is if there is a
00:35.120 --> 00:39.100
political movement to pause AI research
that succeeds within the next five years.
00:40.320 --> 00:44.120
Uh,
I'm going to now describe my technical
00:44.320 --> 00:44.980
I think this.
00:50.580 --> 00:50.589
(electronic music)
00:50.640 --> 00:54.580
Creation of artificial superintelligence
is likely the most important event
00:54.620 --> 00:57.650
of approximately 10,000 years of human
history.
00:58.900 --> 01:00.540
It is more important than Industrial
01:00.600 --> 01:03.680
Revolution/Newton/French
01:03.760 --> 01:06.060
Revolution/printing press.
01:07.200 --> 01:10.940
It is more important than invention of
nuclear weapons, as an
01:11.020 --> 01:14.980
ASI will accelerate creation of weapons
more dangerous
01:15.040 --> 01:15.660
than nukes.
01:16.780 --> 01:20.560
It is definitely more important than the
creation of the internet and all
01:20.660 --> 01:22.180
Silicon Valley startups.
01:23.260 --> 01:27.080
If the takeoff is fast enough,
creation of superintelligence
01:27.120 --> 01:30.820
may be the most important event in the
approximately 14
01:31.000 --> 01:33.420
billion year history of the universe.
01:34.040 --> 01:37.820
It could end up more important than the
evolutionary history of all
01:37.960 --> 01:41.620
other life forms on Earth.
And to the best of our current
01:41.680 --> 01:45.500
knowledge,
Earth is the only place in the universe
01:45.540 --> 01:46.640
intelligent life.
01:47.840 --> 01:51.230
Scaling of RL compute still seems
important and
01:51.340 --> 01:52.020
unsolved.
01:53.100 --> 01:56.500
We still seem to have spent less than $1
million per
01:56.560 --> 02:00.220
inference, and can, in theory,
spend at least $1
02:00.400 --> 02:04.200
billion on it. In my head, it is an open
02:04.260 --> 02:07.370
question whether the labs tried it
and got bad results,
02:08.120 --> 02:11.320
or if there's some other fixable research
or infra
02:11.420 --> 02:15.220
bottleneck that is preventing them from
trying it.
02:15.360 --> 02:19.120
Meta-graph on this still seems useful,
even though it is making
02:19.200 --> 02:20.140
simplifications.
02:21.340 --> 02:22.700
This is still important.
02:23.700 --> 02:27.280
If we have one AI that is superhuman,
we can
02:27.320 --> 02:31.120
probably run at least 100,000 copies of it
at
02:31.420 --> 02:33.780
100 times the speed of human thinking.
02:34.340 --> 02:38.120
This means that once the first slightly
superhuman AI is
02:38.160 --> 02:41.900
invented,
we will go from slightly superhuman to
02:42.100 --> 02:45.720
vastly superhuman in a very short span of
time.
02:46.960 --> 02:50.740
I would nowadays more emphasize the
meta-learning point below,
02:50.780 --> 02:54.580
though,
because that decides what happens until we
02:54.600 --> 02:55.860
superhuman AI.
02:56.840 --> 03:00.240
Deferring to experts
is obviously still important.
03:00.460 --> 03:04.300
I value my inside view a lot,
but I value expert
03:04.340 --> 03:05.940
opinion some amount too.
03:07.160 --> 03:11.020
Many researchers who pretend not to be
doomer also have
03:11.080 --> 03:13.340
timelines in the next five to 10 years.
03:14.040 --> 03:17.400
Examples, Rich Sutton, Ilia Sutskever,
03:17.980 --> 03:19.740
Andrej Karpathy, et cetera.
03:20.760 --> 03:24.460
There is the obvious list of doomer
researchers, example,
03:24.860 --> 03:28.660
Geoffrey Hinton, Yoshua Bengio,
Stuart Russell,
03:28.880 --> 03:29.340
et cetera.
03:30.380 --> 03:34.300
Heads of all frontier labs agree
that their own tech could
03:34.380 --> 03:35.780
cause human extinction.
03:36.640 --> 03:39.840
Example, Elon Musk, Sam Altman,
03:40.460 --> 03:43.960
Dario Amodei, Demis Hassabis, et cetera.
03:45.000 --> 03:48.250
Generalization and meta-learning
are related.
03:48.990 --> 03:52.980
Newer models aren't just memorizing more
skills, but are more
03:53.040 --> 03:56.780
capable of learning new skills on their
own without human
03:56.800 --> 03:57.780
data guiding them.
03:58.750 --> 04:02.620
There is a difference between training AI
such that it learns skills that
04:02.680 --> 04:06.580
humans have already have,
versus training AI to know
04:06.720 --> 04:09.060
how to learn new skills on its own.
04:09.880 --> 04:12.820
The difference between these two
is not binary.
04:13.340 --> 04:15.280
There are levels to generalization.
04:16.260 --> 04:18.640
Example of low level of generalization,
04:19.459 --> 04:23.340
seeing lots of English to French
translations and learning to
04:23.360 --> 04:26.100
become good at English to French
translation.
04:27.080 --> 04:29.580
Example of medium level of generalization,
04:30.440 --> 04:34.420
seeing many solved competitive programming
puzzles and solving
04:34.580 --> 04:38.540
another puzzle with an algo similar to one
it has seen before for
04:38.550 --> 04:39.490
a different puzzle.
04:40.300 --> 04:43.940
Seeing lots of English text
and a little Hindi text,
04:44.440 --> 04:48.430
inferring the grammar similarities
and differences across human languages,
04:48.980 --> 04:52.180
and then learning to speak fluent Hindi as
a result.
04:53.000 --> 04:56.820
Yeah, lol. This is a real result,
and an old
04:56.960 --> 04:57.140
one.
04:58.090 --> 05:00.560
Example of high level of generalization,
05:01.420 --> 05:05.280
seeing a theorem from economics
and realizing an analogous
05:05.300 --> 05:09.240
version of it also applies to biology,
thereby solving
05:09.280 --> 05:11.410
an open problem in biology.
05:12.480 --> 05:16.150
GPT-2 to GPT-5 has led to
05:16.300 --> 05:20.140
immense progress in both learning new
skills and
05:20.200 --> 05:24.010
learning how to learn new skills.
The latter
05:24.060 --> 05:25.290
is more important though.
05:26.280 --> 05:30.240
The best way to test the latter
is to give AI problems that
05:30.300 --> 05:34.080
a human has never seen
and no human knows how to
05:34.200 --> 05:38.160
solve. This could be simple tasks,
like translation
05:38.200 --> 05:41.600
in apocryphal ancient languages,
or complex
05:41.660 --> 05:45.540
tasks,
like proposing novel molecular bio
05:45.560 --> 05:45.840
run.
05:46.800 --> 05:50.540
Continual learning is a hard problem,
but my hunch
05:50.620 --> 05:53.360
is one or two breakthroughs might solve
it.
05:54.480 --> 05:58.080
AI weights are currently static,
and hence, AI
05:58.180 --> 06:00.240
behaves as if it has amnesia.
06:01.060 --> 06:05.000
Once a chain of thought is complete,
the AI forgets it ever
06:05.040 --> 06:08.988
did the task.Naive way to make sure AI
06:09.008 --> 06:12.778
remembers its previous inferences
is to fine-tune it
06:12.848 --> 06:16.718
on this data. But fine-tuning LLMs is
06:16.728 --> 06:18.348
very sample inefficient.
06:19.208 --> 06:23.178
A slightly better approach
is to put this data back into the chain of
06:23.288 --> 06:27.068
thought itself.
But there may be limits to how long
06:27.178 --> 06:30.748
chains of thought can be.
Open research question.
06:31.648 --> 06:33.528
We might discover better approaches.
06:34.708 --> 06:38.488
Operating in the real world can be
expensive in domains like
06:38.548 --> 06:42.428
biotech. Therefore,
we must be sample efficient
06:42.588 --> 06:46.328
in terms of how much data the AI requires
from real-world
06:46.388 --> 06:50.168
experiments before it learns useful
insights from it.
06:50.228 --> 06:53.488
Human language has features not present in
animal language,
06:54.148 --> 06:57.828
and this is likely an important part of
why humans can build spacecraft and
06:57.948 --> 07:00.288
colonize the Earth, but apes can't.
07:01.468 --> 07:04.808
AI already has picked up all these
features of human language.
07:06.048 --> 07:09.768
Go read Hockett's views on what features
separate human language from animal
07:09.828 --> 07:10.328
language.
07:11.748 --> 07:13.888
Humans use language for communication.
07:14.788 --> 07:18.058
Humans might also use language as part of
their reasoning process,
07:18.568 --> 07:21.868
along with other modules,
such as motor skills, spatial
07:21.928 --> 07:23.388
visualization, et cetera.
07:24.368 --> 07:28.328
Both of these are hypotheses for what
separates humans from apes, and
07:28.338 --> 07:29.988
I think there's a good chance they're
true.
07:31.188 --> 07:34.768
Other hypotheses include bigger birth
canal and brain size
07:35.228 --> 07:38.008
and evolutionary pressures to win social
games.
07:38.988 --> 07:42.928
These hypotheses seem compatible with the
hypothesis that language is most
07:42.968 --> 07:43.548
important.
07:44.768 --> 07:48.388
Depending on how you measure it,
AI may now be the second
07:48.508 --> 07:51.388
most complex object in the observable
universe.
07:52.268 --> 07:56.088
More complex than ape brain,
but less complex than human
07:56.168 --> 07:56.488
brain.
07:57.718 --> 08:00.728
Model scaling for text models might be
saturating.
08:01.148 --> 08:04.388
Open research question. But it
is definitely not
08:04.508 --> 08:07.828
saturated for images, video or robotics.
08:09.028 --> 08:12.948
As of December 2025,
text models intuitively feel
08:12.988 --> 08:16.918
better than image models,
and image models intuitively feel better
08:17.048 --> 08:21.008
than video models,
and video models intuitively feel better
08:21.048 --> 08:24.838
than VLAs for robotics. I
08:24.868 --> 08:28.588
think this is mostly just because of
higher compute requirements for the latter
08:28.668 --> 08:29.108
models.
08:30.388 --> 08:33.608
Minor update as of January 6th, 2026.
08:34.308 --> 08:38.178
Robotics has an additional bottleneck
where big enough datasets are not
08:38.268 --> 08:38.688
available.
08:39.578 --> 08:43.288
Manually generating them is expensive,
but probably affordable
08:43.328 --> 08:47.068
given current AI R&D budgets. Most
08:47.108 --> 08:50.988
people seem to IMO suck at forecasting AI
08:51.108 --> 08:55.068
progress even one year into the future,
let alone
08:55.148 --> 08:56.048
five or 10.
08:57.228 --> 09:00.988
Since 2022,
I am used to watching people on Twitter
09:01.068 --> 09:04.668
predictions of some specific benchmark
or skill
09:05.028 --> 09:08.948
X that will never get solved,
only for it to get solved
09:09.028 --> 09:09.808
one year later.
09:10.968 --> 09:14.948
Your specific AI can't do XYZ task
09:15.308 --> 09:18.788
that is trivial for humans
is not impressive to me
Enter "Subscribe: " followed by Signal username, email address or whatsapp phone number. You will receive atmost one update per month. View previous updates here.
Alternatively, you can use RSS, or a third-party service like blogtrottr.
Enter "Comment: " followed by comment. Alternatively, contact me