Hi, everyone. My name is Samuel Shadrack.
Uh, today we are going to be talking about AI timelines.
I have Saksham with me who's a good friend of mine.
Uh, we're going to talk about when, like, really powerful AI could come, like
when in the future. What are the reasons it might or might not be built?
How can we predict this? What are our differences of opinion, if at all we do have any
differences of opinion?
Uh, we have had these types of discussions in private before, uh, but I thought it
would be interesting to put all of these discussions in public and see, you know, how
people react to it.
Uh, Saksham, do you have any thoughts before we begin the discussion?
Yeah, yeah. Looking forward to
disagreeing a lot. (laughs)
(laughs) Yeah.
Or agreeing, whichever, whichever we end up doing.
Uh, yeah, yeah. I'm Saksham. I've thought about AI for a fair amount,
um,
and
I have some-
somewhat, you know, uh,
short-termist and, uh, somewhat extreme views on
where I think this could plausibly lead us.
So,
uh, yeah.
There's a lot to talk about.
Got it. Got it. Makes sense. Uh,
I think we can just start then. Uh,
maybe, I don't know, Saksham, do you want to go first?
Like, if you have to just, like, kind of summarize your opinion on this whole topic and,
you know, just like
two lines of, like... In, like, very brief, like, how would you just summarize your
opinion of the whole topic?
Okay. Uh, I've tried this on my own. I, I tried to, like, summarize my, my whole
worldview that is, uh, built around my takes on AI progress. Um, and it's tough, but...
Okay. So, I, I think AI progress will, uh, continue to yield results, uh, in somewhat
discontinuous manner. But some of the jumps that I would expect to
see
in the next three to five years
will be transformational to begin with, and,
uh,
can be extremely harmful.
And
what that
leads me to is the position
of not wanting to
predict or make longer term,
uh, predictions beyond,
say, the next seven-ish years. Because I expect the world
to look vastly different
based on AI progress.
Got it. Got it. Okay. Fair enough. May, can I also summarize my
position then? I mean...
Yes, I think AI progress is definitely going to continue.
Like, I'll be very surprised if, you know, current models, say, 03 or GPT-4.5
are, you know, the best AI model we're going to get in the next few years.
Like, there will probably be some progress further.
I'm unsure about how much more progress there'll be until, you know, progress stops.
Like, I have a model kind of, of like, progress will keep going on till some point and
then it will just kind of stop or, like, it will just really slow down.
And this is very specifically based on, like, scaling laws.
And scaling laws being this kind of unpredict- like, this prediction which
whose, you know, underlying mechanism we don't understand.
So, I think a lot of my worldview just comes down to, you know, how far this scaling laws
hold versus not hold.
So, I feel, you know, if scaling laws hold for even another maybe five years
or... yeah, five to 10 years at the most. Not more than 10.
Uh, we literally get, you know, super intelligent AI and AI that will be vastly better
than humans at everything humans can do.
I like defining this threshold based on, you know, super intelligent AI, not based on,
you know, AGI, like
artificial general intelligence, which is the more common term which people use.
I don't like using the term AGI and I can talk more about that.
But yeah, I have like, you know, a 25% probability we get
super intelligent AI by 2030. Uh, a much larger
probability, at least 50% that we get more powerf- like, AI
that's more powerful than today, but less powerful than super intelligent AI, you know,
by 2030.
I also maybe lower probability that we don't get much progress at all and we kind of
stagnate, like, that probability is, is pretty low. So, I think that's my view.
Yeah.
Um, so what do you want to, like,
talk about first?
Is there, like, a specific vector of this whole problem that you want to approach first?
What's the... Okay, I can maybe ask a question.
Um,
what do you think is the
single most important factor
that
is, like, very crux-like for your predictions?
Yeah,
let me
try to
understand what you are trying
to suggest before. Yeah, the thing I am basically not understood why you are...
why do you have such... this level of confidence in your prediction that, you know, we
are going to get really transpo-... like, typically for me, it's like, if
you don't have a good model of something, then you just have a lot of uncertainty.
And that's kind of my position, where I have a model but I'm not very certain of it.
So, like, by default, end up with uncertainty and different possible outcomes.
Whereas you're seeing more, like, certain and that...
so the- like, I then end up guessing, you know, you must be having some model faults
going on, and that's why you're certain.
Okay, yeah.
(slurping sound) So,
(clicks tongue) uh, I would say my priors on this are
grounded in,
um, the developments over the past
five, six years.
Uh, key ones being,
uh,
like this assistant/instruction-following paradigm
that we're kind of almost, uh, you know,
moving beyond already.
Which is, you know, how do you take a language model that does well at the
job of prediction, next token,
um,
and turn it into something that can follow instructions and have a conversation,
and, uh, you know,
do all that things that we currently expect cha-
uh, you know, models to do. And we kind of think of it as, to some
extent, a trivial,
uh, trivial thing, because on, on a product level
we've only
interfaced with models that have been,
uh... you know, that have gone through some post-training or something to
make them usable as assistants, which is how most people
still use, you know, these models.
Uh, but before that, we had this just enormous amount of textual knowledge
that,
uh, was just inside this model.
And
over time we've learned that, okay, just like, some amount of post-,
uh, you know, post-training
stuff,
such as RLHF, or now today we're al- also looking at, like,
from, like, cold start reinforcement learning without human feedback
to get a lot of, like, very promising results on,
and very usable results that have basically solved this problem of, like, how do you take
a language model that predicts the next token?
Hello. Sorry, you dropped off there for a sec.
No, no, I'm here. I can hear you.
Yeah. So,
how we take a language model that can do next token prediction and turn it into something
that's usable.
And
the, the follow-on effect of this is like, okay, OpenAI did this first, but
based on, like, similar,
like,
training ideas, we can turn a, like, very, very poor
next token predictor, so like these small models, like the initial LLaMA models
and, you know, the, the billions of...
not billions, but thousands of, uh,
tiny, tiny models that end up on Hugging Face.
Who, if you ask them the question of, "Hey, how are you?" Everyone,
all of these models, like ranging from the three billion tiny model to, up to, you know,
these trillion-parameter models, all will give like a similar,
coherent,
working English answer of, "Hey, I'm, I'm, I'm
all right. How about you? What do you want to work on?" Some standard template response.
I think that was one. The second one was the ability of language models to
do math.
And this is before we're, uh, doing this, uh, current, you know, chain of
thought, uh,
you know, lots of samples,
uh, and verification. Like, even before this, when
trivially you ask
some, uh,
some language model, even if you have some base model that hasn't been,
uh,
hasn't gone through post-training to become an instruction-following, uh, you know,
assistant thing-
Mm-hmm.
...
even that could often
do math. It, it did obviously struggle in a lot of things, like two
very long pieces of text, uh, like two very long, you know, unsigned
integers, ask it to add it up,
hallucinates digits or sometimes gets things wrong.
But the fact that it got some, gets some right
is, uh, is like kind of,
kind of... Yeah. Uh, sorry. So, I was talking about this maths thing where
behind these language models, using
just tokens, there is something going on under the hood.
There are some circuits that are
building,
uh, you know...
uh, basically building the algorithms that are provably correct
for solving simple mathematical problems, like simple mathematical operations like
addition and so on. So one, uh,
one, like, initial result on this was, uh,
like Neil Nanda's grokking,
uh,
like result.
Which was,
uh, he took a very, like, very small,
uh...
yeah, just very small transformer and
used it to...
yeah, used it to solve the problem of, uh, modular addition.
And what he saw was, over-training, there was a very discrete
jump-... discrete jump in the loss. And basically a
point where
the model understood what the problem was and built
circuits
that were interpretable to solve these problems.
So, I mean,
uh, and this, the same principle I would say still applies to, like, the
larger models, although these circuits are largely uninterpretable.
There are, like, some nice write-ups about,
uh, from Anthropic on Claude's inner workings.
And,
uh,
there is a...
I think there's,
there, there's one recent one called Tracing the Thoughts of a Large Language Model,
where they
look at another math-based example, where how
the
language model tries to
do, like, basic addition- addition.
So just, like, two-digit numbers, two two-digit numbers, how does it choose to add them?
And it's, like,
kind of mind-blowing 'cause it's not like
the normal addition algorithms that we're taught.
It's a lot of weird mental math-based tricks
that always tend to,
you know,
uh,
lead to correct
results.
And
it can accurately solve addition even though
it's not, it's not a calculator.
The same way humans learn to solve addition, but using algorithms and slash circuits
that
it finds more useful,
uh, as opposed to the ones that humans understand, uh, natively.
And I think these kind of, these kinds of improvements in the field
suggest to me that there is something
inside these models.
Uh,
it might not be, like, necessarily this architecture that ends up leading to
AGI, but, uh, could very well be this exact
architecture, the transformers,
uh, one that we use,
uh,
that
once you feed it data, it's just hungry to learn.
It finds a way to learn given the right,
uh, you know, given the right data, given the right,
uh,
in, you know... Yeah, just given the right data, it's just hungry to learn.
Um, this is similar to, I think, what...
from what I remember, Ilya Sutskever's,
uh, thesis on
transformers and large language models is
where you just... these models are just hungry to learn and given enough data.
And we are currently with
the, the, the widespread adoption
of AI. We're currently on the process of gathering tons and
tons and tons of data
that would be invaluable for
models later on down the line, much better models that are
scaled more in terms of the initial pre-train- training size
that can do chain of thought in- incredibly well and have been
post-trained to
do that as well as the inference time scaling thing.
But working together, there are, like, these three,
uh,
three vectors of
progress, and none of these I see have been maxed out.
I think pre-training is the one that people think is closest to being maxed out in
terms of how much more we can get out of pre-training large language models further.
But I also,
uh, I also, like, hedge against that,
thinking that some of the gains that we have seen historically from scaling
pre-training more and more
haven't been continuous. And there have been, like, sudden jumps, or what we
might call, like, emergent properties in terms of what the model suddenly becomes
capable of doing once we scale it a hundred times that we couldn't get
signs of if we just scaled it 10 times.
So, I think this, the pre-training scaling law thesis is still valid.
So is this new idea of like, if we know that we are going to be using reasoning and
chain of thought for this model, once we, we can
inform... Uh, based on that, we can
do post-training, which is,
uh, reinforcement learning or, uh,
you know, uh, supervised fine-tuning,
and we can actually scale that back. There is not, like, a simple, like,
fixed size solution. Like, the more you do it, sometimes the better results you get.
And that's one of the, like, somewhat remarkable things that
the DeepSeq models recently, uh, demonstrated.
And, uh, there's also this inference time scaling, where just giving the model time to
think if it has been, you know, uh,
trained in post-training to do more thinking for solving more complex
problems, it does lead to better results, which is this, uh, OpenAI O-
O1, O2, O3 model paradigm.
And I think the room
there is,
is like immense 'cause there's so many different kinds of tasks that we haven't
tried to target
with
providing models with a specific
curated dataset for it to learn those problems effectively.
Uh,
and there is a lot of money
in this space. Like, there is a lot of money in the space.
And
I expect this kind of bubble-like situation to continue until we
get to, like-Uh, either we get to, you know,
some,
like ASI kind of thing,
or
we get to something that,
uh, can
plug and play, replace a lot of knowledge workers.
So there are, like, some sort of, like, milestones, even though, you know, replacing all
knowledge workers might not be the end game in terms of what these companies might want
to do.
But there, that's a milestone where there is a risk of a company might want to
cement their, uh,
their lead, although the market would still remain com- uh, like,
competitive. Uh, there is a chance where one company gains such a big
lead and they have so much money that you're looking at like windfall scenarios,
and progress beyond that gets stalled.
Uh, but
I,
I don't think that's as likely to happen, as much as I believe that it would still
remain like a competitive situation.
And,
uh,
based on like recent,
recent trends of like how we're distilling large models into smaller models work,
works, uh, I think,
uh, there will be like a
pretty decent ecosystem around the large model builders that
try to out, you know, out-compete, as well as, you know, the, the whole China situation
where they kind of have to play the open source game,
uh,
and
how that ends up
just accelerating progress further.
Okay. Okay.
So these are kind of my priors on why I think progress
in AI is going to continue, and
why I think,
uh, there is a lot of overhang in terms of theoretically what's possible but hasn't
been tried yet. And solving that in- in- involves,
you know, training these models,
uh, on tasks that we want them to do, and not just u- you
know, using some out of the box LLM.
Also, as well as that, plus, uh, scaling these models more.
Okay.
Got it.
(laughs) Okay. So, I think there's two things here. One just, you know, exact, like...
yeah, one is probably just, yeah, what are the different outcomes you're predicting with,
you know, what likelihood? And second is, you know, what's the reasons behind them, uh,
reasons behind them?
Yeah, it's just in my head, I kind of bucket it into three different things.
One is like model scaling, second would be like inference scaling, if you want to call
it that. Third would be, you know, some algorithmic improvement, which as of today we
haven't discovered but in future we'll discover.
And I think out of these three you seem most bullish on like inference scaling, like
using RL. Like the first point you mentioned at least was just specifically on like
building RL gives you a lot of gains, and we haven't...
gains in like performance, and we haven't reached like the end of that.
And like just putting more compute into that is going to give us a lot more performance.
I think you seem most bullish on that.
I think your second point you mentioned regarding clogging.
I just wanted to clarify, would you con- count that more as just model scaling or more
RL? Because like I think say Neil Danda's paper if I remember was more on model scaling,
like use bigger models means more clogging.
So would you count that under like model scaling?
Would you count that under like, say, better RL, more compute spent in RL?
Um, I don't see why RL wouldn't necessarily lead to the
same kind of thing. But I think,
uh, I'm not very,
uh,
I'm not very familiar with the technical aspects of reinforcement learning as much.
So I...
like, I'm, I'm trying to think of like the kind of, uh,
solutions self-play robotics or simulated systems,
uh, find as opposed to
like
the kind of systems these huge language models transformers find
in terms of, uh,
when they're asked to solve a specific problem, and- specific problem and we measure
their progress along just that task.
Like asking a computer simulation of, uh, a-
a- a simple, you know, stick figure
and getting it to stand or getting it to run et cetera, et cetera.
Like,
like it... I know that it does happen where if you give,
you know, this, this model, uh, this, uh, stick figure in a simulation
enough RL with enough compute,
it figures out a way to stand, if there is one or something.
But,
yeah, I'm not, I'm not very aware of like what the scaling trends of...
scaling trends on,
um,
on that,
whereas like I don't know how much more scaling in terms of RL compute you need to do.
Whereas how much more algorithmic and theoretical progress do you need in terms of, uh,
uh, finding good ways of updating policy or,
uh, trying to fig- move beyond like...
or try to, you know, figure out how to
correctly reward an agent.
No, okay, got it. Um, but I think even then coming back to whatever maybe just started
asking, like one point which I think I'm definitely getting from you is that, you know,
RL is going to progress either be, you know, just blindly putting in more compute or
maybe some algorithmic improvement in how exactly you do the RL.
Like you're definitely very bullish on RL, I think. That I understood very clearly.
I am asking separately, do you also buy into like model scaling will get us further,
or is model scaling something you are not that optimistic about?...
like, just bigger model- Yeah. ... more data, more compute, and we get results.
The- Like, keeping out of aside from that.
Like, is this model scaling as, like, a smaller factor in your ranking?
Like, in your
I-
I don't necessarily think of these as
specific.
Like, by- Sorry, I don't necessarily think of these as, like, separate things.
So one thing, when I was looking into the DeepSeq model with...
Which is like, it's not a very... It's- it's- it's a lot of different experts.
So a lot of, like, smaller models
coming together. So it's a mixture of experts model, which is unlike the-
Yeah.
...
which kind of has been, like, the trend, uh,
in term... I think since GPT-4. 'Cause GPT-4 was a mixture of
experts model. And I don't think they released the model card on
4.5 yet, or I haven't looked at it, if they have released.
Like, what the architecture looks like.
But dense models generally,
you know, aren't the same. And there's something, like, interesting that happens when
you,
uh, do reinforcement learning on
mixture of expert models,
uh, for the, you know, for like, specific tasks, such as, like,
reasoning.
And you get a lot more gains than you would expect it to with, like, a dense
model.
So,
one way of scaling a mixture of expert setup for pre-training is
more experts of the same small size or fewer experts, but the experts
themselves are bigger model. So imagine a mixture of experts model that is
four GPT-4, like, eight GPT-4 sized models.
That doesn't necessarily require the same amount of scaling
that we have seen,
like, up 'til GPT-3.5.
Or we have seen up to, like, you know, other models.
And...
Yes, but I- I do think,
uh, that reasoning is, like, an important breakthrough.
And,
uh,
reasoning plus RL seems to give us a lot more results, especially on like,
complex but not too long horizon tasks.
Uh,
I think I- I'll be more partial to,
like, doing more post-training on existing models.
But I also think you don't get to, uh,
like ASI level things if you don't scale the pre-train model by
at least another 10X or maybe even like 20X.
Which, yeah, which is like crazy expensive.
Uh, and,
like, might not be feasible given like the economic situation,
uh, at- at the point where, you know, we are looking for
that next step of improvement.
Oh, okay. Got it, got it.
Yeah, uh, and I think I kind of see, like, mixture expert as like a hack.
Like, (laughs) I don't know.
I just see it like, okay, this reduces the amount of,
uh, compute you need. I'm not sure of the exact number, but maybe by a factor of
10X, not more than 10X. And you would have gotten the same result anyway if
you didn't use, like, mixture experts but you just spent 10X more compute.
Like, I just see it as... I don't see it as a thing that scales, like, let's just say
like, now it'll be 10X and then it will be 100X and then it will be 1000X.
I'm, I kind of see mixture experts as like, it's just always just going to be at the most
maybe a 10X improvement and maybe not even a 10X figure.
I think in practice it's less than a 10X improvement.
So, I think that's kind of my view on mixture experts.
Uh, maybe you have a different view that we can talk about that.
But yeah, I think broadly your view then is mostly around, yeah, like,
RL being scaled up and...
Okay, so then that makes, then that means now I need to talk about what are my views on
RL. Uh...
Yeah, I think I am more bullish actually on model scaling and I am more
uncertain on RL.
Uh,
and on the RL part, I kind of see it more as a new trend, whereas I think you seem to be
seeing it more as an old trend that has been continuing for many years.
Uh...
Okay, maybe I should ask you this. Like, would you say, uh, this,
you know, RL HF which was done on like GPT-3 and we've got GPT-3.5, would you say
that had something similar to do with, you know, now O1, O3 and like this type of
reasoning? Like,
are they both, you know, on the same continuous trend of, okay, RL gets the good things,
like RL plus LLM gets the good things?
Or would you say that is a different thing and this is a different thing?
That I, I would say those are different things, because I don't think RL HF was being
used for the same, like, goals. So once we had the, in- reasoning
thing unlocked, like, once we figured that out, okay, just giving the
model more and more time to think about this.
Especially,
uh,
with, you know, th- these models that are very long context
where they can, you know,
have like hundreds of thousands of tokens in their chain of thought
that they can refer to
before answering a question.
Uh, this gives us something to throw reinforcement
learning at.... in the space of verifiable problems, such as math
and coding.
Got it, got it. Yeah, I think the reason I'm asking is, uh, like, if we
want to predict, you know, how far will RL go in the future, you know, say like five
years, like, what are the data points from the past we can use here to predict that?
So
can I just now say GP2, GPQ2 plus RL is to be ignored?
GPT3 plus RL is to be ignored, and only, you know, like GPT4 plus RL, like
O1, O3 only these are the relevant data points for us to predict?
Yeah, I think after O1, the, the reinforcement learning,
like...
Actually, I, I would maybe even say, like, more recent.
I think O1 was, like, the first, like, uh, proof of concept that this
could be done.
And I think DeepSeq R1, a- and the DeepSeq R0 model,
both, like,
tried to
throw as much RL as they could.
Yeah.
And it's interesting, I would, I would, uh,
I would, uh,
or advise people listening to this to try these models out, like side by
side. Like DeepSeq R1. So this is DeepSeq V3, which is a very good
LLM already. It's, like, pretty big, 600 something parameters, 600 billion something
parameters.
Uh, and by itself it's pretty good.
And they did two things with it. One was, they did reinforcement learning, plus
supervised fine tuning on,
uh, high quality examples of what reasoning chain of thought looks like.
That was one way, and that gave us DeepSeq R1, and that's
a very good model.
On the other side, they also did just
reinforcement learning. (sniffs) And
no supervised fine tuning on, you know, the same dataset of, uh,
uh,
good quality
reasoning chain of thought examples.
But... And even that model's, like, pretty good.
So I think, like, those models really shifted my view on, like, what
the potential for more RL is,
and what it can do
once, you know, the bigger labs that have the tons of compute to throw at these
models
start, you know,
uh, implementing them.
Got it. Um-
There have been some recent papers as well on just, like, using pure, uh,
reinforcement learning. I'll try to see if I can find a link, but yeah.
Got it. Got it. Uh, so I
think I'm basically
the, like, relevant historical data for your prediction is just, like, the last one year
of progress. Like 2024 to 2025, basically.
Yeah, that would be fair. Yes.
Yeah. And I completely agree with you that RL has led to,
like, non-trivial amount of progress between 2024, say 2024, um,
Jan, Feb, like, whenever O1 launched, I guess.
Like, since then, till, you know, today, they're in May 2025.
I agree there has been some progress in this last one year.
I think the... And I, I'm also predicting, you know, this progress will continue up to
some point. Like, I don't think, you know, we are right now at the peak of that.
Uh, so,
but I think, yeah, the
maybe difference here is like, where do you get that confidence of like, okay, this one
year can be, you know, extrapolated, say five years into the future?
Like,
I think that's the thing, like I have a lot more uncertainty around this, and you again
seem more confident on, like this one here can really be extrapolated, like, quite far.
Like, where is that confidence?
(keyboard clacking)
Sorry, was that a question?
Uh, yeah.
Could, could you just repeat that?
Oh, uh,
like, given the last one year of data showing that RL has gotten some good progress,
which I completely agree with, uh,
how do you have confidence that we can really extrapolate this trend, you know, like
maybe two years, three years, five years into the future?
Like, I have a lot more uncertainty on how far we can extrapolate this trend.
You seem a lot more confident that we can extrapolate this trends and trying to
understand, you know, where
you get that certainty if we can necessarily extrapolate this trend.
Um, yeah. So
there've been a bunch of smaller models, I think, yeah, from
DeepSeek, but also the, the Alibaba team that is behind the Qwen series of
models. They've been working on, like, provers,
proving, prover models basically. They're just, like, trying to s- s- like
see how much RL they can throw at,
like, basic language models, such that they can do good math,
and good code- coding.
And
it would- would... It's, like, pretty surprising how much, how much they can get out of,
like, very small models.
And I- I assume these, the pre-training on these smaller models was done like,
uh, in a,
in a very,
in a very sound manner. U- They're pro- probably using, like, a lot of, uh,
lot of good data, and they were probably like,
uh...
Yeah, doing the right things in terms of how much data you should feed given a,
given a certain size model.... such that it is able to
improve so much once RL is applied to it.
But especially on verifiable domains, not especially, but I guess only on verifiable
domains like math and coding,
we have found,
like, enormous gains from smaller models
and not just, like,
uh,
and not just bigger models. So it's not like it's unlocking something that was already
inside,
uh,
big LLMs, but
it leverages something that's inside all LLMs to learn
more than it was initially taught in pre-training.
And I believe latter to be more indicative of where we're at as opposed to the
former.
Got it. Uh,
then would you also predict that, okay, let's say
GPT-3 plus a huge amount of RL will get you, let's say, GPT-4 level
performance?
Uh,
on domains such as,
uh, coding and math? I think so.
I think the- you probably have to do, like, some more, uh, post-training work
beyond just RL,
uh, because I don't think
there was- there, there might not have been enough math and coding stuff inside the
original dat- pre-training dataset.
But
yeah, I- I think on, like, these specific verifiable tasks that we apply RL
to,
I think, yeah, we could get,
uh, close to maybe GPT-4 levels of performance,
e- even if it doesn't match
GPT-4 in those specific tasks.
Oh,
I think that seems like interesting crux, because
I have, like,
low probability, like less than 50% for sure that that is possible.
Like, GPT-3 plus maybe more- like, okay, more high-quality data plus huge
amounts of RL gets you GPT-4. Like, I think that's less than 50%
chance possible. I think you are like, "This is more than 50% chance possible."
And, yeah, and this seems like an actual testable hypothesis, like someone can go and
work on this and prove one of us right in the next few years.
I don't know if someone will, but someone can if they wanted to. Yeah.
Uh,
yeah, I think if I bought that, like this specific thing of like, okay, GPT-3 plus huge
amount of RL gets us to GPT-4, then I think
I would also then buy the next step, which you were saying of, okay, you know, GPT-4 plus
huge amounts of RL gets again, like, really far.
Oh, can-
Yeah.
... can I take this even further back then?
Like can I say, okay, GPT-2 plus huge amounts of RL gets, like, a really good model,
maybe GPT-3 or better than GPT-3 level?
Um, I, I would not go ahead and make those kind of statements.
I think that would be- still be a fun experiment to try.
But, uh, the,
the issue I have there is,
uh...
Uh, one sec, yeah.
The issue I have there is, uh,
I don't think
the, the general industry standard of what good
pre-training, like, procedures look like, like the kind of data you need to be
providing it, the kind of data cleaning you have to do,
uh, et cetera, et cetera,
uh,
for these models to,
you know, be good at anything, for example.
I mean, back then they were targeting just language 'cause initially it sprung out of,
like, the translation,
uh,
task where
language to language translation,
but
I think it might be possible. I think it...
Yeah, I think it might be possible.
Yeah. Okay. Uh, then maybe I'll propose it this way.
So I think GPT-2 was like 1.5 billion parameters.
So let's say today in 2025, somebody made, you know, really high-quality
datasets. They trained, you know, 1.5 billion parameter model using,
you know, as much compute as back then.
But we have, like, the knowledge now so they can
use better data, they can use maybe slightly better algorithms.
Like, not, not massively better, but, like, the basic ideas are the same, but they can,
you know, slightly improve things. Like,
can you do that? Like, a model trained today, like, a small model with huge amounts of
RL, will that improve this?
Like, will that...
I think so. Yeah, I think so. I think there, there is like a,
there is probably, like, there exists some post-training recipe that can be applied to
very small mo- I don't know about... So I think when we talk small model, we should
define what we're looking at. So anything below, let's say, 50
billion, would, would you consider that small?
Or are you thinking more like single digit billions or some millions?
Yeah, nothing... GPT-3 was 175 billion.
GPT-4 is, people are guessing, like, what, around few trillion, right?
I
don't know. I didn't get an exact number of how much this thing...
One point something... Yeah, one point something
billion.
Yeah.
It's, it's, it's three... I think it's, uh, eight GPT-3.5s,
uh, in a mixture of expert systems.
Okay, so more than a trillion, but-
So 175 times 8.
Yeah, 1.4 trillion.
Okay. And GPT-2 was, I think, 1.5 billion, right?
Like, kind of, correct?
Mm-hmm.
Just be-
So to give you an example, so to give you an example, um,
there's a leaderboard
for code editing,
and on that leaderboard, Qwen 2.5 coder, which is a
32 billion parameter
model-... outperforms
GPT-4o.
It outperforms o1-mini.
It outperforms GPT Gemini 2, 2.0 Flash
and Claude 3o's.
Okay.
And a bunch of other ones.
Sorry, can we just-
But-
Sorry to interrupt. Can I just Google this? Like, what's the name of the model you said?
Yeah, uh,
Qwen 2.5 4o.
Okay.
Oh, sorry. We can get that. Yeah.
Yeah. But that's not the only model in this class, where there are other.
Like DeepSeek has also released, like, some
small models that
can do a very good job in terms of what they're being measured at right
now. So
I, I would still say maybe,
uh, the bigger model... I mean, it's, this is not the best model, the one I'm talking
about. Like, uh, the
o1, you know, o3-mini,
3.5 Sonnet, 3.7 Sonnet. These are still, like, fairly better than the smaller
model.
But
to imagine a 32 billion parameter model match the
performance on a very useful task, which is code editing.
So taking, like, small snippets of code and editing according to a given
specification.
Comparing that to, like, models that are trillions of parameters,
if not more.
Um,
yeah. That's... I think that's like a
ex- extremely...
Yeah. Just like an extremely,
uh,
extremely crazy result when, when you think about it.
Like in, especially comparing to, like, the initial idea what,
you know, what makes models better. Like two, three or like two years ago, everyone was
like, "Oh, we need more pre-training and we can't scale pre-training because it's too
expensive."
Now we are at, like, a 32 billion parameter model can be used, very
useful
for,
you know, your code editor or whatever. The, uh, in place of copilot.
And you don't need a, a huge model
with a lot of coding-based pre-training data inside it.
Yeah. Uh, have you used any of these models?
Yes.
(phone ringing)
Uh, I... Can you hear me?
Yeah, yeah.
Yeah. I think an actual thing that will help here is just, I'm going to actually try some
of these models which you're saying, which are smaller than 50 billion.
But again, you're claiming that, you know, better than GPT-3, possibly even reaching
GPT-4 levels. And
yeah, I think one is just I have very low trust in benchmarks. That's...
Like I've always... I've never had very high trust in benchmarks.
Whatever trust I had has just gone completely down to zero in, like, past two years.
Like I'm not exaggerating when I say zero.
Uh,
what I would trust is, you know, I actually tried the model myself on a few tasks, you
know, imagined in my head and yeah.
So like, I think if I actually tried some of these models
on, like, tasks that would otherwise be only on GPT-4
and they actually work well, like that would actually persuade me something.
Um,
uh, I don't have a lot of experience trying, like, the smaller coding models.
Uh, one
point of evidence that I have is,
uh, for things like describing an image,
like very small multimodal models. So I'm talking about the Qwen
2.5 7 billion
vision-language model,
is
remarkably good at describing
the contents of an image.
Upload an image,
ask, "Describe this image,"
does a pretty good job.
Okay.
Now, that's a 7 billion parameter model.
And this is something I've tested on a, a decent range of examples.
Yeah. If you've actually already done this, it might help if maybe when you just
share the screenshots or like a report or something and say, you know, "Here are the
examples I tried. These are not in any benchmark or any paper." You know, "I sectionally
made up these examples and it worked well. If you just show me that document that you ...
Like, I am at this point just
debating, like maybe all these examples are fake and, you know, the models were just sent
a benchmark and... Like I am from that kind of opinion right now.
I think that's, yeah, valid. I just don't know if I want to spend the time doing that.
Uh,
I, I mean I sometimes do that for work because I get paid to.
But
otherwise I'm not too interested in, like,
find... Like presenting solid findings for, "This model is
actually good. I have to convince everyone." But I, I do still, like, recommend people
check it out for themselves.
Um,
like the small models are no joke. Although, I mean,
they're not, they're not better at anything compared to the bigger models.
But how close they get
on
a, a decent range of tasks
is, uh, is very promising. Is, is very promising for
what can be done
by just using small models.
Yeah. Yeah.
I'm wondering where to progress the discussion or it's like we really identified the
crux. We are both agreeing, you know, here is the data point that will totally resolve
this argument to self-harm. But like, if you can convince me that, you know, models
less than 50 billion are getting really close to, you know, GPT-4.
Like doesn't have to match, but getting very close to, you know, GPT-4 which is more than
a trillion-parameter model.... that could change my mind a lot and you know
how
far it can be more than scaling the goal versus, you know, other techniques.
Yeah. I mean, I think this is, like, still exploring
explored territory, where I think, uh, the, the GPT-4.5 model that
was released recently to, uh,
a lot of underwhelming responses. Like a lot of people didn't really like the model.
Like, they didn't think it was that intelligent.
But it showed sparks of something,
uh,
in the demos. I didn't access the model myself.
In the demos, it showed sparks of something that
you couldn't necessarily find in other models.
Like,
a more intuitive and difficult to quantify grasp on language,
grasp on, uh, comedy,
et cetera, et cetera.
So
there might be, like, different things happening here
but for the purposes of this discussion, we are talking about, like,
like, tasks that
small models can do pretty well.
Not as well as the bigger models, but surprisingly close.
Uh...
Okay. Sorry to bring this up again, but do you have any ideas for any of this?
Like are any of these hosted? Does any company host these models?
Oh, yeah, yeah. There's so many companies hosting these models. So one, I would...
I think
people should, uh, create a free account on Hyperbolic Labs-
Okay.
... uh, which is one of these, uh, AI provider,
uh,
companies.
You just talk to all these different models. They give you, like, what?
Uh, $1 worth of credits for free.
You can talk to a bunch of these LLaMa models, uh, QWQ, which is Qwen,
DeepSeq,
uh,
et cetera, et cetera.
And
then these are all, like, small models.
Um,
and if... Like, if they're small enough, you'll all, uh...
And if they're popular enough,
you might not find them on these, like, sites like, uh,
Hyperbolic Labs maybe, but, uh,
the general,
the general way to find an inference provider is to first check Hugging Face.
They have a section on their model page
that lists the inference providers that are actively providing
this model.
Uh, or if it's small enough, uh,
yeah, you might even be able to run it on
some, uh, some free collab instances of Google Colab or there
are some other,
uh,
you know, data science related, uh, platforms that offer some free
compute
if they're very small. But yeah, typically you will find, like, inference providers for
these small models.
Okay, got it. I think then I, I want right now, like, after our call, I'll try
some of these models for sure.
Yeah, I guess... Uh, I think I also am independently came to the idea of, like, it would
be nice if more small models are hosted.
Like, I think I wanted what, I think the first LLaMa model to be hosted and could not
find that. Oh, I think GPT-3 has been taken down so we can't really try GPT-3
anymore. Yeah, I think in general, like, bunch of older, older models are just kind of
out there in the ether and nobody's actually hosting them anymore.
And
yeah, maybe even, you know, somebody watches a video and they're like, "This is the thing
they wanted to do," like, go and just host old models.
I think that could change people's minds a bit.
Yeah. Yeah. I mean, I think, uh,
I think you can be very easily desensitized to the, the marvel, the technological
marvel these things are if you only use the state-of-the-art models.
And this is something I... you know, I've gotten very used to where the moment I
realized, oh, GP- Gemini 2.5 Pro is better than 3.7 Sonnet, I
just basically gave up on using Sonnet at all.
I was like, "This is the best, I'll just keep using this." But once in a while it's nice
to take a break and, like, try to use a very
smaller model and see how that influences your workflow.
And, like,
try to think of these models as not something, uh...
Not an adversarial or superior position.
Not in the place of, like, a boss trying to give an employer task, the
same way we are using these assistants, but trying to work with these models and trying
to find their strengths,
um...
Yeah, can be,
can be pretty fun. I don't know if it'll be useful necessarily for whatever people are
doing, but to get, like,
you know, a, a grasp on
what smaller models can do pretty well and what they're horribly bad at,
uh, is... you know, it's a... it's very, uh, useful activity I would say.
Okay. Got it. Got it.
Yeah, I'm wondering then... Okay, here is one clear point you raised.
Apart from this, I'm wondering what else would we be discussing, like where else might be
this discussed?
Um,
so I would... I would say two different topics are, uh...
Well, one is,
uh...
I would say robotics is, like, a very interesting thing.
Uh,
it's not directly related to how we get to, you know,
ASI because I don't think robotics will get us to ASI.
Uh, but I think robotics is, like, a very interesting domain,
uh,
that,
you know, gets slept on because it's not
as readily accessible. Like, the only people who are interested in, like,
funding
more robotics
right now are still people who own factories and operate factories or
warehouses or stuff like that.Uh,
we haven't seen a lot of cases where robots are being
used,
like by offices or, uh, you know, end consumers like you and me
in our homes or something.
And there have been
vast reduced... re- like, reductions in cost
in terms of, like the...
like the tech itself has gotten very, very,
very, very, very fast.
Which is, uh...
yeah, which is a sign that a lot of good products can come out of,
um, a lot of this research. One example would be, like if you think of the
robot dog thing that Boston Dynamics kind of made popular.
Uh, it was this four-legged robot that has a camera attached to it and can walk
around independently without someone else controlling it.
Like, recently a bunch of Chinese firms, one in particular that got a lot of news
headlines, was Unitree. It released a version of that dog that was as cheap as
2,000 dollars.
For con- like, for reference, the initial Boston robotics dog that was available
for the public to buy-
Can I, can I interrupt you? Sorry.
(laughs)
Yeah.
No, I...
No, I'm thinking maybe we should not go down this discussion track of robotics,
because I think both of us kind of agree on
like, uh, robotics is more expensive to do, therefore people will do it, you know,
two years later or like some years later.
And, you know, the state of the art progress we are going to see in next five years is
going to be more impressive in just like more reasoning, more thinking, rather than
robotics. Like, I think we both kind of seem to agree on this.
Like, if we want to f- like forecast, you know, are we getting super intelligent AI in
five years,
I think robotics would not be the thing we are both should be discussing, I think.
Fair. Cool.
But that's... Sorry-
Um-
I know you were interested in the topic and you wanted to talk about it.
I just feel like that's probably not the tracks between us right now.
Yeah.
Yeah, yeah. I thi- I think that's fair. I...
The, the reason I, I, I go into like robotics is when I think of, uh...
when I think of adoption and automation as like
the next step in terms of the, the,
the social place this technology, you know, takes us to,
I think of like two levels of automation where if there's a firm that runs a...
there's a construction company, you know, a GPT-7 can maybe replace
the top level executives, but if the bottom level work is
still being done by human laborers,
then that stuff is not getting automated.
But the higher level knowledge work, strategic planning, et cetera, et cetera, can
maybe be largely automated. But I think the bottom level automation, which is
replacing
the human workers who are like digging holes, laying down cables, et cetera, et cetera.
I think there's also a lot of, uh,
overhang in terms of
applications that... um, sorry, products and robots that can be built
but haven't, haven't been built yet.
Uh, so I think it... the, the robotics thing, I think fa- fits into w- what my picture
of automation looks like
using AI. Uh...
D'you want to go at it? You want to go at it?
Yeah, I think I also agree with you, there is overhang in
robotics.
And also I think I would also be surprised, like very surprised if we get, you know,
ASI on like kind of theoretical tasks.
Like let's say we had an AI use a better phys- like a theoretical physicist than
Einstein, but we don't have, you know, robotics that can build, you know, effectively.
Like,
that would be very surprising to me.
Therefore, I think we both should try to focus on the, you know, can we get a really
good, say, theoretical physicist or mathematician. Like if you can get that, I feel...
yeah, I think if we can get that, I think that already implies like at least in both of
our minds, like a high probability that a lot of physical world stuff will also get
there.
Yeah.
I'd agree.
Okay.
Okay. So yeah, coming back to that again,
I think yeah, we are both bullish on RL. I think...
yeah, I am more cert- uncertainty just because I have seen less data points.
So you know how well did RL plus LLM do in the past?
I kind of see it as just we have just had two, three data points and like that's it over
one year
and even these two, three data points, you know, over the next few years, we might learn
some new things and then reinterpret our current data in a different light.
Like, maybe something else was going on here and there and was kind of masking what was
actually going on here. Like, I think even that is possible because it's like so new data
points we have.
Um...
Okay, maybe I can ask this. Uh, do you see RL again as kind of like a black box where,
you know, you throw something and you get something out or do you have an underlying
model of, you know, how this item is actually getting better results?
Um, I don't... I don't know.
Like, you don't-
Like, I just don't know. Uh...
If
you don't know, then we might be off, like... at least...
I am not saying we can't be a underlying, like yes, no model of how this is getting it
done. I'm just saying I don't have that knowledge.
Yeah, yeah. I don't have enough priors on this to make a comment.
Like, it might be understandable, like what RL is actually doing, it might not be.
Um...
yeah, I just don't know.
All right, then. Okay. So we both agree
RL is kind of like, again, a black box kind of thing.
You throw more compute at it, you throw a few more tricks and it will get more
performance.... but you seem a lot more confident in this side of it
... and, yeah, I think that this,
it's like, kind of unknown-unknown, but in spite of it being unknown-unknown, you seem to
have a high certainty and I think maybe that's where the difference is coming from.
Hmm. Yeah. I mean, I've seen, like, some interesting hypotheses with respect to RL, but
I just don't, uh, I just don't know enough to comment on that.
Uh,
is there any other trend in AI that is at all similar to this
RL plus LLM trend we've seen in the last month?
Like, is there any other data, anything else we can connect to this and say, "Okay, maybe
talking about that would also change what our opinions of this."
Uh, yeah, like, multimodality is, uh, pretty cool.
Uh,
I don't think I've thought enough about, like, what the implications of it might be.
If, uh, we can have,
you know, the- the kind of thing that NVIDIA is trying to build, so
it's, like, basically world simulation models.
Given a frame, uh,
what would the next frame plausibly look like
and how it aids in, uh,
like, simulation for
models that end up in self-driving cars and,
you know, some other autonomous,
uh, pieces of tech.
Yeah.
Because y- you have, uh, models that,
using maybe only images, or I don't think they're doing, like, 3D models yet.
That would be fucking insane, but I think it's mostly through images, predict what
the next thing would be,
the next frame of an image would be based on those type of frames
and trying to, like, reverse engineer physics using that.
Uh,
I think that would be, like,
interesting in terms of, uh,
seeing what a model can learn using,
you know, image data primarily
with some text scaffolding instead of learning through text and then
using that knowledge with images.
Mm-hmm.
And I would not be surprised if this succeeds, but even then, that
doesn't really inform my views on RL plus LLM.
Like, uh, okay, so I think, yeah, uh, predicting, like, short time horizons
seems doable for me at least, but predicting long time horizon seems really hard.
So short time horizon is, you know, like cars in this position after, you know, 500
milliseconds, exactly how many pixels would the camera move.
Like, I think
that seems doable.
Versus, okay, there is a man in this car, you know, after one minute what's
the probability the man will get out of the car?
Like, that requires you to think about, you know, what is this man thinking and
why will this man want to get out of the car?
You know, is the car, like, in a parking spot and that's why the man wants to get out?
Like,
that is a much more complicated kind of world modeling than just, like, you know, ball
moving with this mutual speed, this much expectation it will probably get here after 1
second.
Yeah.
So, yeah, the, like, just predicting, like, you know, one second, just a few frames
ahead, like,
for a lot of driving, like, you can't do driving just with that, but you can do most of
driving with just that, like, of just predicting a few seconds ahead.
And,
yeah, I would not be surprised if that gets done, like, even, you know, in, let's say
2025 someone announces, you know, driving is 95% solved because we just
cracked this. Like, that would not surprise me and I think that would also not change my
opinions on a lot of the other stuff right now.
Yeah. Yeah. I- I think I- I would expect driving to be solvable,
um, just given, like the rate of progress it has already shown.
Uh, plus, like, while doing it, it doesn't feel that difficult.
Like, it feels like it's an easy-ish thing to do
and, uh, with some except- exceptions, I think hardness in terms of how hard
LLMs find something to do versus how hard humans find the same thing to do,
I think there is like a,
the, the, I think both of those things are tracking a similar hardness measure.
So driving doesn't seem necessarily that hard.
I would expect flying a plane
or, like, uh, not flying a plane, or like f- flying a fighter jet
in combatant scenarios
might be a bit more difficult,
um,
than, you know,
following driving. But I think, yeah, I think the timelines on that kind of stuff I
think depends on, uh, in my opinion, like, data.
Uh, so, so I think, like,
companies like Tesla or Waymo
are in the lead not because they have, you know, already some experimental
setups
but
probably because they already have tons and tons of data to learn on.
And they don't need to do as much simulation work, as much they can just
use the data that they have collected,
through having products out there that people use and buy.
Whereas,
uh, you know, another startup that's trying to solve driving would likely have to
do a lot of simulation-based work and won't have access to as much data
to solve the driving problem.
Okay. Okay. Uh, yeah. Okay, maybe I can
put this question to you, uh-Okay. So the data centers
being built right now or maybe even, say, one, two years from now.
Let's say, you know, 100,000 H100 to maybe a million H100.
Like, right now I think 100,000 H100 data center has been built.
You know, we might get all the way up to, like, one million.
It will probably be the next 3P0 H100.
But roughly in this range, like, when these data centers are used, like, what
fraction of the compute do you think will get used for pre-training versus for
RL?
Because right now it seems to be like, you know, more than 99% pre-training and
less than 1% RL. But do you think that, you know, the ratio will change?
Hmm.
I think the, the language model
RL
paradigm is still, like, new.
So we don't necessarily know
what the...
a mature,
like, RL post-training pipeline looks like, and how much data does it actually need.
So in theory, you're like more data, more results.
But, you know, maybe you find out, well, data, you don't actually need that much
data.
Um,
there are some, like, experimental results behind this, where even without any data
just asking a language model to solve a task,
and over time training it,
uh,
you know,
just
ch- on just doing chain of thought
can lead to
very good results.
But again, we, we're, like, very early into this paradigm.
Uh, I don't know how much a
company should try to
spend on,
uh, you know, RL or whatever. 'Cause the idea is like, that you want to produce the,
you know, best next product. But
there is this issue that has pro- cropped up er- er- when it comes to, like, the
incentives to make bigger models, where,
uh, if you spend too much on a model and you make it accessible through an API, someone
collects enough s- you know, examples
of it, of that model doing well on a task.
They take all those examples and they feed all those examples to a smaller model,
and they either fine-tune it... Sorry. Yeah, yeah.
Fine-tune it on that, or they do RL using that.
And they can, you know, kind of mirror or, you know, get close to that level of
performance, which kills whatever comparative advantage the initial
company who spent all that time pre-training,
uh,
got. So I'm not sure what, uh,
the,
the resource allocation for compute looks like
when it comes to
how should
someone with a given amount of compute allocate
their resources.
Um...
Can I say one thing? No, I think that phenomena is kind of fake, like the one where you
said of, like,
you know, bigger model, you get lot of samples and you train a smaller model.
I think the smaller model... Again, I'm not really sure, but I think the smaller model
just again memorizes those examples and then, you know, gives the benchmark.
It's not actually that good of models.
And...
yeah, again, it seems like, again, actually testable.
Like, we can actually go and try this, try a model made this way, like, that was made
this way.
But yeah, my impression, at least until now, was when people did this, it was again the
smaller model just
got those examples and then, like, pulled the benchmark rather than it, it actually
got the 0000.
Yeah. I mean, fair. Uh, I am not,
I'm not too sure. I think, uh,
I think it might, it might be like benchmark gaming and memorization, but I'm
not
sure.
Yeah. Would have to look into it. But again, yeah, seems testable.
Yeah. Also seems related to, you know, our first point we had on those less than 50
billion models, where I feel there's a bunch of benchmark gaming going on there and I
think you are saying it's not happening as much.
So, like, is that the actual difference in our use where I see
more things as giving benchmarks 0000?
So I think... So for, like, there are, like, legitimately good smaller
models out there. But
what my
thing with those is, is like,
uh,
they are a result of... They would never ever lead to any state-of-the-art
improvements. What they would do is, uh, cut your costs,
get you a lot, like, maybe 70 to 90% of the way there
on a c- on certain tasks that are,
you know, that you can break problems down into.
And for those certain tasks, you can have the smaller model
do a better job.
Um,
and I think...
Yeah, I think I, I think I believe that. Yeah. It's not, uh...
I don't think I'm
saying that because on some benchmarks the, the evidence looks favorable.
But I think you can have, like, small models
which is, uh, you can have 10X smaller models than the state-of-the-art model
and get a lot of performance out of it.
Again, this is testable and I think I'm more pessimistic on you, I'm saying, you know,
the actual task, it won't actually be able to, just do the benchmark.
And even if it's a simple task, we give the task, it won't do it, like...
A way to test this, yeah, I guess would be either, again, us trying the models
or if there's any company that has a lot of revenue despite using, let's say,
models smaller than 100 million parameters.
I think
that could also prove things here.
Yeah.
Um,
yeah.
I think I have a hard out in, like, 10-ish minutes.
Okay.
Uh,
so,
yeah.
Yeah. Okay. So con- Maybe I'll come back to your thing you said about, like,
we don't know how much compute can be poured into this new
reasoning paradigm, like, RL is a good reasoning paradigm.
And
we might get pretty far without pouring a lot of compute in, we're not very sure, which
is fair.
Uh, would you then say it's possible we get, you know, five years later we are getting
super intelligent AI?
And even then, you know, less than 1% of that data center was used for RL, but that
small 1% was used because you were so influential that that kind of model was made.
Hmm. Yeah. Fair.
Like, are you saying, like, that can happen?
(taps finger)
So s- sorry, sorry. Could, could you just repeat that?
Uh,
would you consider the following an at all likely scenario?
You know, five years from now, 2030, we have super intelligent AI.
It was trained on whatever is the largest data center around 20, 2029, 20,
2030.
Uh, less than 1% of that compute was used on RL.
However, a lot of those performance improvements are because of that RL which was formed.
That, that feels wrong.
Okay. So where does it feel wrong?
Yeah.
I, I, I don't think that would be...
That w- That, that doesn't feel
plausible.
But it doesn't feel impossible.
Would you say it's then a more likely scenario, like, let's say they're getting super
intelligent AI by 2030, you know, how much compute was spent on what, in that
one?
Uh... Okay.
How much compute do you need for
this superintelligence?
That's a good one.
I'd, I'd say...
Sorry, what?
Fair question, yeah.
Yeah, yeah. I mean, it's a fair question. Uh...
How are we measuring compute in terms of, uh...
FLOPS.
Training data plots. Yeah.
Okay. What would the training FLOPS
for, uh,
GPT-4?
It's 8.4
to 10 to the power, 10 to the power of 24, 25-ish.
I
mean, general rule of thumb is like, uh,
let's say FLOPS per dollar, uh, doubles every 2.5 years.
So five years from now means
FLOPS per dollar will be 4X, but we'll also put a lot more dollars in.
So,
I mean, right now we're putting between, I think, more than $10 million.
Let's say they put in, I don't know, $100 million then.
So that's
maybe, like, 100X more dollars and 4X more, like, FLOPS per
dollar. So that's like maybe
500X to 1,000X more FLOPS we'll have in 2030 compared to right
now.
A- again, maybe someone should check the exact numbers, but roughly I think it's, uh...
Yeah. Yeah. I think it's like, uh...
Basically saying 1,000X, uh,
more,
like, end-point operations speed in 2030 compared to 2025.
Roughly.
Yeah.
I don't feel like I have a, have a grasp on this.
I,
I wouldn't, I wouldn't mind saying something like,
um,
1,000 to,
you know, 50,000X increase would be
necessary in terms of training FLOP, pre-training
compute.
Um,
but I think I very well could be wrong.
I, I, I wouldn't necessarily be surprised if,
in terms of pre-training, we don't need to scale, uh,
that much more
than we already have, than 4.5. I,
I think 4.5 likely is the,
the model with the most pre-training compute spent on.
Yeah.
Maybe like 5.5 to 6-ish.
Like GPT 5.5 to 6. Uh,
yeah, I don't feel like I have a grasp on this, so I don't
know.
But...
it doesn't necessarily feel that GPT-4.5 is that far,
if we do the right things to it. If you use it in the right way.
Like, it doesn't feel like that much of an improvement over the- the previous models,
but it also doesn't feel like it's a step back or it's the same.
I think there is, like, marginal improvement,
and it just needs to be worked on or utilized in the right way.
Yeah, it's, uh... I think I agree with you on there is some marginal improvement.
Maybe the difference in our world is kind of like...
Or one difference is,
ah, how do I say?
Like the world has kind of a bunch of different tasks and you can be improving on some
of the, like, tasks. Like maybe some tasks are, like, medium difficulty and some tasks
are really hard, and the model can be improving on some of the medium difficulty tasks
and therefore loss is going down, but it's not improving at all on the hard difficulty
tasks. Like, I think that's the thing that's possible.
I think that's the thing that has already happened with LMs.
So like GPT-5 could be better than GPT-4.5.
Like, I think that seems likely.
And it could be because on some medium difficulty task it has gone and done even better.
But that could still mean at the same time there's some really hard tasks which have just
not been cracked at all.
I think my worldview looks a bit more
Yeah,
I think that's fair. But I- I'm also like, there's one aspect of, like, uh,
just throwing enough
normal-ish brains at a problem,
um,
can lead to improvements in the hard parts of the problem without,
uh,
without necessarily there being, like, one super genius who- who caused that improvement.
Um, but I don't think that generally applies.
I think, s- in, for some hard problems you need super geniuses.
Um,
but I think there are, like, a set of hard problems that
you can
kind of iteratively improve on and
get better at.
And in theory you can use more compute with, uh...
Not- not on terms of, like,
s- um, pre-training or even (inaudible) scaling, but have more models
dedicated to solving the same problem.
Like, have a research manager break it down into, like, multiple different,
you know, experiments. Have the models do the experiments.
Have the research manager, which is also a model,
like, convene all those results together so it might not...
And, you know, plan the next set of experiments.
It's like, just AI R&D firms
can, I think, improve some of the, like, hard...
Improve performance on some of the hard problems
without needing super geniuses,
super genius models that can just one shot or few shot the hard problem by
themselves.
Uh, yeah, I think I do see some of the hard problems as
more zero to one. Like, either you got them or you kind of don't.
Like, I see them as less decomposable than I think you are seeing them as decomposable.
So to give a concrete example, you know, you and I having this podcast conversation, I
think
there are parts of the skill of... Like, as of today at least, an AI will not be able to
have as interesting conversation as, you know, actually either of us talking to the other
person. I think so.
Uh, also, I think that's not a task you can easily just break down into
sub-tasks and glue a bunch of models together and get it.
It's like the model either who gets the entire thing or kind of doesn't.
I just see it more that way.
Hmm.
And that's a fair crux, and I think I agree
with you. I think that's, like, an important thing to figure out
if we
want to see ASI.
Um...
I mean-
I think I have to drop out, uh,
now-ish.
Oh, sorry. Uh,
and I don't know, maybe I shouldn't be giving people ideas, but I mean, the thing that
kind of makes sense to me is... What should I say?
Where you do, like, specialized things inside the model.
So inside the model people do things, right?
Like, okay, here we'll put a CNN layer, here we'll put, you know, Mixture Experts layer,
here we'll put some other thing at the back and we will, like, end-to-end train the
entire thing.
So like you're doing gradient descent across all of it. I think that makes sense to me.
The thing which does not make sense to me is, okay, here's one AI model we'll train
separately and then we have this another model we train separately, and then you have the
first model will go and talk to the second one, the second one will go and talk to the
first one, I think.
This talking to each other I'm very pessimistic a- about.
Uh, you know, gradient descent, everything all at once, I think that can help.
Hmm.
Yeah.
That seems reasonable.
Um, I'm sorry but I don't have-
No, no. I understand. It was good conversation. Yeah.
Speaker/User: Yeah.
Yeah, yeah.
I think we could have something like this,
another one or something like this in the future.
I think there are, like, some things that
would also be fun to discuss.
Yeah, definitely. We can talk about that. Yeah. It was nice having you.
Yeah.
Uh, another episode of the Sam and Chirag show wrapped up.
Yeah.
Okay. Bye.
See you. All right. Bye.