Ah, hello everyone. My name is Samuel Shadrack.
With me I have Shivam Gupta today.
Hello.
Yeah.
All right. Topic we're going to discuss today is AI infrastructure.
Uh, Shivam, do you have more thoughts on what you'd like to talk about today?
I think, yeah, mostly about AI infra, so we'll be starting like a very,
uh, with a very intro. So like, okay, so like what all is going on like in the infra
space?
So like, open, like ever since chart GPT blew up, right?
Uh, so like I see a lot of challenges in infra, like, like Sam Altman even
tweeting, um, he, "We are facing this issue, that issue."
And like all the infra companies are there.
So, like what kind of challenges are people facing in infra?
Oh.
Yeah. I think that's a fair question.
Uh,
I guess it's only a matter of time. Like most of the challenges at least been, say, one
year. Seems like a matter of, you know, will this take more time or less time?
It's not a challenge in the sense of like this can't be done.
So even if, say-
Yeah.
... people
have in their head, "Okay, we will build, you know, one gigawatt data center." Then that
could be 10 gigawatt. It could maybe even go up to 100 gigawatt.
Like even all the way going up to say 10 or 100, like it just seems a matter
of how much time it takes to build it, rather than whether you can build it or not.
And, okay...
Yeah, so then I guess question becomes if you're building a data center, you know, which
parts take more time to build? Does producing chips
take more time? Does, you know,
like supplying part to the facility take more time?
Is it important to build, you know, one data center versus, you know, m- many different
data centers and you can distribute ta- training across them?
That is, I guess one more important question.
Yeah, but, uh,
think most of these things can be figured out. Yeah, that's just my opinion.
What do you think?
Yeah, I think it's pretty much like the early days of the Web2 thing.
So, I mean, yeah, like in some days, uh, in some, I don't know, months or years this will
be figured out. But like, uh, like what major work is, uh, being, uh, done
in the, like the infra side of, uh, this?
Uh, like are people looking more into how we can do better
GPU optimization by writing better code or more optimized code?
Or people are,
uh, working more on, let's say, the hardware side and how we can tune this for this
s- specific task? Like the hardware is, uh, specifically being designed for transformers
and, um, like where is the money basically flowing in, uh, to solve this
thing?
Mm-hmm. Fair enough.
It's a thing of, uh-
Yeah.
... people are working on hardware, kind of like separate organizations.
People working on like software and research are like separate.
On like software-
Yeah.
... and research side,
I think people are not focused that much on optimizing the software. Like...
Okay.
Uh,
like if I have to guess, you know, a lot of open AI developers are probably writing code
into something similar to PyTorch, like very high level abstraction.
They're not going very low level and really trying to optimize.
I guess DeepSee could be an example of someone where they had limited hardware and
they're only trying to optimize. But
I think mostly AI developers have realized that that is not what they should be
doing. They should be, at least at the top companies, AI developers are like, "Even
if we are not optimizing it a lot, if we are the first, you know, get the next
breakthrough or the latest, like the highest performance results," you know, that is
enough for that company to raise more funding.
Mm-hmm.
And they're focusing more on how do we get the AI model performance up rather than, you
know, optimizing
the software a lot.
On the, uh-
That's like an interesting observation, like, because none of the AI companies are
profitable, right?
So
they are mostly working on things which can get them to more funding.
And seems like AGI is on the line. So,
um,
hardware optimization or GPU optimization might not be a priority for them.
Yeah.
Yeah. I- I think so. Yeah. What do you think?
Yeah, I, I, I also think like, uh, like since you presented this to me, I think that
like,
uh, like a part of the team must be working on reducing costs.
Like let's say G- DeepSee came out. So the, like some people, like I
just saw the, like there was a panic in Meta LLaMA
team. "Okay, how are they able to do this?" And the top management is asking people,
"Hey, how, uh, were they able to reverse engineer this?"
So I think, um,
because, uh, like it's not their priority. Their main priority is make the models better.
So I think if, let's say we have like a 100X, uh, capacity, so a
majority then would go to, uh, making models better and something which might,
uh, give better multimodal capab- capacity,
um, rather than, um, yeah, like, uh, GPU optimization or
something. So like mostly
the people who would be doing this optimizations would be like smaller companies like
let's say, Onslaughts, um,
which are pushing towards like
more,
uh, like running models on less VRAM.
So I think the trend is, uh, like this.
Yeah, that, yeah, that makes sense. And
even on the hardware side, I think on the hardware side, like people want to optimize,
but they're not completely certain like up to what extent they should optimize.
Uh, I think in both hardware and software, the basic......
reason why beyond the point you can't optimize is no one is actually sure, you know,
which research breakthrough will come later and what optimizations will be required for
that. So if they just optimize, you know, the model today and how to get today's model
running really fast with hardware or software,
like that could be, you know, a po- like there's opportunity cost in sort of doing that.
You know, three years from now a different breakthrough might come and that might
require, you know, different hardware, different software, right?
Therefore people want to do more general purpose kind of stuff.
So even in
hardware for example, uh,
like GPUs are basically meant for doing like matrix multiplications or at least
at level of matrix multiplications they will do it really fast,
but they're not very opinionated on, you know,
which matrix multiplications should you do, uh, should you be using more compute?
Should you be using more, uh,
like memory? I think that is a trade-off even in different GPU sizes.
In small- like in a smaller GPU,
you have less memory and,
uh, less memory but still a lot of compute.
In a big GPU you might have more memory, and like the ratio of memory to compute is
different in different GPUs.
And
then for example, people recently realized that memory bandwidth is a problem, that, uh,
GPUs, like multiple GPUs when they communicate with each other, like the memory bandwidth
kind of becomes a bottleneck.
And now I think anybody else like trying to work on that they've recently brought out
this NGP200 cluster
which
is slightly different from like H100, H200 which was before.
So they are optimizing that now because from the software and research side, like the
input has come that's okay memory bandwidth is a bottleneck, this, uh, this needs to be
fixed. Therefore, you know, the hardware companies also decided, okay to move in that
direction.
But even then hardware companies don't want to optimize too much, like the extreme end
of, you know, optimizing this would be like basically like an ASIC.
So an ASIC is just-
Mm-hmm.
I mean you can't really even program it like at the hardware level itself-
Mm-hmm.
... like it's predesigned to only do one specific thing.
So like in, I guess cryptocurrency mining people have like ASICs, like you
have an entire circuit that will only compute, you know, image of a hash, it can't do
anything else.
And similarly for LLM you could in theory, theory have a chip that will only
do, you know, GPT-4 inference and it can't do anything else.
And
that kind of chip would be extremely optimized just to run GPT-4 inference and nothing
else works on it.
But yeah, I don't think anyone wants...
even an adversary wants to optimize up to that extent because they know, you know two or
three years from now a better model will come and
they will have to do something different.
Yeah. I heard like companies like Groq and Cerebrum,
um, they have like very optimized chips so I think,
uh, they might not want to but I guess some companies, uh, might be there
which are doing some optimizations with say Groq
or Cerebrum.
And
for them it will be
we can run the model on, uh, like more cheaply
so that is how they will raise funding.
So
yeah, so I think there are basically two paradigms key, how we can track this key, how
companies are raising funding.
Uh, so one company is raising by making better models and another company
might be raising, uh, by running the models more cheaply.
So I guess these two paradigms might be something to think about.
Yeah, I
think that makes sense. Uh,
yeah, I guess the-
I heard also this in a podcast of Dilip Patel and Lex Fridman.
So they were telling about like infra things where it's like Elon Musk opened this
XAI,
uh, and then, um, they basically, uh, opened it near a power
plant, uh, a chemical power plant. And they routed all the power of the pow- power
plant to this, uh, to power the GPUs.
So and also like Microsoft had their sustainability goals, so they basically reversed
that so they can
jump on this bandwagon (laughs) of, uh, GPU. So a lot of like...
a lot of companies are like going towards this.
And
yeah,
Dilip Patel all... has said many good things, like a lot of optimizations
has to be done on the heat side now, uh, heat, uh, when we maintain
all the temperature, we maintain how many hours we will be going to run.
And one of the companies which I am talking about too in India,
uh, so the problem for them is key. How will they do the,
uh, load balancing thing? Okay, when they have more requests so how do they process it?
How do they distribute it? So the servers are still in US of...
the GPUs are still in US
so like how do they do all this network thing and how to do it?
So basically different problems in different facets.
Yeah.
Got it. Yeah, that makes sense. Yeah, I think reducing cost of inference matters for
then
the companies who are building some application on top rather than doing, you know,
research for the next model.
So
yeah, I guess the, the... it's different for like training and for like inference.
So training,
the people who are really training models are more focused on like research and building
the next model. The people who want to reduce cost of inference are more like, you know,
here's the current model, what application can we build using it?
But yes, that's the main difference.
And
yeah, you're right, like, uh, for people who are building applications they are getting
funding when the actual application has a lot of users, it has revenue traction.
The people building frontier model they get funding when their next model shows a better
research result. It may not need to have a lot of users.
Yeah.
Hm.
I still somewhat do-
Something-
... and I, I'm not... Sorry, I'm not very sure about this but people who optimize
this too much might, at least from the point of view of, you know, who will make more
money in this space, they might actually get left behind
if-
Particularly-
... they might get left behind.
Because they optimized the current model and, you know, two years, three years from
later, a different paradigm might take off and then their code is not working for that.
So, for example, very recently like over in '03, like inference scaling has become a
n- new paradigm that's kind of working.
Let's suppose-
Mm-hmm.
... like I don't know if this will actually happen, but let's imagine like inference
scaling is going to become extremely good over the next two, three years,
and a lot of AI model performance is going to come from that.
Then you know the people until now who have been,
uh, just trying to do
like optimize the inference of like, okay, you know, attention works this way and it will
optimize, you know, attention there and like KV cache and all that.
Like, maybe that ends up being less important and now something else ends up being more
important to optimize it.
Like, two, three years from now I think that can happen because-
Yeah.
... like-
I guess, yeah, we can go more in-depth in, like, what will it take for, uh,
a test time comp- like, for inference scaling to get better.
So, like-
Mm-hmm.
... if I have to break it down into basics, right, so we have the model
and for each layer we have the weights loaded.
So, text will come in, pass through each layer and then we will have an output token,
and we'll have like, uh, self-attention and MLPA, uh, in that.
So-
Mm-hmm.
... like, when it comes to, uh, like RL or inference, so how
will some... Like, how are they basically approaching this thing?
Uh,
how is who approaching? Like, people are doing research or...
Yeah, yeah. Like, how are, like, people making, uh, inference better?
So let's say model training is key. Yeah, like, uh, they might have to just run some
experience weight updation, YouDO, Flora, LoRA,
um, like all these different methods, uh, to improve the capability or to see whether
the same capability can be achieved through, like, less,
um, fine-tuning on less number of weights.
So let's say we have a model which is there.
So how are we going to further optimize it for inference?
Uh, sorry, you mean reducing the cost of (...) or do you mean
like-
Yeah.
... RL and, like, doing RL, like research and scaling RL?
Uh, like, I mean key, like, uh, how are companies approaching it for, for inference?
Like,
um, let's say for, uh, like one key, like, uh, just hardware thing.
You load all the weights in, like, in each part and you just run the models through that
each time. So, like, that is one thing, but let's say we are not involved in hardware
for making, uh, compute better.
So, like, how might, like, companies approach it?
Yeah. How might you reduce the cost of inference using software?
Um,
I mean, one thing you can do is, like...
Okay, so I guess initially when you're starting off, like, your code is probably in,
like, Python, probably PyTorch or on a similar fra- framework like TensorFlow.
Mm-hmm.
It's in, like, that type of high-level framework.
Either you can, like, work in that framework and try to optimize within it.
So
I don't know, there's, I think, one JAX backend.
There are some two, three other different backends in which
you can, like, port your PyTorch code into that backend. So it's...
Like,
lot of your conce- high level concepts which PyTorch has already defined such as, "Okay,
this is what a compiler is like, this is how we actually compute in a backprop on its
own,
uh, this is what a training loop is like" and PyTorch already has a lot of constructs
defined. So you already think with all those high-level constructs but you try to
optimize them.
And then more fundamental approach could be actually try rewriting all of this in a
lower-level language like C,
where, like, you scrap the entire Python project, you rewrite entire,
uh, ML libraries in C and I think there are people who are doing that too.
That takes more time.
But yeah, it's again, all just a matter of, like, how much time, how much effort people
are willing to put in, like,
things, like, eventually after a few years, you know.
Uh,
yeah, it seems obvious at least what a model that's there today, you know, five years
from now that it will obviously be running in a very optimized, like, C code or maybe
even at SMB level people have optimized it.
It's just a question of how long that takes, that probably.
Okay.
And, like, what about, uh, like, I guess key, uh...
There's one thing is infra side for, uh, running, like, training and, uh, inference.
But another thing is infra side for, like, the s- same simple data processing thing so...
I heard key
India, like, has some policies, key, like all the data is going, going outside and, like,
data is the vote. So, like, uh, Reliance and Tata are investing in some factories in
Gujarat and Tamil Nadu. So, like, the data stays within India
so... And that will be reinforced for, like, larger organizations, like, there has
been some reports on this. So that is also going to be very interesting, like, uh, new
jobs, opportunities, like, are going to be created in making, like, a
big data center or inference center or training center,
uh,
um, in Gujarat or in Tamil Nadu areas.
So that is also, like, one very interesting development in terms of, like, uh, Indian
context.
So,
yeah.
Got it. Got it.
Yeah. And other, other major things, uh, I guess so...
Uh, other major things could be, like, how...Like, uh, I'm also very
curious here. Let's say Deep Seek does all these things, right?
So how do like the top companies approach it so they are like, "Okay, we have this, this
strong. We could actually be saving this much percent of our cost." And then just, uh,
uh, like a percent of their team they say, "Hey, do this thing which Deep Seek has done
to our models?" Or how are they approaching it?
I mean, I don't have the exact actual situation of what's going on in any of these labs
right now. They work hard to keep that information secret.
I mean, I can guess if I was, let's say, the head of one of the labs I would...
Even then, I would not focus on how to reduce the cost of training.
I would focus more on how to get the even bigger data center and get the next even bigger
training run going. Like,
I
think their first focus is always still on bigger data center, more training,
rather than on how can we do the, the training on GPU, I'm guessing.
But
yeah, I think we'll need... Once we have actual information, then we'll get to know.
What do you think?
Uh, like were I the head, I don't know, maybe I would have, um,
like hired more people to reduce cost maybe.
Um, or maybe just to market. But I'm sure would have been like...
The focus would have been like gaining
capabilities, but a part would have been to reduce in cost also.
Okay. The cost we reduce here will be used like elsewhere. Otherwise, it's just...
Like that is the bottleneck, right?
The major bottleneck is GPUs like for all the major AI apps.
So
yeah.
Like even if, like even if they get more GPUs, then only they would be able to hire more
people, or that is what like the people are screaming at the top.
So
yeah, let's talk about NVIDIA. So do you think NVIDIA has any competitor
or like how
someone might be able to compete with NVIDIA?
Oh,
uh,
first of all, I think that's kind of difficult.
I mean, at this point like NVIDIA is not really
just an isolated company. Like
it's, at this point, basically like Taiwanese government working with NVIDIA so it's like
you're competing with a country at this point.
Uh,
yeah, there is obviously a lot of like process-specific knowledge which is the inside
people at the company. There's like if you just see the entire manufacturing process of
semiconductors is like 10 to 15 different steps and each of those has a bunch of
engineering knowledge.
And a specialist's processes keep improving, you know, every year.
Uh, even if you go see the public knowledge out there, the public knowledge might be like
four years, five years outdated- outdated engineering knowledge which in most fields is
okay. In AI like five years old like knowledge is like not useful.
So even if someone has to start a competitor, they will have to necessarily hire a lot
of people in the existing community like say which is being hired at NVIDIA or
like existing people who are already at NVIDIA.
If those people can be, you know, poached essentially I think that would be the only way
you can create a competitor.
And
that now there's a question of do you hire those people from the US, do you hire them
from China or do you hire them from Taiwan?
Like, at this point, like governments would also be restrictive on like
okay if say a US lab wants to hire, you know, people from China at mass, you know, will
the Chinese government let those people leave the country?
And these types of things can become concentration.
Mm.
Also one interesting thing I read about Taiwan was key.
It's like, uh,
um, the labor is like, uh, like more direct, cheap and they are like very much focused on
this job, so they are basically
very crazily trained. So I... There's this book, Focus: The ASML Way.
So it tells about the,
um,
like how the Taiwan- Taiwanese operate these machines.
So it's like if one person, uh, has to be the best person in the world in
just doing this job. So example, in just
fixing this screw. So he's the best person in the world and let's say
earthquake happens, right? Or anything happens in Taiwan.
So the person will just, uh,
like they will call this person and he will just come and fix that thing.
So like the book said key, like we can make these factory.
Like even say we import the machine to America or some other place, but the main thing is
the work ethic or the work culture or how dedicated these people are in Taiwan
for this thing. And Tai- Taiwanese government also invested heavily so in that book also
it was like key, like all the other...
Like at that time all the other companies who were like, "Okay, we will also design,
we'll manufacture, we'll do all that shit." So Taiwanese companies saw key,
they, they, they placed a bet. So their economy was going down or something in the '90,
'80s, or I- I don't remember the timeline, '80s or '90s or '70s somewhere,
so they placed very heavily key, "We will just be manufacturing chips." "So
you make the design, you send the design and our only expertise will be in
manufacturing. So we will not do anything.
So we will take a small cut but that will be so large, the volume will be so large,
see it will basically compensate for the other things."
So like that was like a crazy bet which actually paid off.
And also like there are two methods like uh, DUF or uh,
uh, EUV and DUV, extreme ultraviolet photolithography
and
another one is I guess DUV something.
So like it was like a very crazy in- thing which, uh, people
thought initially key how, uh, these ultra small chips
can be manufactured, like less than
6 or 10, um,
I don't... nanometers? I don't remember the scale
but it was like key for the US, uh-... did not agree to, uh,
sponsor them. So, like, Intel came,
uh, like, the US government partially came to, uh, run this experiment,
um, to make the current machines which are, uh, making the GPUs for
ASML. So, um,
that was also an extreme crazy bet which paid off key.
When guy just had an intuition, he might actually just be able to reflect and etch
the, uh, laser onto the chip just by basically reflecting.
Uh, so yeah, like all this like...
Uh, I think it's called the,
uh,
most complex machine,
one of the most complex machines, like the ASML machine which is there.
So,
yeah.
Yeah. That is a very interesting anecdote.
Yeah, and I agree when
back from someone in '80s or '90s, it...
there's no way to forecast that there's- even
computers would have, uh... Like, I know people talked about Moore's Law back then but,
yeah, even then it would have been difficult to forecast that even after 2025 there would
be, you know, this much compute available on Earth.
And,
like, internet would have just completely transformed everyone's lives, like, I think
even sitting in '80s it would be a little bit difficult to predict that.
But, yeah, they seem to have managed to do this anyway.
Yeah.
I think, yeah, like many more things.
So there's this company, uh, it starts with M, I, I don't remember the name.
So it is basically for human labeling by
highest people to do human, uh, data labeling thing, so that is also very huge market.
I think a lot of, like, infras not just tech infra,
like, GPU, better GPU optimization or, or ASICS or whatever.
But I think it's mostly from infra from that side key.
Okay, we would always be needing more data and more data,
and data for different tasks. So the design for that, right, key.
How are we going to train people? How are we going to tune RL in that?
How are we going to make an- make a framework where you, you give this
task and someone will tell what, what is needed and then the process of how to,
uh, get that thing?
So, like, I mean in all the areas, let's say
India's area of, let's say, they basically,
um, India government through sat- satellites and all these things, drones,
they legalize like what's, what boundaries belongs to whom.
So it basically led to a lot of, uh, reduction in number of cases of
non-disputes and it saved a lot of court time just by mapping,
uh, this thing, right? So let's say key, now AI is there, we have to reimagine the
whole thing with AI. So how will we go about that?
Which part of the process will be, will we make better?
How will we use AI for that?
So, I mean, that's a, like, very interesting problem, like, um, like how
are we going to use AI to solve traditional problems?
Let's say, in policies we have to check.
So,
um,
there is this policy, uh, which gives microloans to people in the range of
50,000,
um,
two lakhs or five lakhs,
right? Launched by government back in 2015
but... And it's like every three out of ten small person, like the person who is selling
ice cre- ice cream, right? If the person is selling corn,
the person who has opened a, uh, tailoring machine, so he could be taking
loans from this because it was so effective.
But if we analyze the data, right, so it's like people just take the
initial loan, 50%, 50,000 K, 50 K,
but they don't go, uh, like... but not many get converted to, to
take five lakh loans, right?
So they are basically getting streamlined at 50,000 because if they have to take five
lakh alone, they have to repay their own loan, loan back.
So, like, let's say... And it has created a lot of job opportunities for, like, people
in India who are, uh, who don't earn much, right, to start their own business.
It can be anything, a restaurant or this thing.
So how we might approach solving this problem, right?
Let's say
analytics, whether this policy is effective, not effective,
uh, through the use of AI,
right? Key. How will we do better tracking for these people?
Key. Where are the loans going? How are they spending their money?
What are the major bottlenecks?
So I'm very excited about infra side for this thing, like,
where there already is a lot of money, right? Key.
Let's say a
government is investing in industries, right?
India is basically a services,
uh, nation. Key. We mostly govern our GDP through service, basic
service
but we are slowly shifting to industry also, manufacturing and this thing.
So government has invested a good amount of money, um, in industry, in, in,
like, ten industrial cities. So, like, how to better execute that
thing or, like, very simple things like oil seeds.
So we basically import 60% oil
so government has invested, uh, like, many carloads into reducing our oil
capacity to 70%. Because of the typical household income, 6 to
10% is from oil. So, like, how many minor
you tweak in these things? How will we increase production of something which has...
is already in widespread use
and policies have been launched for that?
So how can we make them better with the use of AI?
I guess, like, these are some more things people are not paying much attention to and
these are often overlooked. So, yeah.
You got it, you got it. Uh...
Yeah.
Yeah. I guess if you're asking specifically infrastructure for all of this, uh,
there's this one article by Sarah Constantin who is I think an employee at
Palantir, uh,
called Great Data Integrations Left and she basically talks
about...Uh, I guess two things. First is
many companies when you tell them that you'll use AI to improve their process they might
be kind of skeptical, and the reason is they don't actually have the dataset on which you
will use the AI. Like, the data is just either it's not being collected or it's maybe
being collected manually, you know, pen and paper or there's a PDF or an Excel sheet.
The data is not actually cleaned. Yeah, so the first is it might not be collected or
it- it needs a lot of, like, cleaning and, you know, reprocessing to put in a very
standard format. And only once you have, like, huge dataset in standard format then you
can apply, like, all statistics and then AI.
To that
I'm guessing that type of problem will appear a lot in this.
Like,
people even if they've collected the data-
But I think it's still solvable, right?
Let's say you break the team
and they are just doing scanning. Like, you just streamline that process.
You have a camera and you just snap, snap, snap. So, like, everything is automated.
It will basically snap pages very fastly or flip pages very fastly and the
camera will record. And
so I mean, like, so basically infra for that, right?
Okay, one person there sets up this thing. They have all these documents.
So I'm saying if that is the major bottlenecks or processed through streamline that
also. Like, that would be, like, very interesting.
Let's say, okay, what can be the input?
It can be the local stakeholders who are there or some documents, right,
or a past document study, right? So how can we combine them
to
make better things, policies and policies which we can get
an idea of very fast key okay like whether this is effective or not?
So
I think that is going to be pretty cool.
Like, I think, yeah, like unstructured data is always a thing,
but,
yeah, like, how let's say
how we might approach this problem. Let's say we have to we have data of who gets loan
who does not get loan. Right?
So we have like name, age written, what is the income everything written and
we have like all these files. So how will we use AI to streamline that?
(laughs)
So I'm, I'm just not saying uploading.
So like the whole thing right you have a machine which will in which you just put papers
and it will flip and it will upload on its own.
Um, it will pick 10 random documents and you can just train a
person. Okay, that is how you do it and it will just analyze the whole data and just show
you.
And then you also set up basic evaluation process.
So without capturing the whole pipeline, right?
So I guess that would be very interesting or let's say people coming and filling the form
on their own, right? So
on paper.
So you need to employ people who can, uh, ask their basic queries.
So what if we can also streamline that or what if one person we make him
so much capable using AI he can manage a lot bunch of things?
So
I'm not saying exactly eliminate humans also.
Maybe
make one person 10x more efficient.
But yeah, I think I completely agree with what you're saying.
Like
there will be huge amount of effort that's required into streamlining these types of
processes of getting data into very structured form.
Like even what you said as simple as
like use state of the art LLMs to scan printed documents.
Like someone could probably make a business just out of that.
Like otherwise people are using maybe three year four year old CR models and
current LLMs are better than those even if you
put in your AI model to scan documents
that could be a business for someone to build.
Yeah,
yeah.
I think it's going to be very interesting like how
the whole infra side plays out.
Yeah.
I think, yeah, we are mostly nearing the end.
So any closing remarks and
mostly what
we might be,
what are some of the things which might be overlooked?
Sorry, what are?
What are some of the things which might be overlooked
in this whole?
Oh, overlooked in infrastructure for AI.
Um, I don't know, I just kind of think it's just all of a
sudden in a very short span of time like two years like so many different at least from
the point of view of you know what can make money so many different opportunities have
opened simultaneously that you know even if everybody goes and picks something or the
other you know there'll be different things left out.
Like
I just think that's the thing that's happening.
Like I can just give a very simple example.
I don't know one year ago I wished I realized okay LLMs are obviously good at voice
translation like if I speak to someone there should be an LLM that can translate the
voice and they should hear in their you know earpiece in their language.
And I was thinking this is so obvious you know someone will obviously build this in the
next six months and then I waited six months nobody did it.
I waited another six months even now nobody's done it and
I think this is just one example there are a lot of applications like this where it's
obvious like okay something like this will end up being built it will end up being useful
it's just yeah no one has done it.
Uh-
Maybe they have built it but not from a
point of view where it is very economically viable.
So let's say whatever anyone's case is someone might have built it but it has not come
to such a large scale that people are using it.
So...
Yeah definitely I mean not just like okay someone made a GitHub repo and posted it but
yeah someone actually acquired customers and built an entire business around it.
Yeah
like that requires like then a bunch of like two three people to really make this their
like kind of life's priority that they want to fix this problem
and-
Yeah.
Yeah like this I think there are a lot of problems again now when you said infrastructure
like exactly what you mentioned earlier about you know scanning documents I have not
checked I don't know if there's any business that you know focuses just just on this one
single problem of how do you scan printed documents.
They might be but I'll have to check.
Uh...
Yeah.
Could be
yeah okay one other thing was which I looked into which only recently I found solution to
was you know data is collected in different formats like file formats like
could be Excel sheet could be PDF uh
could be you know DOCX like basically a bunch of different formats how do you convert
everything into plaintext
and there are some software for that that software is kind of improving it's a very
straightforward problem there's nothing very complicated about converting PDF to text
it's just someone has to write that tool to you know convert 20 different file formats
into text and people have only recently written that.
Now it exists you know on GitHub now someone still needs to make that into like a
business. So-
Yeah.
I think these types of there are a lot of these types of problems for sure.
Yeah
interesting times ahead.
Yeah.
So
I think we are mostly done.
Yeah it was a good conversation with you.
Yeah
nice talking to you. Wow I think just by doing this podcast a lot of
ideas have come to my mind.
Yeah.