WEBVTT
00:01.980 --> 00:05.720
And I came back and I read LessWrong,
and I looked, okay, I looked, two
00:05.800 --> 00:09.440
years have passed
and absolutely nothing has changed.
00:09.480 --> 00:12.300
(laughs) Like, you know,
the capabilities have increased a lot,
00:12.340 --> 00:15.900
from 2022, 2023, to, you know, 2025,
like AI
00:15.980 --> 00:19.800
capabilities have gone further a lot, but,
you know,
00:19.820 --> 00:23.700
all these people talking about, you know,
safety, risk, governance, like
00:23.780 --> 00:27.600
it's the same people,
the same boring speeches, the same,
00:27.680 --> 00:30.730
arguments that happen on LessWrong,
like actually nothing has changed.
00:30.740 --> 00:34.710
There's the same funders, you know, uh,
Muskowitz and 00:40 John Talion doing
00:34.740 --> 00:37.230
the same sort of work,
which I don't really believe in,
00:37.340 --> 00:41.250
uh... Yeah. And I, I am now starting
00:41.300 --> 00:44.720
to realize, look,
if I don't personally do something in this
00:45.600 --> 00:49.540
fate of the entire species might be at
stake (laughs) , and that is kind
00:49.560 --> 00:52.880
of an almost overwhelming sort of
responsibility to be dealing with.
00:52.960 --> 00:56.280
Uh, even by 2035, you're looking at, what,
an at least
00:56.520 --> 00:58.720
40%, 50% chance that we will build
00:58.760 --> 01:02.600
superintelligence. So, at
that point you realize, look,
01:02.660 --> 01:06.600
if you're making a life plan
that does not take into account
01:06.640 --> 01:10.000
the fact that, you know,
superintelligence might be created on this
01:10.060 --> 01:13.960
like what are you doing with your life?
Like, that's just a bad move.
01:13.980 --> 01:16.450
01:23 Less, like the world is on...
01:16.480 --> 01:20.340
Like, there is a difference between, look,
the world is on fire and I'm upset about
01:20.360 --> 01:24.320
this, but I won't, you know,
allow this to overwhelm me,
01:24.380 --> 01:27.200
to the public.
I will be the calm voice in the face of
01:27.210 --> 01:31.020
And there is another way of processing it,
which is,
01:31.080 --> 01:34.360
look, the world is on fire, and look,
I don't care about this at all
01:34.380 --> 01:36.760
going to do my thing,
because I never cared about this in the
01:36.800 --> 01:40.320
place.
01:44.840 --> 01:48.510
Hi everyone. My name
is Guido Reichstagter, and I have the
01:48.580 --> 01:51.980
pleasure of interviewing Samuel Shadrack
today.
01:52.060 --> 01:55.160
It's Sunday, September 21st, and
01:56.300 --> 02:00.260
I first met Samuel online when I heard
02:00.300 --> 02:02.280
that he had started a hunger strike
02:03.740 --> 02:07.240
in India on the issue of AI
02:07.300 --> 02:10.919
risk, because there are companies
which are right now
02:10.940 --> 02:14.340
racing to build superintelligent AI
systems that they
02:14.460 --> 02:17.520
have no reasonable plan for
02:17.560 --> 02:20.360
controlling, and this project puts
02:21.480 --> 02:24.140
all of us at risk. So,
02:25.600 --> 02:28.780
Samuel,
I'm glad to be talking with you today.
02:28.900 --> 02:32.280
Oh, yes, nice to be here.
Glad to speak with
02:32.340 --> 02:33.300
you.
02:33.440 --> 02:37.350
Awesome. Could you, uh,
start out by telling us a little bit about
02:37.360 --> 02:41.060
yourself, your personal history
and how you came to be
02:41.120 --> 02:41.910
aware of, uh,
02:42.920 --> 02:45.420
uh, what- what's going on with AI?
02:46.920 --> 02:50.620
Uh, yeah, sure. Uh, so I mean, I'm, like,
currently
02:50.920 --> 02:54.540
24 years old. Uh, I've
02:55.000 --> 02:58.960
grown up in India my whole life. Uh, I
02:59.240 --> 03:03.180
graduated from IIT Delhi in 2023,
so that's
03:03.240 --> 03:05.450
two years back, with like a BTech...
03:05.460 --> 03:08.260
like a Bachelor's
and a Master's in biochemical
03:08.320 --> 03:11.829
engineering. Uh,
I've always been generally
03:11.860 --> 03:15.400
interested in tech though,
generally interested in, like,
03:15.420 --> 03:19.040
I've been, you know, the... even among,
you know, the people
03:19.140 --> 03:22.940
at, you know, such a college, I was still,
you know, the outlier of, you know, just
03:22.980 --> 03:26.200
a tech nerd who would just, you know,
spend all my time reading online about
03:26.240 --> 03:30.020
various technologies, you know,
that could be, uh, space technology,
03:30.160 --> 03:33.820
could be anti-aging, could be solar,
could be climate change.
03:33.860 --> 03:37.540
Like, just anything really. You know,
I would just, on my own go up and,
03:37.600 --> 03:41.360
read things. So,
naturally at some point or the other,
03:41.400 --> 03:44.940
also got recommended some books around AI
risk.
03:44.980 --> 03:48.020
I think the first time I got introduced to
AI risk,
03:48.240 --> 03:49.820
uh,
03:50.680 --> 03:54.460
was this, uh, recommendation in, I think,
03:54.500 --> 03:57.660
Bill Gates' reading list. Uh,
he had a couple of
03:57.740 --> 04:00.460
recommendations, uh, one of them was,
I think, Factfulness.
04:00.480 --> 04:03.920
There was another one which
was Superintelligence by Nick Bostrom.
04:04.040 --> 04:07.840
Uh, so when I read, you know,
that particular book, that
04:08.100 --> 04:11.630
took AI risk seriously. I realized, okay,
look, okay...
04:11.660 --> 04:14.720
That was the first time where I started
realizing, okay, look, this actually makes
04:14.760 --> 04:18.420
sense and, like,
this could actually be a big deal.
04:18.440 --> 04:21.360
Whereas until then, it
was just kind of like, you know,
04:21.380 --> 04:25.060
fiction thing in my head. Like... Uh,
maybe
04:25.100 --> 04:28.750
another point of background would be like,
uh,
04:29.280 --> 04:33.200
so during my schooling, uh,
I used to use the internet a fair bit.
04:33.240 --> 04:36.960
I was active, like, on Stack Overflow,
Stack Exchange, like those sites were more
04:37.000 --> 04:40.980
popular back then. I think now, like,
AI has kind of, like, reduced the
04:41.000 --> 04:44.620
readership of these sites,
but back then they were pretty popular,
04:44.780 --> 04:48.580
uh, among academia, among even, like,
people who are new to
04:48.600 --> 04:50.860
any, like, scientific field
but want to learn more.
04:50.940 --> 04:54.480
So, my username there was like,
ghostsinthecode.
04:54.580 --> 04:58.420
And this was based on this movie, I,
Robot,
04:58.440 --> 05:00.840
which was fairly popular.
It's like a Hollywood movie.
05:00.960 --> 05:04.520
Uh, Will Smith was an actor. And it is
05:04.560 --> 05:06.600
basically about an AI took- takeover.
05:06.660 --> 05:08.840
Like, uh, there is
05:08.920 --> 05:12.790
this, you know, huge tech company
that builds these, you
05:12.820 --> 05:16.740
know, armies of robots with AI,
and eventually at some point the AIs plan
05:16.780 --> 05:20.620
takeover and, you know,
in very cinematic Hollywood fashion,
05:20.660 --> 05:24.140
at the last moment, you know, the hero,
you know, Will Smith comes and just
05:24.180 --> 05:27.830
physically shoots down a bunch of robots
and, you know, hacks into the center and,
05:27.840 --> 05:31.390
you know, takes it down and stuff. Uh,
I think that was the first
05:31.420 --> 05:34.730
part, even just as like a fun curiosity,
I thought about it through
05:34.800 --> 05:38.740
like,
like just some of the philosophical
05:38.750 --> 05:42.680
them even way back then. Uh, but yeah,
back then
05:42.820 --> 05:45.420
I just thought of it as, you know,
fiction, thought experiment.
05:45.540 --> 05:49.000
I didn't think of it as, oh, literally,
you know, 10 years later we're going to be
05:49.100 --> 05:52.580
having these AIs and these
are going to be actual questions and,
05:52.640 --> 05:54.969
of the world is actually going to depend
on these types of things.
05:56.100 --> 05:59.060
Uh, so maybe that is maybe the-
my first introduction.
05:59.080 --> 06:02.840
My first serious introduction was, yeah,
in college, uh,
06:02.900 --> 06:06.768
reading Superintelligence by Nick
Bostrom.Then I, you know, read
06:06.808 --> 06:09.468
through the reviews of, uh,
superintelligence.
06:09.568 --> 06:09.888
Uh,
06:11.348 --> 06:14.728
there were people saying, "Look, you know,
half of this content is copied from, you
06:14.748 --> 06:18.108
know, Eliezer Yudkowsky.
Just go read what Eliezer Yudkowsky has
06:18.128 --> 06:22.008
instead." And then I was like, "Hmm,
interesting." And then I went and read a
06:22.228 --> 06:26.208
lot of work of, you know,
Eliezer Yudkowsky. I read The Sequences.
06:26.228 --> 06:29.808
That gave me a lot of things to think
about.
06:29.928 --> 06:30.448
Uh,
06:31.368 --> 06:34.148
I think Eliezer Yudkowsky is, like,
a brilliant philosopher.
06:34.168 --> 06:37.198
Like,
I disagree with him on a lot of points,
06:37.228 --> 06:40.928
brilliant, like, this surface area of,
you know, novel topics he has covered
06:40.968 --> 06:44.748
is large, and he has interesting thoughts
06:44.888 --> 06:48.288
on a lot of them, some of
which I agree with and some of
06:48.408 --> 06:48.968
Uh,
06:50.328 --> 06:51.868
so there's that.
06:52.688 --> 06:56.068
So I think that was the point where I
was first sure, okay, look,
06:56.128 --> 06:59.468
some chance, you know, our species
is going extinct as a result of this
06:59.488 --> 07:03.428
technology. Back then, I, this, when I say
07:03.488 --> 07:07.157
back then, I mean, like, you know,
maybe fourth, fourth year of college.
07:07.188 --> 07:09.947
I think back then I assigned like,
you know, a 15%
07:10.668 --> 07:13.498
probability we'll be getting, you know,
superintelligence soon, like, you know, in
07:13.508 --> 07:17.148
the next five years, 10 years. Now
that number has gone up.
07:17.188 --> 07:19.428
Now I'm like, at least 25%, maybe more.
07:19.468 --> 07:22.768
We can talk about, you know,
technicalities of these numbers
07:22.788 --> 07:25.748
basic idea is, yeah. Uh, a couple years
07:25.788 --> 07:29.588
ago, reading Eliezer Yudkowsky's work,
that was the first time I
07:29.628 --> 07:33.568
started taking this stuff like se-
seriously and being like, "Okay, look,
07:33.628 --> 07:37.588
there is some actual non-negligible chance
that, you know, the species may
07:37.628 --> 07:41.278
not survive the creation of this
technology."
07:41.328 --> 07:44.868
After that it's just me trying various
things
07:45.368 --> 07:48.988
around this.
I started the Effective Altruism Club
07:49.248 --> 07:52.908
in my college, IIT Delhi. It did
07:52.968 --> 07:56.928
not go super well, to be quite blunt,
but I learned useful
07:57.028 --> 08:00.308
things from the experience of trying to
start that club over a period of six
08:00.348 --> 08:01.328
months.
08:02.728 --> 08:06.348
What else? I completed this, uh,
two-month, like,
08:06.408 --> 08:10.328
full-time, you know, whatever, 50,
60-hour week intensive computer
08:10.388 --> 08:14.068
science course called ML Safety Scholars
by
08:14.187 --> 08:18.028
Dan Hendrycks who's at UC- UC Berkeley,
and like one of the famous
08:18.068 --> 08:22.048
names in the AI risk space.
That did give me some
08:22.128 --> 08:25.988
technical background, and
that actually scared me a little bit more
08:26.068 --> 08:29.948
it made me understand to the extent to
which, like, AIs are just
08:30.008 --> 08:33.148
black boxes and, like, the people who
are training these things have actually no
08:33.208 --> 08:37.158
idea what's going on inside them. Like,
I had a surface level of
08:37.188 --> 08:40.368
understanding of this, but then actually,
you know, you know like writing down the
08:40.408 --> 08:43.608
equations for a transformer, okay,
obviously, okay, attention is like this.
08:43.628 --> 08:46.648
These are the key val-
query value matrices.
08:46.688 --> 08:50.288
And look, this is, you know,
what a scaling law is, this is how
08:50.868 --> 08:54.368
for the entire field of interpretability
re- research looks like.
08:54.388 --> 08:58.328
You know, this is what people
when they talk about, you know,
08:58.368 --> 09:01.488
ali- other AIs, you know,
at a technical level, this
09:01.508 --> 09:05.218
And just reading a bunch of these things
from a technical level made me
09:05.268 --> 09:08.298
understand the extent to which, like,
AI researchers have no idea what they're
09:08.308 --> 09:12.148
doing. And, like, some of these fields
are so nascent and so much
09:12.188 --> 09:15.278
of what actually works
is just like black box approaches,
09:15.328 --> 09:18.268
like try a thing without understanding why
it works.
09:18.308 --> 09:21.288
It's just, "Look, you know,
the last six months it worked,
09:21.328 --> 09:25.148
again." And that was an intuition
which I really got
09:25.168 --> 09:28.948
through that course
and just studying this, the topic
09:29.028 --> 09:32.268
of AI and, you know, AI alignment,
AI safety at a more technical
09:32.288 --> 09:35.928
level. After that what did I
09:35.988 --> 09:36.968
do? Uh,
09:38.168 --> 09:39.368
after that to be
09:40.188 --> 09:44.088
quite honest I did take like a break of
like at least a year from
09:44.148 --> 09:48.048
these things. So, you know,
in my fifth year, my last year of college,
09:48.108 --> 09:51.458
did not do much about all this. I
was just focused on like graduating.
09:51.468 --> 09:52.828
I finished my thesis,
09:53.648 --> 09:57.588
I, uh, traveled for a few months and at
09:57.628 --> 10:00.128
the end of the travel I
was actually low-key depressed.
10:00.148 --> 10:04.088
I was like,
"What am I doing with my life?" It gave me
10:04.148 --> 10:07.348
of like, "Look,
I actually need to do something useful
10:08.028 --> 10:11.968
this sort of just traveling
and having fun and stuff
10:12.008 --> 10:15.308
to work out for me. Like, you know,
in moderation it's good,
10:15.328 --> 10:16.848
the focus of my life.
10:18.548 --> 10:22.357
And I came back and I read LessWrong
and I looked, okay, I
10:22.368 --> 10:24.568
looked, two years have passed
and absolutely nothing has
10:24.588 --> 10:28.548
changed. (laughs) Like, you know,
the capabilities have increased
10:28.588 --> 10:32.208
a lot, like, you know,
from 2022 to 2023 to, you know, 2025, like
10:32.648 --> 10:36.588
AI capabilities have gone further a lot,
but, you
10:36.648 --> 10:40.088
know, all these people talking about,
you know, safety, risk, governance,
10:40.128 --> 10:44.048
like it's the same people,
the same boring speeches, the same, you
10:44.128 --> 10:47.738
know, arguments that happen on LessWrong.
Like actually nothing has changed.
10:47.748 --> 10:51.688
It's the same funders, you know, um,
Amos Goertz and Jan Talon doing
10:51.708 --> 10:54.228
the same sort of work
which I don't really believe in.
10:54.348 --> 10:54.568
Uh,
10:56.288 --> 11:00.068
yeah, and I- I am now starting to realize,
look, if I don't personally do
11:00.108 --> 11:03.968
something in this space, you know,
the fate of the entire species might
11:04.068 --> 11:07.968
be at stake. And that
is kind of an almost overwhelming sort of
11:08.008 --> 11:11.828
responsibility to be dealing with, uh,
11:11.888 --> 11:14.968
but yeah, I mean, at some level that
is the truth.
11:14.988 --> 11:16.988
Like someone needs to try out new
approaches.
11:17.028 --> 11:20.118
Someone needs to figure out ways this
field can actually grow and like make an
11:20.118 --> 11:24.088
actual impact,
the field of AI risk specifically,
11:24.128 --> 11:27.538
be one of the people to help figure
that out. Yeah.
11:27.568 --> 11:31.508
Mm-hmm Wow. And so you decide to go
11:31.628 --> 11:34.728
on hunger strike.
11:34.738 --> 11:34.738
Uh, yes.
11:34.788 --> 11:38.468
What was your... How did you- how did you-
how did you come to
11:38.508 --> 11:41.468
this decision? What
was your thought process around
11:41.508 --> 11:45.228
this?
11:47.068 --> 11:50.648
That's a good question. Uh,
11:52.228 --> 11:56.088
so I can give you a bunch of reasons from
like political theory and I can
11:56.128 --> 11:59.148
give you a bunch of reasons which
are just, okay, look, I did hit
11:59.168 --> 12:03.008
was like,
"Let's see what works." So the hit
12:03.088 --> 12:07.048
reasons I, look,
I started a YouTube channel,
12:07.088 --> 12:10.548
videos about AI risk, I gave a bunch of,
you know, talks at various
12:10.568 --> 12:14.028
places, I-... wrote this, you
12:14.088 --> 12:17.748
know, database for whistleblowers,
which is kind of, you know,
12:17.828 --> 12:21.048
politically sensitive,
so maybe we don't have to discuss it.
12:21.068 --> 12:24.048
But that is another thing, you know,
how do you support a whistleblower at an
12:24.128 --> 12:28.098
company? I was doing that.
And at some point, yeah, this
12:28.168 --> 12:32.148
idea came to me, like, "Look here,
hunger strike actually makes sense." Uh,
12:33.048 --> 12:35.088
so it's just, you know,
try a bunch of things.
12:35.128 --> 12:38.368
You know, so far what I was trying
was not giving me very concrete, you know,
12:38.408 --> 12:41.788
success.
I couldn't look at any of the things I'd
12:41.808 --> 12:45.708
this is a big success
and it actually made a difference." So I
12:45.748 --> 12:47.828
was like, "Okay, look,
let's just try one more thing
12:49.028 --> 12:52.828
that may have a chance of actually
working." Uh, the reason from political
12:52.928 --> 12:56.588
theory is basically, uh, making
12:56.648 --> 13:00.438
costly public signals. Like
if you signal to the public that, "Look,
13:00.468 --> 13:03.588
willing to make significant sacrifices for
this," like,
13:04.248 --> 13:07.968
that speaks to the public more than just a
few, you know, you go write a blog post
13:08.028 --> 13:11.978
or you make a video. Uh, like words
are cheap, you know, actions are
13:12.028 --> 13:16.008
costly. Uh,
they also have certain thoughts around,
13:16.028 --> 13:19.588
you know, common knowledge, like, uh...
13:19.688 --> 13:23.228
So when... So, like, in general and...
13:23.248 --> 13:27.188
Okay, I'll make a disclaimer here that,
you know, I don't want to try and, you
13:27.228 --> 13:31.208
know, topple over any government
when I say this, but in
13:31.248 --> 13:35.188
the past,
when people have tried to topple over
13:35.248 --> 13:37.798
they have noticed is, uh...
13:39.888 --> 13:41.708
A key factor here is common knowledge.
13:41.728 --> 13:45.508
Like it's not enough for a lot of people,
like if 10% of the population are
13:45.568 --> 13:49.048
like, you know, "We
are unhappy with this government
13:49.068 --> 13:52.448
to be in charge," that
is not enough to actually topple the
13:52.488 --> 13:56.308
It's like, I know that I am unhappy,
and I know the other person is
13:56.367 --> 14:00.288
unhappy, and the other person knows
that I know that they are unhappy,
14:00.308 --> 14:03.958
that sort of common knowledge.
And only when you do that among, you know,
14:03.968 --> 14:06.688
significant fraction of the population,
that's when you can actually topple the
14:06.708 --> 14:10.488
government.
And ways to establish this common
14:10.528 --> 14:14.448
very public things.
So this could be starting with, yes, uh,
14:14.548 --> 14:17.758
let's say, a YouTube channel, you know,
a charismatic speaker has, you know,
14:17.788 --> 14:21.488
millions of likes.
That establishes common knowledge because
14:21.497 --> 14:23.048
"Oh, it's not just me liking this video.
14:23.068 --> 14:26.668
There's also, you know,
10 million other people who
14:26.728 --> 14:30.668
This could be in public demonstrations,
public protests if, you know, if there's
14:30.888 --> 14:34.028
like, you know, there's a city and,
you know, like,
14:34.068 --> 14:36.888
physically on the street
and they all look at each other
14:36.968 --> 14:39.208
Oh, okay,
so I'm not alone in believing this.
14:39.248 --> 14:42.268
We all believe the same thing."
That establishes common
14:42.328 --> 14:46.248
knowledge. The final stage is, you know,
the point at which,
14:46.288 --> 14:50.068
you know, these mass movements can,
you know, actually occupy, you know,
14:50.148 --> 14:54.108
buildings. Uh...
14:54.188 --> 14:58.108
Yeah,
and that again establishes common
14:58.688 --> 15:00.867
buildings are sym- like, symbols.
15:00.908 --> 15:01.398
They're, you know,
15:02.568 --> 15:05.068
like... yeah,
they're just symbolically important.
15:05.148 --> 15:08.928
And when everyone sees, okay,
like the entire country will see, says,
15:08.988 --> 15:12.248
look, this movement has actually gone
and occupied this building
15:12.308 --> 15:15.388
to, you know,
actually stop them," then they realize,
15:15.408 --> 15:18.288
are going to form the new government.
That's just it.
15:18.408 --> 15:18.848
So
15:20.148 --> 15:23.528
I think, yeah, hunger strikes
are one of those costly signals.
15:23.608 --> 15:24.108
And
15:24.948 --> 15:28.208
if a lot of people can come in, you know,
even just like, share, subscribe on
15:28.228 --> 15:31.388
hunger strike, and if hunger strike gets,
you know, a million likes, that can even
15:31.428 --> 15:35.128
establish the common knowledge required to
actually bring about change.
15:36.308 --> 15:38.288
Yeah, I was really surprised that,
15:38.408 --> 15:42.348
um, there was the kind
15:42.388 --> 15:46.218
of reception or the kind of impact
that even this
15:46.228 --> 15:49.928
kind of limited action made. I think the,
15:49.948 --> 15:53.828
the, the post, uh,
that Michael Trauzzi made announcing
15:53.888 --> 15:57.768
his strike was viewed over-
maybe over one and a half million
15:57.828 --> 16:01.708
times. And the one that I made
was viewed also
16:01.728 --> 16:05.648
over a million times,
and it's generated a number of
16:05.728 --> 16:08.808
articles in, you know, press around the
16:08.848 --> 16:11.268
world. So,
16:12.908 --> 16:16.528
definitely was surprising to me the impact
of even this kind of
16:17.208 --> 16:18.188
limited action.
16:19.008 --> 16:20.788
And I think, you know, if this is what
16:21.988 --> 16:24.068
five people,
16:25.188 --> 16:25.828
four people
16:27.028 --> 16:29.988
going, making this kind of costly
16:30.067 --> 16:33.998
action can do,
imagine what 100 people could
16:34.048 --> 16:37.328
do, or 1,000 people.
16:37.387 --> 16:41.308
So I think this is really a crucial part
of building
16:41.318 --> 16:43.468
this common public
16:43.548 --> 16:47.058
awareness. People have to see people
16:47.928 --> 16:51.028
that are acting consistently with what it
means to be in an
16:51.048 --> 16:52.588
emergency.
16:53.568 --> 16:56.387
And, uh, I believe we are in an emergency,
and, um,
16:57.808 --> 17:00.168
I'd be interested in hear-
hearing from you.
17:00.188 --> 17:03.328
What, what is it about this time
that we're in that makes it an
17:03.348 --> 17:05.648
emergency?
17:07.728 --> 17:11.688
Oh, uh, that's a good question. Uh,
(laughs)
17:11.708 --> 17:15.068
I think when used the word emergency,
like there's a rational aspect to it
17:15.108 --> 17:18.988
emotional aspect to it. And, you know,
I'm the type of person who tries to
17:19.028 --> 17:22.938
the two. Uh,
I still have complicated thoughts on,
17:22.988 --> 17:26.677
well... Like,
should these two be separate?
17:26.768 --> 17:30.458
Uh,
so let me just first talk about the
17:30.568 --> 17:32.828
Uh, like rationally, you know,
what do I believe?
17:32.848 --> 17:34.268
What is the state of the world right now?
17:34.308 --> 17:36.648
What could the state of the world be in
the next few years?
17:36.788 --> 17:40.958
Uh...
17:41.608 --> 17:45.468
I think assuming, uh, there
is no such social
17:45.548 --> 17:49.408
movement,
no one actually tries to change the course
17:49.468 --> 17:52.498
labs are trying to take the course of
history, uh,
17:53.548 --> 17:57.248
even in the next five years, there's,
you know,
17:57.288 --> 17:58.808
going to build superintelligence.
17:58.888 --> 18:02.088
Uh, when I say superintelligence,
I mean an AI that is better than all
18:02.128 --> 18:04.888
all tasks. And this
is just the next five years.
18:04.908 --> 18:07.208
You know,
just because they don't do it in the next
18:07.248 --> 18:09.858
won't do it afterwards. If you take,
you know, another five years, so that's,
18:09.888 --> 18:13.868
know, 2035, you can easily put another,
you know, 15 to 20% ta-
18:13.928 --> 18:16.308
like... Again these numbers
are not very accurate.
18:16.328 --> 18:18.068
There's a lot of high uncertainty on these
things.
18:18.128 --> 18:22.068
So-Even by 2035, you're looking at, what,
an at
18:22.108 --> 18:25.688
least 40%, 50% chance
that we will build superintelligence.
18:26.688 --> 18:30.148
So at that point you realize, look,
if you're making a life plan
18:30.808 --> 18:34.388
that does not take into account the fact
that, you know, superintelligence might
18:34.428 --> 18:38.168
be created on this earth, like, what
are you doing with your life?
18:38.188 --> 18:41.948
Like, that's just a bad move. Uh,
if superintelligence is
18:41.968 --> 18:43.968
created, what are the outcomes? Uh,
18:44.848 --> 18:46.908
I think multiple outcomes are possible.
18:46.928 --> 18:49.348
This is the thing I have more uncertainty
over.
18:49.368 --> 18:52.728
Like, I have less uncertainty on like,
okay,
18:52.768 --> 18:56.608
build soon. I have more uncertainty on,
okay, if we build superintelligence,
18:56.648 --> 19:00.588
what happens next?
I think there's a good chance
19:00.688 --> 19:02.888
just has its own thing it cares about.
19:02.928 --> 19:06.868
It can, you know,
make plans over long time horizons
19:06.908 --> 19:10.668
the thing it wants to do,
which probably has, like,
19:10.728 --> 19:14.148
with humans or what humans want
and whatever.
19:14.228 --> 19:16.768
And it'll just take over the earth.
19:16.868 --> 19:17.228
Uh,
19:18.408 --> 19:22.328
it will basically ignore us,
and as a side effect, like, we
19:22.448 --> 19:26.328
might just get wiped out because,
19:26.448 --> 19:29.428
uh, it will have things it wants to do.
19:29.448 --> 19:31.388
You know,
there are resources on the earth.
19:31.448 --> 19:34.258
There are specific, you know,
temperature and pressure conditions on the
19:34.288 --> 19:35.768
which are required for human survival.
19:35.808 --> 19:38.588
There are specific, you know,
food resources, water resources which
19:38.608 --> 19:42.558
for human survival. If the AI decides
that even one of these resources,
19:42.608 --> 19:46.388
it wants,
it will just take those resources
19:46.428 --> 19:50.088
uninterested in what, you know,
humans want or need to live.
19:50.848 --> 19:54.608
Uh, that is a possibility. This
is the other separate possibility
19:54.648 --> 19:58.398
in, is even if you manage to,
with certain techniques, keep the
19:58.448 --> 20:02.008
superintelligence under control, like,
you know, you apply boxing techniques,
20:02.108 --> 20:05.948
uh, for all the, you know,
alignment safety nerds out there,
20:06.208 --> 20:09.868
what AI boxing is. But it's... Yeah,
literally keep the AI in a
20:09.928 --> 20:12.888
box so that even though it's smarter than
you, it just, um, harder to then break
20:12.908 --> 20:16.668
out. Which is, to be clear,
an extremely risky thing to be
20:16.708 --> 20:20.608
trying to do. But even
if you actually managed to pull this off,
20:20.628 --> 20:23.828
I think the outcome with
that sort of thing
20:23.868 --> 20:25.468
take over the world. Uh,
20:26.568 --> 20:30.228
you can, at this point, just, uh,
topple over various governments.
20:30.268 --> 20:33.088
You know, this is a threat to, you know,
the US government, the Chinese government,
20:33.108 --> 20:34.388
the Indian, Russian governments.
20:34.488 --> 20:37.998
If a small group of people have an actual
superintelligence that's in their
20:38.028 --> 20:41.948
control,
and they can do novel things with this,
20:42.028 --> 20:45.818
just build parallel industries.
They can build, you know...
20:45.828 --> 20:46.018
(laughs)
20:46.948 --> 20:50.568
Uh, I mean,
Yudkowsky famously talks about
20:50.968 --> 20:54.708
that is also possible.
We can even just look at, you know,
20:54.788 --> 20:58.648
life, like, we are already not
that far from, away from
20:58.808 --> 21:02.328
being a- able to, you know,
genetically engineer life forms so you can
21:02.348 --> 21:04.868
populate the earth with, you know,
engineered life forms.
21:04.988 --> 21:05.288
Uh,
21:06.508 --> 21:09.988
you can invent new energy sources with a
21:10.048 --> 21:14.037
sufficiently intelligent mind. Uh, like,
we are already doing these
21:14.088 --> 21:15.488
things. Like, forget superintelligence.
21:15.508 --> 21:17.928
Like,
humans might be able to do these things in
21:17.948 --> 21:19.968
Like,
this is not science fiction in my head.
21:20.037 --> 21:23.688
Like, if humans make, you know,
solar energy really cheap or they crack,
21:23.768 --> 21:27.648
fusion energy or something, like,
these things seem totally possible to me,
21:27.688 --> 21:30.108
and the superintelligence
is just going to accelerate these things.
21:30.128 --> 21:33.788
So if a small group of people can just
build things like this, they can
21:33.868 --> 21:37.468
build, you know, their own power grid,
they can be extremely,
21:37.488 --> 21:41.368
extremely persuasive. Like,
just imagine someone like Hitler
21:41.428 --> 21:44.848
way more persuasive, and you have, like,
direct access to millions of people.
21:44.888 --> 21:48.588
You can disperse very people at scale,
you can get really good at cyberattacks.
21:48.648 --> 21:52.028
You know, AI is already, as of 2025,
pretty good at
21:52.048 --> 21:56.008
cyberattacks. You know,
Mark Zuckerberg has openly on the record
21:56.028 --> 21:56.388
you know,
21:57.328 --> 22:00.948
like,
AI is already a threat to the
22:01.068 --> 22:05.058
Uh, so just a fi- you know,
five years more of progress on that,
22:05.088 --> 22:07.968
just basically hack into any system,
any government database and so
22:08.108 --> 22:11.908
on. Like,
there's so many different capability just
22:11.988 --> 22:15.908
open up all at the same time. Yeah,
it seems quite plausible to me a small
22:15.948 --> 22:19.248
of people will just take over the world
and establish their own, you know, like,
22:19.288 --> 22:22.628
their permanent dictatorship of sorts
and just do what they want.
22:22.648 --> 22:26.598
That seems possible to me.
There's also a possibility that
22:26.628 --> 22:29.968
there's just some unknown unknown sort of
thing happens which, you know, I can't
22:29.988 --> 22:32.298
predict. Like, these things
are just really hard to predict.
22:32.348 --> 22:34.728
There are just so many intersecting things
going on.
22:34.748 --> 22:38.178
There's AI capabilities going on, there's,
you know, specific
22:38.228 --> 22:42.208
questions like, okay,
if I had cyberattacking capability, if
22:42.268 --> 22:46.228
I had a fully autonomous military,
if I had XYZ technology,
22:46.268 --> 22:48.508
like,
how much power can I project on the world?
22:48.518 --> 22:51.128
Like, some of these questions
are just honestly open questions.
22:51.148 --> 22:54.138
Like, me sitting in my bedroom is not...
22:54.148 --> 22:57.348
I'm not necessarily going to be able to,
you know, forecast all of these questions
22:57.408 --> 23:00.388
correctly.
So we might just end up with some unknown
23:00.428 --> 23:03.588
unknown, uh, possibility which,
23:03.668 --> 23:06.748
again, m-
doesn't necessarily need to be good.
23:06.768 --> 23:08.248
You know, it could be good, could be bad.
23:08.318 --> 23:08.568
So
23:09.608 --> 23:12.928
it's just, yeah,
it's just going to dramatically change the
23:12.968 --> 23:16.888
And just from a rational point of view,
I think that constitutes an emergency when
23:16.928 --> 23:19.828
you look at these sort of extremely
dramatic
23:19.888 --> 23:23.328
outcomes. Now,
coming to an emotional point of view,
23:23.568 --> 23:27.108
uh,
23:29.428 --> 23:31.648
yeah, I don't know.
I think I'm still working my way through
23:31.688 --> 23:35.468
So there's one set of philosophies
which are kind of popular in
23:35.508 --> 23:39.188
Silicon Valley,
and I have kind of a love-hate
23:39.208 --> 23:43.108
ideas which are popular in Silicon Valley,
which is, yeah, basically a
23:43.168 --> 23:43.928
lot of, like,
23:44.788 --> 23:48.668
stoicism and, like, you know, if
23:48.708 --> 23:51.988
there is a crisis and you
are the leader and you are calm, you know,
23:52.028 --> 23:55.888
good decisions. Like,
there's a lot of emphasis on this sort of
23:55.988 --> 23:59.548
even if the world is on fire,
you need to be calm,
23:59.588 --> 24:03.528
the good decisions here. Like, this
is the thing that's popular in
24:03.548 --> 24:07.488
Silicon Valley. There's- they-
there's a general emphasis on, like,
24:07.528 --> 24:10.708
this is what great people do, and
24:12.768 --> 24:15.748
to part... for part of me,
that makes sense.
24:15.788 --> 24:19.688
Like... but, uh, there is
24:19.708 --> 24:23.488
also another part of me that is like,
okay, if you take that ideology to
24:23.528 --> 24:27.468
its logical extreme,
you really do end up with a person who
24:27.548 --> 24:29.568
not care about these things in the first
place.
24:29.608 --> 24:33.472
Like, the world is on-Like,
there's a difference between, "Look,
24:33.512 --> 24:37.472
is on fire and I'm upset about this,
but I won't, you know, allow this
24:37.532 --> 24:39.992
to overwhelm me," or,
"I won't show this to the public.
24:40.052 --> 24:43.831
I will be the calm voice in the face of
the storm." And there is another
24:44.532 --> 24:47.312
way of processing it, which is, "Look,
the world is on fire.
24:47.331 --> 24:50.492
And look,
I don't care about this at all
24:50.552 --> 24:53.892
I never cared about this in the first
place." And like, I think there's a
24:54.831 --> 24:58.292
dividing line between those. So, you know,
24:58.372 --> 25:02.331
emotionally,
am I looking at it like I'm in an
25:02.372 --> 25:06.132
level? Yes. But at some level,
I'm also kind of leading a
25:06.172 --> 25:10.072
relatively normal life. Like, okay,
I have devoted my career, you know, full
25:10.132 --> 25:12.972
time to working on this,
but apart from my career, what am I doing?
25:12.992 --> 25:16.562
You know, I have a relatively normal soc-
a normal social life.
25:16.572 --> 25:20.392
You know, I have friends, you know,
I have fun activities I do with them,
25:20.432 --> 25:21.872
I travel places. Like,
25:23.272 --> 25:26.372
if you look at, you know,
other aspects of my life,
25:26.412 --> 25:30.232
and be like, "Oh, Samuel is, you know,
in an emergency or a crisis right now."
25:32.492 --> 25:33.072
so
25:35.652 --> 25:38.452
yeah, all in all, I don't know.
I think that just gives you a lot of
25:38.472 --> 25:41.612
on, you know,
to what extent am I looking at this as an
25:41.652 --> 25:44.572
Yeah.
25:46.912 --> 25:49.732
Yeah. For,
for me there's a couple of aspects to it.
25:49.812 --> 25:53.302
One is definitely that there is
25:54.831 --> 25:55.851
an immense risk
25:56.932 --> 26:00.622
in front of us. And
26:01.952 --> 26:05.802
it threatens the lives of everyone I care
about, and really everyone
26:05.851 --> 26:09.792
on this planet. And like you said,
I look at
26:09.831 --> 26:12.952
the trajectory.
If there isn't a movement or if there
26:12.972 --> 26:15.432
effective, um, political
26:17.152 --> 26:17.652
project
26:18.992 --> 26:22.572
to prevent this risk, um,
26:22.591 --> 26:25.572
we're,
we're gonna suffer those consequences,
26:25.632 --> 26:28.972
are. And because there's not
26:29.032 --> 26:32.292
currently any signs that we are on track
26:33.192 --> 26:36.952
to effectively manage it, um, like
26:37.452 --> 26:41.181
to me that's an emergency.
It's kind of like, you
26:41.211 --> 26:44.972
know, we're in a car headed downhill,
gaining
26:45.092 --> 26:47.932
speed and there's no brakes on the car.
26:47.952 --> 26:51.572
It's like, that's an emergency.
26:52.392 --> 26:52.592
So
26:53.912 --> 26:57.792
yeah, it is,
it is hard sometimes to separate the
26:57.851 --> 26:59.831
rational aspect.
26:59.952 --> 27:08.341
Um...
27:08.432 --> 27:12.402
I think for me it,
it boils down to rationally understanding
27:12.412 --> 27:14.402
we're in danger, and
27:15.492 --> 27:18.792
emotionally, the most important thing
that I feel is
27:18.912 --> 27:22.831
not turning away or trying to,
27:22.952 --> 27:26.132
um, deny that fact, and
27:26.212 --> 27:29.952
then emotionally face it,
face what it means, and then
27:30.072 --> 27:33.712
act rationally, appropriately to try
and stop
27:33.752 --> 27:36.252
it. So,
27:37.032 --> 27:37.812
yeah.
27:38.831 --> 27:42.152
H- how are you? We should have,
we should have got this, uh, done in the,
27:42.192 --> 27:45.242
beginning,
but you've been on hunger strike now,
27:45.272 --> 27:49.212
fasting. How did you...
Do you prefer to call it a fast or a
27:49.232 --> 27:52.501
hunger strike? And you've been,
but you've been on it for eight days
27:52.572 --> 27:55.851
now,
so I'm curious how your experience has
27:56.512 --> 27:59.152
personally.
27:59.212 --> 28:02.812
Yeah. Uh, I think I
was initially a bit hesitant to call it a
28:02.892 --> 28:06.032
because, yeah, there were a lot of,
you know, random Twitter trolls who
28:06.072 --> 28:08.061
"Look,
you're not actually putting your life in
28:08.112 --> 28:12.012
Is that a hunger strike or not?"
But at this point I'm like, "Look, I don't
28:12.092 --> 28:14.882
care about that." I do still consider this
a hunger strike.
28:14.892 --> 28:18.632
Like,
I am actually making a costly sacrifice
28:18.692 --> 28:20.672
actually worse as a result. And
28:21.851 --> 28:25.252
like,
if I was just doing what's good for me,
28:25.272 --> 28:29.232
I'm doing this because I believe there's a
bigger, you know, cause that needs to
28:29.272 --> 28:29.612
be,
28:30.492 --> 28:32.371
uh, advanced here. Uh,
28:33.312 --> 28:35.592
so yeah, I would call it a hunger strike.
28:35.692 --> 28:39.312
Uh, how am I doing? Ah, that's a good
28:39.432 --> 28:43.322
question.
I think it's pretty much manageable,
28:43.371 --> 28:46.662
of, you know, today, which
is day number eight of the hunger strike.
28:46.692 --> 28:49.652
I've publicly committed
that I'll be doing this for at least 18
28:49.712 --> 28:51.092
days.
28:51.191 --> 28:52.112
Oh wow.
28:52.251 --> 28:54.581
Yes. Yeah. Uh...
28:57.752 --> 29:01.472
I mean, it is not great, but it
is manageable.
29:01.512 --> 29:05.272
Like, I am definitely extremely, like,
physically weak, like just
29:05.732 --> 29:09.472
less energy in my body. Like
when I walk around,
29:09.512 --> 29:13.092
look, don't bump yourself into things,
don't get yourself injured, you know,
29:13.152 --> 29:17.052
fall over. Uh, I have, I
29:17.092 --> 29:20.632
randomly have nausea at points.
I have randomly, you know,
29:20.652 --> 29:22.492
points. Uh,
29:23.652 --> 29:26.262
like by itself, you know,
none of these things is a big deal.
29:26.272 --> 29:27.952
All of this is very much manageable.
29:27.992 --> 29:28.592
It's just
29:30.192 --> 29:34.092
or, you know, I drink salt water,
it tastes, you know, absolutely horrible.
29:34.192 --> 29:37.612
Uh, all these things by itself,
they're small, but when they
29:37.652 --> 29:40.952
accumulate, you know,
day after day after day just dealing with
29:41.892 --> 29:45.292
shit, it does kind of take a toll on you
29:45.331 --> 29:47.672
mentally. Also, I'm just kind of bored.
29:47.812 --> 29:51.632
I- I'm not doing anything productive
and my mind is not clear
29:51.672 --> 29:53.892
enough to do any, like very focused work.
29:53.972 --> 29:54.132
So
29:55.892 --> 29:59.652
I end up spending a lot of time doing
absolutely nothing, and then
29:59.712 --> 30:01.492
that feels not great. So
30:04.152 --> 30:06.312
yeah, it's just like a m- like a
30:07.132 --> 30:11.052
subpar quality of life.
When you extend that, you know, over many,
30:11.132 --> 30:13.792
it does start taking a bit of a toll on
you.
30:13.812 --> 30:16.992
But all in all,
I think it's pretty much manageable.
30:17.032 --> 30:21.022
You know, I am fine doing this for,
doing this for, you know, another week,
30:21.052 --> 30:24.081
another two weeks if it comes to it. Yeah.
30:24.112 --> 30:27.672
So to be clear, you're just taking, um,
basically
30:27.812 --> 30:31.772
salts for electrolytes,
but otherwise nothing, no
30:31.851 --> 30:34.672
other... no calories, no vitamins,
nothing like that?
30:35.660 --> 30:39.150
Uh, no. Yeah, I'm just drinking water
and I'm
30:39.200 --> 30:42.460
having, you know, table salt, you know,
sodium
30:42.600 --> 30:46.470
chloride, uh, mag- uh, potassium chloride,
which is again,
30:46.480 --> 30:50.340
another salt. I'm having, you know,
magnesium capsules and that's about
30:50.380 --> 30:50.600
it.
30:50.610 --> 30:50.610
Yeah.
30:50.680 --> 30:53.900
I'm not having any calories, no sugar, no
30:53.940 --> 30:55.640
vitamins, so on.
30:55.650 --> 30:55.650
Mm.
30:55.700 --> 30:56.060
Yeah.
30:56.480 --> 30:59.380
Mm. Yeah, I- I wonder
if that's a big difference in our
31:00.140 --> 31:03.920
experiences 'cause I am taking, um,
a multivitamin
31:04.120 --> 31:08.020
and other vitamin supplements, but no c-
no calories, and also
31:08.080 --> 31:12.000
electrolytes. And like, subjectively,
31:12.060 --> 31:15.960
of course, I feel tired
and I have the same nausea
31:16.080 --> 31:19.600
and, um, you know, some stomach pain.
31:19.640 --> 31:23.600
But, um, mentally, I feel like very
31:23.640 --> 31:27.500
there, like very, um, you know,
31:27.560 --> 31:31.360
aware and in full possession of my
faculties and everything,
31:32.600 --> 31:36.460
so.
Or maybe I'm just (laughs) maybe I'm just
31:36.540 --> 31:40.360
(laughs) being there doesn't mean all
that much (laughs) as it may- as it
31:40.420 --> 31:40.940
may to you.
31:42.120 --> 31:42.560
But, um,
31:43.520 --> 31:44.730
yeah, that's amazing. I didn't...
31:44.780 --> 31:48.560
I- I- I missed that part where you had-
had committed to- to going for 18
31:48.620 --> 31:51.960
days, so that's... Yeah,
that's impressive.
31:52.060 --> 31:55.860
Um, I'm curious, how is your... How
are your family and friends,
31:55.960 --> 31:58.630
uh, responding? Do they know about this
or what do they think?
31:58.660 --> 32:00.160
Have you talked to them about it?
32:01.320 --> 32:05.060
Uh, yeah, the response are mixed.
I've definitely got just
32:05.120 --> 32:08.500
support,
solidarity from some of my friends who
32:08.560 --> 32:11.580
thing you're doing
and I wish you all the best," and so
32:11.700 --> 32:15.600
on.
There's certain people who don't really
32:15.620 --> 32:18.900
they don't really buy the case for AI risk
to the extent that I do.
32:18.920 --> 32:22.460
But despite that, they're like, "Look,
you know, you believe in this, you want to
32:22.500 --> 32:24.240
do this, you know, good for you.
All the best.
32:24.280 --> 32:25.940
I hope you, you know,
I hope you stay safe.
32:25.980 --> 32:29.760
I hope your health stays fine, you know,
don't get hospitalized and all that
32:29.840 --> 32:33.400
stuff." Uh, I have gotten a couple of
32:33.440 --> 32:36.650
responses I'm not happy about where I did
realize, okay, look,
32:37.480 --> 32:41.180
these people, you know,
out of a misguided sense of, you know,
32:41.220 --> 32:44.199
me might actually try to, you know,
stop me from continuing on
32:44.240 --> 32:48.000
this. And to them, I have made very clear,
like, "Look, this is
32:48.060 --> 32:50.580
important to me, you know,
don't get in my way.
32:50.620 --> 32:54.420
Like, I know you have, you know,
sort of certain, you know, care
32:54.460 --> 32:58.180
me,
but if you actually truly cared about me,
32:58.220 --> 33:01.960
this is to me. So, you know, don't-
don't get in my way."
33:02.000 --> 33:02.900
Yes, that
33:04.000 --> 33:04.160
I
33:05.900 --> 33:08.980
have gotten again, you know,
even more the responses, they're like,
33:09.020 --> 33:11.960
they're not super happy with me doing
this, but
33:12.960 --> 33:16.440
also end of the day, look, the choice
is not in their hands, the choice is in my
33:16.450 --> 33:20.420
hands and this is what I'm doing. So yeah,
the responses have been pretty mixed all
33:20.460 --> 33:23.350
around.
I think I'm generally happy with the
33:27.040 --> 33:29.360
And, uh, I guess just as a final question,
33:30.160 --> 33:33.970
I'm wondering how do you see yourself, uh,
33:34.100 --> 33:37.460
going forward in the future? What's, um,
33:38.139 --> 33:39.420
you know, what's on your mind about
33:40.600 --> 33:42.420
what part you can play in this whole
33:43.440 --> 33:45.540
situation that we're in?
33:46.800 --> 33:50.720
Uh, that's a good question. Yeah, I mean
33:50.800 --> 33:54.160
if you ask me concretely, do I have like,
a role I want to play and just
33:54.200 --> 33:55.980
consistently play that for five years?
33:56.000 --> 33:59.520
Like, I don't have conviction on what
that role is.
33:59.580 --> 34:02.920
I'm still in a phase of, you know, hit
and trial, try a bunch of things and see
34:02.960 --> 34:04.920
what works, uh-
34:04.960 --> 34:07.800
What's the goal that you're working
towards?
34:07.900 --> 34:11.560
I mean, the high level goal is definitely,
you know, get a pause
34:11.739 --> 34:15.440
on further research development of AI
34:15.540 --> 34:19.239
worldwide across all countries.
This will require
34:19.280 --> 34:23.150
necessarily a treaty between the
government of the United States and the
34:23.199 --> 34:27.080
government of China. This
is going to require a certain
34:27.120 --> 34:29.280
number of en- enforcement mechanisms.
34:29.290 --> 34:32.500
This is going to require a lot of people
actually convinced that their lives are in
34:32.560 --> 34:36.199
danger. Like,
you cannot pull off these sort of large
34:36.260 --> 34:37.670
actually convinced a lot of people.
34:37.670 --> 34:41.520
Like, that is the high level goal. Uh,
I could also
34:41.580 --> 34:44.900
go into like, what
are the more smaller things
34:44.920 --> 34:48.239
that I'm working on, if you'd like to kn-
know about those.
34:48.260 --> 34:51.920
Well, what's your... Yeah, what's your-
what's your kind of, um, big
34:52.040 --> 34:55.940
picture of the path? How do- how-
how generally
34:56.279 --> 34:59.340
do you think we...
What do we need to do to get from here to
34:59.420 --> 35:01.960
there?
35:03.760 --> 35:07.580
Uh, good question. So I think in general,
if you
35:07.620 --> 35:10.500
want to get a person to do something,
there are two ways to do it.
35:10.520 --> 35:13.760
You know, A, you can convince them let-
in their heart that, "Look, this is
35:13.820 --> 35:17.180
actually no- important." Like,
you actually change their values
35:17.240 --> 35:20.080
And B, as you apply incentives on them,
you apply pressure.
35:20.120 --> 35:23.060
Now,
pressure could be actual physical
35:23.120 --> 35:26.160
It could be, you know,
monetary incentives.
35:26.180 --> 35:29.540
You tell them, "Look, if you do this,
you'll be respected," et cetera.
35:29.720 --> 35:33.600
I think this... Solving this problem
is going to require a mix
35:33.640 --> 35:37.060
of both. Like,
you can't use just force
35:37.100 --> 35:40.060
persuasion. There
are going to be people doing a bi- bit of
35:40.120 --> 35:41.920
both. Uh,
35:43.480 --> 35:43.650
but-
35:43.680 --> 35:47.480
You mentioned earlier about, um,
a mass movement.
35:48.690 --> 35:48.690
Uh-
35:48.720 --> 35:52.120
What do you think the...
What do you see the role of a-
35:52.160 --> 35:56.070
process of moving society from where it
is now
35:56.100 --> 35:59.640
to where it needs to be to actually
implement that kind
35:59.720 --> 36:03.480
of a global treaty?
Or even starting with the
36:03.520 --> 36:05.230
United States, right?
36:05.230 --> 36:05.230
Mm-hmm.
36:05.240 --> 36:09.060
Because the United States right now
is nowhere in the ballpark of politically
36:09.760 --> 36:10.020
of
36:11.800 --> 36:15.640
seriously considering
that even as an option.
36:15.700 --> 36:17.790
Yeah, that is fair. Okay, so the...
36:17.800 --> 36:21.120
I think historically, the way that,
you know, effective altruist movement has
36:21.160 --> 36:25.060
approached this is a very softball kind of
approach, which
36:25.140 --> 36:29.060
is, again, I'm not saying like, it is bad
36:29.120 --> 36:31.800
to try this. I'm just saying, you know,
maybe they're failing.
36:31.860 --> 36:35.680
What they tried is essentially, "Look,
we will get people in all the
36:35.740 --> 36:39.531
elite circles of government, you know,
we will..."Uh, establish think
36:39.631 --> 36:42.491
tanks. We'll establish policy.
Policy institutions.
36:42.531 --> 36:45.591
We'll get just people in government and,
you know, just convince people in the US
36:45.611 --> 36:48.732
government this is important
and just make a- action work that way.
36:48.752 --> 36:50.832
So they just went through the persuasion
approach.
36:51.631 --> 36:54.131
And I'm not optimistic on this working
out.
36:54.172 --> 36:58.131
I think the geopolitical power you get
from developing AI is too
36:58.172 --> 37:01.971
large for, uh,
people at the high levels of
37:02.031 --> 37:05.332
US government to be just listening toward
this effective and trust people have to
37:05.392 --> 37:09.192
say. Uh, so just 100% persuasion is not g-
37:09.232 --> 37:12.682
really going to work in my view, uh,
which comes to, you know,
37:12.692 --> 37:16.392
establishing a mass movement
and actually applying pressure on the US
37:16.412 --> 37:18.372
Like, you can't do just persuasion.
37:18.392 --> 37:21.892
You need to actually pressure that, look,
you know, if you don't actually do the
37:21.912 --> 37:25.761
things that we want you to do, like if we,
the public, say we want this and the
37:25.812 --> 37:28.852
government is, like, not willing to do it,
you need to actually apply
37:29.652 --> 37:33.352
pressure and be like, "Look, we
are willing to vote new leaders
37:33.372 --> 37:36.392
We're willing to, you know, bi-
boycott your products if required.
37:36.412 --> 37:39.611
You know,
we are willing to protest on the street
37:39.712 --> 37:43.611
Uh,
we are going to disrupt your way of doing
37:43.712 --> 37:46.531
think a mass movement is a good idea.
37:46.591 --> 37:50.392
Establishing a mass movement, again,
requires persuasion,
37:50.412 --> 37:54.071
different audience.
And instead of convincing, you know,
37:54.111 --> 37:57.772
government,
you're now going to be convincing,
37:57.812 --> 38:00.352
public. Now, even in the public,
who do you mean exactly?
38:00.392 --> 38:03.281
So one is there are people who
are very technically skilled.
38:03.312 --> 38:05.741
This is, again, you know,
the software developers essentially,
38:05.812 --> 38:09.241
Silicon Valley.
So I think convince Silicon Valley
38:09.292 --> 38:13.272
I think convince, you know,
literally every person in the United
38:13.292 --> 38:17.131
is another approach. Uh,
I think multiple approaches need to be
38:17.152 --> 38:20.912
out and we need to see what works.
Even on the, you know,
38:21.111 --> 38:25.031
broader geopolitical scale, like, okay,
look, you convinced US,
38:25.111 --> 38:26.332
like, "You know, what about China?
38:26.352 --> 38:29.872
What about Israel?" You know, uh,
there will be people who will need to
38:30.131 --> 38:34.091
attempt similar things even in China
and I don't
38:34.131 --> 38:38.111
know, for political reasons,
maybe I'm not super comfortable commenting
38:38.172 --> 38:41.832
that, but at some point, through some m-
mix of either
38:41.852 --> 38:45.792
persuasion or- or force,
you do need to also get the Chinese
38:45.832 --> 38:49.392
board with the idea that, you know,
superintelligence is an extremely
38:49.430 --> 38:53.412
technology and you need to respond,
you know,
38:53.471 --> 38:54.672
that it poses.
39:00.611 --> 39:04.511
All right. Well,
thanks for sharing your time.
39:04.591 --> 39:05.161
Yeah.
39:05.912 --> 39:09.652
So there anything else, uh,
you want to say or think is, um,
39:10.172 --> 39:12.591
good- important for our- our audience to
39:12.631 --> 39:15.152
hear?
39:15.372 --> 39:19.131
Uh, no, not much in particular. Yeah,
39:19.172 --> 39:22.812
just in general, look,
if you like the work I'm doing, you know,
39:22.872 --> 39:25.692
like, share,
subscribe to spread this to a wider
39:25.702 --> 39:29.551
I'd especially be interested in being
connected to people, you know,
39:29.752 --> 39:33.721
space, in the legal space,
even just other people who are really full
39:33.812 --> 39:37.582
time committed to this. Uh, there are
39:38.752 --> 39:42.652
ways you can actually, you know,
contribute
39:42.712 --> 39:46.652
to this cause that are, uh, that A,
won't require a lot of your time
39:46.692 --> 39:50.372
and even will require a lot of your time
if you don't have a lot of time or energy
39:50.491 --> 39:54.131
to spare. You know, even just, you know,
sharing, uh, my channel to
39:54.232 --> 39:56.832
others helps,
even just sharing your own views in
39:56.852 --> 39:59.562
Like, don't be like, "You know,
Samuel said this,"
39:59.631 --> 40:03.372
actually believe and you put
that on social media. Even
40:03.452 --> 40:07.372
If you do have a lot of time
and energy to spare, uh, there
40:07.392 --> 40:09.352
are more important things you can do.
40:09.392 --> 40:13.232
You can start a channel of your own,
you know,
40:13.352 --> 40:13.631
Uh,
40:15.031 --> 40:18.852
you can obviously donate money. Uh,
there are things you can do.
40:18.912 --> 40:22.471
So in general,
feel free to reach out to me
40:22.491 --> 40:26.091
you can contribute to the space.
You can also obviously reach out to,
40:26.131 --> 40:29.531
Guido here. There
are a number of people in the space who
40:29.591 --> 40:33.511
if you just openly come and ask, "Look,
you know, I want to contribute.
40:33.551 --> 40:36.241
What can I do?" There
are lots of people who will tell you,
40:36.252 --> 40:37.952
you could be doing.
40:37.971 --> 40:41.582
Sure. Thanks so much for coming on and,
uh, and
40:41.652 --> 40:43.500
speaking, because this is, you know,
this is
40:45.551 --> 40:46.781
probably the most important
40:47.952 --> 40:50.852
issue facing humanity in the near
40:50.952 --> 40:52.752
term.
40:53.772 --> 40:55.571
And thanks for doing your part.
40:55.652 --> 40:58.971
Oh, yeah. You're welcome.
Enter "Subscribe: " followed by Signal username, email address or whatsapp phone number. You will receive atmost one update per month. View previous updates here.
Alternatively, you can use RSS, or a third-party service like blogtrottr.
Enter "Comment: " followed by comment. Alternatively, contact me