I can't, I can't live with myself if I let this
society sleepwalk into disaster, right?
I can't live with myself if I don't do everything in my power
to ensure there's a world for my children. The important thing...
I mean, I go out on the street, I've talked to, I don't know, hundreds,
of people. Many of them, many of them immediately say, "Oh, yes,
Yeah, they're gonna, they're gonna end up killing us all." ...
So for example, in the, in the, in the Birmingham campaign,
that I mentioned earlier, 5,000 schoolchildren were arrested
and spent some time in jail. But it
changed
the public perception of an entire country.
It got attention at the level of the national government,
the introduction of the Civil Rights Act.
That the
core founding principle of StopAI is that
superintelligence is inherently
too dangerous of a technology ever to be built.
This is not, um... this is in contrast to the position of, for
example, Pause AI, which says it should not be built until it
safely. As, as to why Yudkowsky is not on hunger
strike, that's a good question. People can find many different reasons not
things.
Uh, hello, everyone. Uh, my name is Samuel Shadrack.
Uh, with me, I have Guido, uh, who
has been
protesting against creation of
superintelligence. He is on hunger strike.
Uh, I'm just going to ask him some questions,
more broadly, you know, about the issue of, you know,
Why is this a big deal? What can we do about it?
And so on. Uh,
could you maybe just please start by introducing yourself, you know, like
what's your background? How did you, you know, come to this decision where you
are right now of, you know, doing a hunger strike?
Like, where did this begin, and how did that turn into where we
now?
Yeah. Thank you for having me on.
Um, yeah, my name is Guido Reichstadter.
I'm 45 years old. Uh, I'm a father of
two children.
And I first really became aware of the
potential to build artificial general intelligence,
uh, back in university when I was studying math and
physics. I was probably around 20 years old.
And, um,
you know, immediately
I felt that this would be
probably the most dangerous technology that humanity had ever
tried to build. And, uh, just at the
time, the, the people
around me, the information that I could get seemed to point
towards this becoming possible in the second half of the 21st
century. Um, and I kind of just put it out
of my mind really,
until, uh, ChatGPT was
released. And
that kind of served as a wake-up call, uh, that this
process was going a lot faster than I expected it to go.
Not that I thought ChatGPT itself was
there, right? But just what it indicated about the
underlying trend.
And
I remember, um,
beginning to look for more information.
I found this, uh... I found out that people had
been... people had much shorter timelines, even back in
2000. Specifically, uh,
um, there was a paper that was written in the late '90s by Hans
Moravec where he looked at the exponential, uh,
trend in increasing computational power
and used a model of some
neurons to make an estimate of what
the, like, in a... by analogy, what kind...
what, h- how much computation the human brain does.
And said this trend will intersect the computational power of the human brain
sometime in the 2020s. And
that should be a warning sign, right?
That it's becoming, it's potentially becoming
possible now
to create hardware that can do effectively
what a human brain does, even if it's not wired up correctly yet,
right?
Never before in, you know, the history of humanity has it been
to build a machine that could do anything
like the computational ability of the human
brain, and that's becoming possible now.
And so, yeah, um, it's... I go- it
got me worried when I saw Chat GP- GPT come out, seeing the
continuing trends, seeing the billions and billions of dollars in
capital pouring into the field and greater and greater numbers of
researchers being devoted to cracking that problem.
Like, I'm 45. Never in my life
has there been so much
energy, talent, resources,
capital, excitement behind building
this technology. And I feel that
we are in a condition of imminent danger right now because we don't
know where the line is. We don't know how far we
have to go to cross
the barrier
to something that's as generally capable as a human
mind. And we also know that there's a lot of momentum, a lot of
inertia in society.... and that if we want
to have the option to stop this process, we have to
do... we have to change the way the world works,
and that takes time and effort and energy.
So it's kind of like the situation where we're on the Titanic,
huge ship with a lot of momentum and inertia,
and we see an iceberg in front of us,
and we don't know if we are able to maneuver the
ship out of the path of that iceberg in time.
And if so, uh, you know, if it's, if that's not the case,
right now. It's like,
if you're in a car
driving downhill
and you have no brakes, your brakes are gone, you're in an
emergency well before you pass over the edge of that
cliff.
That's the situation which the world is in right now.
There is an uncontrolled race to artificial
superintelligence, which we have good reason to suspect would
have catastrophic or
extinction-level consequences for the human race-
Okay.
... and we have no effective mechanism to stop
currently. So, um,
in 2024, I put my career on hold. Uh, I was a jeweler
in Miami. I started a business with my ex-wife, uh,
years working at that, and,
um, moved to San Francisco
to try and
put some brakes on this (laughs) runaway car,
right? Um,
I landed October... I think it was 17th.
On the 21st, I was sitting down in front of OpenAI
having chained their, their door shut,
right? I believe we all have a right to stop
this race that threatens our lives.
And
in the absence of government action, which is...
has not been forthcoming, we have a responsibility to stop it now
precaution, out of the need to prevent
the risk of devastating consequences to people we love.
So, um,
yeah, I, I don't think there's any...
Any day is a bad day to take direct action to stop this
race. And in fact, we need to see a lot, lot more of it, and that's,
that's
one of the prime motivations for me going on this hunger strike,
is
partly to make a public demand of these companies to stop
putting my life and the life of my children and our
community in danger,
to do everything in their power, which is far greater than
mine, to work towards an end to the global
race which is necessary
for our safety, right?
Yeah. Um-
We need a globally binding treaty which prohibits the development of
this extremely dangerous technology,
and to fulfill their duty to
do everything they can to ensure that our society
is made aware of the urgent danger it's being put
in by this race and by the companies, like
Anthropic, that are participating in it
their responsibility while they're doing so.
It's not enough to
make nice, safety-sounding
words, right?
Mm-hmm.
You've got... If, if this company is
going to put the lives of everyone on Earth
at risk...
They have their reasons. I don't agree with them,
but regardless,
they have a responsibility to make sure that the people whose
lives they're putting at risk understand that their lives are being put
at risk,
and this is not satisfied by giving a few speeches or a few
podcasts here and there, you know?
It's not satisfied until society is made aware of this
urgent threat. So
finally, if Dario Amodei, the CEO of Anthropic,
that, I'm asking him to meet me face-to-face,
being, and explain why, and
I feel that that's not an unreasonable request.
I feel that my life has value just as much as
his.
Got it.
The lives of my children-
Yeah, got it. Um-
... they have a right to life.
Yeah, everything you said was super useful, but yeah,
interrupt. I just, I just want to go to the next question.
Uh, because you mentioned specifically
know, just a disclaimer, I have never visited, but obviously,
Twitter, you know, I see the stuff that gets posted online.
Uh, in general, like, you know, if you ask me personally,
like, for me, it makes sense, like, you know, political action makes
sense. Like, I assume, okay, you know, look, religious leaders have, you know,
influence over society. You know, politicians have, you know, influence over
society. You know, politics actually matters, and, like,
social movements actually matter.
Like, it is not just money that matters.
Like, it's not just, you know, billionaires that matter, you know,
matters. But however, when I look at, you know, the
types of, you know, ideas that are spread in San Francisco, the sense
I get is that, you know, if you're building technology,
like, a tech billionaire, like, that's the only high-status thing to do.
Like, if you do that, you will get respect.
If you, you know, become someone like us who's, you know,
and, you know, start a social movement and so on, like, you know, people in San
Francisco don't respect you. And, like, there's
very clear, like, status dynamics going on here,
even in the minds of people who are inside that bubble, which,
that, "Oh, if I, you know, join these people who are protesting,
are going to look at me as lower status, you know, people who won't respect me," so
on. So-Yeah, I just wanted to ask you, like, what
are your opinions on, you know, how, uh, like, status, social
approval, respect, like, how do these things work in San Francisco?
(laughs)
Like, how do they work currently and be, you know, what can we actually done to,
you know, change that?
Yes. I'm primarily concerned with
respecting myself. I don't really care
about other people's respect. I have to live my life in a
way which I can respect
my own actions in honoring what I believe is
true. And
I think it's
these- s- this thing about status is a pernicious
trap, right?
Um, you know, I- it's- it reminds me of
a story, I don't know if it's true or not,
but, you know, people- people will
willingly, um,
lose their lives rather than lose the respect of their- their peers,
right?
Mm-hmm.
And in this case, we might lose the entire world out of some
foolish...
It's like, I- I- I wonder why people are so concerned about the respect
of others. It makes me wonder. It seems that
it- it's a reflection of a certain self-doubt.
At any rate, it- it's- it doesn't stop me from doing what
I'm doing. Um,
and this is, uh, uh, there's an inter- an interesting point that you brought up.
So people in San Francisco,
it's a- it's a metropolitan area of almost eight
Mm-hmm.
Um, it's very hard to generalize, right?
Yeah.
So there are many people in San Francisco who
recognize that
this AI race is a race to annihilation, and they have no- they
want nothing to do with it, right? The important thing, I mean,
street, I- I've talked to, I don't know, hundreds, thousands of
people. Many of them- many of them immediately say, "Oh yes,
thing. Yeah, they're gonna- they're gonna end up killing us all." Many
understand this. It's- it's not like, um,
some, uh, esoteric
idea. Um, the important thing
is not to
act
based on the expectations of others but to do what is right, to do
what's necessary, and to find other people and
encourage them to do what's right and necessary.
And that's what, uh, you know, this is- there's
a- there's a saying that all- all truth goes through several
stages. First it's ridiculed, then it's opposed, and finally
it's- it's accepted as self-evident, right?
Mm-hmm. Mm-hmm.
So there's always going to be this kind of
opposition, but this is not actually about
necessarily even responding.
The primary thing at least is not responding to the opposition.
The primary thing is doing what is correct and-
Mm-hmm.
... with the understanding that we have that this race puts
our lives in danger.
That's enough. E- even certainty is not
necessary. You wouldn't- I wouldn't let anyone play Russian
children, right? And this is just a risk of death.
We're talking about risk of destroying the entire world, right?
We have rights. We have a right to life, and we
have a right to live unafraid that someone's gonna
destroy the world. So we need to
stand on those rights and defend them because we're actually
up for each other, right?
It's not just our rights we're standing up for.
We can only have these- we only benefit individually from these
rights if we're willing to defend them collectively, because
individually we can't- we can't defend them.
And so this is- this is the thing which, uh, trying to help, um,
people to do.
Got it. Got it. That makes sense, and I am completely supportive of your
efforts. Uh, maybe then the next thing I wanted to
ask is, you know, like, how do we actually, you know,
How do we build this into a social movement?
Uh,
so one thing you actually previously talked to me about, you know, in private,
but I also wondered if they, you know, be, uh, out there in public, which is you
said, like, in order to get more people, you know, in the movement, you
know, against AI, you know, to be actually protesting,
something about it, uh, you said what is important is
not, "Okay, here's a list of technical arguments for,
risk," but what is actually needed is, like, courage.
It's, like, seeing, you know, your friends and people around you,
actually demonstrating that courage and actually, you know,
Uh, like, that intuitively seemed to make a lot of
sense to me. Uh, I just wanted to sort of get more of your
Like, why do you believe this is true?
Do you have some existing examples of this, you know, either in the AI or AI
risk space or, you know, even outside of it?
Yeah.
No, I- I mean I think this is absolutely essential,
right? I mean, it's not the wh- it's not the whole thing.
There are many
interlocking pieces, uh, eh, in- among which
the technical arguments, the awareness
also contribute, right?
But, um,
what in- in my estimation has heretofore been
lacking has been a willingness to act
consistently with the meaning of an
emergency. Even a- a willingness to
acknowledge that there is an emergency on the part of people who
traditionally been concerned
and aware of the threat that-...
AI poses
with regards to extinction.
And
there is a lot of, um,
there's kind of a spirit of uncertainty,
fear, doubt,
self-doubt, and all of this is
totally
fatal
to the creation of a vital en- engaged
movement, right?
We potentially have very little time in which to
accomplish a very large goal, which is
moving the entire world out of the path
of this extremely dangerous technology it's on course
to, to build. And
I have seen the failure of
reformist projects on
social issues for 40 years, on climate change, for example,
with the result that now very devastating, even catastrophic
effects are essentially locked in.
We don't have 40 years to fail on climate change, uh,
on, on AI. We don't have 40 years to fail on
AI. And
we need to act
decisively. If a m- if a, if a man
is putting my life in danger and the lives of my children, I'm not
satisfied merely to ask him to stop
or to ask the government to stop and wait and wait
and wait for a response that may not ever come.
Got it. I-
It's our responsibility to take leadership.
... thing doesn't look very good right at the
To take direct action. We have the right to end this threat
today, right now.
This is our right.
And the only way that it's going to possibly be done is by
beginning to act
with courage, with conviction
on that, on that right, to take direct non-violent action
to stop
the development towards superintelligence, to ta- which could, is
as simple as
non-violently occupying the companies that are doing
it, to lead government, because government is
not leading on this.
Go, the responsibility for our safety
government, but they are abdicating that responsibility.
Where does the responsibility go? It doesn't vanish. It's on us still.
It always was on us. We delegated it to government,
job. It's our responsibility to lead
government, to occupy the offices of
Congress if necessary-
We do.
... to compel them to act.
And we can do this. This is, this has been done by
th- this has been done by social movements
over again.
And
I'm, I'm inspired particularly by this.
There is a, an episode in the civil rights movement in
the Birmingham campaign,
where
5,000 school children
went to jail
for defying the segregation laws in Birmingham,
Alabama.
Yeah.
They filled every jail in Birmingham, they filled the state
fair, the animal pens where they, they held animals at the state
fair,
and this happened over several weeks.
And by the end of it, the chief of police threw his hands in the
said, "There's no way to keep a lid on this
gone."
This is when radical political change happens, when people
lose their fear
and they're willing to act. For me, I'm acting out of love
for my children. If I f- if I have to
face the choice between going on hunger strike
and raising awareness on this issue and mobilizing people to
action or sitting back and, and living a comfortable
life and letting my children's lives be threatened, I'll take the first
one any day of the week. I've been arrested a couple of
times. I'm willing to go to jail.
I'm willing, you know, to do whatever it non-violently takes to
my children are safe, because in my mind there's nothing more
important.
Yeah.
And when people, when people tap into that love, the
fear of, of sanction
becomes much less effective in constraining our
behavior.
Yeah.
I'm sure that many of the people that are concerned about this, if
the, if they, if they knew that simply by
sitting down in front of the AI companies they would stop this race, many would
take that. It's the right thing to do.
The only reason they don't is because they don't think it's going to
work.
And what we need for people to do is to center less on what they think
to work and more on what is right for them to
do, because
paradoxically that's why it hasn't
happened. That's why we haven't stopped this race,
have not en mass led society
and said, "This thing is wrong and it needs to stop."
Yeah.
Got it, got it. No, that is a good answer.
Uh, I wanted to ask you, uh, so there are multiple
groups that are trying to do...
... you know, public advocacy around the idea of
risk from superintelligence. Uh, this could include starting with, you
know, uh, Mary's latest book, you know, Eliezer Yudkowsky and Nate Sawyer
is, they published this book, you know, If Anyone Mutes It, Everyone Dies.
Uh, there is pause AI which was started by,
uh, Meandetsma and Holly
Elmore. Uh, there's, you know, the AI Risk Network by John Sherman.
There's, you know, Liron Shapira who has his own channel
debates. Like, there are lots of people who
need to go to the public. Uh, but I get the sense that they're all
working independently. Uh, so maybe I just wanted to ask you, like,
A, what is something you think like, you know, all these public advocacy groups
agree on, and B, what is, you know, something...
or what other things they disagree on?
Yeah. That's a good question. I'm not terribly familiar with what
e- ev- every other group is doing. I can speak
primarily, uh, for myself, what I'm doing, and
the, the, the work that I've done in support of Stop
AI. Um,
in terms of goals, there is, um... I think there is a
common- a commonality in that
I believe all of these groups
would agree that
the race which is going on now to build superintelligence
is... poses an extreme threat to all of our
safety and wellbeing and our lives, and should
stop now.
I don't think that's... I don't think that would be controversial
(laughs) among any of those groups.
Um, the primary difference
that I see, there are
two. One is kind of in
the goals,
the ultimate goals of those groups, and
one is in the methods which they see
as,
uh, necessary or
effective. Um, in terms of goals, I'd say
that there is a clear
distinction between Stop AI and the other groups which is
that the core
founding principle of Stop AI is that
superintelligence is inherently
too dangerous of a technology ever to be
built. This is not, um... this is in
contrast to the position of, for example, pause AI which says
it should not be built until it can be done safely.
And from Stop AI's position,
there is
no- It's in- it's inherently a contradiction in terms.
This technology cannot ever be done safely
because unlike o- every other technology we've deployed,
been able to test
repeatedly... prototypes
and refine and iterate through engineering.
And
with superintelligence, there is the very real risk that you don't get a
second chance. If you build a system which is
capable enough to take
control of the world and humanity's
fate, you may not get a second chance
at... you may not get a chance to shut that off,
You may have just locked yourself into
inevitable loss of control. And
I don't trust any theory which
gives predictions about the behavior of a system like this.
It's not enough to have theory. It must be tested through
experiment. And this is just not
a regime, the creation of superintelligence,
experimentation. Um,
you know, we... there are airplanes
that leave the ground based on theory
and there are people that get in those airplanes,
pilots. I'm not willing for my
family, against our
will, to be the test subjects
of the AI company's superintelligence experiments, and I
don't think anyone should. So, that's primarily the difference in goals.
There is a s- a- an ultimate goal of stopping
superintelligence permanently, which means creating a world in
which it is possible to do so indefinitely, that we have the
social institutions, the
relations among people which no longer
produce this race and this unstoppable
drive to build ever more powerful AI
systems. And in terms of how, how we stop
that race, like in the near term, there is also like a- a strategic difference
in that
I personally, and it's generally the, the
approach of Stop AI, that
the urgency of this situation demands direct
action,
that the
institution, the... It's a- it's a- it's an estimation that the
institutions of government and of economy, that the
rules of the system as they are set up now
will produce, with overwhelming likelihood,
um, the bad outcome here. And that
there is a phenomenon in social...
in, in this- in the history of, of world society which is able to
produce very rapid changes in government
policy.And in fact, it does so about every two or three
years somewhere in the world. And this is mass parta- participation
civil disobedience. This is when people en
masse, in large numbers, refuse to
allow society to continue as
normal until their demand is met. And
sometimes this is a, a, a demand for a change in the law,
demand for a change in the government.
And like I said, a movement like this succeeds in achieving
in the world about every two or three years. This is a f-...
This is not a one-off thing. It is a social phenomenon,
which is
nearly... It was essentially universal.
It's... It is a way for societies to rapidly
change direction under extreme duress, which we
are in now, and which
we are increasingly approaching,
right? So- so this is the approach
which StopAI has chosen. It is to take direct
action as the catalyst and as the motive
force of building an assertive, vital
social movement, which is not content to go
the reformist route and
parlay
potentially decades of time that we don't have.
Yeah.
Right. So sometimes you, you have to go outside the box.
You have to break the rules. That's okay, right?
Uh, you know, I'm, I'm going to court this, uh, next month
in a case where I obstructed the operation of
OpenAI, which is one of the leading companies
And
we're asserting and defending this right of the public to take
this kind of action, that it's... We are asserting that it
that it is morally justified. And there is this
concept in law called the defense of necessity.
This is
an idea that goes back
probably thousands of years. I'm sure there's something like it in, in,
legal systems around the world. It's the understanding that sometimes it's
necessary to break some laws for...
To prevent a greater harm. Um, just
like you would break the window of a, of a car
to rescue a child trapped inside,
right? There's no other alternative to save
Yeah.
That's the case we're in right now.
We don't know how close these companies
are from the next breakthrough, right?
Remember, it was one big breakthrough in 2017
the field, the transformer.
Mm-hmm.
Right now, there are hundreds, you know, hundreds of billions of dollars and
thousands of the world's best engineers
trying to make that next breakthrough.
We don't know how, how close it is. And this
uncertainty should not be cause for inaction.
It should be cause for preventative, precautionary action,
because the consequences are so large, right?
Yeah, yeah, go ahead.
So, yeah.
Uh, yeah, I'm really sorry. I, I just, I just don't have-
No, no worries. Go ahead. Do your, do your thing.
No, no, I get it. You're... Everything you're saying
Uh, I wanted to dive more specifically into
disagreement, uh, because, you know, I think it seems useful to actually
get people on board, you know, doing things.
Uh, so the specific argument you mentioned,
you know, we don't get, you know, like 50 tries to build superintelligence.
You know, the first time you do it, if you do it wrong, you know,
Like, you can't really experiment with this and,
over many years. Like, this is a pretty popular argument
by Eliezer Yudkowsky. I think he's also completely on board with
this specific argument, uh, that, you know, we can't experiment
with it. You know, we are not going to solve, you know, safety
alignment in, you know, five years, 10 years, probably in 20 years.
Uh, maybe I would just ask, like, uh, how
familiar are you with, you know, Eliezer Yudkowsky's views
on this issue, uh, considering, of course, that he's one of the founders of, you
know, AI risk as a field? And, yeah, I
mean, like, where exact... Like, since, you know, he actually
agrees with you on this core point, like, does that mean you both just,
like, agree on everything? Like, where exactly is the disagreement coming?
You know, why is, you know, Yudkowsky not sitting on a hunger strike
like, you know, you are?
Yeah, yeah, that's a good question. Um, I- I'm not terri...
Uh, I'm honestly, I'm not terribly familiar with, uh,
uh, Eliezer Yudkowsky's, um, uh,
work. Um, I don't think I've... I may, I've read maybe a, a
couple of papers or something. Um,
so I'm not really sure what his position is.
There are some things that he's said publicly that I do disagree
with and, uh,
I think, um, I think I disagree on others, but I'm
not, I'm not sure. So,
I'm not sure where he stands on this issue of alignment
or safety. It seems, it seems to be...
Like, what I'm... My position, you know, is that this
should never be done. And it must, it must never
be done, and we have an obligation to not do it,
stop the people that are doing it.
Mm-hmm.
Um, which s- seems in contradiction to the idea
that one can align a superintelligence
a priori, right? By, by, by ex- by theory.
Mm-hmm.
Also, the, the alignment of,
of such a system is deeply problematic in terms
of,
um...What
goals is it being pointed to? Also, the
idea that such a s- the idea that such a, a
capable, self-reflecting, self-improving system,
um,
s- would, would be faithful to any goal which
is externally,
uh, imposed or structured into it seems
also very suspect.
So I- I- I believe that the- the
concept of alignment in itself is very,
um, misleading
in the sense that it may serve the function of
causing people to be less concerned
about the development of super intelligence.
That there's- there's a natural human tendency to-
want to have to exert one's self
beyond one's normal affairs. And if people
can feel that this problem's being solved for them by
someone else,
it's a very reassuring kind of, um,
story to tell oneself, which excuses
taking action
yourself to stop it.
So, um,
I think there's- there's also the- there's also, um,
a, um
... something that's come to my attention
by, uh,
Yudkowsky about the value, the relative value of
human life and
putative, um,
super intelligences,
which is that the sacrifice of the existence of humanity
would be
acceptable
in order to bring into existence
a more morally valuable super
intelligence, which
I categorically, fundamentally disagree
with. Um,
and
... I don't think this ever can
be or should be done safely. And this
is- this is- this is,
like, a f- this is a- a fundamental motivation for the work
that I do. As- as to
why, uh, Yudkowsky is not on hunger strike.
That's a good question. People can find many different reasons not
do things. Anyways, was that good or was there any part you,
uh, wanna cover more?
No, that is a really good answer. Uh, maybe I
should put it this way. Uh, so ha- I'm assuming you've seen,
like, the statement by, you know, Center for AI Safety, Dan Hendricks,
like, uh, mitigating risk of extinction of AI
and it was signed by, you know, a whole number of people.
And I think that has actually, you know, moved- pushed the movement
forward in, like, bringing this consensus that, okay, look,
know, high credentialed, important, you know, knowledgeable people actually do
agree with this. I wonder if there's value in doing a similar
statement, you know, just even among the, let's say, groups who are protesting
right now or involved in public advocacy that, you know,
look, here is a core statement that, you know, we all do agree on.
Like, you know, we might have, you know, certain various disagreements of our own,
and you know, that's fine, but can we at least, you know, like,
some consensus-building. Like, you know, here is a statement that we all agree on.
Like, do you think s- like, such a statement, you know,
Do you have opinions on, you know, what the exact statement should
be?
Um ...
hm.
Maybe could be valuable. Um, I think the- the first
statement was very valuable in the sense that it provided a reference
point for the public. It, um, and provides,
right? It's- it's very-
it's very useful if you're- when you're mobilizing people to take
to something like that and show that there are,
you know, respected, credentialed
authorities who accept this as
a- as a real risk. And this is, for me at least, this is the
primary value in a statement like that.
It's that
it enables and it facilitates public outreach.
Um,
uh, a similar statement, g- getting,
uh, all these groups on board, uh, I- I don't
think it could hurt per se. Um, except
insofar that it- if it- if
it
obfuscates the- the larger issue of
...
for example, Stop AI w- does not want to give the
impression that it agrees that super intelligence can ever be
built safely, which is the stated position of a lot of
other- these other groups. I think we all can agree that the-
should end now, and the difference is that I think Stop
AI
acts as though it must end now, and we're
encouraging other people to- to take that action, because if
it must end now and if the companies are
not ending it and the government's not ending it, then we
must act, and this is the- this is the core thing.
It's simply not enough to write a letter to
your representative or to call your representative
and another day goes by and we get closer
we get closer to this point of no return where we don't know where it
is. We-People who understand this risk, and this
is- goes for the authorities that signed that letter,
what would be much more useful is for them to go
farther in that sa- in that letter.
Rather than saying it is merely a risk that should
merit some precaution, to actually state,
"We are now, given the circumstances, in an
emergency." And to even go farther, that,
to say that, "Given the ina- inaction by our
government, non-violent direct action is necessary and
justified," this would be incredibly useful in mobilizing
people. And this is, from my point of view, the only real
game in town. There's not going to be the political will
mobilized, strident movement making
demands for an immediate end to this
race. And from my point of view, the most important thing
is what actually assists
in creating that
movement.
Got it. Got it, got it. That makes sense.
Uh, yeah, I think I then wanted to move to a
question, uh, which is, again, coming back to, you know,
the movement? I wanted to ask about funding
So, like, you know, this is something I can imagine, and, you know,
or disagree, but,
like, uh, I can imagine a person who has, you know, a tech
job, they broadly agree with the arguments for
risk, but they're like, "You look, you know, I have, you know, dependents.
I need to make a certain amount of money to, you know, sustain myself, to sustain
people around me. And, you know, if I just come and protest, you know,
there's no money in that. You know, it's just financially,
bad for me." So, I do think it makes sense for,
like, social movements like this to figure out a
Uh, do you have a sense of, like, how that funding can be
raised? Like, would that be primarily, you know, through donations?
Could that be, you know... And by the way, I don't have a strong opinion, you know,
which of these would actually work, but, like, I'm just asking your opinion, you
know? Would that be donations? Could that be, you know, let's say, you know,
a YouTube channel and you run ads on it?
Could that be, you know, you provide consultancy to other groups,
on your actual knowledge expertise you have accumulated?
Like, do you have a sense of, like, how you could actually get
funding to, you know, scale up the
movement?
Yeah, I have, I have a general, a general idea.
Um,
like,
the- this has been done in the past, right?
Um,
the- there have been many social movements in
essentially member-supported. And I think the
important thing is creating a movement which is worthy
of support
and which people ha- can have confidence and
faith
in its ability,
its potential ability, to accomplish these goals.
Okay.
And so there's a, you know, there are, there are literally
millions and tens of millions of people that are concerned about
where this is all going, and some of them are very deeply
concerned. And like you said,
there's two, two sides to the example you brought up.
One is, you can do a lot on a large scale
if millions of people
can be mobilized in support
of a direct action
movement. Millions of people times
a dollar a month, 10 dollars a month, is millions of dollars,
tens of millions of dollars, which can be deployed
to mobilize people for direct action.
So it's, in, in a very real sense, it's a problem of
logistics. My experience with Stop AI is with a handful of
organizers going out into the community through methods of direct
face-to-face mobilization,
and
we have found
several people, right? The, the largest action that
has ever been done by Stop AI involved five people.
It's not a, it's not a large number, but it's also the work of a very small
of mobilizers. If this is systematized and
organized
with 100 mobilizers, I think it's very
realistic to find that within society, you will find it,
100 people to take direct action. Not everyone is gonna be
like your software engineer who has, uh, you know, such a managerial
position that it's very hard for them to
make a very costly sacrifice.
But there are many people who are able to do that, and it's a logistical
question of finding them and mobilizing them and showing and
encouraging them to stand up for their rights, which is what this is really, at a
fundamental level, about. There's no guarantee
rights is going to be free, right? It's often going to be very
costly.
Yeah.
But we're also the inheritors
of generations before us who made very costly sacrifices
for our freedom and our rights. And so there's something that goes
beyond the day-to-day material calculations,
right?
Yep.
100 people, you know,
in front of OpenAI, in front of the other AI companies pushing
annihilation,
would make much bigger headlines, would have a much bigger effect in, in
galvanizing public opinion, and in...
Excuse the, the noise. Okay, are you picking that
up on the-
Yeah, yeah-
... on your end?
Your voice is clear, yes.
Oh, okay. Yeah, there's some background noise. You know?
But with millions of people in support, a movement could
find...... thousands of people to take direct action.
This is absolutely in the realm of th- the immediate present
possibility. The conditions in society are such, it's
simply a question of
interfacing with society on a large enough surface
to find those people and organize these actions.
And
this can have a transformative effect in unlocking the
nonlinear potential of society to ch- change.
So for, for example, in the, in the, in the Birmingham
campaign, the Children's March that I mentioned earlier, 5,000
schoolchildren were arrested
and spent
some time in jail. But it changed
the public perception of an entire country.
It got attention at the level of the national government, and it
led directly to the introduction of the Civil Rights Act.
It's not an understatement to say that the Civil Rights Act of
1964 was a direct consequence of those
5,000 schoolchildren standing up for their rights in 1963.
And I believe that there is similarly a tipping point in this
society, because you can look at
society and see that not much is happening and then draw the conclusion
that nobody is concerned. This is not the case.
Mm-hmm. Yeah.
Many, many, many people are deeply concerned.
And I think you saw some evidence of this with the recent pushback on
the attempt to impose a 10-year moratorium on AI regulation
in the United States, which was killed largely as a result of public
outcry.
Yeah.
Even people who aren't willing, because it's, uh,
opinion, and people are hesitant to hold minority opinions
often publicly.
Even young people who are not willing to publicly acknowledge
that they're worried about this extinction
risk, I believe that in many, many people,
maybe even most, there is an awareness that this technology does
threat. And there is a, a kind of
conflict, even in people who are ve- you
know, might display the most outward bravado
of it.
And it's this, these kinds of very public actions, which
in mass, in a representative, you know, that, that show a
representative subsection of, of the population being
willing to take this contrarian position,
because of their conviction, which can strengthen
in the heart of many, many people, to also
gain the courage to say, "You know
ra- you know what? I don't... This is, this is real.
I don't need to stop denying it, and I've got a right to speak
out, and I've got a right to demand that this situation
end." And so this is part of the, the
mechanism of change, which is not only theoretical.
This is how social movements have changed and do change
the direction a- of entire countries, you know, within a matter of
days or weeks.
So...
Got it, got it, got it. That is a good answer.
Uh, maybe then I'll just go to a really simple
You know, uh, which day are you on
currently? How long do you plan to, you know, continue this hunger strike?
Like, just with the hunger strike specifically,
Uh, yeah. Uh, today after I finish
here, I'll be going out to Anthropic, and it will be the 16th
day which I've been publicly in front of Anthropic.
Mm-hmm. Yeah.
Um, I s- I started, I- I started fasting the day before in
be 17 days that I've gone without food, but 16 days as
a public political act.
Okay.
And
on the, on the first day that I went there, I delivered a
letter to the security desk for Mr.
Amodei,
telling him... asking him
to stop putting my lives, my life and the lives of my
family in danger, which he's publicly acknowledged
this race does.
Asking him instead
to use all his power
to do everything in his ability
to work for a stop to this race globally, which is
the absolutely necessary condition for our survival, a
global end, through government regulation, through international
accord. And Anthropic and Mr. Amodei
personally
could have an immense effect in mobilizing
the public will to demand such an accord.
Thank you.
Also, to fulfill the obligation to ensure that
society is made aware of what's going on in these
companies, to be made aware of the
fact that Anthropic is
contributing to this race, which poses
potentially near-term risk of destruction of our entire
society.
And finally, if he's unwilling to do any of
those things, to meet with me personally
and explain why he feels the right
to put my life, my children's lives, our entire society at
risk. And I told him I'd be waiting outside
his door
for his answer, and that's my intention, to wait
outside his doors
until I'm answered.
And-I'm
also doing this
to fulfill my responsibility to my children and the society
that I'm a part of. I can't, I
can't live with myself if I let this society
sleepwalk into disaster,
right?
I can't live with myself if I don't do everything in
my power
to ensure
there's a world for my children.
And I think all of us that understand this risk have a similar
responsibility, and it's up to each person to question
about
what they are able to do. Not what's
convenient, not what would fit into their schedule,
but what's actually necessary and what's actually right in
this case.
Yeah. Yeah. Got it. That is
(laughs) yeah, something even I need to think more about.
I really appreciate you giving this answer.
Uh-
And thank, thank you for joining in this conversation
putting, you know, a bit of your time and convenience
behind the greater, the greater good.
Oh, yeah.
Right?
Oh, yeah.
I didn't expect that, you know-
Yeah.
When I started this, I had no idea that
this kind of response would, would happen, and here we are having this conversation
now.
Oh, yes. Yes. I completely believe with you on like,
change is possible. You know, the first domino moves,
and you know, before you, you know it, you know, thousands of people are actually
doing something about it. So yes, I just want to be part of
the broader movement, the broader, you know, effort to actually do something
about this. And yeah, I definitely relate with you
on the aspect of like, look, if I was not
working on this, if I was just, you know, trying to lead a comfortable life, you
know, just, you know, get a tech job, you know, have vacations, you
know, have a partner, you know, do just fun things with my life,
like, like I would dislike myself as a person,
you know? I would be just unhappy doing that,
actually done more and I chose not to, and like that was on me.
Like, so like I actually completely resonate
this sentiment of yours. Um-
I mean, that was my lived experience.
I, I, there came a point where I could no longer
continue, you know, setting diamonds, which was what I was
doing before, um,
knowing that other people were making these decisions
that were going to have these consequences for the people I
love. That's what caused me
to put this on hold and start doing
what I think is necessary.
Yeah. Got it. Uh, I just wanted to then end with like a
final question, uh, just for viewers of this
podcast. Like, uh, what is the, like, very concrete, you know,
call to action? Like imagine, you know, this viewer o- of this podcast, you know,
after this like they will spend, you know, maybe 10 minutes, 15 minutes actually
doing the next step. Like, uh, what concretely would you
ask them to do? Would you ask them to, you know, go watch a lecture, you know,
a book? Would you ask them to, you know, email you?
Uh, like just what concrete action would you
Uh, I would maybe split it into two.
So
first I would recommend maybe for viewers who are just not really very familiar
with the AI risk, you know, maybe this is maybe one of the first videos actually
about it. And second with for people who
maybe even agree, but they're not actually n-
for both sets of people, like concretely what do you recommend?
Like, after watching this video, what do you recommend they do?
So I mean, I have, I have very clear ideas about what needs
done, and what needs to be done is the mobilization of mass
participation direct action. Um,
if you're not at, if- if you're just learning about this idea
about AI risk for the first time or you're
not quite, you don't quite understand or you're not quite convinced
that this actually is...
that this is a real risk, um,
probably
you, Samuel, can probably put a- a- a good, put together a good compilation
of resources for people to, you know, take that, that
next step. Off the top of my head,
um, I would suggest
definitely check out stopai.info.
There's a presentation there, um, which does, it's a, it's a recording
of one of the public presentations that, uh,
we've done to mobilize people in-person, and so it
does tr- uh, treat this question in the first part
of the presentation. There's, uh, there's some, part of
it's given by, uh, a Professor David Krueger,
who is a aI, who's, who's a professor in,
uh, at FAI. And, um,
you can also check out, um, you know, there's a great YouTube
channel For Humanity by John Sherman that's got a lot
of AI risk, uh, material on it. Uh,
Roman Yampolskiy is, uh, a guy who's spoken out
very broadly in public and has been on a number of,
um, high-reach podcasts,
and I think these are, these are compelling arguments.
So
for the people that, that do re- are past that point and recognize
that this is actually a real risk and it's urgent and it
threatens the lives of those we love-The
absolutely important thing is to join
these, the, the movement to stop it immediately.
And so there are a number of ways that you can do that,
Uh, almost everyone can give some kind of
of donation to, to keep the mobilization work of Stop AI
going, uh, and increase it. Um,
each person has, you know, unique talents and skills
So there's a join form, uh, stopai.
on, stopai.info. And if you sign up, then an organizer
will get in touch with you and see what the best way
to, to, to further that movement is.
And specifically, Stop AI's goals are to permanently
ban the development of super intelligence.
And to, to, and
pursuant to that, to mobilize a mass non-violent
direct action movement of people who are willing to stand on this
conviction and dem- and, and, and are not willing, not satisfied
to, to say, to sit back and watch the race go on for another
day, but are actually able and willing to take direct action
to stop it immediately.
And so, if you're not able to participate in the
direct action, there are many support roles that are
necessary to facilitate finding those people who
are. And so, um,
you know, financial support is an essential part of
that. Yeah, so check out stopai.info,
definitely.
Got it.
That's where I would send people.
Got it. Thank you for your answer. I will definitely put the link to your
you know, in the description. And-
Yeah. And if anyone wants to personally get in
also in- invite that and would, would be welcome
to, you know-
Got it.
... help encourage and, and, and mobilize people.
Got it, got it. I think that was it for, like, questions from my side.
Is there anything else which you would, you know, either like to ask me or just
bring up on your own? Anything else you feel, you know,
important, this should be, you know, put in
this video. Like is there anything like that or,
Uh-
... end this?
Yeah, I think, I think it's like, uh, I w- I was very...
It was very interesting to see this interview with, um,
Dennis, who's on hunger strike in, uh, London right now,
with Max Tegmark who is, uh, has some reputation in the field
of, you know, making
the public aware of AI risk and has also got credentials.
And what I would like to see, and I think is very
important, is for more people that have that kind of credential or
that kind of influence to speak out publicly
on the urgency
and acknowledge that this is an emergency.
That the fact that we don't know how close the edge
is, the fact that there is so much momentum, inertia in
society,
that maneuvering out of the range of this danger will take time and
is a massive task. And we don't know if we have that time,
and that the consequences of failing are so large
that
all this together is an emer- fo-
Let's just, let's just be honest and tell the truth to the public.
We're in an emergency right now
and we need to act like it's an emergency.
And
fundamentally, this is a question of each of our's right
to li- to life, to live. And we need to
take action today
to defend that right and to stop this race immediately.
It's our right, it's our power collectively
and it's our responsibility. And so let's, let's not le- you
know, the whole future's riding on this.
Let's not, let's not let 'em down.
Got it. Got it, got it. Uh, yeah, I just wanted to
again, like thank you for doing this podcast.
You know, in spite of, you know, you literally being on, you know,
hunger strike, you know, deteriorated, you know, physical strength, you know,
mental strength. In spite of that, you came here, you know,
gave such a coherent interview that I hope would be useful to
all the viewers. Uh-
Yeah.
... of this. Like I just really wanted to appreciate,
actually did this and yeah-
Yeah, absolutely. Thank you so much for putting in the
there. That's something that I need to put more effort into myself
the word out there, so...
Yeah.
Got it. Got it.
It's absolutely necessary.
Yes.
Thank you, Samuel. It was good talking.
Cool. Yeah.
Let's talk again soon.
Yes, sure, sure, sure, sure.