Home | Search

2026-03-12

Would I kill my friend if they plan to kill me out of ignorance?

Disclaimer

Main

Atleast some of the people in these chains of command (AI companies' chains of command, US intelligence chain of command) are intentionally risking murdering me. Atleast some of them are ignorant.
Atleast some of them share a lot in common with me (high openness, atheist, same SV discourse, actually taking ASI seriously when the whole world thinks you're crazy) and this makes it easier for me to empathise with them.
Let's take the worst case scenario as a thought experiment. This person is almost the same as me, they are already my friend, and they are ignorant about the fact that their actions will get me killed. Do I take a gun and shoot them, yes or no? Forget about ASI risk for now, just imagine some other scenario where this happened.
Rationally, I think the answer is yes? Ofcourse I shoot them first.
Emotionally I still feel a bit of, idk, it depends on context? Like, maybe there are contexts in which I would shoot them and contexts in which I would not.

update

Maybe if I was in love with them, I can imagine letting them kill me, instead of me killing them first? But even then it's not super clear to me what I would do. It seems hard to predict in advance.
- Very important - how do I actually get an answer to this question? I could talk to people I care about irl, but there's no one I can talk to, where I care about them enough that I would just let them kill me. (There's some stuff here I can't talk about for privacy reasons.)
- Lol I clearly do have a bias towards women here. Like, even in a thought experiment, I seem to hesitate more at the thought of shooting an attractive woman I care about, as opposed to shooting a male friend. (Assuming both were planning to kill me out of ignorance.)
- I don't have that much bias towards children tbh. I'd be fine shooting a child. Like yes, situation sucks, but it hurts less to do this, atleast in a thought experiment. I think actual parents have stronger instincts towards saving children than I do. Paul Graham is probably right that their brains change at a biological level.

update

honestly I could just cold email some of the people at the AI companies and ask them this exact question

Enter email or phone number to subscribe. You will receive atmost one update per month

Enter comment