Whistleblowers are disprortionately important for changing everyone's minds (collective epistemology)
Here is a giant list of reasons why people might not want to fund this.
(You can add a comment to upvote any reason or add a new one. I will add more arguments and evidence for whichever points get upvoted by more people.)
-
US intelligence circles will significantly underestimate national security implications of AI, lots of information about AI companies will not become classified - Disagree
- I think AI will likely be the number one national security issue of the US by 2026 or 2027 and lots of important information will get classified soon after.
- I'm putting more evidence here because this argument got upvoted.
- Attention of govt and intelligence circles
- Paul Nakasone
- ex-Director of NSA, now on OpenAI board
- Recent talk by Paul Nakasone
- Says: Cybersecurity, AI, and protecting US intellectual property (including AI model weights) as the primary focus for the NSA.
- Likely a significant reason why he was hired by Sam Altman.
- Timothy Haugh
- ex-Director of NSA, fired by Trump in 2025
- Recent talk by Timothy Haugh (12:00 onwards)
- Says: Cybersecurity and AI are top challenges of US govt.
- Says: AI-enabled cyberwarfare such as automated penetration testing now used by NSA.
- Says: Over 7000 NSA analsysts are now using LLMs in their toolkit.
- William Burns
- ex-Director of CIA
- Recent talk by William Burns (22:00 onwards)
- Says: Ability of CIA to adapt to emerging technologies including large language models is number one criteria of success of CIA.
- Says: Analysts use LLMs to process large volumes of data, process biometric data and city-level surveillance data.
- Says: Aware of ASI risk as a theoretical possibility.
- Says: CIA uses social media to identify and recruit potential Russian agents.
- Avril Haines
- ex-Director of National Intelligence, ex-Deputy Director of CIA, ex-National Security Advisor
- Recent talk by Avril Haines
- Says: major priority for her is US election interference using generative AI in social media by Russia and China
- US policy efforts on GPU sanctioning China
- Significant US policy efforts already in place to sanction GPU exports to other countries.
- China is currently bypassing export controls, which will lead US intelligence circles to devise measures to tighten export controls.
- Export controls are a standard lever that US policymaking and intelligence circles pull on many technologies, not just AI. This ensure US remains in frontier R&D of most science, technology and engineering.
- Attention of Big Tech companies
- Leaders of Big Tech companies, including Jensen Huang, Satya Nadella, Larry Ellison, Reid Hoffman, Mark Zuckerberg, Elon Musk and Bill Gates have made public statements that their major focus is AI competitiveness.
- Elon Musk
- Elon Musk is explicitly interested in influencing US govt policy on tech.
- As of 2025-05, Elon Musk likely owns the world's largest GPU datacenter.
- Has publicly spoken about AI risk on multiple podcasts
- Mark Zuckerberg
- As of 2025-05, open sources latest AI models
- Has previously interacted with multiple members of Senate and Congress
- Has publicly spoken about AI risk on multiple pdocasts
- People who understand nothing about AI will follow lagging indicators like capital and attention. This includes people within US govt.
- Capital inflow to AI industry and AI risk
- OpenAI, Deepmind and Anthropic have posted total annual revenue of $10B in 2024. This implies total market cap between $100B and $1T as of 2024. For reference, combined market cap of Apple, Google, Microsoft and Facebook is $10T as of 2025-05.
- All Big Tech companies have experience handling US classified information. Amazon and Microsoft manage significant fraction of US government datacentres.
- If you believe AI in 2027 will be signifcantly better than AI in 2024, you can make corresponding estimates of revenue and market cap.
- Attention inflow to AI industry
- OpenAI claims 400 million weekly active users. This is 5% of world population. For reference, estimated 67% of world population has ever used the internet.
- As of 2025-05, Geoffrey Hinton speaking about AI risk has been covered by mainstream news channels across the world, which has significant increased the fraction of humanity that is aware of AI risk. (You can test this hypothesis by speaking to strangers outside of your friends-of-friends bubble.)
-
AI capability increases will outpace ability of US intelligence circles to adapt. Lots of information won't become classified. - Weakly disagree
- I have low (but not zero) probability we get ASI by 2027. If we get ASI by 2030, I think there's enough time for them to adapt.
- Classifying information is possible without significant changes in org structure or operational practices of AI labs. This means it can be done very quickly.
- Classification is a legal tool.
- The actual operational practices to defend information can take multiple years to implement, but this can come after information is already marked classified in terms of legality.
- US govt can retroactively classify information after it has already been leaked.
- This allows the US govt to pursue a legal case against the whistleblower under Espionage Act and prevent them from presenting evidence in court because it is now classified information.
- The specific detail of whether information was classified at the time of leaking is less important than whether it poses a national security threat as deemed by US intelligence circles. (Law does not matter when it collides with incentives, basically.)
- Case studies
- Mark Klein's 2006 leak of AT&T wiretapping - retroactively classified
- Hillary Clinton 2016 email leak - retroactively classified
- Abu Ghraib abuse photographs 2004 - retroactively classified
- Sgt. Bowe Bergdahl 15-6 investigation file 2016 - retroactively classified
-
Opsec requirements to protect yourself from the US govt is very hard, and my current level of technical competence is not enough to add value here. - Disagree
- The whistleblower guide I'm writing explicitly states that the strategy is leaving the US and publishing, not staying anonymous indefinitely. It is assumed the NSA or AI lab internal security will doxx you eventually.
- A lot of people with cybersecurity background are IMO too focussed on ensuring anonymity indefinitely, which I agree is very hard. My strategy requires as much preparation of legal and media training aspects as it does cybersecurity, because it is not a strategy that relies on indefinite anonymity.
- IMO there's atleast 50% chance a top software developer at an AI company can pull this off even without a guide.
- Part of the value of the guide is simply reminding developers that succeeding at this plan is hard but not impossibly hard, and that with some preparation there's a high probability they can pull this off.
- Increasing probability of success from say 50% to 70% makes this guide worth publishing for me. There's a strong case IMO for why this success probability increase directly correlates with reduction in AI x-risk probability.
- One whistleblower's publication of empirical real world data could be worth more for collective epistemology than hundreds of high-status people making speculative arguments.
-
Should support whistleblowers coming out publicly in the US, instead of going to another country to release the documents - Disagree
- I agree that for people who don't leak classified information, staying in the US with a legal defence is an option.
- That requires a very different guide from this one, and it may be worth writing that too. Once I'm done writing this guide I might consider writing that one too.
- There's also a lot more empirical data for that type of guide, because there are more such cases.
- Case studies
- Every single example of leaking US government classified information in past 30 years has led to the whistleblower getting imprisoned, if they leaked US classified information
- Example: Reality Winner, Chelsea Manning
- Only one person leaked US classified information and escaped prison.
- Examples of people who were not imprisoned despite leaking US classified information are >30 years old.
- Example: Daniel Ellsberg, maybe Perry Fellwock
-
Should privately support whistleblowers leaking classified information, but publicly not talk about leaking classified information - Disagree
- Many aspects of the plan depend critically on whether classified information is leaked or not, as most whistleblowers who don't leak classified information don't get imprisoned, and all whistleblowers who leak classified information get imprisoned.
- Having clear writing and thinking on this issue is extremely important. I can't muddy it by replacing "classified" with some euphemism for it.
- I would like to earn the trust of a potential whistleblower without deception.
- In case of a conflict of interest between doing what is right for the whistleblower, versus doing what is right for the journalist / lawyer / funder / etc, I will prioritise the whistleblower.
- Whistleblowers are the people most likely to go to prison, and this is a factor influencing why I prioritise earning their trust over everyone else's.
-
Writing such a guide is too hard. Any whistleblower who needs your guide is going to get themselves arrested anyway. - Disagree
- Since the plan involves leaving the US, I think it's possible for a potential whistleblower to make some opsec mistakes and still escape with their life.
- The guide explicitly states that indefinite anonymity is not possible, and what is at stake is only the time duration before they get doxxed.
- The guide needs to transmit both the opsec mindset, and a specific set of opsec rules. If both are successfully transmitted, the whistleblower can then attempt to adapt any part of it to their unique circumstances.
- Transmitting opsec mindset, not just a set of opsec rules, is hard to do in a short amount of time. I still think it is worth figuring out how to do this. A lot of employees at OpenAI/Anthropic/Deepmind/etc are software developers and teaching them security mindset may be easier to do.
- A high-quality guide for this set of circumstances does not exist on the internet as far as I know. There are a lot of opsec guides on the internet for everyone from software developers to non-classified whistleblowers to dark web drug vendors to hackers. Obtaining information from those guides and distilling it into a clear set of rules requires time and energy the whistleblower may not have. Most likely, they will benefit from a ready-to-go guide.
- The guide will have to be regularly updated with new information as security landscape changes.
-
Hot war between US and China/Russia is very unlikely. US journalists and youtubers can be trusted to publish the documents, non-US journalists don't need to be involved. - Disagree
- I agree there is a significant probability no hot war happens. I think hot war has atleast 10% chance, and is worth preparing for.
- I'm guessing our actual disagreement is on how likely superintelligent AI is to be built in the first place, or something similar.
- It is obvious to me why a intelligence community and AI lab that has succeeded at building aligned superintelligent AI will try to disable the military and nuclear command of the other country, for instance by cyberhacking, hyperpersuasion, bioweapons, nanotech weapons or so on.
- Even if superintelligent AI has not yet been built, if your country has significant chance of building it first, it makes game theoretic sense to pre-emptively escalate military conflict.
- If any country's govt actually gets convinced that superintelligent AI is likely to cause human extinction, they might pre-emptively escalate military conflict to get other govts to stop building it.
- US journalists will have specific pro-US govt biases in how the publish about the piece.
- This could make it harder to convince the general public outside the US of the issue, even they are aware of it.
-
Publishing original redacted documents is not necessary. Journalists writing a propaganda piece on the issue without publishing documents is fine - Disagree
- Counterexamples:
- Kelsey Piper at Vox publishes redacted documents.
- Snowden insisted on transferring documents to journalists and picking journalists who will report the truth honestly and choose what to disclose.
-
Supporting independent whistleblowers is useful, but supporting independent cyberhackers is not useful - Disagree
- Choosing not to support independent cyberhackers significantly reduces the amount of information that can be published.
- Case study
- Guccifer 2.0 leak revealing private emails of Hillary Clinton about Bernie Sanders was most likely obtained by cyberhacking not by a whistleblower
- This had non-negligible influence on 2016 US election
-
Whistleblower providing clearcut evidence will not lead to an AI ban - Disagree
- Examples of specific evidence a whistleblower may uncover:
- Latest capabilities of AI models, including real examples of AI being using for persuasion (including hyperpersuasion), cyberhacking (including discovering novel zerodays in hardware or software), bioweapons R&D (including inventing novel bioweapons), designs for hypothetical future weapons, and so on.
- Private notes such as diaries and emails where leadership of AI company or leadership of government talks about their true values, including whose lives they prioritise above whose.
- Example: 2016 DNC email leak uncovered some of Hillary Clinton's true thoughts on her campaign backers or on Bernie Sanders. This led to non-zero influence on US election in 2016.
- Example: Assange's collateral murder video uncovered true thoughts of those involved in the shooting. This led to non-zero influence on anti-war protests in US, but has not yet significantly changed US foreign policy.
- Since ideologies (including moral ideologies) associated with AI risk are more extreme, the uncovered information on true values of leaders could also be more extreme.
- Full causal chain behind a certain decision, including all the decison-makers involved and how certain stakeholders conspired to take control away from other stakeholders.
- Example: Snowden explains in detail how the US Supreme Court, members of Congress and Senate, judges on the FISA court, US inspector general, internal reporting channels within NSA were all systematically used or hacked, in order to keep secret the extent to which NSA surveillance has pervaded. This has not yet lead to major changes in this issue.
- There may be a similar causal chain behind important decisons taken using AI, where many nominal decision-makers in US democracy are routed around and power is centralised in the hands of a very small number of people.
- See also: AI governance papers posted on Lesswrong on AI-enabled dictatorships and coups.
-
Instead of mass broadcasting whistleblower guide, consider passing a message to AI employees privately - Maybe
- Someone else can take this strategy while I follow my strategy. I think both are worth doing.
- Me starting my project could inspire someone else to collaborate or start similar projects.
-
Finding journalists in non-US-allied states who cover tech accurately and can adopt latest tech tools may be difficult - Agree
- Agree, but still think it is worth trying.
-
Legal expertise is currently missing on the team - Agree
- Will need to collaborate with someone with legal expertise
It is possible there is misalignment of moral values between me and the funder. This is discussed here.
Enter email or phone number to subscribe. You will receive atmost one update per month