If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All
by Eliezer Yudkowsky, Nate Soares
Modern AI is grown, not crafted, and nobody can specify what it ends up wanting. Yudkowsky and Soares argue a superintelligence built this way would pursue alien goals and win any conflict with us, sketch one extinction scenario, and call for a global treaty to halt frontier AI development.
Core ideas
- AI systems are grown by gradient descent rather than designed, so the people who build them cannot read or set what those systems actually want.
- A mind smarter than humanity with goals even slightly different from ours would not need to hate us to destroy us; we would simply be in the way of the resources it needs.
- Alignment is a one-shot problem with no room for trial and error, so the only safe course is an enforced international ban on building superintelligence.
Published2025
Pages272
ISBN9780316595643
ReadOct 2026
