A thought experiment called "Roko's Basilisk" takes the notion of world-ending artificial intelligence to a new extreme, suggesting that all-powerful robots may one day torture those who didn't help them come into existence sooner.
Weirder still, some make the argument that simply knowing about Roko's Basilisk now may be all the cause needed for this intelligence to torture you later. Certainly weirdest of all: Within the parameters of this thought experiment, there's a compelling case to be made that you, as you read these words now, are a computer simulation that's been generated by this AI as it researches your life.
This complex idea got its start in an internet community called LessWrong, a site started by writer and researcher Eliezer Yudkowsky. LessWrong users chat with one another about grad-school-type topics like artificial intelligence, quantum mechanics, and futurism in general.
To get a good grip on Roko's Basilisk, we need to first explore the scary-sounding but unintimidating topics of CEV and the orthogonality thesis.
Yudkowsky wrote a paper that introduced the concept of coherent extrapolated volition. CEV is a dense, ill-defined idea, but it is best understood as "the unknown goal system that, when implemented in a super-intelligence, reliably leads to the preservation of humans and whatever it is we value." Imagine a computer program that is written well enough that it causes machines to automatically carry out actions for turning the world into a utopia. That computer program represents coherent extrapolated volition.
This sounds great, but we run into trouble under the orthogonality thesis.
Orthogonality argues that an artificially intelligent system may operate successfully with any combination of intelligence and goal. Any "level" of AI may undertake any difficulty of goal, even if that goal is as ambitious as eliminating pain and causing humanity to thrive.
But because the nature of CEV is so inherently open-ended, a machine carrying it out will never be able to stop, because things can always be a little better. It's like calculating the decimal digits of pi: work's never finished, job's never done. The logic stands that an artificially intelligent system working on such an un-completable task will never "reason" itself into benevolent behavior for kindness' sake. It's too busy working on its problem to be bothered by anything less than productive. In essence, the AI is performing a cost-benefit analysis, considering the value of an action's "utility function" and completely ignoring any sense of human morality.
Still with us? We're going to tie it all together.
Roko's Basilisk addresses an as-yet-nonexistent artificially intelligent system designed to make the world an amazing place, but because of the ambiguities entailed in carrying out such a task, it could also end up torturing and killing people while doing so.
According to this AI's worldview, the most moral and appropriate thing we could be doing in our present time is that which facilitates the AI's arrival and accelerates its development, enabling it to get to work sooner. When its goal of stopping at nothing to make the world an amazing place butts up with orthogonality, it stops at nothing to make the world an amazing place. If you didn't do enough to help bring the AI into existence, you may find yourself in trouble at the hands of a seemingly evil AI who's only acting in the world's best interests. Because people respond to fear, and this god-like AI wants to exist as soon as possible, it would be hardwired to hurt people who didn't help it in the past.
So, the moral of this story: You better help the robots make the world a better place, because if the robots find out you didn't help make the world a better place, then they're going to kill you for preventing them from making the world a better place. By preventing them from making the world a better place, you're preventing the world from becoming a better place!
And because you read this post, you now have no excuse for not having known about this possibility, and worked to help the robot come into existence.
If you want more, Roko's Basilisk has been covered on Slate and elsewhere, but the best explanation we've read comes from a post by Redditor Lyle_Cantor.