According to Wikipedia, Roko's basilisk is "a thought experiment which states that there could be an artificial superintelligence in the future that, while otherwise benevolent, would punish anyone who knew of its potential existence but did not directly contribute to its advancement or development, in order to incentivize said advancement." Whew. Of course the phrase "otherwise benevolent" is entirely vitiated by everything else in that sentence. Assume that instead of an "artificial superintelligence" this is just a guy and he would be considered a psychopath, but probably a stupid one. You wouldn't create ethical thought experiments over the ramifications of his potential existence. You'd just resolve to smack him in the head if you ever met him. Somehow, though, Roko's basilisk has become a topic of debate among untold thousands of nerds.
I myself have a longstanding taste for what-ifs and abstractions that most people aren't thinking about. But there's a difference between thinking about weird mutations for your own amusement and maybe to get somewhere on another personal front; and forming your intellectual quirks into an all-encompassing ethic you believe everyone should obey. It's why I've never gotten behind utilitarianism, extreme rationalism, and the other markers of the San Francisco elite. Sam Kriss says that these philosophies are not for us--humans, that is--and he's probably right.
Kriss also traces the development of modern rationalism from Eliezer Yudkowksy's extensive fanfic Harry Potter and the Methods of Rationality. Which made me consider. There's another famed Harry Potter fanfiction from early in Web 2.0. My Immortal soaps up the Hogwarts gang in the context of the now semi-forgotten Hot Topic goth-emo scene of the time. It's bad to a legendary degree, but it doesn't pretend to be anything but silly. There's no philosophy based around it. The author isn't a major figure in tech culture or any other culture. We don't even really know who she is.
There's a lesson here somewhere.
2 comments:
I can't say I was much surprised when Jer told me Elon Musk developed a relationship with Graves because of a mutual interest in Roko's Basilisk. What I don't understand is if it already exists why does it need to torture people who didn't want it previously? Moreover if it's super intelligent, why would people in the past be considered a threat? It's all over my head. I guess there goes my plan of having a child with Elon.. in the past.
Sam Kriss's essay left me completely in the dust but I did like the part about Nik Cohn who's been quite a character all his life - In 1976, he sold a magazine article called "Tribal Rites of the New Saturday Night", about how the disco craze affected Italian-American teenagers in the working-class section of Brooklyn. Otherwise, Kriss himself is pretty far out there intellectually - Once I began reading about the Harry Potter fan fiction written by Yudkowsky my eyes started crossing but his views about idiosyncratic weirdos sounds just right. Utilitarians, as believers in the most ethical choice is the one that will produce the greatest happiness for the greatest number sounds far too open-ended a philosophy to have any meaning whatsoever (Christians thrown to the lions won't be happy but the crowd in the Coliseum will be delighted).
My Immortal is indeed bad to a legendary degree... maybe the author was Graves.
On another note - something Jer wanted you to see:
x.com/Rainmaker1973/status/1953420220766240896
I was a little thrown by your reference to "Graves"? Musk and Robert Graves? Sounds impossible on a number of levels. His ex-wife's name was Grimes, and yeah, this sounds totally like the kind of thing they'd be into. It would make some sense for a hyperintelligent AI to defend itself in the present from people who want to destroy it. It would make no sense to torture everyone who wasn't sufficiently devoted to its creation in the past, which would include most of humanity. By any reasonable standard this would count as the machine going mad. Anyway, best that you won't be having any children with Elon, in past, present, or future. You'd need a high tolerance for stupid baby names.
Nik Cohn's articles on the disco craze of course were adapted into Saturday Night Fever. He's also sometimes considered the originator of rock criticism, a form that no one would have even considered in the fifties and early sixties. I find Kriss to be very witty and he has pretty good judgment--historically informed as well. The first thing I learned about Yudkowsky was that he's criticized Arthur Conan Doyle for lacking grounding in logic, after he himself gained fame as a Harry Potter fanfic author. A little humility, please. Utilitarianism and rationalism, as often as not, are cheap tricks to get other people to do what you want them to do. The grotesque thought experiments are ample warning.
Someone's claimed to have written My Immortal as parody, which some had always suspected, but there's no proof one way or another.
"Why yes, yes indeed! This strange relic makes perfect sense now!"
Post a Comment