what is the correct monkey paw threshold?

One of the great “be careful what you wish for” stories is The Monkey’s Paw in which a family receives a magic item that grants wishes but discovers to their horror that all the wishes are granted in terrible, horrible ways. I can’t remember when I last read the story (though I’m confident I have—maybe in high school?), but monkey paw has stuck in my brain as the metaphor for this idea that wishes can go terribly, terribly wrong, so you really ought to think them through.

One of my favorite variations on this metaphor comes from an episode of the Total Party Kill actual play podcast from The Incomparable entitled “Professional Monkey Paw Operator.” The episode title comes from the clip below, where a player is granted access to a powerful spell that will allow them to bring a party member back to life; because she is very aware of the ways that the Dungeon Master could really mess this up for her, the wording of her wish is hilariously exact.

Generative AI gets me thinking about monkey paws a lot. Reading stories like this article from 404 Media—“Massive Leak Shows Erotic Chatbot Users Turned Women’s Yearbook Pictures Into AI Porn”—fills me with rage and makes me think that if I had access to a monkey paw, I’d be really, really tempted to use it to make generative AI disappear from the face of the earth. Yet, I’m also very aware of the lesson that the monkey paw teaches, that for all of my rage, there could be unintended consequences to this.

First of all, how does one define generative AI? Definitions are famously a problem in the AI industry, with many companies having artificial general intelligence as an explicit goal of theirs… but also struggling to define what it is. (Nilay Patel and David Pierce had fun with this on a recent episode of the Vergecast). AGI and genAI are (clearly) not the same thing, but definitions are hard, and I expect that a definition for generative AI is harder than many definitions. What could I inadvertently wish off the face of the earth by making a rash monkey paw wish about generative AI?

Second, as grumpy as I am about generative AI, I (think) I have the presence of mind to recognize the ways that a cure can be worse than the disease. For example, I’m generally opposed to running student work through an AI detector, because I think cultures of surveillance and consequences of false positives are potentially a bigger problem than using generative AI to do your schoolwork. Likewise, Mike Masnick has written repeatedly (including yesterday about how increasing copyright protections to counter generative AI scraping may create more problems than it solves. For all that I’m angry about generative AI, I think Masnick is right here.

Ultimately, though, this post is less about trying to wrestle with generative AI and more about the idea of a “monkey paw threshold.” At what point would I (or someone else) be justified in making a brash wish in response to a clear problem even though there may be unintended consequences? When it comes to generative AI, I think there are clear problems to the technology and I don’t see a whole lot of societal benefits to the tool (at least as its currently deployed and marketed), and yet I would still be hesitant before trying to monkey paw it away—even at my angriest, I don’t know that we’ve crossed the threshold. The Simpsons’ take on the monkey paw story, on the other hand, ends with Homer angry that the turkey sandwich he carefully wished for was a little too dry—sure, things didn’t turn out the way he expected, but it worked out a lot better than the family’s previous wishes. This seems like a reasonable use of a monkey paw, but a turkey sandwich isn’t exactly a pressing need, either.

No answers in this post, but I’m intrigued by the idea of a monkey paw threshold for helping someone try to think through when a problem is big enough that unintended consequences may be worth it. Maybe you can never reach the monkey paw threshold (after all, part of my anger with generative AI is precisely AI companies’ seeming disregard for unintended consequences), but it’s an interesting question to puzzle out.

• monkey paw
• monkey paw threshold
• generative AI
• Total Party Kill
• The Incomparable
• TTRPGs
• actual play
• David Pierce
• Nilay Patel
• The Verge
• AGI
• copyright
• intellectual property