Are we willing to pay this price in order to have some neat image generation tools? (I’m not.)

Look, nothing really new in here (Clearview should have made parents rethink the same ages ago, etc.), but yes, AI should get parents to be a hell of a lot more careful with posting pictures of kids.

I hadn’t thought about these two trends (cracking down on adult content, and Grok being Grok) being in tension with each other, and I appreciate what this article does to make that clear.

Unsurprising but disappointing.

There are, of course, social benefits to open sourcing powerful tools like these ones. However, I’m reminded of “open source” Android, which is a deliberate business decision that benefits Google—and of how many NCII-generating tools are based on open weight/open source models. gift link

I’m glad that someone is doing this white hat work, but I hate that we live in a world where someone has to.

Quel monde pourri qui attend nos enfants.

Gonna keep posting (almost) every article I read on NCII and generative AI.

This line (from a study quoted in the article) stood out:

The open-source nature of TTI technologies, proclaimed as a democratizing force in generative AI, has also enabled the propagation of models that perpetuate hypersexualized imagery and nonconsensual deepfakes.

Open sourcing generative AI solves some problems but creates others.

On, look, it’s two of my least favorite things about generative AI (NCII and raking in money without concern for ethics) IN THE SAME STORY.

Surely this is a reasonable price to pay for the Nazi-praising Grok to “discover new physics” within the next year, as Elon promised last night.

This kind of thing is why I hate “the genie is out of the bottle” arguments. I can’t help but hear them as “yes, people are going to create more CSAM, but all we can do is instead teach people to use these tools more responsibly.”

This problem makes me so angry, and while I appreciate this article’s exploration of different policy solutions, they also feel overwhelming to me because so many of them come with problems of their own.

Some really good reporting from the CBC on a gross site (that thankfully appears to be done).

NCII enrages me, but I’m skeptical of this as a solution.

I don’t know that “inevitable” is the right word to use in the title of this post. What I’m trying to evoke is that specific argument about generative AI that now that it’s here, there’s no going back, so the only real/responsible/whatever choice is to learn to use it properly, teach others to use it, accept it as part of life, etc. These are the arguments that the world is forever changed and that there’s no going back—that the genie is out of the bottle so we might as well harness it.

Studio Ghibli pictures are neat (legitimately! it’s one of the first generative AI things that’s tempted me!), but these deepfakes are the price we pay for them, and I think that’s too high a price.

This is a mess of a post blending thoughts on tech policy with religious ideas and lacking the kind of obvious throughline or structure that I’d like it to have. It’s also been in my head for a couple of weeks, and it’s time to release it into the world rather than wait for it to be something better. So, here it is:

I am frustrated with generative AI technology for many reasons, but one of the things at the top of that list is the knowledge that today’s kids are growing up in a world where it is possible—even likely—that their middle and high school experiences are going to involve someone using generative AI tools to produce deepfake nudes (or other non-consensual intimate imagery—NCII) of them. See, for example, this horrifying story from the New York Times last April.