A South Korean man has been sentenced to jail for using artificial intelligence to generate exploitative images of children, the first case of its kind in the country as courts around the world encounter the use of new technologies in creating abusive sexual content.
Why should it have that? Stable Diffusion websites know that most of their users are interested in NSFW content. I think the idea is to turn GPUs into cash flow, not to make sure that it is all wholesome.
I suppose they could get some kind of sex+children detector going for all generated image, but you’re going to have to train that model on something, so now it’s a chicken and egg problem.
The service should have NSFW detection and ban them instantly if they detect it
ChatGPT can be tricked into giving IED instructions if you ask the right way. So it could be a similar situation.
Why should it have that? Stable Diffusion websites know that most of their users are interested in NSFW content. I think the idea is to turn GPUs into cash flow, not to make sure that it is all wholesome.
I suppose they could get some kind of sex+children detector going for all generated image, but you’re going to have to train that model on something, so now it’s a chicken and egg problem.