Because LLMs are generating new tokens based on previous tokens, you can “lead” LLMs by crafting “begs the question” style messages. It’s why LLM jailbrakes work. Larger LLMs can be more resistant but are not immune. All of this is because LLMs do not distinguish between its own messages and those of the user. It’s all one giant blob of text it is told to generate even more text repeatedly in response to user input.

I dunno if torturing something that simulates humanity is healthy.

I guess the same could be said about violence in videogames though.

This is much more interesting observation because you are absolutely right. We know what we’re interacting with is fake (like the above), and yet there is something…

I’m too tired to really pull on this thought tonight, but I think there is an interesting psychological phenomenon at work here that various AIs over the decades pull on.