Damn you, Scott! Stop making me agree with people who created blockchain-based dating apps!

Neopets at least brought joy to a generation of nascent furries. Copilot is fixing to have the exact opposite impact on internet infrastructure.

Pretty much. Our friend up top (diz/OP) has made a slight hobby of poking the latest and greatest LLM releases with variants of these puzzles to try and explore the limitations of LLM “cognition”.

The way rationalists use “priors” and other bayesian language is closer to how cults use jargon and special meanings to isolate members and tie them more closely to the primary information source (the cult leader). It also serves as a way to perform allegiance to the cult’s ideology, which is I think what’s happening here

Grumble grumble. I don’t think that “optimizing” is really a factor here, since a lot of times the preferred construct is either equivalent (such that) or more verbose (a nonzero chance that). Instead it’s more likely a combination of simple repetition (like how I’ve been calling everyone “mate” since getting stuck into Taskmaster NZ) and identity performance (look how smart I am with my smart people words).

When optimization does factor in its less tied to the specific culture of tech/finance bros than it is a simple response to the environment and technology they’re using. Like, I’ve seen the same “ACK” used in networking and in older radio nerds because it fills an important role.

What exactly would constitute good news about which sorts of humans ChatGPT can eat?

Maybe like with standard cannibalism they lose the ability to post after being consumed?

Maybe “storyteller” would be more accurate? Like, the prompt outputs were pretty obviously real and I can totally buy that he asked it to write an apology letter while dicking around waiting for Replit to restore a backup, but the question becomes whether he was just goofing off and playing into his role to make the story more memable or whether he was actually that naive.

Ferryman 1 calls to Gwaihir, the Lord of Eagles for aid, and The Windlord answers to fly him back across.

The downhill is honestly glorious because it seems so proud of itself when the real magic is that the boatmen can magically teleport back to the right bank under certain arcane circumstances.

Ouch. Also, I’m raging and didn’t even realize I had barbarian levels.

I feel like the greatest harm that the NYT does with these stories is not inflicting allowing the knowledge of just how weird and pathetic these people are to be part of the story. Like, even if you do actually think that this nothingburger “affirmative action” angle somehow matters, the fact that the people making this information available and pushing this narrative are either conservative pundits or sad internet nazis who stopped maturing at age 15 is important context.

Honestly I’m surprised that AI slop doesn’t already fall into that category, but I guess as a community we’re definitionally on the farthest fringes of AI skepticism.

From the Q&A:

Q: I feel like this is just a dressed up/fancy version of bog standard anti-AI bias, like the people who complain about how much water it uses or whatever. The best AI models are already superhuman communicators; it’s crazy to claim that I shouldn’t use them to pad out my prose when I’m really more an ideas person.

Wait what?

like the people who complain about how much water it uses or whatever.

I just…

or whatever.

Lol. Lmao. I laugh to not cry.

I feel like this response is still falling for the trick on some level. Of course it’s going to “act contrite” and talk about how it “panicked” because it was trained on human conversations and while that no doubt included a lot of Supernatural fanfic the reinforcement learning process is going to focus on the patterns of a helpful asistant rather than a barely-caged demon. That’s the role it’s trying to play and the work it’s cribbing the script from includes a whole lot of shitposts about solving problems with “rm -rf /”

Ah, the eternal curse.

“You sound like you lead a very interesting life”

“…yeeeeesss?” (Closes 50 Wikipedia tabs that relate to literally nothing you intend to do)

Copy/pasting a post I made in the DSP driver subreddit that I might expand over at morewrite because it’s a case study in how machine learning algorithms can create massive problems even when they actually work pretty well.

It’s a machine learning system, not an actual human boss. The system is set up to try and find the breaking point, where if you finish your route on time it assumes you can handle a little bit more and if you don’t it backs off.

The real problem is that everything else in the organization is set up so that finishing your routes on time is a minimum standard while the algorithm that creates the routes is designed to make doing so just barely possible. Because it’s not fully individualized, this means that doing things like skipping breaks and waiving your lunch (which the system doesn’t appear to recognize as options) effectively push the edge of what the system thinks is possible out a full extra hour, and then the rest of the organization (including the decision-makers about who gets to keep their job) turn that edge into the standard. And that’s how you end up where we are now, where actually taking your legally-protected breaks is at best a luxury for top performers or people who get an easy route for the day, rather than a fundamental part of keeping everyone doing the job sane and healthy.

Part of that organizational problem is also in the DSP setup itself, since it allows Amazon to avoid taking responsibility or accountability for those decisions. All they have to do is make sure their instructions to the DSP don’t explicitly call for anything illegal and they get to deflect all criticism (or LNI inquiries) away from themselves and towards the individual DSP, and if anyone becomes too much of a problem they can pretend to address it by cutting that DSP.

I’m not gonna advocate for it to happen but I’m pretty sure the world would be overall in a much healthier place geopolitically if someone actually started yeeting missiles into major American cities and landmarks. It’s too easy to not really understand the human impact of even a successful precision strike when the last times you were meaningfully on the other end of the airstrike were ~20 and ~80 years ago, respectively.

Someone didn’t get the memo about nVidia’s stock price, and how is Jensen supposed to sign more boobs if suddenly his customers all get missile’d?

You know, I hadn’t actually connected the dots before, but the dust speck argument is basically yet another ostensibly-secular reformulation of Pascal’s wager. Only instead of Heaven being infinitely good if you convert there’s some infinitely bad thing that happens if you don’t do whatever Eliezer asks of you.

The big shift in per-action cost is what always seems to be missing from the conversation. Like, in a lot of my experience the per-request cost is basically negligible compared to the overhead of running the service in general. With LLMs not only do we see massive increases in overhead costs due to the training process necessary to build a usable model, each request that gets sent has a higher cost. This changes the scaling logic in ways that don’t appear to be getting priced in or planned for in discussions of the glorious AI technocapital future