How could all these benchmarks be fake, it’s a mystery
https://www.youtube.com/watch?v=KcYZN6sTZjQ&list=UU9rJrMVgcXTfa8xuMnbhAEA - video
https://pivottoai.libsyn.com/20251106-oxford-pretends-ai-benchmarks-are-science-not-marketing - podcast
time: 6 min 16 sec
Reasoning With Machines doesn’t work on reasoning, really. It’s almost entirely large language models — chatbots. Because that’s where the money — sorry, the industry interest — is. But this paper got into the NeurIPS 2025 conference.
A reminder that the NeurIPS FAQ for reviewers says that “interactions with LLMs” are an acceptable way “to enhance your understanding of certain concepts”. What other big conferences are there… AAAI 2026, you say?
AAAI-26 will follow a two-phase reviewing process as in previous years, with two additions: an additional AI-generated review in Phase 1, and an AI-generated summary of the discussions at the end of the discussion phase. The AI-generated content is being used as part of a pilot program to evaluate the ability of AI tools to assist in the peer review process.
I’m gonna say it: The entire “artificial intelligence”/“machine learning” research field is corrupt. They have institutionally accepted the bullshit fountain as a tool. It doesn’t matter if they’re only using chatbots as a “pilot program”; they’ve bought into the ideology. They’ve granted fashtech a seat at the bar and forced all the other customers to shake its hand.
Whatever marginal utility genAI has in mathematics, like being a shitty version of a semantic search engine, is outweighed by the damage it is doing to critical thought at large. “Ooh, the bus to the math conference runs so smoothly on this leaded gasoline!”
I’m gonna say it: The entire “artificial intelligence”/“machine learning” research field is corrupt. They have institutionally accepted the bullshit fountain as a tool. It doesn’t matter if they’re only using chatbots as a “pilot program”; they’ve bought into the ideology. They’ve granted fashtech a seat at the bar and forced all the other customers to shake its hand.
Abso-fucking-lutely. Oxford’s latest “research paper” isn’t marketing - its propaganda. Propaganda for bullshit fountains, and for the ideology which endorses them.
