…and I still don’t get it. I paid for a month of Pro to try it out, and it is consistently and confidently producing subtly broken junk. I had tried doing this before in the past, but gave up because it didn’t work well. I thought that maybe this time it would be far along enough to be useful.
The task was relatively simple, and it involved doing some 3d math. The solutions it generated were almost write every time, but critically broken in subtle ways, and any attempt to fix the problems would either introduce new bugs, or regress with old bugs.
I spent nearly the whole day yesterday going back and forth with it, and felt like I was in a mental fog. It wasn’t until I had a full night’s sleep and reviewed the chat log this morning until I realized how much I was going in circles. I tried prompting a bit more today, but stopped when it kept doing the same crap.
The worst part of this is that, through out all of this, Claude was confidently responding. When I said there was a bug, it would “fix” the bug, and provide a confident explanation of what was wrong… Except it was clearly bullshit because it didn’t work.
I still want to keep an open mind. Is anyone having success with these tools? Is there a special way to prompt it? Would I get better results during certain hours of the day?
For reference, I used Opus 4.6 Extended.
I use it for tedious transformations or needle ones haystack problems.
They are better at searching for themes or concepts then they are at actually doing any “thinking tasks”. My rule is that if requires a lot of critical thinking then the LLM can do it.
It’s definitely not all they say it is. I think LLMs will fundamentally always have these problems.
I’ve actually had a much better time using it for in line completion as if recent. It’s much better when the scope of the problem it needs to “solve” ( the code it needs to find and compose to complete your line ) is like the Goldilocks zone. And if the answer it gives is bad I just keep typing.
I really hate the way LLM vibe coded slop is written and architecture. To me is clear these things have extremely limited conception. I’ve compared it to essentially ripping out the human language center, giving it a keyboard and asking it to program for you. It’s just no really what it’s good at.