Not only would an engineer not need to consider the abstraction layer between their input and the code but they would be unable to fully interrogate that abstraction because the code extruder does not need to show its work.
I think you’re actually right on the money here, nowhere near delusional, especially since you come from a Lisp background. I really appreciate Lisp (and Smalltalk) for the “live-coding” and universal inspectability/debuggability aspects in the tooling. I appreciate test-driven development as I’ve seen it presented in the Smalltalk context, as it essentially encourages you to “program in the debugger” and be aware of where the blank spots in your program specification are. (Although I’m aware that putting TDD into practice on an industrial scale is an entirely different proposition, especially for toolchains that aren’t explicitly built around the concept.)
However, LLM coding assistants are, if not the exact opposite of this sort of tooling, something so far removed as to be in a different and more confusing realm. Since it’s usually a cloud service, you have no access to begin debugging, and it’s drawing from a black box of vector weights even if you do have access. If you manage to figure out how to poke at that, you’re then faced with a non-trivial process of incremental training (further lossy compression) or possibly a rerun of the training process entirely. The lack of legibility and forthright adaptability is an inescapable consequence of the design decision that the computer is now a separate entity from the user, rather than a tool that the user is using.
I’ve posed the question in another slightly less skeptical forum, what advantage do we gain from now having two intermediate representations of a program: the original, fully-specified programming language, as well as the compiler IR/runtime bytecode? I have yet to receive a satisfactory answer.
A lot of younger people who are being conditioned to accept this stuff just weren’t around to experience how unstable and unreliable the vast majority of PC software was in the 1990s, and a lot of more senior-level people must have willfully forgotten. I’ve been thinking about this more and more lately. The difference was that in the 90s, the major PC companies could port their enterprise-grade OSes with proper memory protection down to the consumer level, as hardware advanced and running a more complex OS kernel was no longer a big demand. Even then, it was an uphill battle, especially once you threw widespread networking and dubious internet-sourced malware into the equation.
End-user software has already seen a decline in quality and increase in user frustration during the cloud era, as many apps have become siloed blobs of JavaScript running on top of an extra copy of your web browser engine. I’m concerned that we’re headed firmly back to the bad old days now, without the release valve of better underlying software stacks on the horizon. The main solution will likely be to rip a lot of this crap out and start over (which is already a pretty widespread approach anyway, my credit union is going on their 3rd online banking “upgrade” in 5 years). But that completely zeroes out the “productivity” gains, not that anyone touting such things will ever measure it that way. I suppose the cost of re-stabilizing the software industrial base will be counted as GDP gains instead.