It looks like “AI bad” or “Claude insecure” mantra.
Until you solve prompt injection, they are indeed extremely bad for security and should never be given permissions that would allow them to do anything catastrophic.
I say mantra because there is a large amount of people just hating AI outright, without a grounded reasoning.
Granted, coding agents are insecure by default - they are built to execute remote code - but that does not mean they are generally useless/harmful/bad. I run them in a container, with access to the codebase only.
Also, they hallucinate, produce over-convoluted abstractions, do not know when to reject instead of blindly trying to find a way trough a brick wall.
But also, they can answer questions about gigantic codebases way faster than I could. They can generate tests, find missing test coverage, review code, and many other things.
The way LLMs work is that they actively will make multiple attempts to get past hurdles (because they have no intelligence or methodology) so guardrails need to be extremely tight for them to work, other wise the model will simply see it as one of the challenges to overcome.
That’s the mantra, and that is very poor technology to put in the hands of people who don’t understand how it works.
It’s probably something like “I’ve disabled agent’s
removeFiletool, but LLM figured out that it can use thebashtool, still”.It looks like “AI bad” or “Claude insecure” mantra.
you mean facts?
“It’s my circlejerk - so it’s a fact!”
I hope that you’re hired for long enough to learn what having security means in the context of using LLM “agents” and the like.
deleted by creator
Until you solve prompt injection, they are indeed extremely bad for security and should never be given permissions that would allow them to do anything catastrophic.
I say mantra because there is a large amount of people just hating AI outright, without a grounded reasoning.
Granted, coding agents are insecure by default - they are built to execute remote code - but that does not mean they are generally useless/harmful/bad. I run them in a container, with access to the codebase only.
Also, they hallucinate, produce over-convoluted abstractions, do not know when to reject instead of blindly trying to find a way trough a brick wall.
But also, they can answer questions about gigantic codebases way faster than I could. They can generate tests, find missing test coverage, review code, and many other things.
The way LLMs work is that they actively will make multiple attempts to get past hurdles (because they have no intelligence or methodology) so guardrails need to be extremely tight for them to work, other wise the model will simply see it as one of the challenges to overcome.
That’s the mantra, and that is very poor technology to put in the hands of people who don’t understand how it works.
deleted by creator