cross-posted from: https://sh.itjust.works/post/63615439
The reality of how much damage AI mania is doing to our ability to run institutions effectively.
cross-posted from: https://sh.itjust.works/post/63615439
The reality of how much damage AI mania is doing to our ability to run institutions effectively.
What is puzzling to me is there seem to be no companies which take a deliberate, measured, experimental approach.
No “You can use that but you have to report on difficulties or disadvantages”.
No “you can use it but not when correctness is critical, errors could harm people, or it can directly affect things critical for our business”.
No “Give us first an estimate in how long it will take, and then we will roll a dice, decide if it will be done using AI, and after 50 tries we will compare whether it makes a difference”.
No “You can use AI, but you are responsible that the result works and is correct, no slack on this, and failure to do so gives you minus points”.
No “You can use it, your max token budget is 100 USD per month”.
No “You can use it, but do not input any company secrets, confidential data, medical records, or things like passwords and security-relevant data”.
There seems to be zero risk management.
Imagine pesticides, radioactive substances, or highly toxic chemicals would have been introduced in such a manner.
See also this book chapter by Baldur Bjarnason.
Companies doing that aren’t making headline worthy mistakes
Our company develops software for medical devices. Risk assessment is everything, and we currently do “You can use AI, but you are responsible that the result works and is correct” and “You can use it, but do not input any company secrets, confidential data, medical records, or things like passwords and security-relevant data”.
I guess the difference is we’re too busy making things and we don’t have the time to brag about AI on social media? Also we’re small and don’t have the reach anyway.
And it’s not “news” in the same way that “I drove to work today without incident” doesn’t make headlines.
Why don’t you think it’s not happening? Do you think headlines are the best way to determine what’s happening in the world?
Well, show me any job ads that tell this.
Or is this somehow hidden in some kind of cat whistle?
Sadly, there is no need to imagine. There is no limit to people’s greed. And guardrails are only installed after there is no way to hide damage done.
My employer does the first, second, fourth, and sixth of those. (And I’m in the uncomfortable position of simultaneously building internal tools to make our AI use more effective, and telling everyone who will listen that we’re just walking blindly into yet another Oracle or Broadcom situation.)
These are all policies at the company I work for. Only its $100 AUD, not USD.