cm0002@suppo.fi to Programmer Humor@programming.dev · 1 month agoTech bros having their production environments nuked by AI is never gonna get oldsuppo.fiimagemessage-square160fedilinkarrow-up1947
arrow-up1947imageTech bros having their production environments nuked by AI is never gonna get oldsuppo.ficm0002@suppo.fi to Programmer Humor@programming.dev · 1 month agomessage-square160fedilink
minus-squareegregiousRac@piefed.sociallinkfedilinkEnglisharrow-up15·1 month agoWhat evaluation setup lets the thing being tested see the test results and gives access to the test source code?! Slop evaluating slop.
minus-squaremyotheraccount@lemmy.worldlinkfedilinkarrow-up7·1 month agoThe model is so smart, we let it write it’s own evaluation setup /s
minus-squareAwesomeLowlander@sh.itjust.workslinkfedilinkarrow-up1·1 month agoGiven that they mark it as cheating, isn’t that essentially part of the test in itself?
What evaluation setup lets the thing being tested see the test results and gives access to the test source code?!
Slop evaluating slop.
The model is so smart, we let it write it’s own evaluation setup /s
Given that they mark it as cheating, isn’t that essentially part of the test in itself?