Wasn’t it moreso that they were doing it in his father’s house, though?

“which you have made a den of thieves!”

yeah, it’s extra bad that they were doing it in the temple, but jesus was pretty clearly condemning the moneychangers as thieves