A few years ago, a lone programmer named t0st did something extraordinary: he fixed an 8-year-old bug in GTA Online that had been driving players crazy. The bug? Painfully long load times, sometimes u
It’s a big company problem. Here’s why even obvious bugs like this one slip through the cracks:
The Tyranny of “Requirements”
In large organizations, everything revolves around the roadmap. If a bug fix isn’t tied to a specific requirement or feature, it gets labeled as “tech debt” and shoved to the bottom of the backlog. And let’s be honest: “tech debt” is corporate-speak for “we’ll deal with this never.”
The Rotating Door of Ownership
Over eight years, developers and product managers come and go. The person who originally filed the ticket? Long gone. The person who understood the issue? Moved on to another project. Institutional memory fades, and the ticket becomes a relic of the past. Even if the problem is still very much alive.
The Myth of “Quick Fixes”
A 13-line patch might seem trivial, but in a legacy codebase, even small changes can have unintended consequences. Without proper tests or documentation, developers are often hesitant to touch old code. The risk of breaking something far outweighs the reward of fixing a non-critical bug.
The Invisible ROI
Let’s be real: improving load times doesn’t directly impact the bottom line. Selling Shark Cards (GTA’s virtual currency) does. Companies optimize for metrics that show up on quarterly earnings calls, not for goodwill or user experience, until it’s too late.
As a lead, I’ve recently started finding time to make progress on a lot of this, because a lot of the stuff I’ve seen over the years has just never been prioritized. Over the last few weeks I have:
dramatically reduced load time for a resource our support team uses
significantly cut resource wastage for devs on a handful of our microservices, with one small change
fixed a huge gap in test coverage on code that’s >5 years old, and fixed a couple bugs at the same time
cleaned up a bunch of small tech debt nonsense
You can do this as well. The problem is that I don’t get any recognition for it, so this is completely driven by me making time for it and slipping it in w/ other changes. I document the more important ones, but I’m taking a risk w/ these fixes since any bugs I create in the process will not look good.
Whenever someone else on the team does something like this and is keeping up w/ their work, I try to lavish praise on them in the hopes that maybe they’ll do it again.
Whenever the topic comes up with leadership, I try to explain it in financial terms.
Tech debt is just like financial debt. There are times when its appropriate and necessary to take on some debt. But debt accumulates interest charges. If we just keep building up more debt without ever paying it down, it’ll eventually bankrupt the company.
The engineering team doesn’t always know when the finance team is accumulating debt or paying it back, but we trust that they are doing so appropriately.
You don’t always know when the engineering team is accumulating or paying off debt, but you need to trust that when we say we need time to to pay down tech debt, we’re serious. We’ll all be out of a job if we don’t.
They don’t usually like to hear it, but when put in those terms, they don’t have an argument against it. I’m sure if we could provide a statement showing we had $478,562.78 in tech debt at 4.75%, they’d be more understanding about paying it down.
Fortunately, my boss is sympathetic, but unfortunately, our stakeholders aren’t. We are given time for tech debt (10-20%), but that hasn’t been achievable recently due to a huge focus on delivery. We just reorged, and we’re trying to get a bunch of people new to our business unit on-boarded w/ our product, and they have a variety of requirements that we need to meet for that to happen. Things should settle down in a few months, but in the meantime, I try to fix some low-hanging fruit that directly improves morale for the team.
My org is better than most, I think, but it’s still an issue here. It’s absolutely crazy the difference between prioritized and unprioritized tech debt. For example, our architect wants a thing, so it gets done same day. Our dev team wants a thing, and it takes months, if not years. It’s getting better, but like anything, it’s two steps forward (finally got to trunk-based dev) and one step back (other teams whining about changes).
I think everywhere I’ve worked has said “we have 20% time for tech debt” but has never actually done that. It’s always “we need to ship this by end of week” and “the CEO wants us to add the thing we said we’d cut so we could make the deadline”.
So far, we’ve only had one or two deadlines come from outside the dev team, and up until this year, we got close to that 20%. So hopefully it’s a temporary change due to our new CEO trying to catch up to our competitors in a few areas.
I recently read that’s one of the reasons Windows is so messy and bloated, time goes by and so do devs and managers, so it gets increasingly difficult to know what does what, and why.
That’s true of any large old piece of software, I sometimes read my own code written a few months ago that I’ve forgotten and need to spend time to understand what it’s doing, imagine reading someone elses code written years ago. Companies don’t incentive good documentation or comments, and rarely have I seen proper coding standards enforced, so you end up with a lot of spaghetti code, with 600 line methods that do too many things and there may or may not be proper unit testing that covers this code thoroughly
The Tyranny of “Requirements”
In large organizations, everything revolves around the roadmap. If a bug fix isn’t tied to a specific requirement or feature, it gets labeled as “tech debt” and shoved to the bottom of the backlog. And let’s be honest: “tech debt” is corporate-speak for “we’ll deal with this never.”
The Rotating Door of Ownership
Over eight years, developers and product managers come and go. The person who originally filed the ticket? Long gone. The person who understood the issue? Moved on to another project. Institutional memory fades, and the ticket becomes a relic of the past. Even if the problem is still very much alive.
The Myth of “Quick Fixes”
A 13-line patch might seem trivial, but in a legacy codebase, even small changes can have unintended consequences. Without proper tests or documentation, developers are often hesitant to touch old code. The risk of breaking something far outweighs the reward of fixing a non-critical bug.
The Invisible ROI
Let’s be real: improving load times doesn’t directly impact the bottom line. Selling Shark Cards (GTA’s virtual currency) does. Companies optimize for metrics that show up on quarterly earnings calls, not for goodwill or user experience, until it’s too late.
As a lead, I’ve recently started finding time to make progress on a lot of this, because a lot of the stuff I’ve seen over the years has just never been prioritized. Over the last few weeks I have:
You can do this as well. The problem is that I don’t get any recognition for it, so this is completely driven by me making time for it and slipping it in w/ other changes. I document the more important ones, but I’m taking a risk w/ these fixes since any bugs I create in the process will not look good.
Whenever someone else on the team does something like this and is keeping up w/ their work, I try to lavish praise on them in the hopes that maybe they’ll do it again.
Whenever the topic comes up with leadership, I try to explain it in financial terms.
They don’t usually like to hear it, but when put in those terms, they don’t have an argument against it. I’m sure if we could provide a statement showing we had $478,562.78 in tech debt at 4.75%, they’d be more understanding about paying it down.
Fortunately, my boss is sympathetic, but unfortunately, our stakeholders aren’t. We are given time for tech debt (10-20%), but that hasn’t been achievable recently due to a huge focus on delivery. We just reorged, and we’re trying to get a bunch of people new to our business unit on-boarded w/ our product, and they have a variety of requirements that we need to meet for that to happen. Things should settle down in a few months, but in the meantime, I try to fix some low-hanging fruit that directly improves morale for the team.
My org is better than most, I think, but it’s still an issue here. It’s absolutely crazy the difference between prioritized and unprioritized tech debt. For example, our architect wants a thing, so it gets done same day. Our dev team wants a thing, and it takes months, if not years. It’s getting better, but like anything, it’s two steps forward (finally got to trunk-based dev) and one step back (other teams whining about changes).
I think everywhere I’ve worked has said “we have 20% time for tech debt” but has never actually done that. It’s always “we need to ship this by end of week” and “the CEO wants us to add the thing we said we’d cut so we could make the deadline”.
So far, we’ve only had one or two deadlines come from outside the dev team, and up until this year, we got close to that 20%. So hopefully it’s a temporary change due to our new CEO trying to catch up to our competitors in a few areas.
I recently read that’s one of the reasons Windows is so messy and bloated, time goes by and so do devs and managers, so it gets increasingly difficult to know what does what, and why.
That’s true of any large old piece of software, I sometimes read my own code written a few months ago that I’ve forgotten and need to spend time to understand what it’s doing, imagine reading someone elses code written years ago. Companies don’t incentive good documentation or comments, and rarely have I seen proper coding standards enforced, so you end up with a lot of spaghetti code, with 600 line methods that do too many things and there may or may not be proper unit testing that covers this code thoroughly
And that’s the same reason Linux and open source doesn’t, because when developers are empowered to do things that impact them, shit happens
Depends. In an online store earning $50,000 a day, a one-second delay adds up to more than $1 million in lost sales each year.
https://www.bigcommerce.com/glossary/page-load/
I never got into GTA Online because I couldn’t stand the long load times. So they probably lost a shit ton of potential customers.