Your programmers are not fantasizing about a rewrite if you notice a new feature takes 3x longer time to ship than before or they start taking frequent coffee breaks during a sprint.
Once upon a time, I was a bystander in a heated argument among the senior engineers and the non-tech top management of a big-data tech company. The engineers were trying to convince the top management to go with a complete rewrite of a critical part of the backend – the data processing pipeline.
As a new hire, barely 3 months on my job, I did not know much about the details of the project. So I could only watched with apprehension as they argued among themselves.
There were those who were for and those who were against the proposal. The business guys did not want the product development to stop for 3 months. That was inconceivable to them. And then there was ego at play. The engineer who wrote most of the code of the data processing pipeline refused to acknowledge that his code was beyond salvation.
I noticed one thing clearly.
The engineers were only telling the management what they wished for, what they thought were good for the business if they have completed the rewrite.
“We will be using a new framework.”
“The code will be cleaner.”
“The new data model should make queries simpler.”
“Roll-out of new features will be faster.”
“The data processing performance should be faster.” etc. etc.
Nobody said anything about the impact to the business if they skip the rewrite.
Traditionally, management does not view technical debt as a bad thing. The term itself suggests it is like an investment or a loan.
You incur technical debt so as to go to market fast, learn about your customers’ needs fast, capture market share faster than your competitors. It is a necessary evil of tech startups.
Somehow the management thinks if the startup succeeds, it will be straightforward to repay the “debt” later.
A standard advice is that engineers are free to spend extra time on refactoring while working on new features.
They are told to factor in the time into their effort estimate. You can view this as some kind of installment plan to repay the technical debt gradually.
That was the boss’ final decision. He wrote it in an email that concluded the meeting. He also attached in his email a story about Jeff Bezos.
Amazon’s CEO chaired this type of meeting once. He reportedly told his perfectionist engineers about the difference between an academic nuclear reactor and a nuclear reactor in production. The meaning is that you’ve got to live with technical debts in real-life engineering projects.
Fast-forward two years. I took over the project from the previous engineers who by then had all left the company. The pipeline was so buggy that the data it produced was close to garbage. Data cleanup on the production database took 1 month to complete. We had to do a cleanup every 3 months. The customers were churning because they noticed the errors in the data.
Eventually my boss tasked me with the rewrite.
I gave him an estimate of 6 months: 3 months to hire more capable engineers who are experienced enough to deal with bad code, another 3 months to rewrite the system.
On hindsight, it is obvious that accepting a 3-month moratorium on new feature roll-outs was a better decision for my ex-boss. But what could he have done in the meeting? How could he have figured out if the programmers were fantasizing about a rewrite or if they were dead serious about it.
The answer I think is that he should have directed the engineers to assess carefully the projected impact of the technical debt on the business.
For that, we needed data. We needed metrics. We need to measure and track many things: the estimates, the code coverage, the data quality, the conversion rate, the churn rate, the number of reported issues on the bug tracker, and engineers’ productivity, etc.
Once you have all these data charted, it is easier to understand if the bug list is growing, how existing issues in the code affect the estimate on a user story. You will be able to make an informed decision, whether to bite the bullet and perform a rewrite, or repay the technical debt bits by bits.
In other words, listen closely to your engineers, and do check the facts.