Beware the Complexity Critical Point, June 14, 2015

disaster icon

One of the engineering problems faced by designers of battery-powered electric vehicles is the problem of thermal runaway. Simply put, when certain types of batteries are charged, there is a risk that part of the battery will overheat. This can start a positive feedback loop where more and more of the battery gets hotter and hotter, with potentially disasterous results.

The way I see it, software developers face a similar problem. A software developer worth his salt knows that one of his prime responsibilities is to manage the complexity of any system he is responsible for. Think about your favorite programming language. Whether it is C, C++, C#, Java, Python, Ruby, Javascript, or whatever language it is that you like, there is a syntax and a limited number of keywords and used to express logic in that language. It is not a particularly difficult matter to learn the syntax and keywords of any language. However, mastering the syntax and keywords of C++ no more qualifies a person to write a large codebase in C++ than does mastering the syntax of the Russian language qualify a person to write a sequel to "War and Peace".

I'll repeat: the syntax of a programming language is simple. The programs we use programming languages to describe are mind-bogglingly complex. If you write a program, it's your job to manage the complexity. If you fail to do this, your program will be unreadable and unmaintainable. Vast sums of money may be spent hiring consultants to clean up your mess. I once worked on a project where a company spent well into the six figures of US dollars on using contractors to maintain a messy codebase, and even then, the result was that a handful of critical bugs were fixed while the company budgeted even more money to hire another agency to provide a replacement.

What does that have to do with overheating batteries setting people's houses on fire?

The answer is simple. In my years as a software developer, I have worked on a number of different projects. Some of these projects, I built from the ground up. Some other projects, I worked on code that was very old and had been worked on by many different teams in the past. I noticed a pattern: there are good codebases and there are awful codebases. I can't recall that I ever encountered a codebase for a large project that seemed to be low-quality, but not too terribly bad. It's fire and ice.

I believe that what's happening here is that as long as a codebase is well-maintained by competent developers who work hard to manage the complexity of the project with good code practices and documentation, the codebase will stay readable and maintainable; but when bad practices start to creep in, when someone fails to do documentation because they're in a hurry, or someone copy-pastes code from one part of the codebase to another, or an incompetent developer starts tinkering with it, watch out!

Once a certain amount of unnecessary complexity enters your codebase, the damage may be difficult or impossible to reverse. Just because the quality of your codebase is going down the toilet doesn't mean that the business it serves doesn't have need for new features, and while you try to balance fixing the problem with adding new features demanded by management, the nastiness in your codebase will prove an awful stumbling block. In short, bad code breeds more bad code.

And, if you're like most developers, you'll do one of two things. Either you take advantage of the situation to build "job security" because "no one else can maintain this code", or you high-tail it out of there and let it be someone else's problem while you move on to the next disaster.