Over-engineered is a phrase I hear often when talking about code (and designs in general) but what exactly is over-engineering? I ran into an article today with this to say:

Over-Engineering is when someone decides to build more than what is really necessary based on speculation.

by Fagner Brack - How To Accept Over-Engineering For What It Really Is

It goes on, but I think this is the most concise definition I've heard. Essentially, over-engineering is solving problems you don't have. [ SO ] That makes sense and seems to gel with what I think of as over-engineering, but the real question for me is how do I prevent myself from over-engineering? And why do so many programmers -- myself included -- keep doing it?

I think the answer is simple if we think about it in terms of risk and reward, because we want to invest in the future. We spend time, money, and complexity now to solve a problem we anticipate in the future, for potential profit. The problem is that not all bets pay off. Over-engineering is more precisely sunk costs with no benefit and likely a more complex design because of it.

Seen this way, over-engineering is less a vice of amateur programmers than it is a bad bet that anyone can fall for. You might be tempted to avoid over-engineering by simply designing for today and never planning for the future, but I think this too is a mistake. The world is filled with examples of software that no longer fits our needs, but which is difficult to change due to backwards compatibly, cost, mind-share and dozens of other reasons (e.g. Python 2 vs 3, JavaScript's primitives, etcetera).

This doesn't leave us with an easy solution, but it does suggest we should think carefully about how we plan for the future. For example, using API versioning in your first release is a good idea. It gives you a clean escape hatch to make breaking changes, while supporting old code.