When increasing complexity means you're really holding out for the missing piece

Before I went freelance, one of the perceived impediments to my plan was our home office space. My wife works from home too, so we had to work out how to make the most out of a room that was 2.82 metres long and 1.82 metres wide, and needed to house two people and a cat for most of a working day. Very quickly we got as far as we felt we could in just simply throwing things out, and the next step was perversely to buy more furniture.

Along with a lot of shelving we bought an Alex drawer unit, with nine flat, long, deep drawers. Initially this definitely made the office more cramped, and we wondered for a while if we'd made the wrong decision. But we rearranged and rearranged—it didn't feel much like we were throwing anything out—and suddenly we looked around to find that the contents of the room had somehow, as we stacked and restacked it like the Tower of Hanoi, drastically reduced in quantity and clutter.

Recently I've been working on a project with a half-complete test harness, and was asked to refactor some quite complicated functionality that wasn't under test, in order to bring in a new library. Tempted as I was to continue with it in its untested state—especially as the refactoring was meant to reduce the code's complicatedness—I nonetheless laboured on with a set of tests, getting it all right: even in the knowledge that a lot of what I was writing might end up redundant with the old code.

Once the test harness covered the code, though, I was able to not only refactor, but also clear out a whole tier of practically unused complications, safe in the knowledge that the ultimate behaviour was still correct. Line after line vanished, and eventually the temporary increase in codebase size meant we could safely decrease its size in the long term.

Our desire to simplify often leads us to start hacking away at what we see as a quantity of complications in a complicated system, under the assumption that simplicity is merely the near-absence of complications. But simplicity is instead a qualitatively different state, which is absent because the abstractions and assumptions of the system aren't optimal. Merely reducing quantity as if you were hacking back brambles is unlikely to ever transition the system, from incorrectly abstracted complexity, to somehow-correctly abstracted simplicity; it's much more likely to break the system, potentially irrevocably, requiring a rebuild of at least some of its processes from scratch.

Instead, a complicated system sometimes suits being temporarily more complicated, but with a clear plan in mind: permitting abstractions and assumptions to shift gradually; supporting both old and new paradigms until the old one can be removed entirely. Once the new abstraction is safely embedded, its succinctness can permit far greater simplification than would have otherwise been possible, and the entire system can eventually shed all its baggage, old and new. 

Sometimes if you want to get lines to disappear from your Tetris board, you have to add the right new pieces, one after the other, until the plan is completed successfully.