Striking the Balance in Software Development

When it comes to software development, there’s always that internal tug-of-war between under-engineering and over- engineering a project. I’ve been on both sides of that rope before, and I can tell you, neither extreme feels great. Let’s dig in.

Under-engineering is like rushing to throw together IKEA furniture. We’re thinking, “Yeah, I’ll skip the manual. How hard can it be?” But fast forward a month, and suddenly, you’re lying under a collapsing bookshelf. In the software world, that’s technical debt: the bugs, the brittle code, the constant firefighting. I’ve dealt with projects where hasty decisions and lack of planning turned every bug fix into a significant event. We were trying to repair a plane while flying it—frankly, it was exhausting and demoralising.

But I’ve also seen the other side of the spectrum: over-engineering. Spending days, weeks, or months planning every detail can feel like trying to reinvent the wheel… and then adding rocket boosters to it because “what if?” I’ve sat in endless meetings, whiteboard markers running dry, feeling the slow creep of analysis paralysis. We’d over-plan until the project started feeling like a tangled ball of yarn no one wanted to touch. By the time we got to actually building anything, momentum had already died, and the initial excitement was just a distant memory.

The real question is, how do we get the balance right? I think about it like preparing for a camping trip. You want to bring enough gear to stay safe and comfortable, but you’ll never get on the trail if you pack your entire house. The trick is to be prepared but adaptable. One memorable project that drove this point home involved rapid changes due to shifting market demands. All our careful plans were tossed out, and what worked in theory didn’t fit in reality. The flexibility to adapt was what saved us—and it made me realise that sometimes you need just enough structure to get started, but room to improvise as things change.

I’m a fan of using analogies, and city building is a great one for this. Cities evolve—they don’t pop up fully formed overnight. Over decades, what started as a few basic shops slowly can become a hub of activity. When there were just a handful of people, no one bothered with skyscrapers. But as the population grew, so will the infrastructure. The same logic applies to software. Don’t try to build a sprawling metropolis with only a few users. Build as you need, and be ready to adapt.

This brings us to the microservices vs. monolith debate. I’ve had moments when sticking with a monolith seemed like the right call—simple, efficient, and great for getting things off the ground. But it’s a bit like those houses all clumped together. One bad leak can make the whole thing collapse. With microservices, you’re building independent pieces— each with its own “house.” When one has a problem, it’s like your neighbour’s plumbing issue—it doesn’t flood your living room.

There was one particularly hairy issue with an e-commerce system I worked on. Our monolithic architecture meant that a bug in the checkout process took down the entire site, like knocking over a house of cards. If we’d used microservices, that bug would have been isolated, keeping most of our functions operational. The ability to tinker with one service without fear of breaking everything else makes microservices powerful and, honestly, a bit liberating.

The lesson is this: start with what fits your needs now, but be ready to change course. Build what you need and know that you can scale up—or tear down—when the time comes. That balance between planning and doing is where all the magic (and chaos) happens. It’s what makes this field exciting and endlessly challenging.

Article by Dave