Should shared problems have shared solutions? Deciding so requires us to understand them at a deeper level
Throughout my career, one scenario has reared its head over and over again: multiple components seemingly sharing one problem in complex systems.
From software architecture to macroeconomics, the question always arises: “Should shared problems have shared solutions?” This question is deceptively simple on the surface. At the start of my career, like many engineers looking to simplify problems, my answer would have been, “Of course!” Why do the work twice?
If a problem exists in two places, why on earth would I establish two different solutions? I’d be reinventing a wheel I invented in the first place!
As many readers who have been bitten by an overzealous approach to DRYness know, most problems that look alike… rarely stay alike. Problems are some of the most mutable things in a system. They are the amalgam of often complex and evolving feedback cycles of inputs and outputs. What is needed one day from one part of the system changes with that part of the system.
In all situations where a shared solution fails, it is because the needs of the components that utilise it have changed somewhat. Often this leads to the scope of the solution expanding to address two separate problems, normally still pretending they are the same one. We see this everywhere, from the lowly spork (which neither pierces food nor scoops liquid particularly well) all the way up to whatever shared problems Google+ thought it was trying to solve for Alphabet’s various products.
The summary is that shared solutions can denature from their founding forms to something that doesn’t address either component’s needs well.
So what’s the alternative… isn’t it obvious?! Everyone goes and builds their own bespoke solutions to every single problem ever experienced, and we all live happily ever after!
Facetious sarcasm aside, complex systems like the ones we live in aren’t going to get very far on primitivism (no matter how much Ted Kaczynski may have wanted it). The fact is that at some layer of abstraction, two components will benefit from sharing a solution. The question is, how do we come to determine that layer?
There is one key element about components that we need to ascertain to answer this:
Stability: What is the rate of change of these distributed components? Remember, these could be microservices or teams of people (Conway’s law would suggest they’ll often mirror each other anyway).
If two components are highly stable, you are most likely taking a low risk by applying even a relatively intricate solution to both, provided you’ve established that it initially addresses both well in the first place.
However, if any one of these elements is lacking, then you run the risk of coupling them to a solution that will end up addressing both’s needs poorly. So what should we do if stability is lacking? Well, it will be no surprise that my initial suggestion would be to review the effort needed to improve that.
Whilst it is true that in complex systems, something like stability is often a rare commodity, we shouldn’t let that stop us from evaluating the degree of leverage we can have over this. Our main lever for this is Intent, the component’s purpose that restrains its scope of operation.
If you can tightly limit the intent of a component, you can limit the rate of change in its requirements.
I can’t lie, though: maintaining the “intent” of something in a complex system is some simple task. Agile software development revels in the acceptance of eternal change. As much as our monkey brains may want to put pretty little shapes on what the great Alan Watts described as a “Wiggly World,” we must accept that this will often fail. What we thought customers, consumers, and subscribers wanted one day will change the next.
So if we can’t always stabilise the intent of our components, how then should we approach problems that they seemingly share?
Well, as mentioned before, this is more than just asserting what problem they share… it’s about defining the layer of abstraction at which they most stably share problems. This represents what I call the Highest Abstraction of Stability (H.A.S).
Whilst cars have evolved significantly over the last 100 years, some things have stayed highly stable: they, mostly, still have four wheels, they still have a transmission, they still require suspension, and they still drive on roads. Each of these addresses problems shared by the vast majority of cars. Yes, there is some variation, but it is much lower than the variation at the “car” level of abstraction. That is because it has remained much more stable over time.
If we take roads as another example, asphalt and concrete have stayed the main materials for road construction for over a century.
What I’m getting at here is that in all complex systems, there will be shared problems. As systems thinkers, our leverage exists in identifying the layers of abstraction at which they are most stable. This allows us to build systems that are both flexible and interoperable.
When approaching shared problems, always ask, “How long has this been a problem for these components?” If the answer is “not very long,” then go down a level of abstraction, find the shared problem, and ask it again.
From there, you will be able to build the foundations of greatness!
The Problem With Sharing Problems in Complex Systems was originally published in Better Programming on Medium, where people are continuing the conversation by highlighting and responding to this story.