Simplify! – Part 12

In the previous post, I completed the consideration of accidental complexity at the architecture level by discussing potential countermeasures to the drivers of accidental complexity which I discussed in the post before.

When I started to discuss the different drivers of accidental complexity in IT and what we can do about it several posts ago, I wrote that I would move “from the outside in” to provide a bit of structure. In this post we finally reach the most inward level, the implementation level.

As on the other levels, there would be a lot of topics to discuss. I will focus on a few drivers that from my experience create a lot of accidental complexity on the implementation level.

Breaking the existing designs

The very widespread driver of accidental complexity is breaking the existing designs. This comes in many different flavors:

• Circumventing a regular module interface and access the module implementation via a non-official access path. In single process boundaries this is common practice. If the IDE and the compiler allow it, it will be used. This is the probably biggest driver of the clumping of applications, the effect known as “big ball of mud” and why monoliths have such a poor reputation.
• Neglecting a design rule. Each project has common design rules. E.g., there might be an agreed upon way how to resolve external dependencies. If someone uses a different way, it creates confusion and increased intellectual load. Code becomes harder to read, you have to double check what is going on. Bugs become harder to spot.
• Using a different library. Instead of sticking to a single approach to solve a given problem, several approaches are used. This happens often if developers who join a project or an application later have a preference for a different library than the one used. All other developers have to learn two approaches now. The number of dependencies (and potential vulnerabilities) increases.
• And so on …

There are more flavors than the ones listed, but it should be sufficient to understand the driver. Sometimes it happens due to not understanding the existing design. Sometimes it happens due to external delivery pressure (see also next driver). Sometimes due to ignorance. Sometimes due to a diverging opinion.

But whatever the reasons are: All variants of breaking the existing design lead to increased accidental complexity. Effect: The solution becomes harder to understand, to maintain, to evolve, more error-prone, less secure, …

Implementation shortcuts

Another widespread driver are implementation shortcuts, also known as “workarounds”, “dirty hacks”, etc.

Most developers are under constant delivery pressure. We discussed this several times before in this series. Additionally (or consequently), they are incentivised for completing their tasks fast. They are not incentivised for creating good solutions ¹. As a result, developers in such a setting tend to complete their tasks as fast as possible.

They do not care if the solution is maintainable, if it is of good overall quality. The only thing that counts is to close the corresponding ticket as fast as possible to evade the constant high delivery pressure.

Some developers even make a career of it, building a reputation of quickly getting things done, being pulled by project decision makers to “save” their projects if they are late, in the worst case leaving a terrible mess behind while “saving” the next project.

A terrible example I have seen several times in such settings are database fields that are allocated several times: An existing database field that is not used in a certain context is “reused” by attaching a different semantic to it, saving the efforts for database schema changes ². Of course, this double semantic leads to lots of confusion, drives error likelihood up, makes it harder to detect data corruption, increases cognitive load for all other developers, and more. But it saved a bit of time when implementing the feature.

In harsh words:

Every shortcut is trading 1 development hour now for 100 hours later – plus tons of additional bugs in production.

Still, the project decision makers usually try to save the single hour now, and for many developers it becomes habitual over time – and as mentioned before: Some even turn it into a career.

Design by Stack Overflow

A variant of the implementation shortcuts is the “Design by Stack Overflow” (also known as “Google-driven Design”). The typical pattern is:

1. A quite straightforward task needs to be solved.
2. A simple solution is not obvious.
3. A Google search for the problem is started, usually leading to a post at Stack Overflow.
4. The code snippet is copied, the required Maven (or whatever your build system is) snippet is also copied to include the required libraries.
5. The included code snippet is tweaked until the solution “works on my machine” (and the unit test becomes green if one exists).
6. The code is checked in, ticket is closed and the task is “done”.

While Stack Overflow is a wonderful source of knowledge that also I do not want to miss anymore, it should be used carefully. If we blindly copy and paste code and libraries to the solution, hardly knowing what they do, accidental complexity explodes.

Show-off implementation

Then there are “rock star developers”. And for a real rock star developer it is essential that everyone immediately sees how great and “cool” they are. So, they try show their genius by writing so “advanced” code that nobody but them understands it:

• 2nd order functions? For starters! At least 5th order functions! See how concise my code has become!
• Garbage collector? For losers! Check out my brilliant off-heap implementation!
• One framework? That’s nothing! Watch me weave together 4 frameworks in this highly sophisticated way bypassing the conflicts of different thread handling strategies in this exceptional (and highly brittle) way!
• One paradigm? How lame! See me creating this awesome elegant code by combining object orientation, functional programming and templates in a single line of code! Imperative programming? Only over my dead and cold corpse! That’s for noobs and other losers.
• To be continued …

While I deliberately exaggerated a bit, quite likely you have met several persons in your career that acted in a similar way. Sometimes they poorly hide their attitude behind humble-bragging. Sometimes they show off in a quite offensive way. Sometimes they just do it for themselves, simply because “they can”.

But no matter why and how they do it, they increase accidental complexity a lot. Instead of focusing on the task to solve, other developers have to waste most of their thoughts understanding the “brilliant” show-off implementations of those “rock stars”. Overall, this attitude makes code harder to understand, to maintain and to extend, increases error likelihoods and slows projects down.

Keystroke theater

A variant of show-off implementation is the keystroke theater. There is this persistent misconception that the shorter code is, the better it is. As a result, developers often go to great lengths to make their code shorter, in developer-speak: “make it more concise” or “get rid of the boilerplate code”.

While overly lengthy code for a given problem can indeed impede understandability, overly short code usually usually is even worse. If you measure your code brilliance in the number of characters not typed, you miss an important point: In the end, it is about understandability, not the amount of characters.

Perfect code is code that is invisible.

It does not stand between you and the problem it solves.

In other words: Perfect code is code that does not force you to think about the implementation but allows you to solely focus on the problem it solves. It does not stand in the way, does not impede understanding.

The key point is that there is a sweet spot between overly chatty and overly short code. If it is too long and chatty, the code stands in the way. If it is too short and cryptic, it does so even more. Understandability and number of characters are only correlated on the “too chatty” side. On the “too short” side they are inversely correlated.

Quality theater

The last driver I would like to mention is quality theater. In many organizations, developers are deprived of all intrinsic motivation factors:

• No Autonomy – They do not have the freedom to decide how to do things. They are squeezed into a tight corset of regulations, prescriptions and guidelines.
• No Purpose – They do not know what they are working for. They are just given tiny pieces of work, never seeing the big picture behind it, let alone the purpose for the job.
• No Mastery – They are not rewarded for doing good work, but only for doing their work quickly.

As an understandable consequence, some developers start to strive for mastery without a clear direction, usually for the sake of their self-esteem and mental health. Unfortunately, this often results in various sorts of quality theater – cargo-cults, that are religiously defended. There are fierce fights over method lengths, unit test coverage, “clean” code, using the “right” patterns, and more.

While all the subjects of debate have a value, losing sight of their original intentions and overdoing it does not add more value. Instead it adds to accidental complexity. Developers around such people need to spend a lot of their mental capacity to make sure they adhere to the “rules” because otherwise they will be dragged into endless debates and blaming.

This is unneeded complexity, binding capacity that is not available at more important places. Besides, it increases the stress levels of the people affected, if they have people around who dogmatically “fight” for their personal quality beliefs. ³

In the worst case, such a quality theater can lead to missed project goals. Quite often the acting persons only have their default project context in mind, like writing and maintaining core enterprise software. If those people get placed, e.g., in a product exploration project where gathering as much feedback from the customers as quickly as possible to validate the product assumptions, they typically will start to act as usual, insisting in high test coverage, clean code, and alike, ignoring that their current context requires very different quality properties.

Again: The general idea behind quality theater approaches is a valuable one. But overdoing it destroys value and the motivation of the people around. The line between creating and destroying value often is a thin one, and without seeing the big picture it often is hard to understand where the line is. Especially if you assess the situation just from your local context, chances are that you will end up on the wrong side of the line.

Improving the situation

These were some of the most popular and widespread drivers of accidental complexity on the implementation level. As written before this list is not complete.

But again, the question is: How can we do better?

I think there are a lot of ideas we could discuss. I will just discuss a few ones, that are quite powerful based on my understanding:

• Holistic understanding – We had the first recommendation in several variants already: Leave your comfy chair. Here it is about understanding the needs of the solution from different points of view, not only the developer view to create a more holistic picture. It will help to make better decisions overall, not only with respect to accidental complexity.
• Consider effects over time and people – An old developer proverb says: “Code is written once, but read a thousand times”. Understand that the key to good code is understandability, not brilliance or misunderstood “elegance”. And it is irrelevant what you consider understandable. The thousand developers who have to read your code decide what “understandable” is. Try to have them in your mind while you write your code. BTW: One of them is your “future you”. That person will be the most skeptical judge of your work today. You simply do not yet know … ;)
• Become egoless – This is the extension of the previous recommendation. Egoless code does not try to shine. It aims for invisibility, i.e., it does not stand in the way between a reader and the problem it solves. Perfect code is code where the reader says “That’s obvious. I could have written that”. But you need to get your ego out of the way to judge such a statement as a big compliment.
• Test enough, but no more – Testing is important. Overdoing it is harmful. Too much testing is waste and compromises other important goals. In the worst case it creates a competitive disadvantage. Thus, make sure that testing does not become and end in itself. The goal of testing is not 100% certainty (economically this would be pure madness), but to be sure enough that problems in operations will be on an acceptable level. This is risk management. This is not about all or nothing. It is about finding the right balance.
• Resist the pressure – Try to resist the constant delivery pressure. Exercise a bit of “civil disobedience”. Take that bit of extra time to get your solution right and as simple as possible. Often you will need to find a working compromise, but even a compromise is better than nothing. At the same time make sure that you do not fall for quality theater. Again, it is a delicate balance, but worth working on.
• Code review – On a very practical level, code reviews and pair programming can help to avoid accidental complexity. It helps to detect widespread anti-patterns like circumventing existing designs and implementing shortcuts. It can also help to reduce design by Stack Overflow and show-off implementations (the latter admittedly can become quite hard as a lot of ego tends to get in the way).
• Train and explain – Do regular training sessions where you explain and train your project development practices. Both parts are important: To explain to help people understand the “why”, and to train to practice the “what” and “how” in a controlled and save environment. It is important to bear in mind that the goal is not to reveal people who are doing it wrong but to establish an agreed upon common set of practices.
• Document practices – As an extension of the previous recommendation, document your practices to have the “what” available whenever needed. But bear in mind that the documentation alone will not be sufficient. You also need the train and explain sessions to establish and maintain the common set of practices.
• Learn 10-finger touch typing – No irony intended! Surprising few developers ever learned touch typing, which is a hidden driver of keystroke theater and other character shedding practices, including missing documentation where needed ⁴. Thus, to get rid of this trivial impediment: Learn touch typing if you are not proficient in it. I also only learned it many years after I started coding, but ever since I ask myself why I did not learn it in the beginning. That bit of training effort pays every single day.

Summing up

In this post, we discussed several drivers of accidental complexity on the implementation level and what we can do about it. This was the last and most inward level of my original structure to discuss accidental complexity.

In the next post, I will discuss one final source of accidental complexity that did not fit in the original schema. It is about the layers of technology that build up over time in an IT system landscape. It did neither fit the technology discussion nor the architecture discussion as it is primarily about understanding effects over time. Stay tuned …