The reusability fallacy – Part 2

In the first part of this blog series about reusability I discussed why all the reusability ideas break that root in the physical world and that are based on the idea you can save money in the production process by using reusable/standardized parts. We have seen that the actual production process in IT is already practically optimal. Thus, if reusability can help to save time, costs and efforts at all, it can only do so in the design process, which – as we have also seen – is everything until the last line of code is written.

The costs of reusability

Before we can discuss if and how reusability can help to save money, we first need to understand how much it costs to create a reusable part in the first place. This creates a good lower bound when reusability starts to pay off.

Fred Brooks discussed the costs of creating a reusable part in his essay “The tar pit” ¹. He starts with an observation:

One occasionally reads newspaper accounts of how two programmers in a remodeled garage have built an important program that surpasses the best efforts of large teams. And every programmer is prepared to believe such tales, for he knows that he could build any program much faster than the 1000 statements/year reported for industrial teams.

Then he raises the (rhetorical) question:

Why then have not all industrial programming teams been replaced by dedicated garage duos?

which he immediately counters:

One must look at what is being produced.

Brooks then explains that the newspaper accounts compare apples and oranges.

He starts his explanation by calling the original asset ², the “programmers in a remodeled garage” created, a “program”. This is a module suitable for the context it was created for. In our normal work as software developers, we create modules of this kind all the time.

Brooks continues that you need to apply two independent transformations to the original module to turn it into a reusable module:

• On the one hand, the program must be turned into a “programming product”. A programming product is a generalized module that is suitable for multiple contexts, not only the single context the original module was written for.
• On the other hand, the program must be turned into a “programming system”. A programming system is a module that is ready to be used in an ecosystem of interacting modules, i.e., a module you can easily access and use.

Brooks writes that both transformations require approximately 3 times the effort it required to create the underlying module. As those activities are orthogonal to each other, the efforts multiply up. This means, creating a reusable asset costs 9 times more than creating an asset that is suitable for a specific single context.

The obvious next question is where these numbers come from. Creating a “programming product” means generalizing and completing the functionality for arbitrary usage scenarios. This includes activities like:

• Hardening the implementation
• Handling edge cases
• Thorough functional testing
• Creating design documentation (for the sake of adequate maintainability) ³

Creating a “programming system” requires embedding the module in a technical ecosystem to make it easily accessible. Additionally, it needs to be usable, i.e., users of the module must easily understand what the module does and how to use it. Providing such a kind of accessibility includes activities like:

• Creating a precise interface definition
• Defining a clear behavioral contract
• Thorough integration (or alike) testing
• Providing excellent API Documentation ⁴

Pondering these activities, it becomes clear that an overall effort multiplier of 9 probably is a lower bound, if you take reusability serious. In the end, it does not matter if the factor is 7, 9 or 13. The key point is that creating a reusable asset means several times the effort of creating a non-reusable asset.

Shirking the costs

A common anti-pattern in that context is trying to shirk the extra costs for creating a reusable asset. The asset is created for a single context (a “program” in Brooks’ terminology) and then declared “reusable”. This means that other parties face an asset that only works in a single context (that usually is unknown to them), probably does not handle edge cases well, is not well tested, poorly documented, not integrated well in the ecosystem and so on.

As a result, each party that tries to use that asset, needs to spend extra time, costs and efforts to understand how the asset works, if it can handle their context, if they need to change it to make it work, to test it and so on. In other words: Each user of the “reusable” asset has to make the asset usable for their context. In the worst cases this costs more than writing a solution for the given context from scratch.

The more parties try to “reuse” that asset, the more expensive the asset becomes and after several “reuses” the parties that tried to “reuse” the asset paid more than it would have cost to make the asset reusable in the first place. Additionally, those parties suffer from lots of unexpected and thus unplanned costs of efforts. They planned for a reusable asset and found something they had to rework from scratch to make sure that it works in their context.

Unfortunately, in most places where someone tries to introduce a new architectural paradigm, the accompanying business case is based a lot on reusability but ignores the extra costs that creating reusability implies. All assets are calculated with a factor of 1, not 9 (or whatever your reusability factor is).

In such a setting, the only hope for the poor engineers is that they are not forced to use the “reusable” assets and find ways to circumvent them.

Reusability suitability

If we assume that the costs to make assets reusable are paid, we are still left with the question if a new architectural paradigm is needed to support creating reusable assets. This leads to the reusability suitability of different types of assets and the question which type of assets the new paradigm would support.

To answer this question, it makes sense to start with a look at reusability promoters and inhibitors. Reusability promoters are properties of an asset that increase the chances that the asset is worth being made reusable (i.e., likely to be reused multiple times). Reusability inhibitors on the other hand are properties that reduce the chances that an asset is worth being made reusable.

Typical reusability promoters are:

• Simplicity – the functionality is simple to use ⁵
• Stability – the functionality does not change a lot (from a usage point of view)
• Fine granularity – smaller chunks of functionality tend to be more reusable than bigger chunks
• Universality – the functionality solves a general type of problem
• Technical solution – the problem solved by the asset is an IT-specific problem (e.g., file access or network communication)

Typical reusability inhibitors are:

• Complicatedness – the functionality is hard to use
• Volatility – the functionality changes a lot from a usage point of view (i.e., the asset provides an unstable interface)
• Coarse granularity – bigger chunks of functionality tend to be less reusable than smaller chunks
• Specific functionality – the functionality solves a specific type of problem
• Business solution – the problem solved by the asset is a business-specific problem (e.g., contract or claims management)

This list is not complete, but it should give an idea regarding reusability suitability of assets. Also note that these are drivers, not binary criteria. If, e.g., your solution is a technical solution that solves a general problem and is simple to use (e.g., file system access), it is a lot more likely that it is worth making it reusable than if it solves a very specific business problem and exposes a complicated interface (e.g., a solution for risk hedging via complex financial products).

The drivers create a range from very good reusability suitability to very poor suitability. It is easier to understand the range by looking at a few arbitrary examples:

• Collection classes are great as reusable assets. They are technical, solve a universal and well-explored problem, are relatively small and offer a stable and simple to use interface.
• Technical frameworks (e.g., Spring in Java, Angular in JavaScript or Django in Python) are more specific, quite a bit bigger and are harder to use than collection classes. Still, they make good reusable assets.
• An embedded HTTP server is even more specific than a technical framework. The simplicity of the user interface varies from implementation to implementation. Still, they tend to be fairly good reusable assets.
• Moving to the business domain, currency is fine-grained, well-understood, universal and can be exposed through a simple and stable interface. Interestingly, while being a perfect candidate for creating a reusable asset from that perspective, you see a lot less reusable currency implementations than, e.g., embedded HTTP servers. So, the business domain acts as a strong inhibitor of reusability.
• If we move to a higher-level abstraction like accounts & transactions or even a complete ledger that are coarser-grained and more specific than currency, the reusability suitability goes further down.
• An example of rather poor reusability suitability is campaign management as it typically tends to be very specific, changes a lot and exposes a complex interface. ⁶

This were just a few examples and their exact positioning on the range might be worthy of discussion. Still, it should help to develop a general idea regarding the reusability suitability of an asset.

Reusability rarely pays

If we take that range and look around, we can make some interesting observations:

• All things on the left side, the ones with good reusability suitability already exist as reusable assets. They are part of existing language ecosystems. Most of the times they are also OSS solutions. In other words: Almost everything worth being made a reusable asset already exists. You just need to download it.
• Right of that is a place where you find assets that are too specific to provide them as public assets. Still, quite some of them are worth being created in a reusable way inside a company. Many companies have technical and business problems that are very specific for them but occur in many places. It would be a good option to create internal reusable assets for them (e.g., as libraries or alike) and provide them in an “internal OSS” fashion inside the company. Unfortunately, this approach is not very widespread.
• Most of the initiatives to introduce a new architectural paradigm focus on the far-right side. They target rather coarse-grained and specific business assets. This is the place where reusability usually does not make a lot of sense.

To emphasize the last statement a bit: In the later (already more critical) days of SOA, I had the opportunity to find out the reuse factors of a few big SOA initiatives ⁷. Most of them were around 1.1 or 1.2. One time I heard about a factor of 1.8. To explain these numbers: 1.0 means the service is used exactly once. 2.0 would mean, a service is used twice, and so on. 1.1 means, most services were used exactly once and just a few of them 2, maybe 3 times. ⁸

If we compare these factors with the factor 9 from Fred Brooks to implement an asset in a reusable way, we come to a simple conclusion:

Introducing a new architectural paradigm with reusability as business case does not pay.

The domain the new architectural paradigms typically target (company-specific business functionality) is not suitable for reusability.

A reason for this is what I call the efficiency of the business domains: Companies strove for many years to make their organization and processes more efficient. As a result you rarely find any business functionality left that is needed in many places. Instead all these types of functionality were identified and extracted to different organizational units. As a result, all business units tend to do quite unique work.

All functionality that is needed in several places is located in a single place and can be used from the other places if needed. The other places do not implement it as part of their work. This is the difference between “use” and “reuse” that I will explain in more detail in the next post.

This pattern repeats in the IT implementation of the business functionalities and is one of the reasons why going for reusability rarely pays in the business domain.

Again, what is left for reusability

In this post, we have looked at the costs of making assets reusable, the price of shirking the costs and reusability promoting and inhibiting properties of assets. Based on that we have seen that most of the “new architectural paradigm with better reusability” initiatives cannot work because they aim at the places where reusability does not pay.

The whole notion of needing to introduce a new architectural paradigm to establish higher productivity via decoupling and increased reusability is absurd anyway: We have everything, we need to achieve this in place for more than 60 years meanwhile. It is called modularization. Modularization is the underlying concept that enables decoupling and reuse.

We do not need any new paradigm to harvest the benefits of modularization. If we were not able to harvest them in the last 60 years, what makes you think that introducing a new paradigm, also based on the idea of modularization, will change anything?

This is a quite long post again. Thus, I will leave it here.

In the next part of this series I will discuss the price you pay for reusability in distributed systems as most paradigms that promise better reusability are distributed (e.g., DCE, CORBA, EJB, DCOM, SOA or microservices). Additionally, I will discuss the subtle, but essential distinction between “usable” and “reusable” assets and why confusing them leads to very bad designs.

Again, I hope this post gives you some food for thought and stay tuned for the next part.