Open Source Software – Part 2

In the previous post, I discussed the rise of OSS, the positive effects it had and how it changed software development. At the end of the post I mentioned that also some misconceptions sneaked in. These I will discuss in this post.

Misconceptions sneaking in

While the rise and success of OSS was really important for the whole software development domain, it also fostered an “OSS is good, commercial software is evil” attitude and boosted a DIY (Do it yourself) attitude across developers. Note that creating a dichotomy “OSS vs. commercial” already is a misconception in itself ¹, but very often “OSS” still is misinterpreted as “strictly non-commercial” – including the misleading conclusions.

As a consequence, many software engineers did not even bother to look if a commercial solution would be more appropriate in a given context (often including commercial solutions built around OSS). It was either “pure” OSS or nothing – “nothing” meaning building it all on your own. This meant when building a new solution, everything on top of the programming language had to be either OSS ² or self-coded.

As always if you develop an attitude, you also develop an explanation schema to justify your attitude (at least if you do not have a severe mental disorder). This is true for useful attitudes as well as for useless ones. In the case of OSS and “OSS or nothing”, it lead to some misconceptions that are simply not correct.

I do not mean the ideal of a free, open world with open access to everything. While I personally see a value in open access for everyone (and a few risks of course), it will only get you so far in an economically driven world.

If you give away everything for free, you need to get the money you need to live from somewhere else, not your work. Donations, you might say. But if everyone lives from donations, where do the donations come from? I will stop the reasoning here. It should be clear that more is needed than just making all software free to implement the ideal (which does not make it a bad ideal).

Also this is not how you convince a decision maker in an economically driven company. You need a different kind of non-idealistic, but rather economic reasoning to “sell” OSS in a company. This kind of reasoning is then typically used to justify the aforementioned “OSS is good, commercial software is evil” attitude.

Thus, I will only discuss OSS from an economical point of view in the remainder of this post. More concrete, I would like to focus on the two most widespread misconceptions.

OSS is for free

Misconception #1: OSS costs nothing.

The fact that you can download the software for free, does not mean that overall OSS does not cost anything. The license costs of a software are just one of many components in a TCO (total cost of ownership) consideration. Things that are typically forgotten are, e.g.:

• It takes time and effort to integrate the software in your own environment.
• Usually it needs to be customized, sometimes even extended to fit your own needs.
• Often commercial solutions provide you with additional features on a higher abstraction level that you need for your problem which are not part of the OSS solution, i.e., you need to build them on your own.
• Sometimes you get new, enhanced features with commercial product upgrades that you would have to build on your own.
• You need to take care of maintaining the resulting solution including the OSS parts. This includes, e.g., fixing security vulnerabilities on your own, taking care of upgrades, and more. Some of the biggest known security breaches happened because companies ran versions of OSS components that contained known CVEs, i.e., the respective companies did not properly maintain their OSS-based solutions. ³
• Especially today, you need to buy the commercial variant of many OSS solutions if you want to use them in production as the OSS versions lack crucial production features like, e.g., monitoring or HA capabilities.
• Operations administrators need to learn how to run the new solutions. E.g., it is still fashionable to integrate a NoSQL or event streaming solution in “modern” solutions, usually neglecting that running those solutions is at least as complex as running a HA RDBMS cluster. You need to find the right people. You need to train them a lot. Often you need to hire new people – who are not available at the job market, or if they are, they are really expensive. This is nothing, the “already there” administrators can pick up on the fly by reading a bit of (often relatively poor) documentation.
• Even if the administrators are available and trained, many OSS solutions have a significant amount of operations efforts.
• If you are running your solutions 24x7, you additionally need to make sure that people are available at all times. Going with Google’s recommendations from their SRE book that means at least 8 administrators, or 2x6 administrators if you have locations in different time zones that are far enough apart (at least an 8h time zone shift).
• Last, but not least, lost opportunity costs can easily be orders of magnitude bigger than all of the aforementioned costs together. The additional time you need for setting up and customizing the OSS solution compared to buying a commercial solution can mean that you are later with your new feature or solution. In a competitive market, 2 or 3 months delay can make the difference between success or failure, can make a difference of tens of millions in the revenue stream.

There are some more costs relevant for a TCO consideration. But if you just take the ones listed here, assess them honestly and add them up, it can happen to you that you realize that your “free” solution actually costs you a lot of money in development and even more in maintenance and production – which can be more than a commercial solution would cost you. And we have not yet considered the lost opportunity costs due to the release delay.

I had project situations where the considered “free” OSS solution would have caused extra costs of roughly a million Euros per year. This does not mean that using OSS is always that expensive and that you should not consider OSS options anymore. But OSS definitely is not for free.

Reality: OSS is not for free. You need to look at all costs, not only license costs.

Still, I have rarely seen such economic considerations when discussing the costs of different solution options.

OSS prevents lock-in

Misconception #2: OSS means no lock-in.

Let me start this with a brief rant: From what I see the whole vendor lock-in debate is one of the most groundless debates in IT. Some IT decision makers in the past deliberately bought everything from IBM, SAP, Oracle, you name it. Some probably did it because “You do not get fired for buying <vendor X>”. Some used a different reasoning. But fact is that nobody forced them to buy from those vendors. They did it of their own accord.

Some of the vendors tried to maximize their profits, using the product dependencies of their clients as a lever. The decision makers complained because someone did to them what usually they only wanted to do to their clients. But they did not take any serious action to get away from those vendors, because “You do not get fired …” and so on.

Again, nobody forced the companies and their decision makers to buy from those vendors. Nobody forced them to put all eggs in a single basket, to make themselves totally dependent of one or a few vendors. It was their deliberate decision – probably for the wrong reasons, but still their decision.

But if you learn that your decision making process is flawed, fix your process and do not generally condemn “lock-in”.

Maybe the software engineers then designed solutions without considering a potential exit strategy and ended up in a place where the vendor solution is inextricably interwoven with the rest of your IT landscape.

But if you learn that your software design process is flawed, fix your process and do not generally condemn “lock-in”.

It does not make sense to exclude valuable options of today because of the wrong decisions you made on the past. Fix your decision making and software design processes instead.

Because every single decision you make for a tool or technology creates a “lock-in”, no matter if it is a commercial or an OSS solution.

But it does not hurt if it is an OSS solution? Well, maybe you save some license costs but it works the same way.

Generally, opposed to what most people think lock-in is something good because it makes you faster. You decide for a solution because it provides you with capabilities you need to solve your problems.

By using the capabilities of that solution you do not need to implement them yourself and can focus on the remaining – usually value-creating – parts needed to solve your task at hand. The more capabilities you do not need to implement yourself, the faster you can move.

Yet, every capability you do not implement yourself means a lock-in. You decided for a solution that provides that capability, to use it instead of building and maintaining it yourself. So, lock-in gives you a competitive advantage. Up to here everything is fine.

The problems start if you need to go away from the solution you chose. It takes you time and effort to remove the solution from your IT landscape and replace it with something different. Depending on the way you integrated the solution in your landscape, this takes more or less time and effort. In the worst case, time and efforts are so high that economically it is not feasible to go away from the solution.

This is equally true for commercial as well as OSS solutions. Assume, e.g., you are a company that heavily built on Spring, one of the most widespread OSS solutions in the Java ecosystem ⁴. Now assume that for some reasons, the development of Spring would be discontinued (remember: Formerly successful OSS solutions eventually abandoned are legion).

Assessing the future risks you might decide to replace Spring with a different solution. Even if you do not need to pay license fees for using Spring, chances increase massively that security vulnerabilities in the abandoned solution will be discovered in the future that nobody will fix anymore ⁵.

If you relied heavily on Spring in the past and integrated it everywhere in your system landscape, removing Spring from your system landscape would basically mean a major rewrite of all your Java-based applications.

While this is a hypothetical example (personally, I wish Spring, its maintainers and community a long and happy future), it should help to understand that also OSS solutions of all kinds create a lock-in.

The key point is: It is not important if you create a lock-in. Actually, you want lock-ins because you need them to create competitive advantages. Just imagine how long it would take to create a typical application of today on top of nothing but a 3GL programming language (also without all the huge standard libraries that are shipped with most programming languages today). That would multiply time and efforts to develop that application by orders of magnitude which is simply not an option.

So, instead of trying to avoid lock-in by all means or confusing OSS with not having any lock-in, it is a lot more important to understand what a potential exit from a lock-in means in terms of time and money.

For a sensible lock-in decision making process you need to determine the following (simplified decision making process ⁶):

0. Create a baseline by determining time, money and risks of building, running and maintaining the solution on your own (no exit efforts or risks involved).
1. Calculate time and money it takes to implement and run the solution based on the tool or technology you want to use (Remember: Also OSS is not for free).
2. Calculate time and money an exit, i.e., replacing the tool or technology with a different one would take. Also determine potential particular exit risks involved.
3. Determine time you expect to take advantage of the solution, i.e., not needing to go away from it.
4. Derive money and risk advantage you expect by using the solution over the expected period of time compared to your baseline (the competitive advantage).
5. Compare expected competitive advantage to expected exit costs and risks.
6. If the expected competitive advantage outweighs the expected exit costs and risks, go for it. Otherwise, think again.

Usually, it makes sense to make this reasoning and calculation for more than one option and to reassess your results if you decide to use the selected solution in additional places, i.e., if you increase your dependencies. This would give you a solid economic foundation for your decision making process ⁷:

Reality: You always have lock-in, also with OSS.

Lock-in is not the problem. The problem is not to base decision making and software design processes on sound economic foundations.

Yet, I still wait to see such an economic reasoning regarding lock-in.

Shadow boxing

Instead, most of the reasonings I have seen in the decision making and software design processes were flimsily disguised versions of “You do not get fired for buying …” (if the decision maker was in favor of a commercial solution), “OSS is good” (on the developer level), “OSS is for free” (on the decision maker level), and alike.

If people wanted to push through an OSS-based solution, typically they used the aforementioned pseudo-economic misconceptions to justify their preferred decision. Overall, it were more gut feelings and personal preferences than actual economic reasonings.

I do not condemn gut-based decision making. Our lives would be a lot less complicated if all of us would admit that most of our decisions are much more gut-driven than we think they are. I only find it difficult if people start a pseudo-economic discussion to disguise their personal gut-based preference.

These diversion tactics deprive the people involved you from the possibility to find the solution that balances the real needs best. Instead, they tend to lead to far from optimal solutions – especially if they are based on wrong assumptions like in the case of the misconceptions I just discussed.

Summing up

In this post we discussed the two most widespread misconceptions about OSS, that OSS is for free and that it does not create any lock-in. We have seen that both are not true and are based on an incomplete treatment of the economic foundations.

Again, this does not mean at all that OSS is bad. It just means that the misconceptions are wrong. To make the decisions that suit you best you need a sound economic foundation which almost everywhere is missing.

Having this in mind, in the next and last post of this little series we will rethink the role of OSS in the IT world of today. So, stay tuned …

[Update] August 9, 2020: Added short hint and footnote #1 explaining that the whole notion of “OSS vs. commercial” that can often be observed and I picked up in my post is already a misconception in itself. (Kudos to Gunnar Morling (https://twitter.com/gunnarmorling) for the hint, as I missed to point this out in the original version of the post)