The public cloud revolution - Part 2

In the previous post we discussed the evolution of the public cloud offerings from their beginnings until around 2015, starting at the plain compute and storage level and working their way up to the PaaS level.

In this post, we will continue our cloud evolution journey and see how the evolution turned into a revolution.

Public cloud ~2017

Around 2017, the dynamics of public cloud started to change dramatically – unfortunately still unnoticed by many companies. Before that time, most public cloud offerings mostly affected the operations side of IT. All the stuff needed to build and run software has become available on demand.

But after having worked its way up to the PaaS level, public cloud moved on and started to heavily affect the way, software was developed. And this was a game changer – at least for those who understood it.

The foundations of the revolution are what often is summarized as “serverless”. Just for the sake of completeness: “Serverless” does not mean, servers do not exist as some (IMO ignorant) people still try to ridicule serverless. Instead, it means you do not need to care about servers, infrastructure, platform, applications, hardening, updating, patching and alike, but that you consume all this on a higher abstraction level: As managed services.

Also, “serverless” is not a synonym for FaaS (Function-as-a-Service) as some other people still tend to claim. Functions play a role in serverless, but they are not the core of it. Its core are the managed services.

Around 2017, a huge ecosystem of managed services at all levels was available and ready to consume on demand. Not only infrastructure and platform services were available; more and more business level services became available.

Many of the former SaaS offerings turned into full-fledged managed services: They were not limited anymore to manual subscription and consumption via their Web UI. You could also provision, consume and decommission those services via API, i.e., programmatically. Additionally, more and more managed services emerged that were “headless”, i.e., only accessible via API.

This made all the formerly isolated SaaS offerings combinable. You could combine a content management service with an e-commerce service with a payment service with infrastructure and platform services as needed. Instead of building custom software solutions or implementing standard software solutions, you picked the required managed services and integrated them via a small service integration layer.

The “glue” for this service integration were functions. FaaS turned out to be a very powerful and versatile integration technology. Of course, functions also have other usage scenarios, but this one enabled a completely new kind of software development.

It enabled combining available managed services and build your applications in a mix-n-match fashion from existing managed services, offering functionalities up to the business level. As a consequence, you could massively reduce your vertical integration depth in software development. Non-differentiating parts of your software offerings you needed to build yourself in the past (or buy as poorly integrable COTS software) could be easily integrated in your custom software solutions now. ¹

If you used these new options wisely, it changed your development speed and efforts dramatically. Years turned into months, months turned into weeks, weeks turned into days. Adding serverless to the public cloud offerings crossed a tipping point where the evolution turned into a revolution. Now public cloud not only transformed operations but also development. This created new options on the business side that simply did not exist before.

The reduction of the vertical integration depth up to the application level freed a lot of IT capacity. IT was not completely occupied anymore with building and running basic services. If you did things right, i.e., also transformed your organization and processes more towards a DevOps culture, the organization had enough capacity left to explore completely novel (business) ideas.

Public cloud 2020+

Basic services could be provided mostly based on the combination of existing managed services. Sometimes a relatively thin integration layer (often implemented via FaaS) was needed. Sometimes the managed services were sufficient as they satisfied the respective basic customer need. Development has become at least an order of magnitude faster if done right (including the required transformations of the organization and processes). Capacity was freed. People were ready to try out novel ideas.

Such a setting strategically enables an innovation move which is one of the most interesting movements that Wardley maps help to exhibit. The pattern is always the same:

• You push something further down the value chain from custom-built towards commodity.
• This frees up capacity.
• The not so smart companies then only think about laying off people to reduce their costs. They will not make the innovation move. Most likely, over time they will develop a competitive disadvantage.
• The smarter companies on the other hand use the capacity available to make a move to the left and up – the innovation move. Those companies most likely will develop a strong competitive advantage.

The move to the left and up means that you use the capacity that was set free on a lower level of the value chain to explore new ideas and options further up the value chain. I.e., you go to the left towards “genesis” and up closer to the customers and their needs on the Wardley map.

Here it means you use the capacity not needed anymore to build and run the basic services to explore higher-level services that match the actual needs of the customers better. E.g., to pick up the insurance example from the first post once more: You try to offer your customers safety, e.g., regarding living or traveling. You combine several of your core services and complementing services to create a novel service that fits the actual needs of your customers a lot better.

The whole IT value chain, i.e., the time needed from having a new business idea until the customers experience the (IT-based) result, has been accelerated by an order of magnitude due to the reduction of vertical integration depth and a corresponding transformation of organization and processes. This means you can explore a lot more of those ideas and options than your competitors who did not follow that path which gives you a competitive advantage. ²

The private cloud fallacy

Before wrapping up, let us have a quick look at private clouds. There were – and still are – quite some companies that placed their bets on private cloud for various reasons. The reasoning I usually heard went along the following lines:

“We want to go cloud, but we do not trust the big cloud providers because of (choose one or more):

• security concerns
• data privacy concerns
• lock-in concerns
• competition concerns
• …

Due to these concerns, we cannot use a public cloud offering. Instead we will build our private cloud. This gives us all the cloud advantages, but it still is under our control.”

And so these companies built – and still build – a private cloud. Usually, “private cloud” means to install and run a PaaS-type software like Red Hat’s OpenShift, KubeCF (based on Cloud Foundry) or alike.

While there is nothing wrong with the products I mentioned, you will not get the effect I described in the previous section of this post.

First of all, you still need to run all the stuff yourself, i.e., it does not free any capacity. On the contrary, usually the new products do not replace anything of the existing tool stack, but are added to it. This means in the worst case you need more capacity to run a private cloud.

Secondly, the private clouds stop at the PaaS level at best (I have seen handcrafted private clouds that did not even offer reasonable compute services). But the secret sauce of the public cloud revolution are the managed services you can easily combine and orchestrate. You do not get these in your private cloud. Each service, you want to provide in a private cloud, you need create, set up, operate and maintain yourself. Usually, this means you need more capacity instead of less.

Thus, no revolution with private clouds.

Basically, all you get if you run a private cloud are containers. If you use a product like OpenShift or Cloud Foundry, you also get a quite nice platform on top of it (at least if the respective platform capabilities suit your needs). This is a good thing if you did not have containers in place before.

The rise of containers was a very relevant development in the last decade. As I described in one of my prior posts, containers are a very powerful technology to break the vicious dependency hell between applications and the OS that plagued us all those years. And they have some more nice traits. Thus, you should definitely go for containers.

But containers do not fuel the public cloud revolution.

Thus again, you will not benefit from the public cloud revolution in a private cloud. You will not be able to leverage the possibilities of serverless and experience sort of an exponential productivity boost.

The same is BTW true if you decide to use a public cloud offering with a “cloud-ready” strategy. This will also deprive you from the possibility of experiencing the revolution I described. “Cloud-ready” means you limit your tooling to the subset of cloud services, all providers (ideally including private cloud providers) offer alike. This basically leaves you where public cloud was around 2011 (including containers) at best – no revolution to be seen.

Summing up

In the first post of this little blog series we discussed the evolution of the public cloud offerings from their beginnings until around 2015 and how it laid the foundations for the public cloud revolution we discussed in this post.

We also have seen that a private cloud or public cloud implementing a “cloud-ready” strategy will deprive you from experiencing the benefits of the public cloud revolution: Reduced vertical integration depth, becoming a lot faster, having capacity to explore new business ideas and options in short time, creating a competitive advantage – especially if you live in a dynamic, fast-moving market.

Of course, just going for a public cloud offering without adopting your organization and processes accordingly will also not let you experience the benefits of the public cloud revolution. The new options, serverless offers require new organizational and processual approaches to really leverage them.

Finally, you might ask yourself if I am a strong advocate of the three big cloud vendors ³ because this post basically states: You need to go all-in with one of the big cloud vendors to experience the serverless advantages.

To be frank: Personally, I am neither a fan nor an opponent of the three big cloud vendors. It is just that they currently offer possibilities you do not get anywhere else: not on-premises, not with any other cloud provider (most of the competition is still stuck at the IaaS or PaaS level) and not by using any other IT sourcing concept currently available.

Personally, I would really like to have more options than just the three big cloud vendors. Living in Germany, I would especially love to have some strong options offered by European companies. Unfortunately, it does not look as if this will become reality in the next years. All “competing” offerings are at least 5 years, if not a decade behind.

Thus, I am afraid, at the moment we need to go for the big three vendors if we want to leverage the public cloud revolution for us, like it or not …