The industrialization of IT fallacy

Probably you also heard it several times before. Someone comes along saying that IT is a young and immature domain. That we are not yet an engineering discipline. That the way we write code sucks: Slow, not enough throughput, error-prone, not easily repeatable.

Hence, we need to learn from other domains like car manufacturing or house building how to professionalize our domain, adopting their engineering experience. We need to implement more of the industrial practices of, e.g., car manufacturing in our software development process. We need to set up efficient software production assembly lines and make writing code as efficient as building a car. And so on.

The core message is: Software development is highly inefficient. We need to industrialize and streamline our software development process, especially the coding part, guided by car manufacturing assembly lines and alike.

Well, from how I understand it, this could not be any wronger. ¹

Why?

Distinguishing design and build phase

To understand why I think the whole “industrialization of IT” ideas are wrong, we need to start with the distinction of the design and the build phase of a product. ²

It all starts with the idea of a new product. Before you can build it in its final form, you need to design it. The result of the design phase is a build specification – all details relevant for building the product specified and laid out.

Then you can take the build specification to set up the build process, setting up the required tooling, instructing the people involved, and so on. Especially if you want to build many instances of a product, you set up the build process carefully to make sure you can efficiently build many instances of your product.

Design and build phase in car manufacturing

Let us look at the design and build phase in car manufacturing as this is one of the poster children of the “industrialization of IT” advocates.

The design phase typically is a “creative chaos” that is optimized for quick learning:

• Cross-functional teams collaborating in big “war rooms”
• Walls covered with design sketches
• Prototypes and more spread across the tables
• Many experiments with short feedback cycles
• Lots of discussions with people from multiple domains, all struggling to find the best solution within the given constraints

Not only engineers are part of those cross-functional teams, also people from finance, marketing, management, sales, design experts and many more, i.e., a truly cross-functional team.

Their work is done after every curve of the hull is fixed, every cable is in position, every weld seam is defined, etc.

This completed specification is then translated into an assembly process and an associated setup of the construction site: How to lay out the assembly line, where to place the robots, how to program them, and many things more. The goal is to have a highly efficient production process in place that can repeatedly produce cars of high quality. The whole setup is optimized for throughput and repeatability:

• The assembly process and the associated construction site setup are predefined
• Division of labor principles are applied wherever possible
• Automation is maximized
• Opposed to the design phase collaboration and feedback are minimized to maximize throughput

And then the cars are built.

Of course, in reality things are more complex than I sketched it here ³. Also, sometimes you will only discover a design flaw after the production has started – which means cycling back to the design phase. Still, we can quite clearly recognize the very different characteristics of the design and the build phase:

• The design phase is optimized for quick learning.
• The build phase is optimized for maximum efficiency (throughput and repeatability).

Design and build phase in software development

Now let us look at the design and the build phase in software development.

This is where the big misunderstandings usually start. Many people think that the design phase consists of requirements capturing and high-level design, while the build phase consists of coding. The specification would be complete after the last architecture diagram has been drawn.

But this notion is completely wrong!

The product we create in software development is not source code.

Our product is a running application, i.e., one or more binary files we can execute. The means we use to specify this application is (primarily) source code. ⁴

If we let this sink in for a moment, it leads us to the following observation:

The specification in software development is only complete after the last line of source code has been written.

Architecture diagrams and alike are just an interim step on our way to create the complete specification. Sometimes they are a bit like the prototypes in car design. Sometimes they augment the source code on a higher abstraction level describing general structure and guiding principles. But they are never the complete specification. Only the code is. You need the complete source code to create the product, the running application from it.

If you do not believe it: Try to feed an architecture diagram into a compiler. Take an architecture diagram and try to extract all the details from it that are needed to build an application which works in all details as your users expect it to. It does not work.

But if writing code is not the build phase, what is?

The build phase in software development is creating a running application, the binary file(s) from the specification, the source code.

We call that a compiler.

Compilers (and the associated linkers) exist for more than 60 years. Our whole build process is completely automated for more than 60 years (at least, if we are doing it right). Building a product from the specification in software development is merely pushing a button.

In today’s CI/CD world we do not even need to push the (figurative) button anymore. As soon as we check in new code into our repository, a fully automated build, test and deployment cycle is triggered.

This leads us to the second observation regarding software development:

In software development, the build phase is so unparalleled efficient that many people forget it even exists.

At least we have everything in place for more than 60 years to fully automate our build phase. Industrial production sites only dream of such an efficiency level.

If companies do not leverage this potential, it has nothing to do with the immaturity of our domain. It is just a serious mismanagement of some imo irresponsible decision makers in those companies.

Revisiting the ideas of the “industrialization of IT” advocates

After having looked at car manufacturing as the usual example of a “mature engineering discipline” and software manufacturing, let us revisit the proposals of the “industrialization of IT” advocates as I sketched them in the beginning.

They typically draw the wrong line between design and build phase in software development. As the build phase in software development is so unparalleled efficient, they tend to forget it exists and confuse writing code with the build phase.

Hence, they consider the design completed after some high level descriptions of the target system have been created. The car manufacturing equivalent of that would be expecting to build cars from some design sketches and other high level specification you may find on the car design “war room” walls.

As a consequence, the “industrialization of IT” advocates try to apply the build phase practices of industrial production to design phase activities of software development because they consider writing code “inefficient” and thus “immature” – neglecting that it is part of the design.

I can only guess how, e.g., a designer in car manufacturing would respond if you would try to industrialize her detailed car design activities and deprive her from all possible feedback. I assume, not only her but the whole cross-functional design team and everybody up to the top management would immediately expel you if you would come up with such an idea.

They would probably tell you that on the contrary they need the cross-functional collaboration and short feedback cycles to come up with a truly great solution. Instead of minimizing collaboration and feedback, they need to maximize it.

Still, the suggestion of the advocates is to minimize collaboration and feedback in the design phase of software.

At the same time they often ignore the unparalleled automation potential of our build phase in software development, probably because they forgot its existence due to its efficiency.

Thus, if you follow the ideas of the advocates of “IT industrialization”, you end up in an absurd situation that reflects the opposite of what you can learn from industrial production:

• Your design phase is set up in a division-of-labor fashion with minimized collaboration and feedback. Any responsible person from car manufacturing would probably throw their hands up in horror.
• Your build phase consists of many manual steps and poor automation. Any responsible person from car manufacturing would probably shake their head in disbelief.

That is why I say, it could not be any wronger and the whole idea is a fallacy.

But strangely enough, this is what we often see in IT departments:

• Software development processes with many roles that only communicate via artifacts along the lines of some process framework which minimizes direct collaboration and feedback in a division-of-labor fashion.
• At the same time, poorly automated build, test and deployment processes – not even starting with the still manual operations processes to provision infrastructure and alike.

Doing it better

How can we do better?

The answer should be obvious.

First, we need to understand that writing code is part of system design. The product, i.e, the application to be built is only fully specified after the last line of code is written.

Then, we should indeed learn from a domain like car manufacturing: If we want to create great solutions, we must foster cross-functional teams, collaboration, many experiments and quick feedback in our design phase and optimize for fast learning.

If that reminds you of the original ideas behind agility (not the cargo-cult implementations you often see implemented in companies): Surprise! We have everything in place we need – for many years. We only need to apply it in a sensible way.

Finally, we need to leverage our unparalleled automation potential in our build and operations phase. We can automate all these processes for more than 60 years.

Not having this in place in 2021 (at least for the majority of the system landscape) for me borders on criminal recklessness. It not only deprives companies from many options (see, e.g., my blog post “The value of speed”), it means a real competitive disadvantage just based on ignorance.

If we have this homework done, we can leverage the unparalleled efficiency of our build phase: We can seamlessly build, test and deploy our work while we are still designing it. This gives us new options to test product and feature ideas, to gather customer feedback very early, to learn which of our ideas are valuable and should be developed further and which ideas should rather be dropped before we waste a lot of money on them.

If that reminds you of the original ideas behind DevOps (not the cargo-cult implementations you often see implemented in companies): Surprise again!

These are options, all the poor people who build physical products do not have. This is something, every car manufacturer in the world envies us. They would love to have these options. And we tend to throw them away because we do not understand how powerful the tools are we have in our hands.

Summing up

The “industrialization of IT” advocates keep repeating the idea we need to industrialize and streamline our software development process, especially the coding part, using car manufacturing assembly lines or house building sites as examples.

We then looked at the distinction between designing a product and building a product. We saw that the characteristics of designing a product and building a product in industrial production are very different. The design phase is characterized by collaboration of cross-functional teams, experimentation and fast feedback. In contrast, the build phase is characterized by division of labor, automation and minimal collaboration and feedback.

Moving on to software development, we saw that writing code is part of the design phase. The source code is the specification of the product to be built, i.e., it is the outcome of the design phase. Building the product is so unparalleled efficient in software development for more than 60 years that most people forgot it even exists. It is starting a compiler (or today a CI/CD pipeline) which fully automated translates the specification into the product, the executable program.

The demands of the “industrialization of IT” advocates basically mean destroying the required collaborative design process while at the same time often neglecting the unparalleled automation capabilities of our build processes, i.e., leaving build, test, deployment, infrastructure provisioning and management non-automated. This is the opposite of what, e.g., car manufacturing tries to accomplish.

Again, this is why I say, those ideas lead the wrong way. ⁵

Moving on

So, is everything fine? Are we better than all these other domains?

Well, not really.

While I think we are not as bad as many of the “industrialization of IT” advocates and some other parties try to make us believe, there is still a lot of room for improvement in our domain.

Very often we do not leverage our possibilities at all and tend to do the opposite of what we should do. I wrote about quite a lot of topics I think we need to address in IT in my “Simplify!” blog series. And there are a lot more things where we can, should or even must improve in IT.

Therefore, I think while doing fine in some places we should remain humble and always strive for better ways to do software development.

But, “industrializing” coding does not help …