Software - It’s not what you think it is - Part 1

The current AI hype is accompanied with a lot of predictions that software development will be taken over by AI solutions soon and most software developers will lose their jobs together with most other white collar workers. While I agree that AI solutions will have a significant impact on software development, I disagree with the notion that software development will be taken over by AI solutions anytime soon.

Based on my observations, this narrative is strongest among people who do not really understand software and what it means to create software. To be a bit provocative: I think, it will be a lot easier to replace each highly paid manager who eagerly awaits to replace all their software developers than to replace the software developers they want to get rid of. I mean, most of those managers act in very predictive ways, not showing any signs of actual creativity. Software development on the other hand is a different story.

Putting provocations aside: What do I mean with that?

From all I see, most people do not understand software and software development at all. Especially, most fail to understand the following six misconceptions regarding software:

1. The assembly line fallacy (this post)
2. The broken abstraction dilemma
3. The greenfield fallacy
4. The value preservation dilemma
5. The invisibility dilemma
6. The malleability curse

There are more persistent misconceptions regarding software and software development, but from what I have seen those six hurt us most. So, let us go through those misconceptions one by one. I will discuss the first misconception in this post. The other misconceptions will follow in subsequent posts of this little blog series. At the end, I will summarize everything and what we can (or should) learn from it with respect to modern AI solutions and – even more important – with respect to the humans affected.

The assembly line fallacy

The probably biggest and most persistent misconception is the assembly line fallacy. This fallacy stems from the attempts to compare software development with the construction of physical goods like, e.g., cars. Those attempts in turn stem from the software invisibility dilemma we will discuss in a later post.

The comparison and the resulting metaphors basically all work like this:

• Creating a software architecture, especially creating some complex diagrams with lots of boxes and lines, corresponds to creating the blueprints for a physical good.
• Writing code corresponds to building the physical good along an assembly line.

Hence, a software developer is nothing but an overpaid assembly line worker, the reasoning goes. Therefore, for at least 40 years, people try to automate this “assembly line” and get rid of the workers, the software developers – the whole “industrialization of software development” movement is about this.

The problem is that all those people totally misunderstand software development.

Best assembly lines of all industries

If we try to uphold the broken metaphors for a moment and take a closer look, we realize that IT has the best and most efficient assembly lines of all industries. All other industries would happily spend billions of Euros to get such assembly lines.

The point is that we do not ship source code. Our products are executable software programs. We create these programs (our products) using a compiler and linker – or these days rather a CI/CD pipeline. Those are our “assembly lines”: Compilers, linkers and CI/CD pipelines. We merely need to push a button (or enter a command) and we turn our specifications into ready-to-ship products.

And if we need multiple instances of the same product, we simply create a copy. We can create as many copies of the same product as we like at virtually no cost. Try this with a car (or any other physical product)!

Source code is design

But if the executable is our product what is our source code?

Well, the source code is the complete specification of our product, the complete “blueprint”. To stress the car manufacturing metaphor once more: Only after the last line of code is written, we know the position of each cable, the exact curvature of each metal sheet, the exact positioning of all instruments on the dashboard and their wiring, and so on. Only after the last line of code is written, we are able to build an instance of our desired final product. ¹

Architectural designs are high-level designs

But if the source code is our design what are our architectural designs?

Are all those architectural diagrams, specifications, constraints, styles, etc. just unnecessary waste as some software developers claim? What are they if they are not the blueprints we use in house construction, car building and alike?

(Mis-)using the car building metaphor again: The design of a car does not start with a blueprint. It contains many iterations, many ideas, many discussions, looking at the car from many different angles like exterior design, interior design, material engineering, powertrain design, electric design, air resistance, cost to build, software, safety, and many more.

This results in a series of high level designs and prototypes (often just to discuss a single aspect of the car) which support a long series of discussions involving all the needed subject matter experts with the goal to figure out a high level design that supports all the relevant drivers and forces. This high level design is not yet the blueprints used for construction but a commonly understood basis for the detailed design work that leads to the blueprints.

Sometimes, while working on the blueprints someone may find a flaw in the high-level design which in turn leads to an update of the high-level design, including the discussions needed with the various subject matter experts to agree on the updated design. The high-level designs create a common basis for all the detail work of the different subject matter experts involved.

Again, the car building metaphor is not too useful when it comes to software development. Still, the car high-level designs are something that resembles software architecture designs. Thus, you may consider software architecture being something needed to create a common ground for everyone involved and that everyone should agree upon. We all know that for many reasons this often is not the case but this is a different story outside the scope of this post.

Confused by the waterfall

One of the reasons for the die-hard misconception is that the whole software development process often is still executed in an assembly-line fashion, often called “waterfall”. This fashion stems from a phase in IT around 50 years ago when it needed to develop from its pre-industrial setting into a more advanced setting which was the industrial setting.

If the whole process looks like a big assembly line, it is easy to confuse software developers with simple assembly line workers who just need to implement the designs architects designed at the preceding work station. However, IT arrived in a post-industrial setting many years ago, unfortunately neglected by many companies.

Reversing the reasoning

Now that we have seen that writing code is actually part of the design and not part of the construction process, let us return to car manufacturing and at least once use that metaphor for something useful, which is to understand how broken the notion of a software developer being an assembly line worker is.

Cars are designed by highly cross-functional teams: exterior designers, interior designers, material experts, mechanical engineers, electrical engineers, air resistance experts, finance, marketing, sales, etc. ². The whole design process is highly collaborative. Of course, the different experts withdraw once in a while to do some work on their own. But all the learnings are shared and most of the design process is highly collaborative and sometimes may even feel a bit chaotic for a person from the outside not familiar with it.

Now let us apply the “industrialization of IT” recommendations to car design: Let us go to car manufacturers and tell them, for the sake of making things more efficient they need to re-organize the whole design process. Tell them, they need to have a few lead designers who completely create the high-level designs based on the requirements all the other subject matter experts had written down in detail upfront. And then some “implementors”, all coming from the same area of expertise, need to fill in the gaps, carefully kept away from the subject matter experts.

How do think, the car manufacturers would respond to this as-a-matter-of-fact presented advice?

If they had a really good day they would let out a guffaw – and if you were smart you would join in as quickly as possible, try to make it look as a deliberate joke and never mention it again. But usually, they would stare at you in disbelief for a few seconds and then simply kick you out without another word and bar you from ever coming back because they know this is probably the worst way designing a good product. Well, at least for their physical products they know.

Unleashing the real power of software development

However, building a physical product is very expensive. It is not only about the production costs themselves. It also is about setting up the assembly lines. Also, updating thousands or millions of already built products later usually is by orders of magnitude more expensive than building them correctly in the first place. Thus, the big design effort upfront makes a lot of sense when it comes to physical products.

Updating software is a completely different story. Our build process is so efficient and inexpensive, also our distribution process these days usually has become very inexpensive, too. We also can create wildly incomplete products that still make a lot of sense. This means, in software development we have completely different options to move forth and back between design (including implementation) and construction most manufacturers of physical goods do not have.

We even can do continuous design while evolving the product. Manufacturers of physical products can only dream of such options. However, way too often we still do not leverage our unique options and try to organize our software development based on broken metaphors and misconceptions derived from them.

Overcoming the assembly line fallacy and unleashing the real power of software development by turning it into a highly collaborative, truly cross-functional, continuous design and exploration process would be a great starting point. It would also help us to understand better where AI can truly support this process and which parts it cannot take over.

First interlude

In this first post of this series we discussed the assembly line fallacy, the misconception that software development is the same as, e.g., building a car. We have seen that building software is so efficient that most people have forgotten it exists. Our product is running software, not source code. We build our products using a compiler and linker, or more recently a CI/CD pipeline. Writing code on the other hand is part of the design process. The design is not complete until the last line of code has been written. Therefore:

Software developers are designers, not assembly line workers.

In the next post of this series, we will discuss the next misconception, the broken abstraction dilemma and its consequences. Stay tuned …