The purpose of architecture

Software architecture is one of the things in software development that is important. At least that is what many people – especially software architects – claim: It is about the “important decisions”, the “decisions that are hard to change later”, and so on.

To be frank, doing architectural work myself for more than 25 years meanwhile, these statements always left me a bit unsatisfied. It always gave me a bit of a “We also do not know what architecture is about. Somehow we believe it is important and to secure our jobs we need to talk everybody else into believing they cannot do without us.” feeling.

The consequences of a missing purpose

Also, this pretentious “our work is so important” attitude led to a bad development in the mid-1990s when leading architects proclaimed the “age of architecture-centric software development” where architectures for eternities will be designed by super-smart architects and dumb developers then can fit the irrelevant business logic into these ingenious architectures. Okay, the architects back then did not phrase it exactly as I did here, but the message they sent was a lot like it.

Of course, it did not take long for the tit-for-tat response to follow: In the late 1990s the agile movement started to gain momentum. One of the first popular agile methods was XP (Extreme Programming), developed by Kent Beck.

When Kent Beck traveled around and presented XP, it sounded like sort of an antithesis to architecture-centric software development. A bit simplified he said that the architecture will be created along the way while implementing, testing and refactoring business functionality, that there is neither need for nor value in a detached upfront architecture design.

A bit later Scrum became the predominant agile method. Scrum only defined three roles: The product owner, the scrum master and the team member. This gave rise to sort of a “follow-up” misunderstanding, amplified by several “Agile” Pied Pipers who tried to attract developers. The story was like “Scrum does not define an architect role. Thus, architects are not needed in Agile software development projects.”

Well, Scrum being a domain-agnostic method also does not define a developer role, but nobody cared. There seemed to be a several decades old thorn in the developer’s flesh, that architects in the past often behaved as being “superior” to developers, fueled by broken career models that placed architects above developers on the career ladder. And the agile movement finally provided the basis for retaliation. At least that is how I often perceived it.

Without trying to judge here who is right and who is wrong (personally, I think there was a lot of right-doing and wrong-doing on all sides involved), this led to a situation where architectural work often was considered a byproduct of development work.

So, within 10 years we went from “Architecture is the center of the IT universe” to “Architecture is an incidental byproduct of some other work”.

And still we have not clarified what it is good for – which IMO is the core reason why the perception of the topic could turn from “super-important” to “totally irrelevant” in just a few years. How can you know if something is relevant or not without understanding its purpose? Without understanding the purpose, we are merely limited to opinions.

In search of a purpose

In the recent years, the pendulum swung back a bit from architecture being a byproduct. Many agile teams learned the hard way that it is not that easy, that there still seems to be a value in explicit architectural work. But we still lack a purpose. Being “about the important decisions” or alike is too vague and also presumptuous in most cases.

Another problem is that without understanding the purpose – the “Why” – of a task the effectiveness of the activities and thus the outcome is accidental. If you do not understand why you do something you cannot judge if what you are doing creates value, is a waste of time or even destroys value. It is random. You lack direction. We need the purpose to create focus and effectiveness.

So, what is the purpose of architecture?

A purpose I came up with is:

We need architecture to minimize the overall costs of a solution over its lifetime without violating the required runtime properties (which may change over time).

This is a quite dense sentence that needs a bit of unpacking.

Let us start with looking at the software quality attributes. In a good software architecture education you will learn that these attributes are central to architectural work, being one of the essential prerequisites that you need to understand before you can start with software design. ¹

Looking at the quality attributes you find two distinct types of attributes:

1. Attributes describing the properties of the solution at development time (including build and test), e.g., evolvability, maintainability, changeability, and more.
2. Attributes describing the properties of the solution at runtime (usage and operations related), e.g., availability, performance, security, usability, observability, and more. ²

The first set of attributes basically targets the cost of change, how complex (and thus expensive) it is to extend and change the given solution at any given point in time. This includes a lot of imponderabilities as precise decision making would require knowing all future requirements affecting the solution over its lifetime, which obviously is impossible but in some very rare special situations.

Discussing this in detail would require at least another post (and I will pick that topic definitely up in some future post). Here I will leave it with the observation that optimizing for cost of change is not a deterministic process but basically based on heuristics, i.e., it is a probabilistic process with all its intricacies (and thus often subject of heated discussions).

The runtime related attributes are of a different kind. E.g., it can be measured precisely if the response time of the system is below a given threshold for a given percentage of all requests. It can be tested, it can be measured at runtime. Here we do not optimize properties in a non-deterministic way, here we make sure that measurable properties stay inside predefined boundaries. This is a deterministic process.

The two different kinds of attributes and related properties are the reason why the aforementioned purpose definition consists of two clauses – the first one addressing the development optimization part and the second one addressing the runtime constraints.

Still, the first part is not yet complete. While the quality attributes only address cost of change, it is not sufficient just looking at them. Two little examples to illustrate the problem:

• A required change would cost 100.000 EUR. By integrating a commercial product to the solution the cost of change can be reduced to 30.000 EUR. Additionally, it is estimated that the commercial solution will save additional 10.000 EUR per year in the next years regarding cost of change. Unfortunately, the license costs of the product would be 100.000 EUR per year. While it would perfectly make sense from a pure cost of change point of view to integrate the commercial product, an overall cost evaluation reveals that economically this solution does not make sense.
• Investing in better UX design for a B2E solution, making sure that users of the system can pick up changes easily would cost additional 200.000 EUR per year of development costs. As immediate value is not seen and it only makes the solution more complex, the money is not spent. This means that twice a year (a new version of the solution is rolled out twice a year) all users need to attend a full-day training to learn to use the new version. The full costs of an employee using the solution are 500 EUR per day in average. 2.000 users are using the solution. Without calculating the losses for the users not being able to do their regular work for the two training days per day, the training costs alone are at least 2.000.000 EUR per year. Again, while a pure cost of change consideration suggests not to invest the money for better UX design, the overall cost evaluation reveals that economically it does not make sense.

It is a bit like in “The Principles of Product Development Flow” by Donald Reinertsen: You first need a comprehensive enough economic framework you can use to evaluate your decisions and actions against. Only then you can understand if they economically make sense.

Regarding architectural decisions an economic framework just taking cost of change into account is not enough. All types of costs need to be taken into account across the whole lifecycle of an IT solution. Besides development costs, this incorporates testing, deployment, operations, maintenance, license, infrastructure, usage, administration and more types of cost.

You also need to take lost opportunity costs into account, i.e., the revenue that you will not make if your solution or a certain feature of it will be delayed. Especially in a winner-takes-all market this can be more important than all other types of costs. Also penalties for not meeting a given date, e.g., in the context of legal constraints, need to be considered. And so on.

Overall, this economic framework is not trivial and as written before, optimizing for it is a complex, non-deterministic task. Still, if architectural work shall unfold its whole value it needs to be done.

Putting humans back into architecture

We could leave it here as we have a nice purpose of architectural work. It is nicely actionable and we can use it to decide if certain activities create value from an architectural point of view or not.

But even if I use this purpose as a basis for my architecture trainings, it still left me a bit unsatisfied. First, it feels a bit cold and technocratic. All decisions reduced to money and metrics to be met. Quite some people get a shiver when pondering it. I know that many people think that being “professional” requires cold and technocratic thinking, but still …

Besides it still leaves room for better or worse interpretations. Especially the optimization for overall costs combined with a non-deterministic approach based on heuristics leaves a lot of room for harmful interpretations where misinterpretations of cost efficiency lead to bad architectural decisions.

Thus, to me it still felt as if my definition is not complete, as if something is missing to nudge it in the right direction.

That missing piece came to me by chance. I watched the documentation anthology “Abstract” at Netflix where different designers were portrayed. Those designers are not software designers. Still, looking at people in other domains often gives you a fresh perspective on your own work.

One episode was about Ilse Crawford, a quite famous interior designer, and she said a very interesting sentence:

Ultimately, design is a tool to enhance our humanity. – Ilse Crawford

That sentence immediately resonated with me and I wrote it down. I let the sentence sink in over the course of the next days and it became clear to me that this was the missing piece:

Design is always about humans and their needs. Machines do not care about design. As long as the computer finds an interpretable next instruction it will continue to do its work ³. Humans care about design. They care about easy changeability, quick response times, privacy, high availability, correct functionality, good learnability and all that. All the quality attributes in the end are about human needs.

Thus, it perfectly makes sense to put human needs in the focus of architectural work:

The purpose of software architecture is to maximize the well-being of all people affected by the given software over its lifecycle.

If you think it through, it covers the other definition I presented before, and additionally it supports to nudge the interpretations in the right direction:

• Developers want a system they can easily understand and change without having nightmares.
• Users want simple approachability and a solution that supports them smoothly with their tasks.
• Administrators want an easy to operate system that reports its state at all times and signals in an easy to understand way if it needs support by the administrator.
• The business departments want a system that allows for quick changes and supports continuous learning.
• The CFO wants a system that does not cost more than needed.
• Users also expect quick response times and high availability. Additionally, they expect privacy and adjustable authentication levels (without compromising privacy).
• The audit department needs legal compliance.
• …

All the different stakeholder groups have different needs and if they are not satisfied they feel bad ⁴. Even if a CIO wants to use the solution to boost his or her career, it is a need – maybe not a desirable one but still a need (and you may find it more often than you would expect).

The job of the architectural work is to balance all the needs and to find a design that supports the needs as good as possible without loosing the lifecycle of the system out of sight.

In the end, a good architecture maximizes the overall well-being of the people involved now and in the future.

I like this purpose a lot because it brings the human back to architecture.

You may argue now that it is very hard to implement this purpose in a good way, and I agree: It is hard and a continuous struggle. Due to the non-deterministic nature of this optimization task you never know for sure if you really met the optimum. Still, you can use my first, more technocratic definition to make the decision and then use my second, more humane definition to validate and potentially readjust it.

Summing up

With the purpose in mind – both the technocratic and the humane one – it is a lot easier to determine which ideas and activities point in the right direction, which point in the wrong direction and which of the widespread architecture “best practices” turn out to be just ends in themselves.

I do not know if the purpose of architecture I have described here is the best one possible. Most likely it is not. There is always room for improvement and maybe I will find a better purpose in the future. But up to then I am quite happy with the one I have.

After many years of searching, for the first time I have found a purpose that I find satisfying and that helps me to better shape my activities than all the purposes I have seen before. Also, it puts humans in the center of architectural decisions what I personally like a lot. ⁵

I hope, I gave you a few ideas to ponder. Maybe you have another good idea to add to the discussion. If you do, please share it. We need a good purpose. I like the one I found but that does not mean that there is not a better one …