The continuous amnesia issue

In this post I want to discuss an issue that I, being for a longer time in IT meanwhile, observe over and over again. It is the observation that as an industry we continuously forget what we have learned.

What do I mean with that claim?

The very first discussion – again and again and …

Probably I best illustrate this by sharing an experience I had. About two years ago I attended an unconference. It was not the first time I attended it. I liked it a lot. It always had a very energetic atmosphere. The participants were very active, loved to share and discuss. But unlike the previous times, I decided to do something different that year.

As an experiment, I decided not to share myself, not to offer sessions myself where I drive the topic of discussion, but to listen only. I know that this is not the idea of an unconference, but I was curious. As I wrote: It was a little experiment, I wanted to conduct.

I knew that always a very active group of people gathered at this unconference and I wanted to use that as an unbiased opportunity to learn about the state of IT, how people think about topics, what moves them, and so on. So, I looked forward to what I would learn.

To be frank, it was a devastating experience for me.

The session that really killed it was a session about reusability. A group of people came together to discuss reusability. I was really curious as I dealt with the topic for many years already. A great community. A topic I am particularly interested in. I was looking forward to learning something new. I did not expect a lot of new ideas, but maybe one or two.

So, I listened … and could not believe what I heard.

The discussion moved at a level as if nobody ever had said or written a single word about reusability in the past 50 years. I felt set back to a pre-1968 discussion, set back to a time before the NATO Software Engineering Conferences were held. Where had all the ideas about reusability from the 1970s, the 1980s, the 1990s and all the years after gone?

Obviously, nobody in the room ever had heard anything about what we as an industry already had figured out about reusability. They discussed the topic completely from scratch, and due to that in a totally naive way, not touching any of the important learning of the last 50 years. I sat there listening and did not understand what happened. I felt the urge to jump up and scream: “Shut up, all of you! You do not know sh*t about reusability!”

Of course, I did not. Besides the fact that it would have been very rude, it also would have been unfair.

As I wrote before: It was a very active community that gathered there, all of them adding with their best intentions to a vibrant discussion. So, I eventually decided to leave. Not all sessions felt as devastating as this one but I had similar experiences in most of the sessions I attended.

I was quite confused and had a lot to ponder. I went to the conference to learn something new and … well, I learned something new. But it was definitely not what I expected to learn. I learned that the people in the sessions I attended did not seem to have any knowledge about the discussions that we had before in our industry about the very topics.

Continuous collective amnesia

What happened? It kept me a while thinking. Eventually, I realized that I had observed a disease of our whole industry in its purest form: We continuously forget what we have learned. We always reinvent everything from scratch. My personal observation is that discussions in the IT community start over about every 5 years ¹. That is how long we remember as a community. After that we need to rediscover your insights from scratch.

Or how I like to phrase it in a bit provocative way:

In IT, we suffer from continuous collective amnesia and we are even proud of it! ²

Again, this is not about the people who discussed in that room. All of them did their best, they eagerly discussed hoping to gather new insights. They were simply victims of a widespread disease in IT:

We do not value the wisdom of the past.

We suffer from extreme youth obsession, not only regarding the age of people but also regarding how we value knowledge. Old knowledge is considered “worthless”, while new knowledge is considered “great”.

A tweet that Mark, founder of Bitmen Studios sent me in a little conversion we had via Twitter IMO really nicely describes the effect:

We reinvent the wheel over and over again, making the same mistakes again and again, not learning as a community. And as Mark correctly pointed out, the growing technology complexity amplifies the problem – and, as I would like to add, vice versa: Our continuous collective amnesia amplifies our growing complexity problem.

Why is it this way? Why do we loose our collective memory every 5 years?

I am not sure about the causes. My current attempt of an explanation consists of 5 factors:

1. We consider ourselves a dynamic, fast moving industry. New concepts, tools and technologies emerge every day. How can knowledge of yesterday still be applicable to problems of today? That is what we keep telling ourselves. We wallow in our perceived vigor and speed of innovation, neglecting that we keep solving the same problems over and over again, just with different tools and technologies. Yes, some things change, but usually 80%+ of the problems stay the same. But we refuse to see that. We prefer considering ourselves the “masters of new”.
2. We are technology believers. We are convinced that we can solve any problem by simply applying the right tool or technology. E.g., if people have a collaboration problem, the typical reflex is to look for a tool that solves the problem, neglecting that it almost certainly is not a tool problem. The same is true for all other kinds of problems. Whenever we face a non-trivial problem, we try to solve it by applying a tool or technology to it instead of solving the actual problem. This has become such a natural pattern that we do not realize it anymore. And as past tools and technologies did not solve the problem, we look for new ones.
3. Insights grow slower than knowledge. Our knowledge continuously grows, the longer we work in IT. Of course, not everything we learn is valuable. Quite often we learn that some of our knowledge is plain crap. If I, e.g., think back to SOA: Wow, what a crap! Would never do it again this way! But not everything was bad. I also took some great insights from the SOA times. It takes times to separate the wheat from the chaff. In the beginning of our careers we basically learn new stuff all the time. It takes some years until we start to see the recurring patterns, to separate real insights from plain knowledge.
4. We face a continuous stream of new developers. I feel as if I face a new generation of developers every 5 years ³. Maybe that is a very subjective experience. But for me, people just coming from their IT education, being 5 years in business, 10 years or 15 years and more feel very different, like different generations. All those new people in IT have to build their own insights from scratch. They have nothing to build on as in IT we created a cult(ure) of new, celebrating the new and despising the old.
5. If you combine the “cult of new” with what I call “the arrogance of the youth”, you get an explosive mixture. Before you think that I want to talk bad about younger people: I just talk from personal experience. I was a graduate myself years ago and believe me: I was arrogant! I knew better! I had to learn humbleness the hard way, and I am still not sure if I arrived (it is a lot easier to judge an old self than to judge the current self). So, I do not want to reproach any young person for being somewhat arrogant. I guess that is natural and also healthy to a certain degree because it helps against persisting in outdated ideas as an industry. But in a culture of new, this habit works as an unhealthy accelerant.

If we put these factors together, we end up with an environment where we only value new things and despise old things. If we face a problem, we never look if someone solved the same problem before ⁴. We only look for a shiny new solution, ideally a new tool or technology. If we learn a good solution for a problem, we forget it after a few years and start again from scratch.

It is not that individuals do not learn and do not grow insights. They do.

But as a community, we do not.

Unlike other engineering disciplines, we do not create our body of knowledge, foster our timeless insights, work to extract the essence from the solutions we found yesterday and make it available to the community of today. We do not only not create a body of knowledge, we despise it. We always look to the horizon hoping to spot a silver bullet (which does not exist as we know) instead of looking back once in a while and trying to learn from the wisdom of our ancestors.

Moving on

This is what I observe all the time. Again, this does not mean that I do not meet individuals who act differently. But in general, I see the same discussions recurring again and again about every 5 years.

To be honest, I do not have an actual idea how to change this. This cult of new is so deeply embedded in our culture and self-perception that it would take a long time and effort to change it.

Personally, I try to share some of the old wisdom, e.g., with my talk “Excavating the knowledge of the ancestors”. There is also the “Papers We Love” movement that tries to share such knowledge.

But overall this is just a drop in the ocean. It would require a radical rethinking to stop our continuous collective amnesia. We would need to accept that most of our problems are not new, but that most of the times we solve the same problems again and again, just with different technology.

But until we learn this, I am afraid we are not yet an engineering discipline, but rather a bunch of people obsessed with “new”, not learning.

As so often, there would be a lot more to write. But I leave it here. I hope I gave you something to ponder.

And maybe you will come up with a great idea how to change it. If you have one, please share it! We need it – more desperately than most people are aware of …