Solving the wrong problem

I think a lot about AI-assisted and AI-based coding. The first one is a human who writes code with more or less support of an AI solution. We see it all the time already now. The second one is a human leaving the coding part to a fleet of AI agents. If the human does not even look into the code created but treats it as a black box and looks at the solution only from the outside, i.e., only watches what it is doing, it is usually called vibe coding.

Especially in the second situation where the AI agents took the driver seat regarding coding, it is impressive to see what kind of solutions these AI agents are already able to create.

Sure, sometimes they behave like a 4 year old wunderkind with a huge attention deficit that regularly gets distracted and forgets what it was about to do and what kind of constraints it should adhere to while doing its work. Then you have to constantly remind them: “No, this is not what I told you. You were meant to do that.”. Then, the agent apologizes and tries to undo the mess it created while being oblivious of everything around – until it becomes distracted again.

And sometimes, the agent just adds some plain wrong fragments into otherwise perfectly fine code and you shake you head how it can be so far off the mark in just that one place while doing a good job everywhere else.

But overall, it is impressive what kind of ambitious tasks these agents are already able to master and it can be expected that many of the current unreliabilities will be smoothened out over time.

Still feeling wrong

However, it always leaves me with the nagging feeling that something does not fit, that in the euphoria what has become possible we stopped to ask if it all makes sense, if we are addressing the right problem. I do not talk about the fact that AI-based coding does not help to solve the actual problems we have in IT. I wrote about this including some possible consequences of AI-based coding at length in a prior blog series. While all I have discussed in that other blog series IMO is relevant, too, it is not what I mean here.

Here, I pushed all these other considerations aside and simply said: “Okay, let us do it.”. But even then, there is still that nagging feeling that something is not alright. I am still not sure if I am able to put my finger on it precisely but a few issues popped up while pondering this feeling.

The basic functional principle of a LLM

If we step back several steps (which rarely anyone does these days based on my perception), we run into a very different series of questions. In their core, LLMs run on probabilities. They receive some input and based on what they have seen during their training, they provide the most likely next token, the one that makes most sense based on what they have seen during their training.

They respond in much smarter ways than traditional multi-layer perceptrons or traditional RNNs like LSTM networks because their attention mechanisms helps them to make much better sense of the input they were provided by selectively paying more or less attention to different parts of the input (including their output generated so far). This is similar to what humans do. If provided with some textual input, we do not pay the same amount of attention to every single word. While creating our answer, different parts of the input are relevant for different parts of the answer, our output.

Nevertheless, their answer is based on what they have seen during their training. Even if we add tools as it has become the norm with agents, in the end their responses are still based on what they have seen during their training. If we ponder this fact, it leaves us with a conundrum.

Agents create code that is based on what they have seen during their training. They cannot be creative in the traditional way. Without useful information in their training data, their functional principle makes them interpolate between areas where they have training data, which is usually called “hallucinating”. This means, if the agents create useful code most of the time, they must have seen something similar during training.

They do not need to have seen the whole solution in their training data. Instead, they might have seen fragments of it and ways to recombine those fragments. And they must have seen the stuff, the fragments and how to recombine them, not only once but many, many times to recall them in a reliable way.

To illustrate the point: Some people told me that their agentic AI solutions completely got stuck creating some Rust code from scratch. To work around that problem they told their agent to create a solution in Python first and after this worked properly, they told the agent to translate it to Rust.

The problem was that the models have not seen enough Rust coding during their training. Therefore, generating Rust code from scratch did not work properly. However, they have seen lots of Python code during their training which made it easier for them to create proper Python code. They also had enough data in their training corpus to know how to translate Python code into Rust code. Thus, the detour over Python worked while the direct approach did not.

Reinventing the wheel time and again

This leads to the question: If the code the LLM uses as basis for the generation is so often contained in the training data of the LLM why is it not available as a higher level building block like, e.g., a library, a framework or even some piece of standard software in the first place?

Are we so busy reinventing the wheel that we forgot to improve our situation in software development?

If it is so easy for an AI agent backed by a LLM to create working code, it additionally points to the fact that we rather do the same stuff over and over and over again instead of creating sensible higher level abstractions. This leads to the question:

Are we doomed to reinvent the wheel over and over again?

As I wrote further up in this post: I cannot put my finger on it precisely but this feels sort of off: We use LLMs to create the same code that has been created a million times before over and over again instead of thinking of better ways of dealing with the issue? Hmm …

Compensating for a lack of knowledge

Additionally, when it comes to the latter question, we must be a bit cautious. If we, e.g., go to LinkedIn, the meeting point of all those self-marketers that claim to be at the forefront of innovation and see them blaring how unbelievable crazy it is what they just vibe coded with some AI agents in a few hours which would have cost them weeks before, we must carefully look at who is blaring and what they are doing.

Almost always, we see some people who do not code for a living. They might have coded in the past and maybe they still do some fun coding. But barely anyone of them writes production code that needs to run reliably and must be changed continually over the next 20+ years. Most of the time it goes like: “I just wrote an iOS app. I have no idea of Swift but was able to create a working app in just a few hours, a task that would have cost me months otherwise. Vibe coding is the future!”

If we let this sink in for a moment, we quickly realize that most of the barkers usually reinvent the wheel. They basically implement something that has been implemented a thousand, if not a million times before. Their only problem was that they did not know the language or its ecosystem.

In other words: They were able to create a fun application that would have cost them a lot more time otherwise, but not because the coding would have taken so long but because of their lack of knowledge and expertise. ¹

It is obvious that AI agents can speed you up massively if you do something that has been done a million times before but you lack the required knowledge and expertise to do it yourself. Still, it is reinventing the wheel.

The hustlers game

Side note: Some of those self-proclaimed hustlers of course use AI agents to implement their “groundbreaking” software solutions that will “make them rich in no time” without knowing anything about software development and either not being willing or not having the money to pay software developers. However, it has still to be proven that this approach will work beyond the stage of early prototypes needed to validate product-market fit.

Do not get me wrong: It is great that people can turn their ideas into software without needing to learn software development, programming languages, ecosystems and all that before they can create their first line of code – or having to pay software developers to do it for them which can be inhibitory expensive. But this has nothing to do with writing production code that needs to be secure, to run reliably in production, to be maintainable and evolvable over a long period of time, etcetera.

Writing production-ready code

This brings me to my second nagging feeling. Writing production code is a completely different game than (vibe) coding a prototype. To get this right, you need profound knowledge of the programming language, of its ecosystem, of the dos and don’ts of good software design and development, of the additional bolts and screws needed to make the software production-ready, and much more. This is a game where it does not only count what the code does but also how it does it, where details can make a huge difference and you need a lot of knowledge and experience to get it right.

Interestingly, you rarely hear experienced software developers gushing how much AI agents would have sped them up. Quite some of them also use AI agents or at least AI assistants but they do not talk much about incredible productivity gains. They rather talk about how these agents can take over the boring parts and leave the juicy and hard ones for them. They do not talk about speed or productivity gains but about convenience. The difference between them and the LinkedIn hustlers is that they know their craft.

I think, it is fine if those people use AI agents for a higher degree of convenience, for having more fun at work. Having fun at work is a massively undervalued motivation factor. But the key point is that for those experienced software developers AI is not the revolution many people want to make us believe. It is nice, it makes things more convenient and more fun – which is good – but the “revolutionary” part is missing.

Crappy code

But most of the software engineers are not like the ones I just described, you may say. Otherwise, there would not be so much hastily written crappy code being deployed to production that is not secure, does not run reliably in production, is neither properly maintainable nor evolvable, and misses all the other properties of production-ready code. This kind of code is the majority of code, you may say. Or as Paul Graham recently tweeted on X:

“The reason AI coding works so well is that the source code of the median app was already slop before LLMs.”

Thus, would it not make much more sense to use that as a baseline for average production code?

If we use that as our baseline, then even if AI agents would not make the aforementioned experienced developers faster, would they not make less experienced developers faster? Would it not be cheaper to create at least the same subpar code quality we just declared our baseline with AI agents?

I mean, if decision makers see that the code they get either from their own or some external developers is usually crappy and that it takes a perceived long time until the crappy code is written, is it not understandable that they go for the promises of the AI vendors and push towards AI agents to take over software development?

And yes, crappy code prevails. Most enterprise software code is just good enough at best. Often, it is not. But addressing the problem by leaving the coding to AI agents at least for me feels like solving the wrong problem.

How hard can it be?

From all I see, the problem starts with a lack of education of the average software developer. For many years, IT in general and software development in particular were considered a secure job machine, a safe bet when it came to jobs. Therefore, many people turned towards software development when their original jobs did not work out: “If I cannot make ends meet with my original education, let me become a software developer. How hard can it be?”

For some reason, there was (and still is) also the illusion that it is easy to become a software developer: “How hard can it be?”. But as learning to build a box by sawing a few planks and nailing them together without sawing your fingers off does not turn you into a proficient carpenter, just learning to code a bit does not turn you into a proficient software developer.

Nevertheless, the idea that learning software development is easy persisted and to meet the seemingly endless demand for software developers, lots of retraining courses sprung up that promised to turn anyone into a proficient software developer in just a few months. A result of this development was (and still is) that most people in IT lack the very basic software engineering skills needed to create, run and maintain good software when they start their IT careers.

I mean, go to an average IT department and ask the people what education they have. The fewest of them will tell you, they have a computer science education. When I asked people, I heard virtually everything, even studying philosophy. While studying philosophy can help you to learn a lot about humans, their behavior and the world in general, it for sure does not teach you how to write production-ready software.

Even if people studied computer sciences, it is still questionable if they learned how to write production-ready software. But at least they learned some relevant foundations that make it a lot easier for them to understand how to write production-ready software.

All this leaves a huge gap between the average eduction of a software developer when they start their careers in software development and what is needed to write production-ready software – which brings us to the continuous training needed keep our skills sharp.

Paring down education

IT is a domain that never stands still. Even if we mask out the short-lived fashions, IT is in a steady motion. Skills we have picked up 10 or 15 years ago may have become counterproductive meanwhile. We need to continually adapt. This means, as a company it is important to train your employees regularly to keep their skills sharp.

Then we look around to see how much money the average company invests in training its people and keep their skills sharp. And we realize somewhat disillusioned: Barely enough that they can do their jobs at all – and sometimes not even that.

Training people costs money and does not create short-term profits. If people do work instead of attending trainings they create immediate output which in an efficiency-centered environment is used as the measure of expected revenue. Consequentially, employees including software developers are massively undertrained and often barely fit for their jobs.

They usually start their jobs without knowing how to write production-ready software. They usually do not get the training how to do it while they are on their jobs. And even if they knew it at one point in time, they often lose the skill because IT moves on and they are left behind without getting the time and training to catch up.

The consequence of all this, plus the urge to churn out features as quickly as possible is exactly this: Crappy code everywhere.

Handing over software development to AI agents because software developers are too poorly educated in average to create solid, production-ready code at a reasonable rate, leaves me with the nagging feeling we are solving the wrong problem. For me, all this feels like:

We perfected a system that creates crappy code. Now, let us use a machine to create crappy code faster.

If we would create much better code with AI agents, it would be a different story. But the agents and LLMs are bound to what they saw during their training – which is exactly the crappy code we suffer from and which creates problems of all kinds, from becoming slow in software development to non-reliable software in production.

Creating waste faster

We could now dive into the question why companies always want to speed up software development. But as I discussed that question in many posts already, here I will touch it only briefly. For more details, see, e.g., my “Simplify!” blog series where I discuss a lot of drivers of accidental complexity and how we could address them. It is mostly this disproportionally growing accidental complexity that fuels the continuous demand to speed up software development, to deliver more features in less time.

Most of the features demanded do not create any value, i.e., they are waste (see, e.g., my “Forget efficiency”) blog post for more details). But instead of asking the question that if the desired business impact does not manifest, might it be because of requiring features that do not create value, a self-reinforcing cycle is triggered which – put simply – goes from “We need more features!” to “Our actions do not show the desired impact” and back to “We need more features!”.

The implicit assumption in efficiency-driven companies (i.e., almost all companies) is that every feature requested creates value (and thus impact/revenue) per definition Thus, according to this school of thought a lack of impact can only mean that not enough features were implemented.

At the same time, this huge number of valueless features clog the codebase. They make it increasingly harder to understand what the code does. Every new feature must be programmed around all those useless (and the few useful) existing features which slows development down. All this results in a vicious circle: More and more features are requested in less and less time, making it harder with each (usually) pointless feature to implement the next one.

But now we got agentic AI: AI-based agents that can create software at a much faster rate! Now we finally can create as many features as the business departments want to have. Now everything is going to be fine!

To be honest: I doubt things will be fine. The vicious circle will just spin faster. To put in other words:

Most of the time, we build useless features. Now, let us use a machine to build useless features faster.

Another nagging feeling that we are trying to solve the wrong problem.

Better documentation to the rescue

Finally, I would like to mention an irony. If you talk with people who use AI agents a lot for software development, they will tell you that the quality of the requirements and the architectural design and documentation make a huge difference regarding the quality of the code produced by the AI agents. Thus, it would be necessary to put more effort into phrasing requirements and designing a good architecture.

This is the point where I do not know how to respond to it. I really do not know.

I mean, for at least the last 3 or 4 decades, one of the biggest impediments we had regarding fast and reliable software development were poor requirements and poor architectural design. When pointing it out, improving the situation was (and still is) always turned down because: “We do not have the time for this. We need to deliver the features. C’mon! Don’t make a fuss! It can’t be that hard.”

But when an AI agent comes around the corner that struggles to create reasonable results if the requirements or the architecture are poorly designed and documented, suddenly it is perfectly fine to take the time to improve the design and documentation while it was turned down all the time if human developers asked for it?

Human software developers had the same problems with poorly designed and documented requirements and architecture for the last decades, but nobody cared. Instead, the only complaint was that writing software would take too long.

Again, it adds to the feeling that something is off balance.

A powerful solution for the wrong problem

As I wrote in the beginning: Even though it is very impressive to see what we can do with AI in software development these days, I still have the nagging feeling that something does not fit:

• It starts with the functional principle of LLMs and the agents based on them that lets me wonder if we failed to create abstractions for ever-repeating tasks and thus are doomed to reinvent the wheel over and over again.
• It also makes me cautious that most of the loudest fanboys (and fangirls) who blurt out how unbelievably more productive agentic AI would be are not people who regularly write production code. What I see there is usually just a lack of knowledge, mitigated by the machine, not actual productivity gains.
• When looking at experienced software developers, I do not hear them talking about huge speedups so much but rather about an increase of convenience.
• When I look at the argument that AI agents would help to improve the overall software quality, I see a lack of education, reinforced by a very short-sighted investment strategy that misses the reality (not only) of software development as primary reason why we see so much crappy code.
• Additionally, AI agents are bound to create software that resembles what they have seen during their training – which is mostly exactly the crappy code people think to get away from with AI agents. Thus, it is unlikely that software quality will improve significantly with AI agents.
• When looking at the demand to deliver features faster all the time, I see people not understanding post-industrial markets, running on outdated governance models, business departments demanding pointless features, etcetera, all leading to disproportionally fast growing accidental complexity in software that does not deliver any business value but massively slows down software development speed and reliability in production.
• As AI agents struggle with poor requirements and architecture design and documentation, it is commonly accepted as “best practice” to invest more effort into these tasks while the same was turned down all the years for human software developers who struggled equally with such a lack of quality.

And the solution to all these issues should be “Use agentic AI for software development”?

Maybe you understand that nagging feeling a bit better now.

To me, it feels like AI-agent-based software development is used (again ²) as a welcome excuse not to address all the aforementioned, highly relevant issues.

Instead, it is used to address the perceived problem that software development needs to become faster, i.e., to deliver more features in less time without pondering if this our actual problem.

That is why I think we are solving the wrong problem (again).

Moving on

LLMs are an impressive piece of technology and it is also impressive what AI-based software development solutions are capable to do. But if we just apply them mindlessly everywhere, I am afraid it just cements the status quo and amplifies the problems we currently have in IT in general and in software development in particular.

Personally, I think it would be nice if we were able to use such an impressive technology in a smarter way. Just assume, we would solve the aforementioned issues and then apply agentic AI. I think, that would be really impressive …