Agile Mythodology
That “y” in the title is not a typo but a reference to the myths many believe in when they choose to apply the “Agile Methodology”. I promise I won’t make it any easier for you to make decisions. The only hope is that it will become clearer. Still here? What does “agility” mean to you? In this business, many only think of “speed”, as it appears to be the ultimate goal of other, more correct definitions. Let’s run with it.
Can you speed up art?
Think about an artistic process. How does it work? Can you prescribe it, or script it? Try an easier question – how to recognize art? I think that human history has taught us that this is very hard or impossible. Were it easy we would not have had so many artists recognized only posthumously. We’d also be able to engineer that recognition and, whenever we are able to do that, we’re very close to being able to create it too. Would you say that generative AI can produce art or is it the prompts and subsequent recognition and choices that humans make that drive it? Art requires something more, a context, a goal, a perspective, something we can’t yet quite define but that evokes individual thoughts and feelings. It isn’t about perfection, which is why it may be the only thing that can be perfect.
How does art compare to engineering? Engineering is about making some choices about how to work with physical realities, followed by the actual building of the creation. In engineering the building part is prescribed, in art it is a part of the creative process. In engineering the decisions are much more limited and straightforward. If a bridge needs to be built, a bridge is what needs to be built, not something anything that may evoke thoughts of it whichever way. Art has many more degrees of freedom and decision making in its creative process. We can see the two intertwine, when we let art percolate into engineering, for example to produce striking buildings. When it goes the other way, and we let engineering affect art, it is usually seen as a necessary but undesirable compromise.
Isn’t making software just plain engineering? Let’s think this through. It is almost as if it does not deal with the real world. The hardware limitations today are laughable compare to what they were at the beginning. Other factors became much more important than the physical, engineering kind of efficiency of the code. It matters how the code will be thought of, evolve, interact with other, future, yet-unknown code. There is no building part in a traditional sense. The moment the creative process is done, the software is done. It’s not like anyone has to dig the foundations, lay bricks or connect the beams. The one and only, creative part of making software is much closer to art than to engineering.
In pure engineering we can reasonably well estimate how much time will be needed to make whatever is required. We have good data on how long decisions take and even better about simple mechanics – say how long does it take to paint walls, cut wood or record sounds. Now switch to what mechanically appears similar … but isn’t: how long does it take to paint an oil painting, carve a sculpture or create a symphony? Suddenly we run out of answers.
So, why is everyone trying to prescribe how to make software or, at least, estimate and plan for how long it will take? Why are some intent to apply purely engineering practices? “We have a greater supply of the nuclear green paint and it sold well. It must be good to use more of it in your paintings.” Could that work? Let’s stretch our imagination further and assume that it could. Should other “artists” apply the same suggestion because it worked for one?
The software development business is …, well business. Business has its own realities, and constraints and strives for better ROI. Pressures are high and mean the difference between making it and… failing. Imitating the success of others seems like a less risky path than doing something very different, disruptive. Trouble is that businesses aren’t all equal. Those that differentiate well tend to do better. What made one successful may be the cause of failure for another.
I’ll call out a major example that was rather significantly, yet often incorrectly, applied to software development. Toyota, a car manufacturer, found its lean manufacturing practices to be of great value to them as they optimized the logistics of parts needed for the assembly, thus reducing overhead. Here are some questions for you:
Did Toyota apply the same practices to the pre-manufacturing process of designing their new products? If you think they did, how far?
Which part of the software development lifecycle do you think the assembly and part sourcing logistics maps to? The creative part or producing, stamping physical media with ready-made software on it? When was the last time you saw that done? Do you even know what I’m talking about? Hint if you don’t: software used to be sold in physical boxes, usually cardboard, with CDs and/or diskettes and printed manuals inside. These had to be manufactured and assembled and each came from a different source.
Am I bashing “lean software development”? I’m trying not to. I am trying to point out what I observed to the frequent misapplication of it. There are many good lessons to be learned from it, but they need to be learned and understood well. Without understanding what we’re applying we make errors. I’ll go through those that I observed. You may find them controversial as they are meant to be provoking and make you think, reassess your opinions and allow you to either double down on them … or adjust them accordingly.
Common Errors
Repeated warning: These are not about what authors of well-known practices wanted but about my observations “in the wild”, the real-world experience of what is actually done and the consequences suffered.
K.I.S.S. of Death
Is it “Keep it Simple, Stupid” or “Keep it Stupid, Simple”? I’m sure you know what it should be, but does that match the application? Is it, actually, good in any form? These “kisses” are frequently brought up in conversations involving something like the following:
There are no requirements for the immediate work at hand. Yeah, we don’t care about any other requirements that are not for right now, after all they may change. We should be lean. KISS!
That “kiss” is then usually followed by an assumption and, sometimes, arguments, that any future possibility is equally likely and, given that, the simplest one should be assumed. Depending on how you count, you may find that there are one or two (or, perhaps, more) related false premises in there:
We don’t know or care where we’re going(er) long term at all. Only the next step matters.
Choosing what appears to be the easiest at the moment is good enough.
Imagine travelling somewhere far, and not flying. You need to make many stops. You need to eat, do some work along the way to replenish your funds. Do you:
Pick the nearest / best work and eating opportunity regardless of where it is relative to your intended destination?
Steer your travel in the general direction of your destination, even if there are nearer ones, that may appear better.
Bonus question:
Do you believe in yourself at all with respect to picking a destination or are you just roaming around, picking the lowest-hanging fruit?
Do you see where this is going? Option (1) will make you either stuck in the local minima or cause you to roam around in random directions. Don’t do that unless you haven’t decided what your business is about yet. That translates to not picking the simplest solutions as they will, in the long run, very often turn out to be more “stupid” than “simple”. Make informed decisions. When you don’t know enough at the moment, either don’t do it at all or abstract it out reasonably well to cover possibilities beyond the simplest one. Do keep it as simple as possible, but not simpler than that!
Type as fast as possible, add more hands! (TAFAPAMH!?)
This is the often seen “application” of the “extreme programming”, thriving on “KISS”. It justifies the means by the immediate ends only. That’s all great when there are no “further ends” after that one. What happens when the result needs to evolve beyond the first “release”? What happens if maintainability is not considered a requirement? Well, you get an unmaintainable product. For some vendors this seems advantageous as it can bring more work if new requirements come. Making things simpler, easier, may not appear to be the goal. That’s not the case in my “book”. How about you?
Fire-fighting as the only priority (FFATOP?)
“Fixing burning issues is all we do” is not a motto you want to have. How/why did you end up in that situation? How could / should you have prevented it? That’d be a distinct topic. Let’s say that you are in this predicament. Are you investing all your budget in fighting those fires? Is there an end in sight, truly? Are you giving yourself some time and bandwidth to reassess the situation, notice patterns and try to solve many issues with a single fix? Are you investing anything in growth or are you letting your competition leapfrog you?
Some think of this as final life support and “milking the cow” while it can still be done. Consider the outcome and what would happen next. Is it profitable at all? Would that next “gig” be continuing fighting your fires or take a different approach? Would it (not) make sense to start working on that approach immediately?
“Jacks of all trades = no bottlenecks”
There is a very common, yet very shallow assumption among those trying to apply agile processes they read about, that having many small teams with equal and broad expertise and responsibilities means that there are no bottlenecks. It seems true – if one team is busy, another one can be allocated. Better yet, this appears to manage the employee churn risk as everyone, including entire teams are easy to replace (or dismiss) with minimal impact. Having individual experts, SMEs and specialized teams is seen as creating bottlenecks because there will be less people/teams that any particular work can be assigned to at any moment. Is all that true or does it just hide some other effects under a proverbial rug?
Let’s flip that around. Does that mean that experts are not needed or wanted? No. It just doesn’t mention the hope that experts are easily available everywhere. You shouldn’t even need a wake-up call to realize that this isn’t the case. Many decades ago it was possible for people to actually know everything that could be learned about computers and software. That time is long gone. You can no longer have any one person be an expert for everything, let alone multiple or a team of those. Even if you could, you probably wouldn’t want to. Why?
Suppose you do live in a fairy tale and all your people happen to be experts in everything. How do you get them to agree? Oh, pardon me, that’s a part of the fairy tale too. They just do because their equal expertise will yield equal opinions, every time. They won’t even need to discuss it; it’ll just happen on its own. They are perfect clones of one another. Let’s take it from there.
As you assign different work to different teams over time their experience and, thus, expertise will start growing apart. Will you try to manage that by distributing work in a way that will attempt to maintain equal experiences? Does that not bring a bit of a bottleneck back? How costly, even how possible do you think this is? How quickly do you think the expertise will be growing considering constantly shifting focus? They work on A at one time, then B at another, and so on. But let’s not stop there. Think about the following:
1. How much effort will it take them to remember the last state they saw, to pick up from?
2. How much effort will it take them to discover what others did to that state since then?
3. How will decide on the approach to take? Will they have some sort of vision, direction to follow? Whose? Where does it come from? Will they form their own or follow someone else’s?
4. In light of (1)-(3) what motivates them to stay?
Bottlenecks do not disappear. A risk of a bottleneck is a risk of not being able to do some work at desired time. With the practice as described above, that risk is traded for a guarantee of a continuously slow performance and elimination of ownership due to constantly shifting, divided focus. Both reduce motivation, one due to slow progression, another due to lack of power (and responsibility) obstructing career growth and misaligning with individual interests.
In the world in which perfect generalists (know-it-alls) are impossible, all that remains as the career growth is specialization. This specialization can take many forms, “vertical” or “horizontal” or even “diagonal” 😊, technical or managerial. It must be allowed and leveraged, not prevented. While general expertise remains desired and everyone needs it to better understand how their specializations fit in the bigger picture, it mustn’t be the only driving force. I’ll happily accept a risk of temporary bottlenecks if it comes with happier, more able workforce, accomplishing their goals better and in less time. And, no, that is not a fairy tale. It is possible.
Bonus: The Space-Time Delusion
How much work can a team accomplish in a sprint? This, of course, is important to avoid taking work that cannot be done together with other or, worse, on its own. What do you use to figure this out? This is always a numbers game – if an estimate is not greater than the limit (“capacity”) you’re good. I don’t care (here) about the process you use to get those numbers but about what these numbers represent – what is the meaning of the “story point”? If you think or research this you’ll find that:
There is a strong desire to avoid relating the points to units of time to avoid giving a specific time commitment.
Ideally these points should be additive: the sum of points of individual work items should closely match the points for the sum of work.
New teams always struggle to synchronize on what the points are, but they eventually do… until new members join. Think about how each individual contributor ends up understanding the meaning of a “point”.
The concept of velocity is introduced in some form as the number of points that can be accomplished in, say, a sprint. Sprints are usually of constant duration.
Now, we’re not stupid. We study enough Math even in elementary schools to understand that, to travel 60 km a car driving 60 km/h will need 1 hour. That’s the basic equation of distance, speed (velocity*) and time. We’ve got the point velocity (points per sprint). We’ve got the distance too – a single sprint, which is usually some number of weeks. Divide the two. You get how many points there are in a week. That becomes a unit conversion exercise. Whether appropriate or not, points become time units even within the team (capacity planning) and it is trivial for others to do the same. We may as well stop pretending this is not the case as pretending takes time away and does not accomplish what it was supposed to.
I’ll go further. Is it supposed to? Should only individual teams be agile or the entire business? Should you not be able to align the meaning of points across separate teams? You see, the business side of the “story” also has its own need for agility, sprints and capacities with very real definition of points: money; capacities translate to budgets and sprints are bounded by times by which work needs to be accomplished to produce a desired effect. Customers don’t care about understanding someone’s abstract “points” – its mostly about cost and availability at the right time. Can the business be agile if it isn’t given the same opportunity, you are? If it isn’t agile, what does it mean for you? Can you really be, or are you just pretending?
Cogito, ergo sum
“I think, therefore I am”. What works for others may not work for us. Our decisions are our responsibilities. Pointing fingers at others after your own failure won’t help. Due diligence is due, repeatedly – one must stay on top of the ever-changing game. The most a blind following of someone’s recommendations can get you is an asymptotic approach to what they are already doing. To leapfrog them we need to do better than that.
