This is a tricky subject. Sometimes it can be hard to understand the difference between being unnecessarily reactive and when we are flexible due to uncertainty and are able to adapt to the new situation, or when we are being comfortably happy that we found something new that we can learn.
The reason for reactiveness is often an overuse of uncertainty, a buzzword nowadays. Maybe today is more uncertain than yesterday, and the future will be more uncertain than today, but we can definitely not blame all our reactiveness on the need of handling extreme uncertainty. Another buzzword today is complexity, where many people states that we need “to embrace complexity”, which is only correct if we have complexity. But unfortunately, we, despite this statement, have reactiveness due to that we do not reduce the complexity enough in product development at scale, especially for software. As we will see soon, different domains can have very different uncertainty, while it in product development does not differ between different product development domains, instead we have different levels of complexity. We also have another even bigger difference, and that is that the level of uncertainty is context independent, while the level of complexity is context dependent.
First, we need to go back to our definitions of uncertainty and complexity, as we defined in the introduction blog post to this series; uncertainty relates to developing the right product due needs outside the organisation and its friend complexity relates to developing the product right which is a need inside the organisation (and product).
Uncertainty (do the right things)
We first start to dissect uncertainty, since there is a big difference in the level of uncertainty about what product our customers want, where the uncertainty completely depends on the domain.
Of course, it would be great if we can have a “one size fits all”-way of working to take care about uncertainty, but if we use our common sense, we understand that is not realistic. And if we think about it a bit, we have situations which are very different: 1) we have low probability to know if the customers even want the product, we are developing 2) we have quite high probability that our customers want it, which can be our next generation of our popular product, or 3) our customers (users) do not have an option. So, we have the span from the former that has high uncertainty, via the middle one that has low uncertainty, to the latter one with no uncertainty. The former one really need to have loops of early feedback from the functionality to understand if the customers want the product in the end or not. Of course, also the two latter ones need early interactions with the customers, in order to get fast feedback, since we still want to make an awesome user-friendly product, even if we already know that we are developing the right product.
In a world that is uncertain, being able to predict the exact date for a delivery, is many times impossible. This has led to that in the agile world, planning has really become a non-mantra. Planning has become something ugly.
But planning is not synonymous with time planning, it is so much more than just the time plan. Planning also gives us our future needs, that we need in order to be able to deliver at all, even though we cannot specify exactly when we will use our needs. The later we will understand our needs, the more reactive we will be. This means that if we only refer to the world as always having high uncertainty, and therefore cannot be planned, we will become reactive compared to traditional waterfall way of working. So, to throw out all babies with the bath water is never a good idea. And when we look at our examples with high and low uncertainty above, the former one makes time planning very difficult. But still, even a short-term planning gives us indications of our needs, meaning we will get less reactive. The two latter ones, are also the ones that we quite straightforward are able to do a reasonably good planning to understand our need, which means minimising also the reactiveness.
Another deficiency related to planning, and that will increase our reactiveness instead of being more proactive, is when we are using some of the agile scaling frameworks that use methods originating from a production context. They often divide the releases into increments of functionality as small-scale agile does, which many times means that the details for the long-term planning is not done. In turn this means that it is very high risk that every increment starts with a planning from zero, as the memory of the planning with all needs, hinders and risks from the last increment, has been reset. This kind of non-planning gives a very reactive behaviour of needs through the whole organisation; resources, competences, education, technology, etc., which of course gives a bad flow, which in the end means late/missed releases, frankly low efficiency. Not to mention, that the very same needs can be brought up again and again at every planning of a new increment. This differ enormously from small-scale agile where the teams, that is also is the whole organisation, having the Big Picture and all needs in reach.
Complexity (do the things right)
Complexity, and in this case, transdisciplinary complexity/ complicatedness, has a completely other feedback loop than uncertainty. The feedback loop for transdisciplinary complexity/ complicatedness we also need to close as soon as possible, so we are sure that we are developing our product right. The bigger system product we have, the more complex our system product is, and the more difficult it will be to use incremental methods for adding functionality (and non-functional requirements) to the customer as mentioned above for the feedback loop of uncertainty. The reason is that the systems design is how we reduce the complexity for our system product, to be able to understand which parts we need to be able to integrate them to a coherent and cohesive whole. But we cannot specify the perfect systems design even if we have all requirements (the non-functional requirements are vital for the systems design), so we need early feedback, where we can think about an endoskeleton of the systems design. Without this early feedback of the coherence and cohesiveness of the total system, we are risking years of work to achieve a big product that does not even work, since we did not develop the product right due to ignorance of the systems design.
Reactiveness that originates from not taking care of complexity, by ignoring systems design when developing new products, software, hardware or a mix, is also common today. This neglection has increased steadily, once again, by the introduction of agile scaling frameworks that use methods originating from a production context, where complexity never need to be handled.
Let us look into how this can be done instead in big systems. If the initiatives are new big systems or are enhancement of existing big systems, we always need to iteratively make or there is (or should) already be a proper systems design. Since we in big systems most certainly now what to do, the backend can simply be separated from the frontend with an UX component. If our collaboration with the customer gives the opportunity for new functionality, we have most probably iterated the enhancements many times by using other tools with very short feedback loops, like pen and paper, power point, etc., then (only) implementing code directly in the UX component. Either way having a new system or enhancements, by using a UX component, we can be systematic and structured when we are planning our work. There is therefore no need for rush, or for sending half-finished functionality to the customer or end user. This is especially valid for the backend parts of any initiative, since it is where the main part of the difficulties with setting a proper systems design are. By separating this backend work from the frontend work, i.e., using a UX component in the architecture, we can be systematic and reduce the transdisciplinary/ complicatedness complexity by explorations of the systems design, at the same time as we together with the customer and end users can explore and develop an awesome user experience. And of course, frontend also to some extent will give input to the systems design, to have the full transdisciplinary view from start. By having a UX component, we most of all, to a great deal, have separated the two different feedback loops, the one for reducing the transdisciplinary/ complicatedness complexity by iteratively making the systems design for the backend, and the feedback loop for the uncertainty on how to fulfil the customer needs. We can from this also conclude that if we are going to make a big product, but we do not have a clue regarding what the customer need, that is the only time that an MVP (Minimum Viable Product) can be considered. And in a big system, an MVP strategy always will be messy, due to the totally different needs of the two different feedback loops.
But, let us also look into why it can work in a small company, doing agile. In a small company, with one product, developed by a few agile teams, they will for sure have control of their needs due to the short distance (both hierarchical and physical, even though we can never forget about Conway’s law) between the teams and the product owner, they are really sitting in the same boat. And they must have control, because there is no other one to ask, they need to plan and fulfil their own needs themselves. It is worth mentioning that the T-shape of these teams are high (and need to be since they are the only ones in this small company). They also need to have the overall Big Picture of the product, and they have themselves done the systems design to some extent in order to reduce the complexity in a suitable way, with architecture, design and everything regarding test as output. If this small company find themselves to be too reactive, they will take care of that together, by finetuning their way of working, and refactor both the structure of the code, and the code itself.
We stated earlier that in many agile scaling frameworks, the functionality is planned increment per increment, often only at the start of the increment. For the complexity side this means that the requirements for the total product is seldom considered, leading to that proper systems design cannot be done from the beginning. This means a very high risk that, especially all the non-functional requirements, have not been part of the systems design, which means that the complexity has not been reduced properly. Actually, this means that the parts are not designed to integrate to a unified whole, even if they fit together like the pieces in a jigsaw puzzle. This also means that the test environment activities for the whole product will not start, due to the believe that all the parts instead can have built-in quality. This is an extreme risk taking; first we not reduce the complexity properly, then we do not even plan for testing the whole product, or the big parts of it, so we could be able to find our blunder.
This in turn means, that regarding quality issues on the wholeness, the whole product, no one can see them coming, since the teams add tests that only test their added functionality that is incrementally implemented. In worst case, quality issues will not be until the release in production, since no one has been responsible for the whole. This reactiveness could have been avoided, or at least the risk of reactiveness could have been reduced, if systems design with the right level of details had been done. Preferably with mocked components on the whole, to be able to start early system tests in order to close the “develop the product right” feedback loop as early as possible. As we can see it is not only bad planning that make us reactive, because if we omit the systems design, we will have big problems with reactiveness too.
We have seen that we cannot always brush aside when we all the time are reactive to new needs, because many times we could have understood the need much earlier. These needs are also often our need from other parts of the organisation, putting them in a bad situation, since they are never given any information in advance so they can be proactive instead. The whole organisation then gets severely inefficient.
Of course, there are many unexpected things that can happen that make us reactive. But the conclusion is that the mentioned two, planning and systems design, are two ingredients that if we do them, we can reduce a lot of the unnecessary reactiveness. And to blame uncertainty for our continuous reactiveness, we must really avoid, and instead seek for the root causes to our reactiveness, where non-planning and non-systems design clearly are two of them, even if it of course can be others.
If we put uncertainty and complexity together, it results in something interesting, but also quite funny, and that is that the complaints on the waterfall way of working in software, was that the output was the wrong product, but it was developed correct. But, now with agile scaling frameworks, it is the other way; we (maybe) develop the right product, but we develop it wrong. It becomes rather clear, that by throwing away all old way of working, means that we did not learn anything, which never is a wanted mindset, especially not in agile.
C u soon again.