This is a tricky subject. Sometimes it can be hard to understand the difference between being unnecessarily reactive and when we are flexible due to uncertainty and are able to adapt to the new situation, or when we are being comfortably happy that we found something new that we can learn.
The reason for reactiveness is often an overuse of uncertainty, a buzzword nowadays. Maybe today is more uncertain than yesterday, and the future will be more uncertain than today, but we can definitely not blame all our reactiveness on the need of handling extreme uncertainty. Another buzzword today is complexity, where many people states that we need “to embrace complexity”, which is only correct if we have complexity. But unfortunately, we, despite this statement, have reactiveness due to that we do not reduce the complexity enough in product development at scale, especially for software.
First of all, we need to go back to our definitions of uncertainty and complexity, to refresh the meaning of them in System Collaboration; uncertainty relates to developing the right product due to needs outside the organisation and its friend complexity relates to developing the product right which is a need inside the organisation (and product).
Different domains, can have very different uncertainty, where software normally have much higher uncertainty than hardware when starting an initiative. But, within a context and its different domains, we always have the same complexity, no matter if the uncertainty can differ between the domains within the context. For example, in product development there is no difference regarding transdisciplinary complexity/ complicatedness between the domains when making a new initiative (we have passed the disciplinary research phase present only for hardware), where making a new product or system, always is very complex, no matter if the domain is hardware, software, or hybrids. It is worth to add, that the uncertainty level we have from start when making an initiative, will be the same until we have delivered the product or system to the customer, even if we through the initiative has reduced the transdisciplinary complexity/ complicatedness to a minimum, when we start to produce and deliver the product or system to the customers. That means that the uncertainty level is context independent, while the level of complexity always is context dependent.
Now it is time to dig into the details of uncertainty and complexity.
Uncertainty – do the right things (products/systems)
We first start to dissect uncertainty, since there is a big difference in the level of uncertainty about what product our customers want, where the uncertainty often depends on the domain.
Of course, it would be great if we can have a “one size fits all”-way of working to take care about uncertainty, but if we use our common sense, we understand that is not realistic. And if we think about it a bit, we have situations which are very different: 1) we have low probability to know if the customers even want the product, we are developing 2) we have quite high probability that our customers want it, which can be our next generation of our popular product, or 3) our customers (users) do not have an option. So, we have the span from the former that has high uncertainty, via the middle one that has low uncertainty, to the latter one with no uncertainty in what the customer wants. The former one really need to have loops of early feedback from the functionality to understand if the customers want the product in the end or not. Of course, also the two latter ones need early interactions with the customers, in order to get fast feedback, since we still want to make an awesome user-friendly product, but, can we separate that, we will give ourselves an extreme advantage.
In a world that is uncertain, being able to predict the exact date for a delivery, is many times impossible. This has led to that in the agile world, planning has really become a non-mantra. Planning has almost become something ugly.
But planning is not synonymous with time planning, it is so much more than just the time plan. Planning also gives us our future needs, that we need in order to be able to deliver at all, even though we cannot specify exactly when we will use our needs. The later we will understand our needs, the more reactive we will be. This means that if we only refer to the world as always having high uncertainty, and therefore cannot be planned, we will become reactive compared to traditional waterfall way of working or Lean product development. So, to throw out all babies with the bath water is never a good idea, not this time either. And when we look at our examples with high and low uncertainty above, the former one makes time planning very difficult. But still, even a short-term planning gives us indications of our needs, meaning we will get less reactive. The two latter ones, are also the ones that we quite straightforward are able to do a reasonably good planning to understand our need, which means minimising also the reactiveness.
Another deficiency related to planning, and that will increase our reactiveness instead of being more proactive, is when we are using some of the agile scaling frameworks that use methods originating from a production context. They often divide the releases into increments of functionality as small-scale agile does, which many times means that the details for the long-term planning is not done. In turn this means that it is very high risk that every increment starts with a planning from zero, as the memory of the planning with all needs, hinders and risks from the last increment, has been reset. This kind of non-long-term-planning gives a very reactive behaviour of needs through the whole organisation; resources, competences, education, technology, etc., which of course gives a bad flow, which in the end means late/missed releases, frankly low efficiency. Not to mention, that the very same needs can be brought up again and again at every planning of a new increment. This differ enormously from small-scale agile where the teams, that also are the whole organisation, have the Big Picture and all needs in reach.
Complexity – do the things (products/systems) right
Complexity, and in this case, transdisciplinary complexity/ complicatedness, has a completely other feedback loop than uncertainty. The feedback loop for transdisciplinary complexity/ complicatedness we also need to close as soon as possible, so we are sure that we are developing our product right. The bigger system product we have, the more complex our solution to our system product will be to find. This automatically gives that it will be more difficult to use incremental methods for adding more and more functionality (and non-functional requirements) to the customer, that was mentioned above as a solution for the feedback loop of uncertainty. The reason is that the systems design is how we reduce the complexity for our system product, to be able to understand which parts we need to be able to integrate them to a coherent and cohesive whole. Due to the high complexity when making new products, we can neither specify the perfect systems design even if we have all requirements (the non-functional requirements are vital for the systems design). This means that we need early feedback also for the systems design, where we can think about an endoskeleton of the systems design, which is completely different from the early feedback from the customers. Without this early feedback of the coherence and cohesiveness of the total system, we are risking years of work to achieve a big product that does not even work, since we did not develop the product right due to ignorance of the systems design. This means that for uncertainty we need early (rather just-in-time) feedback from the customers that we are building the right product, and for complexity, we need early (rather just-in-time) feedback from our own system, so we are building our product right.
Reactiveness that originates from not taking care of complexity, by ignoring systems design when developing new products, software, hardware or a mix, is also common today. This neglection has increased steadily, once again, by the introduction of agile scaling frameworks that use methods originating from a production context, where complexity never need to be handled.
Let us now look into how this can be done instead in big systems. If the initiatives are new big systems or are bigger enhancements of existing big systems, we always need to iteratively make the systems design or make appropriate updates to the existing one. Since we in big systems most certainly know what product or system to do, the backend can simply be separated from the frontend with an UX component, which means that we are separating the two feedback loops for uncertainty and complexity. If our collaboration with the customer gives the opportunity for new functionality, we have most probably iterated the enhancements many times by using other tools with very short feedback loops, like pen and paper, power point, etc., before implementing the code directly in the UX component. Either way having a new system or enhancements, by using a UX component, we can be systematic and structured when we are planning our work for backend and frontend. There is therefore no need for rush, or for sending half-finished functionality to the customer or end user. This is especially valid for the backend parts of any initiative, since it is where the main part of the difficulties with setting a proper systems design are. By separating this backend work from the frontend work, i.e., using a UX component in the architecture for the frontend, we can be systematic and reduce the transdisciplinary/ complicatedness complexity by explorations of the systems design for backend, at the same time as we together with the customer and end users can explore and develop an awesome user experience. And of course, frontend also to some extent will give input to the systems design, to have the full transdisciplinary view from start. By having a UX component, we most of all, to a great deal, have separated the two different feedback loops, the one for reducing the transdisciplinary/ complicatedness complexity by iteratively making the systems design for the backend, and the feedback loop for the uncertainty on how to fulfil the customer needs. This is also valid for any domain in product development, where hardware would consider it as industrial design, the appearance of the product. We can from this also conclude that if we are going to make a big product, but we do not have a clue regarding what the customer need, that is the only time that an MVP (Minimum Viable Product) should ever be considered. Remember though, that not knowing what big product or system to do for the customer, fortunately is a very odd case, and that through its extremely expensive initial costs, really should be reconsidered.
But, let us also look into why it can work in a small company, doing agile. In a small company, with one product, developed by a few agile teams, they will for sure have control of their needs due to the short distance (both hierarchical and physical, even though we can never forget about Conway’s law) between the teams and the product owner, they are really sitting in the same boat. And they must have control, because there is no other one to ask, they need to plan and fulfil their own needs themselves. It is worth mentioning that the T-shape of these teams are high (and need to be since they are the only ones in this small company). They also need to have the overall Big Picture of the product, and they have themselves done the systems design to some extent in order to reduce the complexity in a suitable way, with architecture, design and everything regarding test as output. If this small company find themselves to be too reactive, they will take care of that together, by finetuning their way of working, and refactor both the structure of the code, and the code itself.
We stated earlier that in many agile scaling frameworks, the functionality is planned increment per increment, often only at the start of the increment. For the complexity side this means that the requirements for the total product is seldom considered, leading to that proper systems design cannot be done from the beginning. This means a very high risk that, especially all the non-functional requirements, have not been part of the systems design, which means that the transdisciplinary complexity/ complicatedness have not been reduced properly. Actually, this means that the parts are not designed to integrate to a unified whole, even if they fit together like the pieces in a jigsaw puzzle. This also means that the test environment activities for the whole product will not start, due to that this aggregation thinking in itself, leads to a believe that all the parts instead can have built-in quality. This is an erronouos thinking which is leading to an extreme risk taking; first we not reduce the complexity properly, then we do not even plan for testing the whole product, or the big parts of it, so we could be able to find our blunder, until very, very late, maybe years later.
This in turn means, that regarding quality issues on the wholeness, the whole product, no one can see them coming, since the teams add tests that only test their added functionality that is incrementally implemented. In worst case, quality issues will not be until the release in production, since no one has been responsible for the whole. This late reactiveness when understanding that the whole is not united and unified, could have been avoided, or at least the risk of reactiveness could have been reduced, if systems design with the right level of details had been done from start. Preferably with mocked components on the whole, to be able to start early system tests in order to close the “develop the product right” feedback loop as early as possible. As we can see, it is not only bad planning that make us reactive, because if we omit the systems design, we will have big problems with reactiveness too.
We have seen that we cannot always brush aside this reactiveness, when we all the time are reactive to new needs in order to solve new problems, because many times we could have understood the need much earlier by proper planning, or we could have avoided problems by a proper systems design. This reactiveness often spread to need from other parts of the organisation, putting them in a bad situation, since they are never given any information in advance so they can be proactive instead. The whole organisation then gets severely inefficient.
Of course, there are many unexpected things that can happen that make us reactive. But the conclusion is that the mentioned two, planning and systems design, are two ingredients that if we do them, we can avoid a lot of the reactiveness that these unnecessary problems gives us. And to blame uncertainty for our continuous reactiveness, we must really avoid, and instead seek for the root causes to our reactiveness, where non-planning and non-systems design clearly are the two prominent ones, even if it of course can be others.
C u soon again.