Are you stuck in a rabbit hole of hypothesis-driven everything?

We need to be very careful when we use one successful method or technique from one context in another totally different context. Hypothesis-driven everything now seems to be the new hype, with its roots in production. But, hypothesis-driven everything is now starting to be used in other contexts like product development, transformations, problem-solving and other areas with high complexity, which means that we need to be vigilant.

The need for action instead of a never-ending analysis is understandable. The expression analysis-paralysis is apt, when having a difficult situation where overthinking leads to non-decisions due to too many variables. But, to take a too fast decision and just make any hypothesis, is not good either, since it can lead to extinct by instinct.

So, we need to understand more about the problem we are standing in front of. We need to understand in which cases we can analyse a result we actually already have or to find the root cause(s), before we take action, and the cases where we have no result or too little information and therefore need to take an action, for example experiments, sometimes even trial-and-error, to get enough results to analyse. So, we will not get rid of analysing in any case, even though it is a lot of gaslighting trying to diminish it.

The hypothesis-driven approach has its roots in the PDCA cycle, or the PDSA cycle as Deming changed it to. The reason for Deming’s change from a C to a S was in order to emphasize on analysis as in study, and not inspection as in check. This is understandable, so we not just do new hypothesis within a blink of an eye without any consideration, just because it is the name of the game. But unfortunately, it seems to have become the case anyway, despite Deming’s effort. Analysis has become the big bad wolf (but that is another story), and since it is used in projects, it must clearly be wrong, which means that we can justify the lack of any analysis at all by saying the magic word hypothesis.

But there are a few hunches in the case of the hypothesis-driven everything approach, that needs to me mentioned.

The hypothesis-driven technique comes from Japanese (Lean) manufacturing, and has been used as a technique in order to achieve continuous improvement, usually with PDCA cycles to achieve the set goal. As many techniques that are used in manufacturing, they are also adopted (unfortunately not adapted or even better transformed) by agile software development and especially in agile scaling frameworks. But manufacturing is a simple context, where every process makes its unique part, as they do in Lean manufacturing. On the other hand, all product development is a complicated, complex or/and uncertain context. In a simple context like manufacturing, it is not only easy to divide bigger things to smaller parts and improve them one by one in respective process, it is legitimate too. Why is it legitimate in a simple context? Because there are clear and specified dependencies to the closest processes and no interdependencies to other processes, and all processes are stable and standardized as well. This means that we can use a hypothesis-driven approach on each and every process and improve them separately, since the whole is not affected.

Let us now look into some other areas with high complexity mentioned before, where a hypothesis-driven approach can be deeply problematic with extremely high risk-taking if performed.

Product development
In a context like product development, we often, at least from the start do not know if we are able to develop our new product, which means that we need to gain more knowledge on the whole, when making our modules/subsystems of the whole. This knowledge can be gained directly in an experimental and hypothesis-driven approach by a number of techniques like PDCA/PDSA, Set-Based design, MBSE, Multiple concepts, etc.

Since product development have high complexity, our way of working (context dependent) to get the solution on the whole would most probably start in the Complex domain if we do not even know if we can develop the product (our problem to solve) with (our) existing knowledge. Sometimes we can start in the Complicated domain, but only if we are surer about the solution and think that we can gain the knowledge in an iteration or two. Examples of gaining knowledge by iterations are prototyping in hardware product development, or some sprints with User Stories in order to solve the Feature in software product development. If our product is big, we will recursively apply the same patterns as above on the modules/subsystems.

But, here comes a hunch. We can only use experimental and hypothesis-driven techniques on the parts, only if we have started top-down on the whole first, and then recursively on the parts if necessary. Why? Because, in product development we have high complexity, which need to be reduced with a systems design, in order to get parts that can be integrated together to a unified whole. But, although a proper systems design has been done, it is still very complicated with a lot of interdependencies between our teams and between the modules/sub-systems of our product that we need to keep track of. This means that trying to improve one part or making team improvements are difficult due to the high connectivity. The complexity makes it very easy to sub-optimise, and therefore it is not legitimate to use hypothesis-driven approaches on the parts without thoughtful consideration on the whole first. To be tangible, it is the non-functional requirements on the whole, like for example system performance and fault handling, that need to be part of the systems design on the whole, to achieve the parts that with some planned iterations (prototypes), finally will integrate to a unified whole. And attentive readers that had examined the earlier blog post Aggregation vs Integration vs false Integration, will understand that if the systems design is neglected, it will as well lead to the treacherous false integration, which is the result of trying to make product development look like manufacturing.

Problem-solving (always caused problems)
First we need to distinguish a problem from a problem, where we have two different types of problems. If our problem is to develop our product from start, see product development above.

But, if we instead have caused the problem (ourselves) for example when developing our product, the problem is commonly called a complex or wicked problem. A caused problem means that the problem instead is a symptom of something deeper, one or several root causes*. A caused problem is context independent and will be put in the Disordered domain in the Cynefin framework, since we cannot solve it directly. Why can we not solve it directly? Because any try to solve symptoms will sub-optimise on the whole, meaning it will generate unintended consequences anywhere and anytime on the whole. And here we have a hunch again. And not a small one. This leads to that all techniques mentioned above will stand short since they are solution techniques, which also means that we will go into context. None of them work on symptoms, and since symptoms are unsolvable, no solution technique will do the job. And as you know we instead need to find the root cause(s). And the only way to reach a root cause is to use root cause analysis techniques, simply ask why we have the problem(s). But of course, there is hunch. We cannot move root cause analysis techniques successful in one context to be used in another context without being careful. And another hunch, if we do not know what the possible root causes** are (root causes are context independent (context-free) too), we do not even know when to stop our root cause analysis.

A short summary regarding a hypothesis-driven approach to problem-solving is that the problem cannot be a caused problem (by ourselves), since then we need to find the root cause first. When we know the root cause, we can start examining the most appropriate solution, where of course a hypothesis-driven approach sometimes can be the answer.

Transformations (of organizations)
This is really a sad story, where some agile scaling frameworks divides everything in an organisation into parts in any imaginable direction; teams, team of teams, maintenance, application, software, hardware, economy, roles, analysis, architecture, design, test, requirements, portfolio, culture, efficiency, effectiveness, flow, innovation, line management, deliveries, strategy, operations, etc. for later combining to a whole. This is false integration at its worst since no thinking at all about the whole has been considered first, even though it cannot be much more complex than the complexity in an organization doing product development. Why can this happen? The believe is, which is contrary to existing science (now and forever), is that we can divide anything in small parts, optimise them one by one, and put them back to an optimised whole. And by oppressing our common sense, we start to believe that this is true. But we will face tough times when we wake up from the fairy-tale. Be sure.

Anyone, without an oppressed common sense and even without looking into science, can see that trying to put up a hypothesis to make any part working/better/optimised without affecting another part is only a wild goose, it is only guessing. Not to mention how in earth we can know and tell our top management that our transformation is on track.

As long as a hypothesis-driven approach is done on the whole (any context) or recursively on parts or completely independent parts (like processes in manufacturing), it is a good and fast way to gain new knowledge if needed. And as mentioned there are many methods that can be used. The feedback-loop is important and for example regarding the uncertainty of what the customer need really is, leading indicators can be helpful metrics. It is never possible to have a hypothesis-driven approach when trying to solve caused problems, because it is impossible to solve symptoms. This goes for both organizational problems, when we have problems with our way of working, or product problems. Instead, we need to get down to the root cause(s) first, which is where we can apply solutions, depending on the context, domain and complexity level.

As we now can see, it is of utterly importance to understand that a caused problem only can be solved by solving its root cause(s), and it seems like this knowledge need to increase. Without this understanding, expressions like analysis-paralysis can emerge and hypothesis-driven everything can become a hype.

And now when we have this knowledge, we can stop an overuse of the hypothesis-driven everything, because who wants to get into the fatal hypothesis-nemesis cycle, self-inflicted or not. No one, we can presume.

C u soon again.


*root causes are simply only non-fulfilled principles (laws). Think about Archimedes’ principle; if we make a boat that is heavier than the water it pushes away, it will go under in no-time, meaning that our root cause is that we did not fulfil Archimedes’ principle.

**and, of course we need to understand where the solution on the root cause can be executed, in the team, in the team of teams, etc. up to the enterprise level. This makes it for example very easy to make improvements or solving problems within the processes at manufacturing, since the lack of interdependencies makes the hypothesis local to the process only.
Which also means that a solution in product development can be at any level, from locally within the team to globally on the whole enterprise. Applying the solution on the wrong level, especially a too low level, means that we are trying to solve symptoms, which only will lead to unintended consequences which we sometimes can see pretty soon, but in most cases much later.
But, when we finally have found the root cause(s), we apply the appropriate solution techniques, depending on what root cause to solve. Experimental techniques if the solution is in the Complex domain, or collect some knowledge when in the Complicated domain, or just use straightforward techniques since the solution many times will be in the Clear domain, when we have found the root cause(s) to the caused/complex/wicked problem we started with. Probably the words complex or wicked in front of problem, is just because we for a long time have been trying to do the impossible; to solve symptoms.
The last hunch about root causes is when the science has not yet understood the root cause to a problem, meaning the question why does not give any answer, because we are only trying to solve a symptom, so experimentation is not possible. An example is the unsuccessful blood transfusions with many people dying, during 300 years, before the 1940s when the last piece was found, the Rh system, and we could finally understand why people were dying even though we already year 1900 understood blood groups. To focus on the blood transfusion with further experimentation would never have found the science of the blood group and finally the science of the Rh system.

Leave a Reply