Are you stuck in a rabbit hole of hypothesis-driven everything?

We need to be very careful when we use one successful method or technique from one context in another totally different context. Hypothesis-driven everything now seems to be the new hype. This yields hypothesis-driven used in product development, transformations, problem-solving and other areas with high complexity.

The need for action instead of a never-ending analysis is understandable. The expression analysis-paralysis is apt, when having a difficult situation where overthinking leads to non-decisions due to too many variables. But, to take a too fast decision and just make any hypothesis, is not good either, since it can lead to extinct by instinct.

So, we need to understand more about the problem we are standing in front of. We need to understand in which cases we can analyse a result we actually already have before we take action, and the cases where we have no result or too little information and therefore need to take an action to get enough results to analyse. So, we will not get rid of analysing in any case.

The hypothesis-driven approach has its roots in the PDCA cycle, or the PDSA cycle as Deming changed it to. The reason for Deming’s change from a C to a S was in order to emphasize on analysis as in study, and not inspection as in check. This is understandable, so we not just do new hypothesis within a blink of an eye without any consideration, just because it is the name of the game. But unfortunately, it seems to have become the case anyway, despite Deming’s effort. Analysis has become the big bad wolf (but that is another story), and since it is used in projects, it must clearly be wrong, which means that we can justify the lack of any analysis at all by saying the magic word hypothesis.

But there are a few hunches in the case of the hypothesis-driven everything approach, that needs to me mentioned.

The hypothesis-driven technique comes from Japanese (Lean) manufacturing, and has been used as a technique in order to achieve continuous improvement, usually with PDCA cycles to achieve the set goal. As many techniques that are used in manufacturing, they are also adopted (unfortunately not adapted or even better transformed) by agile software development and especially in agile scaling frameworks. But manufacturing is a simple context, where every process makes its unique part, as they do in Lean manufacturing. On the other hand, all product development is a complicated, complex or/and uncertain context. In a simple context like manufacturing, it is not only easy to divide bigger things to smaller parts and improve them one by one in respective process, it is legitimate too. Why is it legitimate in a simple context? Because there are clear and specified dependencies to the closest processes and no interdependencies to other processes. This means that we can use a hypothesis-driven approach on each and every process and improve them separately, since the whole is not affected.

Let us now look into some other areas with high complexity mentioned before, where a hypothesis-driven approach can be deeply problematic with extremely high risk-taking if performed.

Product development
In a context like product development, we often, at least from the start, need to gain more knowledge on the whole, when making our modules/subsystems of the whole. This knowledge can be gained directly in an experimental and hypothesis-driven approach by a number of techniques like PDCA/PDSA, Set-Based design, MBSE, Multiple concepts, etc. If we look at this kind of problem in the Cynefin framework, we would put it in the Disorder domain, which is context independent (context-free), before we understand what solution to apply to solve our initial problem. Since product development have high complexity, our solution on the whole (context dependent) would most probably end up in the Complex domain if we do not even know if we can solve the problem with (our) existing knowledge, or in the Complicated domain if we think we can gain the knowledge in a few iterations. Examples of gaining knowledge by iterations are prototyping in hardware product development, or some sprints with User Stories in order to solve the Feature in software product development. If our product is big, we will recursively apply the same patterns as above on the modules/subsystems.

But, here comes the hunch. We can only use any experimental and hypothesis-driven techniques on the parts, only if we have started top-down on the whole first. Why? Because, in product development we have high complexity, which means a lot of interdependencies between our teams and between the modules/sub-systems of our product that we need to keep track of. This means that trying to improve one part or making team improvements are difficult due to the high connectivity. The complexity makes it very easy to sub-optimise, and therefore it is not legitimate to use hypothesis-driven approaches on the parts without thoughtful consideration on the whole first. To be tangible, it is the non-functional requirements on the whole, like performance and fault handling, that need to be part of the systems design on the whole, to achieve the parts that with some planned iterations (prototypes) will integrate to a whole. And attentive readers that had examined the earlier blog post Aggregation vs Integration vs false Integration, will understand that if the systems design is neglected, it will lead to the treacherous false integration.

Problem-solving (always caused problems)
First we need to distinguish a problem from a problem, where we have two different types of problems. If our problem is to make our product from start, see product development above.

But, if we instead have caused the problem (ourselves) for example when developing our product, the problem is commonly called a complex or wicked problem. A caused problem means that the problem instead is a symptom of something deeper, one or several root causes*. The problem is still context independent and in the Disorder domain in the Cynefin framework, but it is a symptom, meaning we cannot directly solve it. Why? Because any try to solve symptoms will sub-optimise on the whole, meaning it will generate unintended consequences anywhere and anytime on the whole. And here we have a hunch again. And not a small one. This leads to that all techniques mentioned above will stand short since they are solution techniques, which means context dependent. None of them work on symptoms, i.e., since symptoms are unsolvable, no technique will do the job. And as you know we need to find the root cause(s). And the only way to reach a root cause is to use root cause analysis techniques, simply ask why we have the problem(s). But of course, there is hunch. We cannot move root cause analysis techniques successful in one context to be used in another context without being careful. And another hunch, if we do not know what the possible root causes** are (root causes are context independent (context-free) too), we do not even know when to stop our root cause analysis.

A shot summary regarding a hypothesis-driven approach to problem-solving is that the problem cannot be a caused problem (by ourselves), since then we need to find the root cause first. When we know the root cause, we can start examining the most appropriate solution, where of course a hypothesis-driven approach sometimes can be the answer.

Transformations (of organizations)
This is really a sad story, where some agile scaling frameworks divides everything in an organisation into parts in any imaginable direction; teams, team of teams, maintenance, application, software, hardware, economy, roles, analysis, architecture, design, test, requirements, portfolio, culture, efficiency, effectiveness, flow, innovation, line management, deliveries, strategy, operations, etc. for later combining to a whole. This is false integration at its worst since no thinking at all about the whole has been considered first, even though it cannot be much more complex than the complexity in an organization doing product development. Why can this happen? The believe is, which is contrary to existing science (now and forever), is that we can divide anything in small parts, optimise them one by one, and put them back to an optimised whole. And by oppressing our common sense, we start to believe that this is true. But we will face tough times when we wake up from the fairy-tale. Be sure.

Anyone, without an oppressed common sense and even without looking into science, can see that trying to put up a hypothesis to make any part working/better/optimised without affecting another part is only a wild goose, it is only guessing. Not to mention how in earth we can know and tell our top management that our transformation is on track.

Summary
As long as a hypothesis-driven approach is done on the whole (any context) or recursively on parts or completely independent parts (like processes in manufacturing), it is a good and fast way to gain new knowledge if needed. And as mentioned there are many methods that can be used. The feedback-loop is important and for example regarding the uncertainty of what the customer need really is, leading indicators can be helpful metrics. It is never possible to have a hypothesis-driven approach when trying to solve caused problems, because it is impossible to solve symptoms. This goes for both organizational problems, when we have problems with our way of working, or product problems. Instead, we need to get down to the root cause first, which is where we can apply a solution, depending on the context and complexity level.

As we now can see, it is of utterly importance to understand that a caused problem only can be solved by solving its root cause(s), and it seems like this knowledge need to increase. Without this understanding, expressions like analysis-paralysis can emerge and hypothesis-driven everything can become a hype.

And now when we have this knowledge, we can stop an overuse of the hypothesis-driven everything, because who wants to get into the fatal hypothesis-nemesis cycle, self-inflicted or not. No one, we can presume.

C u soon again.

 

*root causes are simply only non-fulfilled principles (laws). Think about Archimedes’ principle; if we make a boat that is heavier than the water it pushes away, it will go under in no-time, meaning that our root cause is that we did not fulfil Archimedes’ principle.

**and, of course we need to understand where the solution on the root cause can be executed, in the team, in the team of teams, etc. up to the enterprise level. This makes it for example very easy to make improvements or solving problems within the processes at manufacturing, since the lack of interdependencies makes the hypothesis local to the process only.
Which also means that a solution in product development can be at any level, from locally within the team to globally on the whole enterprise. Applying the solution on the wrong level, especially a too low level, means that we are trying to solve symptoms, which only will lead to unintended consequences which we sometimes can see pretty soon, but in most cases much later.
But, when we finally have found the root cause(s), we apply the appropriate solution techniques, depending on what root cause to solve. Experimental techniques if the solution is in the Complex/Complicated domain, or just straightforward techniques since the solution many times will be in the Obvious domain, when we have found the root cause(s) to the caused/complex/wicked problem we started with. Probably the words complex or wicked in front of problem, is just because we for a long time have been trying to do the impossible, solve symptoms.