This article consists of two examples of some common combination of root causes from waterfall with projects respective organizations that has transformed to agile at scale, and how to dissolve the root causes. It will then also be easy to understand that Continuous Improvement will not solve any root causes to the common organizational problems, no matter the way of working they originate from.
This series of blog posts, can also be recommended; a deep-dive into Dr. Russell Ackoff’s “panaceas”, antisystemic methods that still are in use, but still having the same pain points. The reason why it is possible to give examples, are because there are not so many root causes. This means that even if the symptoms and consequences are different between two organizations doing product development with waterfall projects, the root causes are most probably the same. This is why the examples below are apt, originating from real-world problems.
Common root causes for big organizations doing waterfall with projects
Here is a file, pain points in silo organizations, with some big pain points in an organization with a line hierarchy doing the waterfall methodology with projects.
Before we are getting into the pain points, we can note the following, which is also what we should expect! We can see that there are no problems having a line hierarchy and also adding a virtual cross-functional delivery structure (projects) when the purpose is to do product development. The reason is that the line hierarchy is necessary for reducing the complexity when having many people working, with all the normal responsibilities of a line hierarchy, except the delivery, since that is the responsibility of the virtual delivery structure (a project in this case). The line hierarchy takes care of all the problems that emanating from an unstructured and unordered bunch of people, where we of course must follow the Organizational principles for people, regarding the size of the teams and total size of the teams of teams. To dissolve these problems, the line hierarchy is therefore a must, and is valid for all organizations, it is context independent. In product development we need to remove the delivery responsibility of a decided initiative (project) from the line hierarchy and introduce a virtual cross-functional delivery structure that takes care of the delivery, HOW to deliver it. The line hierarchy can now be seen as a dynamic resource pool, a flexible solution for getting the needed resources to an initiative (project). A resource pool can also be static as many times with agile at scale, but where we still need the line hierarchy to avoid sub-optimization as stated above. The cross-functional delivery structure, dynamic or static, is thus a necessity, dissolving many of the organizational problems regarding sub-optimisation in the line hierarchy, where the sub-optimisation here is about the lack of the planning and prioritization for the initiative (project). But, the management in the line hierarchy still have the responsibility for WHAT to deliver. And since there in big organizations will be more than one initiative, when need to plan and prioritize also these multiple initiatives, and that is why we need a portfolio, which is the responsibility of the management.
The pain point in the middle (left) gives common problems in hardware product development organizations doing waterfall with projects. Especially when the size of projects is reduced due to small competitors intruding on parts of the business, that is normally not the core business. The small companies can have an efficiency of 2-3 times higher than the big organization, mostly depending on the too low T-shape competence profile for the employees and the long chains of interactions between the employees (which also gives the Chinese Whispers effect, which means information forgotten and/or corrupted) at the big organization. Dissolving these problems and their root cause is to make flatter organizations for the virtual cross-functional delivery structures, which will induce a lot of T-shaping for the employees, making the broader, and by this, it is possible to have a lot more full-time employees in small projects even in big organizations. This also means that the managers need to let go their requests on their employees only working with their respective managers area of competence responsibility. This will also give enormous benefits to big projects, since fewer part-time employees will be needed in the projects. Having more T-shaped employees therefore lead to less Project Administration waste, see this blog post for more information.
The two middle (right) pain points are about that there sometimes can be too many stakeholders and with too low agreement between them, which also can relate a lot to uncertainty of what product to make for the customer. This in turn gives late requirements or late start date, which in turn gives too little time to reduce the complexity in the product. The latter is also connected to the sometimes too big time-boxes in waterfall projects, unable to reduce the complexity fast enough even though prototypes are used. This is all about making a proper pre-study and systems design, and not rush into the execution phase too fast, which will easily happen when we lack the time that we really need. If we start too early with execution, but we have not really reduced the complexity, it will only take a longer time. Stakeholders that cannot agree due to uncertainty, means that we need more interactions with the customers, to understand what the really want, which requires iterations in order to acquire more knowledge about WHAT product to make. Here we need to add that waterfall with projects sometimes have too loose connection to the production, which is a very important stakeholder, since we need to be able to manufacture the product easily. Lean Product Development has since long figured that out, and a production team is always within the product development from start.
The right most pain point is two-fold; the inability to make platforms, which gives the possibility to make incremental deliveries of new “better” modules continually, as well as that a phase need to be completed before next phase can continue, which gives long lead times. One part of how to dissolve these problems, are as mentioned above to have a flat organization, to induce more T-shaping, but also the needed iterations for being able to make the platform by reducing the transdisciplinary (integrative) complexity, which is the strength of Lean Product Development, a phase that does not exist in the waterfall method.
The final problem and root cause, is the one to the most left in the bottom of the picture. The lack of an organizational problem-solving method inside the way of working, that solves organizational problems within the method or framework itself, goes for all methods and frameworks. Except Toyota’s Toyota Production System and Toyota Product Development System, which are built non only on the Kaizen pillar, but also on Toyota’s Jidoka pillar, where the latter means to automatically find problems and solve them as well.
Common root causes for organizations transform(ed/ing) to agile at scale
Here is a file, pain points for agile at scale, with some big pain points in an organization that has transformed or is under transformation to agile at scale.
Agile at scale often has Continuous Improvement as its mantra, with the aim to make each team or team of teams-constellation efficient. But Continuous Improvement was a method used for organizations already three decades ago, and was brought up by Dr. Russel Ackoff as manager’s panaceas, methods that Ackoff considered antisystemic. Continuous Improvement is dissected in this blog post, where it is stated that a clear sub-optimization will be the effect when used in organizations, especially product development organizations. It is simply not possible to optimize the whole, by optimizing the parts, which is exactly what is happening when doing Continuous Improvement, for example with the PDCA method, commonly referred to when doing agile product development.
The word continuous in Continuous Improvement gives the impression that it is good to do improvements as fast as possible on any part in a system. This has led to tremendous misconceptions about the meaning of Continuous Improvement and when and how it can be performed. This can be seen especially in Agile product development using the Continuous Improvement from Toyota/Lean production, with their, close to autonomous, production processes, without consideration about the change of context; from a clear to a complex context. This together with a focus on queues in Agile, due to the misunderstanding of WIP inventories in Toyota/Lean Production, has many times led to a culture of non-planning. Planning is context free and is something our specie has known since ancient times, a necessity to avoid a mess. Since queues are only a symptom, neither WIP Limits, nor Process Cycle Efficiency calculations nor other measurements, nor any other method, can solve the problem with queues directly. Only taking care of the root causes to queues can and no Continuous Improvement in the world can make it better either, it will only sub-optimize on the whole. All problems are generated by non-existent mid- and long-term planning, and the focus on that teams with their Continuous Improvement for the respective team, that they can fix these problems (symptoms), can be seen exactly with two pain points in the middle of the picture.
But, as always, the smaller the system is, the easier it is and sometimes possible to take short-cuts. This means that for a few teams doing a small software system it is possible to understand when PDCA can be done and when it cannot be done, in the same way that we can have emergent architecture/design as well as very little documented information for small systems. But, when the software system is big, the complexity and all the interdependencies between the teams and the dependencies within the software code, makes it impossible to differ an organizational problem, from something that is working, but which we can make better. It is then no longer possible to judge when Continuous Improvements can be done or not.
The two pain points to the right in the picture, shows the problems occurring when a proper systems design is neglected. This leads to many problems with the requirements traceability, lack of documented information, lack of a proper architecture, many bugs, etc., frankly to a tremendous lot of problems, as we discussed earlier in this article. And if we do not have any proper systems test on the whole product as well, we will find our problems within the product very late, if we are not doing SPPA continually. Without SPPA the risk is instead that we are trying to solve symptoms, which leads to that all teams must plan at the same time (due to non-existing mid- and long-term planning), that everybody in the team need to know everything (due to non-existing mid- and long-term planning, as well as that the team need to do all the work, since no people working with the whole is left in the organization anymore, etc.). That all teams need to plan at the same time is interesting, since that is a context independent root cause (false cadence), valid both for production and product development (in any domain), and apparently it cannot be the solution. It is obvious that this is an attempt to solve the symptom of non-existing mid- and long-term planning. And all symptom solving is sub-optimization which gives nasty unintended consequences, i.e., more symptoms, that can come anywhere and anytime. This sub-optimization is a really clear one, since false cadence on some parts of the organization really makes things worse on the total. See also this blog post for a detailed root cause analysis.
Unfortunately, we will also get problems with our flow efficiency, as well as the risk of making the product wrong, which are the top two pain points in the picture, which can take longer time for the organization to realize. These problems are still many times apparent and visible, which will start making a lot of friction within the organization between people that need to continue on the agile track, and people that are objective, stating that symptoms cannot be solved. That is a clear fact. But, by doing a SPPA, it will be clear that agile at scale have many built-in problems, that we need to hunt out.
The final problem and root cause, is the one to the most left in the bottom of the picture, which was already presented above, but worth to repeat. The lack of an organizational problem-solving method inside the way of working, that solves organizational problems within the method or framework itself, goes for all methods and frameworks. Except Toyota’s Toyota Production System and Toyota Product Development System, which are built not only on the Kaizen pillar, but also on Toyota’s Jidoka pillar, where the latter means to automatically find problems and solve them as well.
Kaizen can neither solve organizational problems
Even though we now in the beginning of this SOSD series of articles, have acquired some theory and therefore understand the output from SOSD, as well as some important aspects about product development for especially big systems, implementing a systemic dissolving of the organizational problems (by eliminating the organizational root causes) can sometimes be a bit tricky. This depends on the impact of the changes, i.e., how bad the way of working was in the first place. Thanks to the System Collaboration Deductions our updated systems design of our way of working will directly achieve astonishing results. It is important to understand that while having unsolved root causes, especially when not knowing which root causes, Kaizen cannot be used (yet). The reason is that we cannot understand the impact of the improvements we are doing, since the improvements are then done on symptoms, and trying to solve symptoms are impossible and only give unintended consequences, i.e., more and meaner symptoms. And this goes hand in hand with Toyota having two different pillars for Jidoka and Kaizen, and not using Kaizen until the process is well-functioning, stable and standardized, which is required for Kaizen to be effective. Related to this, that we need to be vigilant to, is when scaled agile transformations take years to implement, and then after that, continuous improvement will be done, for many more years. To frame the solution of organizational problems to be found by the team, within the team, is exactly what sub-optimization is all about. This is really no difference to the traditional sub-optimization within silos, where a silo is the frame instead. That means that the solution is the same in both cases, which means virtual delivery structures that deliver wholes, not partly deliveries which is the case of silos or agile teams. To accept this sub-optimization, i.e., the fact that the root causes to the original problems are never looked for, is really risky, since the mal-functioning way of working can never be cured with sub-optimization, meaning of course that Kaizen will of course neither help. Only Jidoka and problem-solving can.
This was all for this article, the next one of this series of the SOSD method is the summary article.