Welcome to the series of articles for the deductions in System Collaboration. The deductions are made from the existing science, i.e., our organizational principles, and is the foundation for SPPA – Systemic Problem Picture Analysis, SOSD – Systemic Organizational Systems Design and TDSD – Test-Driven Systems Design for software and hybrids. Actually, any way of working need to follow these principles in order to make the organization flourishing.
This series consists of many articles with different content, covering all the important parts of the deductions. It deep-dives into the overall thinking, as well as the parts, where the parts needed always depends on context. In this series of articles, we will cover the following subjects, which is in a kind of chronological order; introduction, top-down synthesis, starting from the organizational purpose is a must, line hierarchy, systems design, virtual delivery structures, portfolio and a summary. The reminder of this introductory article, will also be a short introduction and summary of the whole series. Here is also the compromised list of all the deductions made in this series.
Introduction
In System Collaboration, the focus is always on achieving flourishing way of workings, especially for product development, in the domains of software (pin-pointed), hardware or combinations of them both. This means that the focus is on the Complex and Complicated context, with reference to the Cynefin™ Framework. Remember that our organizational principles are domain independent, even if we choose a context, which is the reason why there is actually no difference between product development for software, hardware, or their combinations, since we still need to follow the organizational principles for the complex and complicated contexts.
We will see that we always need to start from the organizational common purpose to get the context, leading to the need of having a top-down synthesis approach for our way of working, all to be able to fulfil the organizational principles. And if we fail with fulfilling our organizational principles, we will get built-in root causes, that only a new organizational systems design can eliminate. This means that it requires organizational systems design already from start, when implementing any way of working in any context, to be sure to be free from built-in organizational root causes. This need of a systems design of the way of working, in combination with the narrow solution space in a complex or complicated context when developing new products or new platforms, gives a very prescriptive order of HOW (for any domain) in these non-clear contexts. This domain independence, means that this prescriptive order is valid and must be fulfilled also for hardware product development, before the hardware domain and the domain knowledge of the organization, will nail down to for example waterfall with projects, or Lean Product Development (or why not a future TDSD for hardware). For development of big software system, this is of course also valid, which is the reason for the birth of TDSD for software (and hybrids), where we with the deductions as a foundation, further will go into the domain specific HOW, to be able to make a complete method for developing big software systems. Included in this order, is also the important top-down synthesis approach for the actual products that is needed for all product development, which we normally call systems design. And it is actually already here, as we soon can see, when deep-diving into systems design of our products, that we understand that we really need fast feedback also on our system architecture for big systems, not only on the design of the parts. It is here the understanding of the concept of test-driven systems design starts to grow, as a solution in order to get fast feedback, which is actually possible for any new product development in any domain.
With product development is meant the whole product life cycle of product development, from the first ideas and all the way to maintenance, i.e., completely new product development and not only maintenance or some small enhancements of existing products with their existing systems design and system architecture. If we think about that, we per se need to be vigilant to any bottom-up transformation (aggregation) in product development, since the start often is maintenance or easy and small enhancements which is a fairly easy context, and therefore more production like. This is due to that the systems design, system architecture and systems test (and its test cases and environment) of the product are already there. It has earlier, in a top-down synthesis approach, maybe with the waterfall method, already been scaled down to coherent and cohesive parts to make a united and unified system. So, even if it feels more comfortable to start in a bottom-up manner (aggregation), in small scale and then scale up, that is a very treacherous way to try to achieve a way of working for the total product life cycle. This is due to the fact that product development of new products that has the highest complexity, and maintenance has very low complexity, since the latter already has a working solution on the wholeness for the product. Instead, we always need to use a top-down synthesis approach, originating from the organizational purpose of the organization, that in turn gives us the context; the full life-cycle of product development, production, service, etc. By doing this, we can with very few iterations, also due to former experiences, establish our way of working (including people structures), to be able to fulfil the organizational principles for our context. This means that every way of working is simply an organizational systems design. This organizational systems design for our way of working is needed for every context, complex or non-complex, but where the context exactly specifies what organizational principles that are activated, and therefore need to be fulfilled by the organizational systems design.
Since traditional waterfall with projects, as well as trying to scale-up agile, are common solutions today for product development, but both have built-in root causes (=built-in deficiencies). In SOSD will also show them as examples and how we can rescue organizations from them, with some pretty easy changes of the way of working.
Short summary
A main topic, which is extremely important, is the need to understand the complexity of product development of a new initiative, especially the need of reducing transdisciplinary complexity and complicatedness for novel products, which also includes the theory behind. This understanding gives tremendous information to the possible solutions of the total way of working for product development, for example that the whole product life cycle must be considered already from start. Other important topics that this leads to are the people structures, the static line hierarchy and the dynamic virtual delivery structures are always necessary. The portfolio needed for all product development in bigger organizations, the strategical and tactical management within it and above it, is also a topic that will be discussed, since it is a necessary part of the total way of working as well.
It is necessary to grip the concept of System Collaboration, that even if it is easy, it many times means that we need to change our patterns of thought, since we are not used to follow science also for any way of working. The concept is all about taking care of the total way of working in a top-down synthesis approach, which originates from the common organizational purpose of the organization, and where the different structures of our people, also are a part of way of working. Since we need to have the context, the organization will operate in, the common organizational purpose must be our origin. This means that we need to plan the way of working top-down, from the common organizational purpose, which also gives us that we need to involve the whole product life cycle from start, which is especially critical for product development. Starting with the purpose in a top-down approach has been used by us humans for probably hundreds of thousands of years or even more. This is an approach for us to reduce the complexity on what we are going to do, by being systematic and structured in an organization, or in any human complex adaptive system. This planning needs to be done top-down to understand WHAT to do and HOW to do it, and it will soon be clear why, so let us start to elaborate on the need of planning of the whole way of working first. The top-down synthesis approach is needed, to be sure that we take care of also the complex and complicated contexts in our way of working for product development, especially systems design of novel products. This because it will have heavily impact also on the systems design of our way of working. And any systems design, no matter if it is a product or a way of working, of course needs to be systemic. But, beware of that it is not transdisciplinary complexity that is reduced with our systems design of our way of working, i.e., it is not something completely novel. This is due to that the science (our organizational principles) are already there, as well as our extensive experiences since ancient times, which means that with the help of SPPA, our experts can easily guide us to a solution. This is vital to understand, not only in order to understand the method TDSD for software, needed when developing big software systems, but also for the hardware product development domain (and generally for any domain regarding product development). For other kind of product development of big systems, like waterfall and Lean Product Development, they will also from start, as when using TDSD, need to follow these deductions, i.e., to be sure to fulfil the organizational principles (science) in the complex context of product development.
Making this required top-down systems design of the way of working, means that we for example cannot start in the middle of the total way of working, and add the portfolio later. This is because the portfolio stands for planning and prioritization, which is on a strategic and tactical level, and then gives the operative responsibility for the initiatives to virtual structures in product development. But, in order to be able to be responsible for structure, planning and prioritization, the portfolio also has responsibility for the money used for each initiative. This means that the portfolio will get most of the organization’s money in a product development company, even in a hardware company, where the portfolio gets much more money than the research part of the organization.
In the same way the portfolio responsible cannot start directly with designing the portfolio, without understanding the systems design needed, the need to reduce the transdisciplinary complexity for novel products. Transdisciplinary complexity means that a lot of new knowledge need to be achieved at integration of the systems designed (novel) parts, which means that we neither know the parts, nor when we can be ready. Our new initiatives therefore need to be broken down and integrated, verified and validated, in order to iteratively reduce the transdisciplinary complexity for the product we are going to build. All this is heavily affecting the portfolio, that need to understand risks, failures and that the product from start, including the undefined parts, cannot be planned in time, which also means money consumption. Unfortunately, many scaling frameworks for agile do not consider this transdisciplinary complexity (or transdisciplinary complicatedness, or planning) for either the product or the organization. Without consideration, they instead just divide the work and organization into parts, a construction sometimes called value streams, no matter transdisciplinary complexity. This gives an enormous risk for sub-optimization of both the initiative and the organization. This sub-optimization is even worse than for the old (but non-common) projectized organization (see this blog post), which at least only sub-optimized on the organization, since the total scope of the project/product/initiative was not split. The believe that the whole can be divided into smaller parts as a first step, without consideration of the whole (ignore context), also leads to the erroneous thinking that making the top-levels, including the portfolio, can be done as the last step. But, since new product development is novel, and therefore always has transdisciplinary complexity that need to be reduced, this thinking is putting the transformation and the whole organization at an extremely big risk for failure. We need to have the complete picture for our products/systems life cycle from start, in order to get organizational levels and the portfolio right. From the portfolio’s need of understanding the solution of how to reduce the transdisciplinary complexity (and complicatedness), we also have other things necessary for the portfolio to understand, like; when the investment in the initiatives is returned, with what quality, how uncertainties of what products to do, how unpredictable and unsure it is to reduce complexity and complicatedness, how the feedback will be produced, etc.
Every organization need to prioritize which work that should be done and also to plan the work, finance the work etc., which is derived from the common purpose of the organization. Planning and prioritization of initiatives are valid for all organizations and will be strictly formal with a portfolio for a big organization and more informal in a small organization, where the initiatives are few. So, the reason for discussing the portfolio is therefore two-fold; 1) we need to know the way of working for the product development life cycle first where new development is the most complex, until we can know the implementation of the portfolio, and 2) we need to understand and reduce the risk taking in many agile transformations at scale, often leaving the portfolio as the last step.
Moreover, other parameters of the purpose needed for our system as; security level, size of the system, size of the organization, length of the life cycle for the system, etc. are also very important input in order to achieve a flourishing organization. Depending on these different parameters, other necessary qualities of the system need to be taken into the equation, for example requirements traceability if our system starts to get big, which also will be further discussed below.
This also leads to that it is difference between qualities and qualities, as well as qualities and non-functional requirements. All put together will give some different ways of working, as already presented, which actually is not so many as one can expect, even if every one of them can have some flavour depending on the organization. We can then see that the solution for the top level of big systems is domain independent as well. We will also be able to understand that talking about emergent architecture and design, i.e., bottom-up software development, is easier said than done when it comes to big software systems, or many teams in a big organization that is working within the same system.
Worth mention is also that for hardware systems, emergent architecture/design is not possible, so a proper systems design is needed, no matter the size of the system. The line between small and big systems depends on many parameters. But, a few teams and a total of up to 50 persons, is where an organization many times is said to go from small to bigger, needing a formal setup with a line hierarchy and virtual delivery structures, a proper systems design and documented information. For big systems, built from scratch, it is seldom the uncertainty of WHAT the customer needs that is the hard part. Rather it is reducing the transdisciplinary complexity/complicatedness, i.e., HOW to make the big product, which is the case for many service organizations, like banks, governments, insurance companies, finance institutes, or retail. It is rather how to iteratively reduce the transdisciplinary complexity/complicatedness of especially the backend part of the big system developed, all in order to avoid risk taking, with a resulting proper systems design, see this article about the difference between uncertainty and complexity. This also means that most of the requirements for big systems, both the functional and non-functional requirements, normally can be acquired early on, meaning that we do not even need to take the risk with emergent architecture and emergent design, which easily leads to sub-optimization. With that said, we can understand that systems design is always a vital part when doing big systems. But, before we deep-dive into the world of systems design, we need to go through some theory of how to dissolve organizational problems, showing why sub-optimizations are never the way forward.
In the end, when putting all the bits and pieces together, it becomes clear that for big product (system) development in any domain, there is on a high level, only one track forward with already predetermined steps, as a result from the deductions. The steps on the track, are the result of following the organizational principles, and are presented in a picture that is finalizing this article.
Next article in this series will cover the necessary top-down synthesis approach in detail, a necessity when making a new way of working in any context.