As stated earlier, the ability of systems design (and systems test) to reduce transdisciplinary complexity/ complicatedness when making new products or platforms, is one of the most important ones in product development. If this is not done correctly, it is easy to end up with just a system sketch (more like a jigsaw puzzle) and not a system architecture, see this blog post for a deep-dive how to be sure to not get it wrong. See also this article, where the difference between complexity (product right) and uncertainty (right product) are defined, which is important when reading this article.
A resulting system architecture is also a necessity for us so we can get an overview and understanding what we are doing and how our system actually looks like, which is also an important complexity to reduce, especially regarding development of bigger products. The system architecture therefore gives us the needed structure and overview, as well as how the parts fits and interact with each other, from our first thinking to a ready product. A very big product (system) can also need sub-system architectures, which means systems design per sub-system as well.
Here is an attempt to explain why reducing transdisciplinary complexity/ complicatedness is so important in order to achieve a proper system architecture:
The specified requirements on the respective parts, to be able to fulfil their respective functionality, as well as their respective non-functionality, i.e., the parts’ interactions to become a unified and well-functioning whole, all in order to fulfil the specified functional and non-functional requirements on the whole, can never be set in advance when we have transdisciplinary complexity/ complicatedness. This is what is meant with transdisciplinary (integrative) complexity/ complicatedness; we cannot specify the solution of the respective parts from start. Instead, we need to integrate the parts (from a well-though-out systemic hypothesis) to a whole, and verify and validate the whole, in order to see if our assumption or hypothesis, the systems design on the whole with all its parts, were correct. And regarding this reduction of transdisciplinary complexity/ complicatedness, there is no difference between hardware and software. Reducing transdisciplinary complexity/ complicatedness often means prototypes in order to gain new knowledge from the tests on the integrated whole, until we have all the knowledge needed for developing our product. This is very different from disciplinary complexity, often called research, which is about digging down into a discipline in order find new science or new technology, and therefore mostly is a part of the way of working for hardware. But, as this also is about reducing complexity, reducing disciplinary or transdisciplinary complexity has one thing in common; we do not even know if we will find what we are looking for, and if we find it, we will not know when.
So, we can never fully specify the requirements for the subsystems (parts) when having high transdisciplinary complexity, since it is impossible to have that knowledge in advance, which is what Alicia Juarrero [1] explains as “trying to reach new cohesiveness”. This is why all new product development, no matter domain, is highly transdisciplinary complex. Simply, we can only achieve new knowledge at the synthesis (integration) of the parts to a whole, which also includes the systems tests of the whole, since we can never specify us out of any complexity. From this we can also deduce that a part in itself can never be validated, until the whole is both verified and validated.
If we have not done a systems design on the total system (the WHAT), we cannot claim that we have taken care of the non-functional requirements especially, since lower levels then do not know the non-functional requirements on their respective parts. Systems design is about making HOW the parts fit together, interact together, to become not only a united, but also a unified and well-functioning whole, which means the what the respective part is responsible for according to their respective functional and non-functional requirements. Just to divide the functional requirements of the WHAT into smaller functional requirements of what per respective parts, will not do the job of systems design, see this picture, Systems design is always top-down.
It is also important to point out that the outcome from the systems design is not only specific functional and non-functional requirements that will be put in the requirements specification for the respective subsystems. There will always be general specifications that all of the subsystems need to follow, where especially interface specifications, like communication protocols, including cyber security functions like authentication and authorization requirements should be mentioned. These general specifications are also important for achieving a proper traceability, as well as that they also represent the intentions of the systems design for communication between the subsystems, that all subsystems need to follow, otherwise the solution on the whole will not work. If the subsystems themselves are the owner of these general requirements, by putting them in their local subsystem specifications, changes in the systems design regarding the communication, for example by adding extra cyber security functionality, will not be reflected in the subsystems automatically. It is then also a high risk that these general requirements are modified or deleted locally in the subsystem specifications, by mistake, or due to that the intention has been lost.
It is also worth mentioning that general specifications for communication, also need to consist of the high-level requirements as well as the description of the total communication solution. The reason is that when implementing the SW communication module, it will be on the lowest level of abstraction for testing, since it is a module, meaning it is not reasonable or in many times not meaningful to verify the low-level requirements, instead the higher-level requirements will be tested against the solution. A communication module must also be tested thoroughly locally, since the test of the whole subsystem that is performed later, will test the whole subsystem’s functionality, including communication and all the other functionality at the same time. This makes any communication module somewhat different with a clear function that can be seen from the outside, in comparison to all other modules within the subsystem, that together achieving the other functionality of the subsystem, i.e., the black box. A control algorithm, to maintain vehicle speed, is an apt example. It consists of many different subsystems, that together implements the solution, but where no subsystem can be understood from the outside, since they together achieve the function “maintain vehicle speed”. See the following picture, communication protocol testing including cybersecurity.
We will now show two different scenarios of what will happen when we are not taking care of non-functional requirements. Note that no one of the two scenarios, and their examples, will of course work at the integration of the parts and verification of the whole, due to neglecting the need of a well-thought-out systemic hypothesis.
In the first scenario, common from agile at scale, which just divides down the functions on the whole into the parts, without taking care of non-functional requirements like maximum 100 milliseconds respond time on the whole systems. By neglecting this non-functional requirement, this will end up in functions that will look like an easy and fast implementation within each of the parts, since the respond time has not needed to be considered per the respective part, actually totally neglected, since there is no systems design done. This will of course give tremendous performance problems, since all the parts are under-worked, which will not be visible until system tests of the whole system have been done. Unfortunately, systems test for the whole, is also another common neglection in agile at scale, which means that we will not even understand that we have an inferior system until very late, most probably too late.
Therefore, the plan for the whole initiative must always consist of deliveries of the parts to Integration Events – IEs, where we integrate the parts and verify the whole, in order to get knowledge of our systems design, our hypothesis, i.e., for each layer. In the beginning, when the transdisciplinary complexity is high, we just need to put mocked parts together, to get knowledge about if they are united and unified as expected. Later on, these IEs will continue, but instead every team is then delivering their important piece, to be integrated, verified and validated to the whole on the wholeness level for big initiatives. Note that the term CI – Continuous Integration, is not synonymous with one or many IEs, not even the last IE before release. CI is about adding new tested code to the existing code, to avoid Big Bang Integration (and of course also a big bang for the verification and validation) at the end, but does not automatically include that the total code is verified and validated. This induce that huge problems still can be the case, especially for doing the big system right, see this blog post for a deep-dive. The systems design and systems test can therefore be seen as necessary iterative experimentations, but from well-reasoned hypotheses continuously updated. By that approach, we are able to gain new knowledge, so we can reduce the transdisciplinary complexity (unpredictable) to transdisciplinary complicatedness (predictable), before starting an initiative.
Making a new platform has even more transdisciplinary complex complexity, compared to making a new product or system, due to the fact that more parameters need to be taken into consideration. This is also valid, when we for example are combining different existing monoliths (some kind of disciplines), to a common modular platform instead, since we only have the disciplinary science and technology for each monolith. This makes the new platform a novel product (system), since the new combination, i.e., putting all the non-functional requirements for each monolith together, means that the new solution is novel. Put another way it means that, we can never aggregate the respective solutions of the monoliths to a whole, even though we can aggregate their respective non-functional requirements to the full non-functional requirements for the whole.
Changes on an existing product or variant of it, instead need only exploitation which means reducing transdisciplinary complicatedness. This means that the results from a few prototypes analysed by experts, will give us the needed knowledge so we can do the necessary final updates, integrate, verify and validate to a unified and well-functioning whole. This is why an iterative approach with prototypes as a solution to this, are self-evident in hardware product development, an integral part for us humans, since probably hundreds of thousands of years, in “hard” domains when reducing transdisciplinary complicatedness. These iterations can be seen as exploitations, but from well-reasoned hypotheses of course, to be able to gain the last needed new knowledge, and by that approach, reducing transdisciplinary complicatedness, still iteratively, until the product is finalized.
It is also necessary to consider that the way of working is following this top-down and transdisciplinary approach, considering all needed disciplines and competences in order to do a proper systems design. This top-down approach is very important, so we do not skip (ignore) any steps from start, or trying to start on a level in the middle of our product, without the right requirements figured out on the parts on that level. Here below we will show some typical scenarios that are common, but that we must avoid, in order to not risk our product development. The first picture, systems design to achieve the architecture, shows the intended total way of working, and is just a variant on earlier pictures:
Our total way of working of course need to support this top-down approach needed for achieving our system, our product, with all the different how, i.e., top-down and level per level. This to avoid generating symptoms, that when having them, often occur very late in the PD process, especially when early testing of the full system also has been ignored. Since product development is transdisciplinary work level for level as shown above, and domain independent as well, here is another picture, with a little other approach, showing also the transdisciplinary work needed by all the disciplines within the organization, Transdisciplinary Way of Working needed – showing the disciplines:
As we can see in all transdisciplinary work, all the disciplines within the whole organization are dependent on each other, in order to be able to know what to do on the next level, regardless if there is about planning or reducing complexity. And since a lower level depends on the output from the higher level right above, that means that we always need to start from the top, i.e., which gives the top-down approach. The reason why the systems design is a fluffy cloud, is because in order to make a new or especially novel product, we always need to gain new knowledge, which means reduce the complexity. This is why the method TDSD has been developed, including its system skeleton thinking, per level. This gaining of new knowledge, means iterations of a level, as well as go back from a lower level to a higher, since the systems design on the higher level, needs refinement. Remember also that sub-optimization will always occur when trying to focus and optimize the parts themselves, i.e., ignoring the needed transdisciplinary work in any product development way of working. From that we can also deduce that changing the parts from disciplines to dimensions, i.e., any other way to get parts (overlapping or not) of the whole, without any top-down transdisciplinary work, will still only sub-optimise the whole.
It is also important to point out, that when we do not have any way of working, the requirements (What) for any level, also means that we need to make our way of working at the same time. This means thinking about the total PD process and its sub-processes, which can be different per level, until we reach the teams’ level. This gives that the What is not only requirements valid for the product, but also requirements required for the total PD process and its sub-processes. We therefore, as shown, at the same time need to think about how we will work with traceability, release management, planning, quality, documentation handling, tools etc., and level per level as well. Making pilots is one way to test the PD process, but given the above, means that it is important that the total PD process is nailed out top-down, at least as an overview, all in order to avoid sub-optimization. But unfortunately, many times it only becomes more of dry swimming of many sub-processes which are not properly connected, which then will only become a lot of different sub-optimizing activities, that only will lead to even more and more severe symptoms.
But we should also be firm, that we have the needed science about the organizational principles as a solid foundation, and where this series, with all its deductions, for example shows the needed top-down approach, as well as that the solution space for a PD process is dramatically decreased. This gives that it is secure, to make a total PD process implementation based on science. It is also preferred, compared to doing pilots that fully neglect science, and therefore only will lead to sub-optimization and a mal-functioning PD process.
There are of course times when we already have a unified, united and well-functioning system, with an awesome systems-designed architecture. Then we have two cases where the above setup with agile at scale, with decentralized thinking, actually can work.
- The first example is for non-complex additional functionality or maintenance work (bug fixing). This is valid since in this case, we do not need to consider the non-functional requirements and we already have systems test for the whole as well. Non-complex tasks can of course (almost) any way of working successfully solve, which will look like this, non-complex tasks in product development:
A huge problem here, which is also the most common approach, is when you introduce agile at scale by showing how it can work when solving non-complex tasks, which properly can be done by decentralized and local work. But, since that is not the reason why you introduce agile at scale in the first place, it really does not show any performance at all regarding solving complex tasks, like developing a novel product or a platform.
- In the other example it is, on a lower level in the architecture, also possible to add, not only local functional requirements, but also local non-functional requirements. A bus system for communication, where you can add new hardware units, with software functionality, is an apt example. It will then look like this, Adding modules to an existing architecture:
The problem with this last example is when the day comes, and it will for sure do, where we need to make a novel product, a new platform, or need to introduce functionality that is affecting all parts (like cyber security), which means that a top-down systems design for sure is needed. Because, this then mean that we do not have any way of working, no responsibility, no competences, no capabilities, no experience, no job positions, no tools, etc., for the needed work in the top layers, since we for a long time have not needed it. This is of course understandable, but will in this case stall any organization. This especially because there is not even someone responsible for the total way of working, or maybe not even the total product. There is therefore no one that can take the decision to achieve this top-down approach.
Here are two scenarios where the top-down approach is definitely not followed, and that will generate infinite severe symptoms, unfortunately discovered very late:
- Only the functional requirements are considered, and divided into smaller and smaller pieces, without considering the non-functional requirements. This approach is common for agile at scale, and looks like this for novel products, Way of Working – system design on teams level:
With only systems design at the team’s level, there is no proper systems architecture, on the above levels, and which also means that the teams work in their architectural parts are like isolated islands. There are neither testing on higher levels, since the non-functional requirements have not been considered from start. The lack of proper systems design, means lack of proper requirements, which in turn means lack of traceability, etc. Systems design is really a hypothesis of how to solve all the requirements, where the non-functional requirements are the tricky ones. Systems design means first analysis and then synthesis, where synthesis is integrating and not aggregating, in order to fulfil the requirements (what), i.e., parts can never be validated themselves. All this will of course lead to tremendous inefficiency and low quality, and most probably a low performance product, probably it will not work, which of course make the approach not useable for products where high safety and security is mandatory. Note, that due to the high complexity to reduce in bit systems, refactoring will not help us.
- One variant of this is when systems design explicitly is added on top of an agile at scale way of working, due to that the organization understands the need of systems design, due to the fact that the organization started with hardware product development, where it is obvious that systems design is always needed for novel products. But, even if the systems design is done earlier, it is still too late if it is not done from the top, since the non-functional requirements are still not considered from start (and probably not in the systems design actually done at the team of teams’ level). Then it will look like this for a novel product, Way of Working – system design on team of teams level:
Even if we in this case have systems design on the team of teams’ level, the architectural parts on that layer are still isolated islands. This means that even if the architectural parts on the teams’ level fit together, we still do not know if the whole will work, since we are missing the first two upper levels.
Here is a last picture that shows what will happen when we are missing the transdisciplinary work, and thinking, in an organization. Without the transdisciplinary work, there are only the disciplines that actually can take actions, which they of course will do, but regardless of the What on any level, missing the transdisciplinary work:
We can see that it is not many similarities with systems design top-down level per level shown in the background. This transdisciplinary work is always needed for systems design, or even planning or release management, which are found in production as well. It is important to point out, that even if we start from the top as in this picture, this is only from the top of each discipline, which does not mean the needed transdisciplinary solutions within in our product development way of working, that we are longing for.
We end this article as we started, with stating how important the ability of systems design is, since reducing transdisciplinary complexity/ complicatedness, is key when we are developing novel products and platforms in any domain; software and hardware, as well as building, bridges, etc.
Next article in this series, is about the need of virtual delivery structures in product development.