System Collaboration Deductions – Systems Design – Reducing transdisciplinary complexity/complicatedness (product right)

As stated earlier, the ability of systems design (and systems test) to reduce transdisciplinary complexity/ complicatedness when making new products or platforms, is one of the most important ones in product development. If this is not done correctly, it is easy to end up with just a system sketch (more like a jigsaw puzzle) and not a system architecture, see this blog post for a deep-dive how to be sure to not get it wrong. See also this article, where the difference between complexity (product right) and uncertainty (right product) are defined, which is important when reading this article.

A resulting system architecture is also a necessity for us so we can get an overview and understanding what we are doing and how our system actually looks like, which is also an important complexity to reduce, especially regarding development of bigger products. The system architecture therefore gives us the needed structure and overview, as well as how the parts fits and interact with each other, from our first thinking to a ready product. A very big product (system) can also need sub-system architectures, which means systems design per sub-system as well.

Here is an attempt to explain why reducing transdisciplinary complexity/ complicatedness is so important in order to achieve a proper system architecture:
The specified requirements on the respective parts, to be able to fulfil their respective functionality, as well as their respective non-functionality, i.e., the parts’ interactions to become a unified and well-functioning whole, all in order to fulfil the specified functional and non-functional requirements on the whole, can never be set in advance when we have transdisciplinary complexity/ complicatedness. This is what is meant with transdisciplinary (integrative) complexity/ complicatedness; we cannot specify the solution of the respective parts from start. Instead, we need to integrate the parts (from a well-though-out systemic hypothesis) to a whole, and verify and validate the whole, in order to see if our assumption or hypothesis, the systems design on the whole with all its parts, were correct. And regarding this reduction of transdisciplinary complexity/ complicatedness, there is no difference between hardware and software. Reducing transdisciplinary complexity/ complicatedness often means prototypes in order to gain new knowledge from the tests on the integrated whole, until we have all the knowledge needed for developing our product. This is very different from disciplinary complexity, often called research, which is about digging down into a discipline in order find new science or new technology, and therefore mostly is a part of the way of working for hardware. But, as this also is about reducing complexity, reducing disciplinary or transdisciplinary complexity has one thing in common; we do not even know if we will find what we are looking for, and if we find it, we will not know when.

So, we can never fully specify the requirements for the subsystems (parts) when having high transdisciplinary complexity, since it is impossible to have that knowledge in advance, which is what Alicia Juarrero [1] explains as “trying to reach new cohesiveness”. This is why all new product development, no matter domain, is highly transdisciplinary complex. Simply, we can only achieve new knowledge at the synthesis (integration) of the parts to a whole, which also includes the systems tests of the whole, since we can never specify us out of any complexity. From this we can also deduce that a part in itself can never be validated, until the whole is both verified and validated.

If we have not done a systems design on the total system (the WHAT), we cannot claim that we have taken care of the non-functional requirements especially, since lower levels then do not know the non-functional requirements on their respective parts. Systems design is about making HOW the parts fit together, interact together, to become not only a united, but also a unified and well-functioning whole, which means the what the respective part is responsible for according to their respective functional and non-functional requirements. Just to divide the functional requirements of the WHAT into smaller functional requirements of what per respective parts, will not do the job of systems design, see this picture, Systems design is always top-down.

It is also important to point out that the outcome from the systems design is not only specific functional and non-functional requirements that will be put in the requirements specification for the respective subsystems. There will always be general specifications that all of the subsystems need to follow, where especially interface specifications, like communication protocols, including cyber security functions like authentication and authorization requirements should be mentioned. These general specifications are also important for achieving a proper traceability, as well as that they also represent the intentions of the systems design for communication between the subsystems, that all subsystems need to follow, otherwise the solution on the whole will not work. If the subsystems themselves are the owner of these general requirements, by putting them in their local subsystem specifications, changes in the systems design regarding the communication, for example by adding extra cyber security functionality, will not be reflected in the subsystems automatically. It is then also a high risk that these general requirements are modified or deleted locally in the subsystem specifications, by mistake, or due to that the intention has been lost.

It is also worth mentioning that general specifications for communication, also need to consist of the high-level requirements as well as the description of the total communication solution. The reason is that when implementing the SW communication module, it will be on the lowest level of abstraction for testing, since it is a module, meaning it is not reasonable or in many times not meaningful to verify the low-level requirements, instead the higher-level requirements will be tested against the solution. A communication module must also be tested thoroughly locally, since the test of the whole subsystem that is performed later, will test the whole subsystem’s functionality, including communication and all the other functionality at the same time. This makes any communication module somewhat different with a clear function that can be seen from the outside, in comparison to all other modules within the subsystem, that together achieving the other functionality of the subsystem, i.e., the black box. A control algorithm, to maintain vehicle speed, is an apt example. It consists of many different subsystems, that together implements the solution, but where no subsystem can be understood from the outside, since they together achieve the function “maintain vehicle speed”. See the following picture, communication protocol testing including cybersecurity.

We will now show two different scenarios of what will happen when we are not taking care of non-functional requirements. Note that no one of the two scenarios, and their examples, will of course work at the integration of the parts and verification of the whole, due to neglecting the need of a well-thought-out systemic hypothesis.

In the first scenario, common from agile at scale, which just divides down the functions on the whole into the parts, without taking care of non-functional requirements like maximum 100 milliseconds respond time on the whole systems. By neglecting this non-functional requirement, this will end up in functions that will look like an easy and fast implementation within each of the parts, since the respond time has not needed to be considered per the respective part, actually totally neglected, since there is no systems design done. This will of course give tremendous performance problems, since all the parts are under-worked, which will not be visible until system tests of the whole system have been done. Unfortunately, systems test for the whole, is also another common neglection in agile at scale, which means that we will not even understand that we have an inferior system until very late, most probably too late.

Another scenario is when the implementation of parts become much more difficult, and many times impossible to do for each part by itself, when systems design is neglected, where we will give three different examples. Remind that the parts anyhow, implementable or not, still will not integrate to a whole.
The first example is for hardware products that need to have protection from becoming too hot or being electrified on the surface reachable for the customer, where there are many rules to follow in the standards. With only functional requirements in mind and without taking care of the non-functional requirements for a product, this gives no understanding for the protection level that is needed for the respective parts. This is of course necessary, since the needed protection level depends on where the part is situated in the systems design, which other parts it is connected to, as well as current technology for achieving a protection level, etc. For example, an exhaust pipe can be several 100 degrees, but is due to its position, not regarded as dangerous, as well as an electric part in a dish washer is not touchable, and therefore not considered as dangerous. Of course, in both cases, there will be warning signs on the products, as well in the instruction book.
In the coming two examples for this second scenario, we will take an onion and its layers as example of the need to make the systems design from the top, or another way to put it; from the outside and towards the middle of our system.
The first example is about functional safety, regarding protection of the driver in a car, when adding the airbag as a subsystem for reducing the risk lever at collisions. When making the airbag, we already have a car, which has technology for protection by other subsystems, according to the current technology. This can of course be taken into account when making the airbag, which means that the driver and passengers already are protected by many layers or levels of protection already; iron, damage zones, girders, windows, safety belts, etc., and that there also are a proper place where the airbag can be attached to the car, i.e., a proper interface towards another subsystem of the car. This means that the risk level for the car driver to get severely injured, is decreased for every layer. This means that we can take that into account during the development of the airbag, as well as we need that in order to connect the airbag. Here is a picture showing how the risk level is decreased for damage scenarios, Decreasing the risk level for damages.
The second example is about cyber security over the air, where we also can talk about layers of protection, due to the fact that the cyber security threat can only come from the outside, see also the picture below. This means attack interfaces like Bluetooth and Wi-Fi that are short range, or via cellular that is long range, making the latter one the most threatful one for cyber security. This means that we need to add authentication mechanisms to this communication with our product, to be able to protect the properties of the data (CIA – Confidentiality, Integrity, Availability), and the requirements of course need to be common for the modules using them. The current authorization function, need to be reviewed, in order to add functionality also for increasing the cyber security. This can be additional functionality, like adding a schedule when entrance is possible and by whom, as well as other conditions, making it even harder for an attacker to intrude. For the actual data, there can be check sums, hidden identifiers within the data, or other measures to increase the chance to detect an attack. This needed thinking is continued for each layer, when the data originating from outside the product, continues to be sent within our product, until it finally reaches the module the data was aimed for in the attack, see the picture, Decreasing the risk level for cyber security threats, below. Even if it is not as obvious as in the example with the airbag, that needs physical connection to other subsystems of the car, increasing the cyber security can neither be done local, since all subsystems have interface to at least one other subsystem. It is all about a systems design for every level from the outside towards the “middle”, to achieve parts that are united, unified and well-functioning for achieving cyber security. In a system, there is never a one-man-show. Spamming is a good example of the need of a top-down/outside-and-in-to-the-middle systems design, that we need to be able to protect from already at the first layer. This due to the fact that letting to many non-authorized messages into our product will choke the internal communication bus inside our product, and our product will then stop due to that the integrity of critical functions within our product, is lost.
In all these four examples for both the scenarios, we can see that we really need to do the systems design top-down (outside-and-in-to-the-middle). This due to the fact that the implementation of non-functional requirements directly on the parts will lead to over- or underworked for the whole, in any product development, no matter domain. This will generate many symptoms due to the fact that we did not properly take care about our systems design; symptoms for the parts, like:
– too expensive
– too slow, bad performance
– too long time to implement
– technology, experience, educations, knowledge is not enough
– the parts will be too big, making the total product too big
– extremely bad quality
– etc.
But the worst of all, is of course still that without the systems design top-down, the parts will not integrate together to a united, unified and well-functioning whole, no matter the two different scenarios above, i.e., no matter if it was easy or difficult to implement the part itself. It will still not work.
Here it is worth mentioning some words about the difference between complex tasks and symptoms, which means the difference between unsolvable and insolvable. Complex tasks like are unsolvable, which means that we need have more information/data about our proposal (hypothesis) of the solution to the task, which is usually achieved by iterations and/or prototypes. Symptoms instead, are impossible to solve, they are insolvable. This means that more information/data will not help us, since we need to find and solve the root cause(s) that are the origin of the symptoms. This implies that we really, really need to be careful when we use Big Data, since even not Big Data can solve a symptom. This also implies that we need to be careful with measurements of parts/teams when there are unsolved root causes on the whole, since the measurements then do not give any useful information/data, since any information/data cannot solve a symptom. Here is a picture showing the relationship between neglected complex tasks and the generated symptoms, neglecting the systems design gives insolvable symptoms.
We can also combine the systems design/architecture picture with the layers of the onion picture, into the picture below, since this is actually what is happening. This exemplifies that there is actually no difference at all between the scenarios, or systems design of any other non-functional requirement, or combinations of non-functional requirements. Remember also that non-functional requirements are always present, there is no implementation of anything that do not have any non-functional requirements. Every new level also means different new systems-designed non-functional requirements for the respective parts, which we of course cannot understand until we have verified our well-thought-out systemic hypothesis, see this picture, Systems design is always from outside and towards the middle.

Therefore, the plan for the whole initiative must always consist of deliveries of the parts to Integration Events – IEs, where we integrate the parts and verify the whole, in order to get knowledge of our systems design, our hypothesis, i.e., for each layer. In the beginning, when the transdisciplinary complexity is high, we just need to put mocked parts together, to get knowledge about if they are united and unified as expected. Later on, these IEs will continue, but instead every team is then delivering their important piece, to be integrated, verified and validated to the whole on the wholeness level for big initiatives. Note that the term CI – Continuous Integration, is not synonymous with one or many IEs, not even the last IE before release. CI is about adding new tested code to the existing code, to avoid Big Bang Integration (and of course also a big bang for the verification and validation) at the end, but does not automatically include that the total code is verified and validated. This induce that huge problems still can be the case, especially for doing the big system right, see this blog post for a deep-dive. The systems design and systems test can therefore be seen as necessary iterative experimentations, but from well-reasoned hypotheses continuously updated. By that approach, we are able to gain new knowledge, so we can reduce the transdisciplinary complexity (unpredictable) to transdisciplinary complicatedness (predictable), before starting an initiative.

Making a new platform has even more transdisciplinary complex complexity, compared to making a new product or system, due to the fact that more parameters need to be taken into consideration. This is also valid, when we for example are combining different existing monoliths (some kind of disciplines), to a common modular platform instead, since we only have the disciplinary science and technology for each monolith. This makes the new platform a novel product (system), since the new combination, i.e., putting all the non-functional requirements for each monolith together, means that the new solution is novel. Put another way it means that, we can never aggregate the respective solutions of the monoliths to a whole, even though we can aggregate their respective non-functional requirements to the full non-functional requirements for the whole.

Changes on an existing product or variant of it, instead need only exploitation which means reducing transdisciplinary complicatedness. This means that the results from a few prototypes analysed by experts, will give us the needed knowledge so we can do the necessary final updates, integrate, verify and validate to a unified and well-functioning whole. This is why an iterative approach with prototypes as a solution to this, are self-evident in hardware product development, an integral part for us humans, since probably hundreds of thousands of years, in “hard” domains when reducing transdisciplinary complicatedness. These iterations can be seen as exploitations, but from well-reasoned hypotheses of course, to be able to gain the last needed new knowledge, and by that approach, reducing transdisciplinary complicatedness, still iteratively, until the product is finalized.

It is also necessary to consider that the way of working is following this top-down and transdisciplinary approach, considering all needed disciplines and competences in order to do a proper systems design. This top-down approach is very important, so we do not skip (ignore) any steps from start, or trying to start on a level in the middle of our product, without the right requirements figured out on the parts on that level. Here below we will show some typical scenarios that are common, but that we must avoid, in order to not risk our product development. The first picture, systems design to achieve the architecture, shows the intended total way of working, and is just a variant on earlier pictures:

Our total way of working of course need to support this top-down approach needed for achieving our system, our product, with all the different how, i.e., top-down and level per level. This to avoid generating symptoms, that when having them, often occur very late in the PD process, especially when early testing of the full system also has been ignored. Since product development is transdisciplinary work level for level as shown above, and domain independent as well, here is another picture, with a little other approach, showing also the transdisciplinary work needed by all the disciplines within the organization, Transdisciplinary Way of Working needed – showing the disciplines:

As we can see in all transdisciplinary work, all the disciplines within the whole organization are dependent on each other, in order to be able to know what to do on the next level, regardless if there is about planning or reducing complexity. And since a lower level depends on the output from the higher level right above, that means that we always need to start from the top, i.e., which gives the top-down approach. The reason why the systems design is a fluffy cloud, is because in order to make a new or especially novel product, we always need to gain new knowledge, which means reduce the complexity. This is why the method TDSD has been developed, including its system skeleton thinking, per level. This gaining of new knowledge, means iterations of a level, as well as go back from a lower level to a higher, since the systems design on the higher level, needs refinement. Remember also that sub-optimization will always occur when trying to focus and optimize the parts themselves, i.e., ignoring the needed transdisciplinary work in any product development way of working. From that we can also deduce that changing the parts from disciplines to dimensions, i.e., any other way to get parts (overlapping or not) of the whole, without any top-down transdisciplinary work, will still only sub-optimise the whole.

It is also important to point out, that when we do not have any way of working, the requirements (What) for any level, also means that we need to make our way of working at the same time. This means thinking about the total PD process and its sub-processes, which can be different per level, until we reach the teams’ level. This gives that the What is not only requirements valid for the product, but also requirements required for the total PD process and its sub-processes. We therefore, as shown, at the same time need to think about how we will work with traceability, release management, planning, quality, documentation handling, tools etc., and level per level as well. Making pilots is one way to test the PD process, but given the above, means that it is important that the total PD process is nailed out top-down, at least as an overview, all in order to avoid sub-optimization. But unfortunately, many times it only becomes more of dry swimming of many sub-processes which are not properly connected, which then will only become a lot of different sub-optimizing activities, that only will lead to even more and more severe symptoms.

But we should also be firm, that we have the needed science about the organizational principles as a solid foundation, and where this series, with all its deductions, for example shows the needed top-down approach, as well as that the solution space for a PD process is dramatically decreased. This gives that it is secure, to make a total PD process implementation based on science. It is also preferred, compared to doing pilots that fully neglect science, and therefore only will lead to sub-optimization and a mal-functioning PD process.

There are of course times when we already have a unified, united and well-functioning system, with an awesome systems-designed architecture. Then we have two cases where the above setup with agile at scale, with decentralized thinking, actually can work.

  • The first example is for non-complex additional functionality or maintenance work (bug fixing). This is valid since in this case, we do not need to consider the non-functional requirements and we already have systems test for the whole as well. Non-complex tasks can of course (almost) any way of working successfully solve, which will look like this, non-complex tasks in product development:

A huge problem here, which is also the most common approach, is when you introduce agile at scale by showing how it can work when solving non-complex tasks, which properly can be done by decentralized and local work. But, since that is not the reason why you introduce agile at scale in the first place, it really does not show any performance at all regarding solving complex tasks, like developing a novel product or a platform.

  • In the other example it is, on a lower level in the architecture, also possible to add, not only local functional requirements, but also local non-functional requirements. A bus system for communication, where you can add new hardware units, with software functionality, is an apt example. It will then look like this, Adding modules to an existing architecture:

The problem with this last example is when the day comes, and it will for sure do, where we need to make a novel product, a new platform, or need to introduce functionality that is affecting all parts (like cyber security), which means that a top-down systems design for sure is needed. Because, this then mean that we do not have any way of working, no responsibility, no competences, no capabilities, no experience, no job positions, no tools, etc., for the needed work in the top layers, since we for a long time have not needed it. This is of course understandable, but will in this case stall any organization. This especially because there is not even someone responsible for the total way of working, or maybe not even the total product. There is therefore no one that can take the decision to achieve this top-down approach.

Here are two scenarios where the top-down approach is definitely not followed, and that will generate infinite severe symptoms, unfortunately discovered very late:

  • Only the functional requirements are considered, and divided into smaller and smaller pieces, without considering the non-functional requirements. This approach is common for agile at scale, and looks like this for novel products, Way of Working – system design on teams level:

With only systems design at the team’s level, there is no proper systems architecture, on the above levels, and which also means that the teams work in their architectural parts are like isolated islands. There are neither testing on higher levels, since the non-functional requirements have not been considered from start. The lack of proper systems design, means lack of proper requirements, which in turn means lack of traceability, etc. Systems design is really a hypothesis of how to solve all the requirements, where the non-functional requirements are the tricky ones. Systems design means first analysis and then synthesis, where synthesis is integrating and not aggregating, in order to fulfil the requirements (what), i.e., parts can never be validated themselves. All this will of course lead to tremendous inefficiency and low quality, and most probably a low performance product, probably it will not work, which of course make the approach not useable for products where high safety and security is mandatory. Note, that due to the high complexity to reduce in bit systems, refactoring will not help us.

  • One variant of this is when systems design explicitly is added on top of an agile at scale way of working, due to that the organization understands the need of systems design, due to the fact that the organization started with hardware product development, where it is obvious that systems design is always needed for novel products. But, even if the systems design is done earlier, it is still too late if it is not done from the top, since the non-functional requirements are still not considered from start (and probably not in the systems design actually done at the team of teams’ level). Then it will look like this for a novel product, Way of Working – system design on team of teams level:

Even if we in this case have systems design on the team of teams’ level, the architectural parts on that layer are still isolated islands. This means that even if the architectural parts on the teams’ level fit together, we still do not know if the whole will work, since we are missing the first two upper levels.

Here is a last picture that shows what will happen when we are missing the transdisciplinary work, and thinking, in an organization. Without the transdisciplinary work, there are only the disciplines that actually can take actions, which they of course will do, but regardless of the What on any level, missing the transdisciplinary work:

We can see that it is not many similarities with systems design top-down level per level shown in the background. This transdisciplinary work is always needed for systems design, or even planning or release management, which are found in production as well. It is important to point out, that even if we start from the top as in this picture, this is only from the top of each discipline, which does not mean the needed transdisciplinary solutions within in our product development way of working, that we are longing for.

We end this article as we started, with stating how important the ability of systems design is, since reducing transdisciplinary complexity/ complicatedness, is key when we are developing novel products and platforms in any domain; software and hardware, as well as building, bridges, etc.

Next article in this series, is about the need of virtual delivery structures in product development.

Leave a Reply