TDSD – Test-Driven Systems Design – Introduction

Welcome to first blog post about this series about TDSD – Test-Driven Systems Design, which is a method for all kinds of product (system) development in the software domain, especially big software systems.

TDSD is built on SOSD, and in SOSD is where the test-driven systems design concept starts for any product development domain, and is where all the theory, WHY and explanations can be found. SOSD explains also the prescriptive HOW steps that need to be followed and WHY they need to be followed, all in order to fulfil the organizational principles in this complex context (all principles activated), which in turn strongly decreases the solution space for our way of working. In TDSD we are going one step further, by going into the domain and domain knowledge, skills and experience of the organization, further enhancing the HOW. Of course, TDSD also fulfils all the organizational principles, to achieve the highest quality assurance possible for TDSD, with no built-in root causes from start, which gives us the best possibilities to achieve quality assurance in our products. TDSD is by that, taking care of and solving the current, and unfortunately, too many common problems of today, when developing big software systems with current commercialized methods and frameworks. As the observant readers we are, we can understand that even though TDSD is focusing on software, it can with small adaptations, also be used for hardware, since the organizational principles activated in a specific context, always are domain independent. The reason is due to the fact that the test-driven systems design approach from SOSD is valid for all product development.

That most commercialized methods are antisystemic, was brought up by Dr. Russell Ackoff already in the 1990s, where he listed twenty of them from his mind, and not a complete list. Some of them have been dissected, to understand why they are failing, and can be found in this series of blog posts, with reference to Ackoff, and here is also a general part on how and why it is important to dissect methods in this blog post. Even though science about organizations have been available for decades, and even already in the 1990s, this list of antisystemic methods available on the market today is unfortunately considerably longer. This also yields recently new methods and frameworks for product development of big software systems, where some of them have induced even more, as well as added new problems, that we did not have earlier. It is always important to understand the past in order to build new methods on old experience. For example, why waterfall way of working with project succeeded to fly to the moon, but why it failed to make big software systems already in the 1980s. The failure depends not only to limitations in the waterfall method not understood, but also due to not following the waterfall method properly, mostly lacking the virtual delivery structure. The difference between hardware and software product development, between Technology Push and Consumer Pull and the reason to why there sometimes is failure, is deep-dived in this blog post series.

TBD: Add the coming “chapters” with a brief explanation to the Introduction!
In the chapter “TDSD has SOSD as its foundation”, we go through why it so important that TDSD for software and hybrids originates from SOSD as far as possible. It is also important to understand that when working in a complex context, there are certain step that need to be taken, which is the reason to that we always have methods for any way of working in product development, and that it per se cannot be a framework, presented in the chapter “TDSD is a method – cherry-picking from a framework is wrong per se”. To be extra clear about the need of TDSD, is dissected in the chapter “Why TDSD? Do we not have too many software methods already?”. In the last chapter, we go through the different levels in the organization, and the need for having them.

TDSD has SOSD as its foundation
As mentioned in SOSD, in product development, the line hierarchy is to be seen like a resource pool and the virtual structure is the delivery structure, which is a necessary setup in order to avoid sub-optimization in product development, due to that the interactions can never be predetermined in advance. From the theory behind the method of SOSD, we can draw the conclusion that in order to properly implement even the line hierarchy with its silos, we first need to understand the overviewing solution of the context we are operating in. This means in this case the basic understanding of product development, that it is about reducing disciplinary complexity (only for hardware), as well as transdisciplinary complexity/complicatedness, before we can understand how to implement the virtual structures and a portfolio, as well. And without fully understanding the systemic organizational systems design solution of the line hierarchy, the virtual structures, the systems design and the portfolio, it is too impossible to fully implement a flourishing method. But, with common sense and the experience of past problems, mistakes and failures in antisystemic methods and frameworks, and with the help of the organizational principles, we can be guided by our experts. This means that the implementation of TDSD in order to get cohesiveness between the parts of the organization, is straightforward, where the prescriptive HOW in SOSD helps a lot. But, as SOSD, TDSD is also to some degree prescriptive, but only to the level that the organizational principles are followed. This gives the organization and its people the best prerequisites and understanding on how to guide themselves in the implementation/change/transformation, since they best know their domain and their own domain skills, knowledge and experiences.

By going through the diagram in the end of the blog post about SOSD, a diagram that starts at the top with the domain and purpose, we will come all the way to the appropriate method for the actual product development, i.e., go into the domain, and use TDSD in our case. As we can understand, the experience for mankind, that in many cases has turned into common sense, makes it much easier for us to also make a good implementation of TDSD already from start as already stated, where the one-way track in the diagram, gives a sunny start on our Walk in the Park. And where, as stated earlier, the problems of antisystemic methods and frameworks gives tremendous valuable information, so we are sure how shall not do that. The reason is that their failures are showing anti-Best Practice, that is failing due to their inability to fulfil the organizational principles, for example problems when scaling agile, which also is exemplified in the SOSD blog post. From this we can understand that our chosen method TDSD, with a systems design fulfilling all the organizational principles, is the only way forward.

TDSD is a method – cherry-picking from a framework is wrong per se
First of all, it needs to be stated that TDSD is a method and not a framework, and here below the reasons are presented.

The first reason for this is that it is not claiming one-size-fits all, the focus is on big software systems, even if it with small changes for example can be used for hardware. For different hardware development and for small software systems, there are already methods, even though TDSD also successfully can be used for all software systems, also small ones, especially when security, safety is needed or/and when the life-cycle is long. Many frameworks on the other hand claims context independency, and many times in combination with a collection of Best Practice.  But, the more information the better, is necessarily not the best concept, since excessive information is hard to navigate. Also, the constant updates on the home pages, makes us want to raise the question if we yet can find the Best Practice information we need for our domain and situation.

The second reason for TDSD being a method, is that it, for any organizational design (not only one including product development), no matter context and therefore also no matter domain, is impossible to just cherry-pick processes, sub-methods, tools and artifacts, even if we state that we have them all in our framework. So, apart from the excessive information and the updates mentioned above, the really important consideration is that it is never stated what parts in the framework that will become a unified and well-functioning way of working when put together.

The third reason, with another twist than the second one, is that in order to follow the science, the organizational principles, not all combinations of different solutions, when cherry-picking processes, sub-methods, tools and artifacts, would be satisfactory, and most of them probably mal-functioning solutions.

But most of all, is the fourth and final reason, that TDSD is not a framework depending on the need of a systems design when making any way of working in order to reduce the inherit transdisciplinary complicatedness. Systems design always has to be done top-down, which was stated in the method SOSD, taking care of the total way of working for an organization, starting with its common purpose. SOSD is the foundation not only for TDSD, but also hardware product development methods like waterfall and Lean Product Development, that already have similar foundation with virtual delivery structures, systems design and a portfolio.

The conclusion of all this, means that we can never make our own framework for product development, which we could see in SOSD. Just having a particular set of rules, ideas and beliefs, home-made or collected, and to that add all famous small methods available and try to aggregate everything, no matter combination, is wrong per se. The reason is that we need to state validity (the need of fulfilling the organizational principles) for any combination (systems design) of these small methods, and we could see in SOSD that there was really only one overviewing solution that follows the organizational principles. But unfortunately, this is not done (since we already have frameworks), and the frameworks instead let their users cherry-pick small methods and other needs from a smorgasbord, what the users with the help from framework coaches, think is valid for their organizational way of working at scale. And if the organization fails, the answer from framework coaches will only be that they picked the wrong methods, or implemented them wrong, or the latest “it will take 5-10 years to get it right”, leaving the organization with the failure. The cherry-picking-from-a-framework approach is invalid for any systems design of a product (cherry-picking components) or a way of working (cherry-picking sub-methods). Instead, we need well-informed hypotheses, that during some iterations are integrated, verified and validated, to understand the validity of the way of working, i.e., that we have fulfilled the organizational principles.

Why TDSD? Do we not have too many software methods already?
The aim of TDSD, is to focus on fulfilling the science of the organizational principles, since there are too many methods and frameworks with too low level of fulfilment of them, generating an immense number of problems in the organization. And as described in the previous chapter, cherry-picking methods from a framework, trying to aggregate to a total way of working for product development, is wrong per se.  By only focusing on the fulfilment of the organizational principles, TDSD is giving the people in the organization the right prerequisites for achieving a flourishing way of working. In this way, TDSD is also not more prescriptive than necessary, leaving slack to the organization depending on its domain, skills, competences, capabilities, experiences, people, tools, etc. The lack of appropriate methods or frameworks for software development of big software systems for the complete software life cycle, generating a tremendous number of built-in problems for today’s organizations, is simply the reason for why TDSD has been developed. TDSD is completely derived from fundamental science, the organizational principles of people and activities. This is the same way as we need to follow Archimedes’ principle during the product development of our new boat, if we want it to survive the launching, and some more science if it is going to survive the first storm. But TDSD is also suitably backed-up of the available evidence-based problems from methods and frameworks that are antisystemic. When built on science, showing this anti-systemicness of other methods and frameworks, is of course not needed, but it gives us the experience of what is not an appropriate solution, a kind of anti-hypothesis, i.e., what solutions we should not bother. This gives hopefully more confidence for you as a reader, in your struggle to change your though patterns. By finding the root causes to these built-in problems, due to the anti-systemicness of a method or framework, gives invaluable experience on not only how to not do the solution. By deducing, from all these experience from the past, TDSD will have solutions from different old methods, and can therefore definitely be said to be a hybrid. In short it can be said that from waterfall way of working with projects, the very important top-down systems design and the need of long-term planning of the product (and organization) is taken, from Lean Product Development, the iterative thinking with fast feedback loops by exploration of the systems design is taken, and finally from Agile, the iterative thinking with fast feedback loops to understand what the customer really need, to reduce uncertainty, is taken.

The team of teams-level (operational)
If we continue on the economic aspects on initiatives, for example to understand the total cost of an initiative, or to be able to make economical follow-up on an initiative, the following thinking is suitable in order to have a proper solution, and easy as well. When estimating the cost of the initiative, the work needed by the wholeness level and the number of implementing teams, or team of teams on the team of teams’ level, is calculated. In that way we know the resources needed from both the wholeness level (by percentage of the total cost for the wholeness level over time) and the team of teams’ level (we simply count the cost for a team over time), and we understand from the current prioritization on the portfolio level, when we can start the initiative. Since the resources already are in the virtual structure of the wholeness level, the managers do not need to follow-up their resources exactly, the prioritization on the portfolio decides where the resources need to work if there are delays. We will also decrease the pressure on the resources on the wholeness level to fulfil percentage on different initiatives, which is a trap from waterfall with projects, that we need to avoid to not have sub-optimization. Of course, it is still possible to follow-up on individual level, but due to the risk of also other sub-optimization, that is not to recommend, since it will hamper whole teams, broad competences (due competence is outside manager’s area of responsibility) and full-time resources. This hampering will then only lead to many part-time resources in too many initiative, leading to Project Administration waste, which is common in waterfall with projects implementations, i.e., due to KPIs on the managers to have 100% utilization of the employees, see this blog post for a deep-dive.

Another important matter, for example for governments, is to be able to differ new or further development from maintenance, many times called grow and run. For making a difference, the run has percentage numbers agreed with the portfolio. The percentage numbers will of course be a small part for the wholeness level supporting the teams, since the systems design is already set, but of course will increase when we are having a crappy systemization. For the teams the percentage number is higher since they will fix bugs and refactor the whole code, which includes of course also decided changes in the systems design, agreed with the wholeness level. The prioritization done in the portfolio level also gives the direction when deciding which is most important, so that maintenance for a higher prioritized product/initiative is done before new functionality in a lower prioritized product/initiative.