As stated here, every big organization needs a line hierarchy, no matter context or domain. But, for product development, which is in the Complicated and Complex domain of the Cynefin™ Framework by Dave Snowden, the line hierarchy by itself, is not enough. This depends most of all on the high probability for suboptimization due to the silo-KPIs on the managers on different levels in the line hierarchy, as well as the fact that no one having the overview, or the responsibility of the total delivery (every silo makes its respective sub-delivery). These are the major reasons for the need of virtual delivery structures, also cross-functional ones, in product development, and some of the reasons for the introduction of projects in the 1950s, which is further explored here.
Except for sub-optimization, other important reasons for having virtual delivery structures, is the need of decreasing the number of intermediates, in order to be fast, not to lose too much information during transmissions and reduce Project Administration waste. This means that the fewer levels we have within the virtual delivery structure, the better. Of course, not bigger than teams of teams (< 150 persons) that deliver something together is of course the best, which is a very flat structure of only two levels, where the line hierarchy can need four levels or more. This means that no matter the size of the line hierarchy, three intertwined levels for our virtual delivery structures are appropriate; Portfolio level, Wholeness level and Team of teams-level, where the Portfolio level is a virtual structure, but not for delivering products. The Portfolio level is strategical/tactical, the Wholeness level is tactical/operational and the Team of teams-level is operational. This also means a flexible and adaptable solution, that can solve many different kinds of initiatives regarding both size and complexity. The teams needed for these different levels are further handled in the TDSD method.
The bigger system, the more levels, but normally the wholeness level and a team of teams-level should be the absolute maximum for any initiative, to not go over the limits for the people structures mentioned above. The best is that an initiative never is bigger than a team of teams-structure of 150 persons, but which also depends on other things like dependencies between the teams, as well as the size of the teams. The wholeness level has as the name tell us, responsible for the whole, the systems design of the non-functional requirements, that can be security, safety, jurisdiction including GDPR, fault handling, that give us the architecture, and of course systems test of the whole. The virtual delivery structure of any initiative, always start at the wholeness level, which is responsible for delivering the initiatives. The absolutely best is that a team of teams-structure can be responsible for the delivery of an initiative, as a complete system by their own, where they together with support from the wholeness level, which is still overall responsible for the delivery of the system within the total system.
Working close to the virtual delivery structures will also be supporting teams, that due to their context has another team setup, compared to the agile teams within a team of teams-setup for software development. The important part here is not how the teams work, but what they deliver and when. For example, an “in-house network and infrastructural operation” silo mentioned above can have their own virtual structure with many teams that sometimes deliver themselves, but many times together. This silo can then be seen as wholeness level of their own, with their own Kanban board and their own way of working, where every team probably only has a team leader and not a Scrum Master and a Product owner. They can also sometimes have individuals integrated in the team of teams-structures, which totally depends on the need and not prescription. Preferably they have as much similarities as there are need for, which can be morning meetings, retrospectives, etc. The most important is that the manager is not also the team leader, all to avoid micro-managing and sub-optimization, which KPIs on the silos on different levels, always lead to.
The virtual delivery structure for an initiative, including the collaboration with the supporting teams, can of course look in many different ways, depending on the prerequisites, like domain knowledge of the organization, but always need to be properly constructed top-down, in order to be able to fulfil our organizational principles. Top-down means to reduce transdisciplinary complexity/ complicatedness and consider how the hypothesis for the sub-systems (components) also iteratively can be integrated, verified and validated to a cohesive and well-functioning whole, where a novel whole has the highest complexity. Top-down is needed not only to get a proper organizational systems design et al, but also in order to get the total team feeling for the whole initiative to be delivered by them together.
Worth adding is also that in product development, the line hierarchy not only consists of capabilities for taking care of a predefined WHAT, but it also needs to consist of capabilities for HOW, to be able to make the systems design (in order to reduce transdisciplinary complicatedness/ complexity). As a result, we also must add additional silos taking care of this HOW, not only regarding systems design, but also regarding systems test. This is the top level of the initiative, and of course requires the wholeness level team that takes care about the wholeness of the initiative. This wholeness level is most often missing when agile is scaled in a bottom-up transformation of big organizations, which also implies that if resources for HOW did exist in the old way of working, they will be removed before the organization even understands that they were still needed. The wholeness level can never be found inside the agile teams, more than in small organizations and small systems, since refactoring of code and architecture can be done gradually for small system, but not for big systems. We also need to be aware that the portfolio level requires support functions, that neither can be part of the teams.
When having virtual structures, these structures can be static or dynamic, which need to be considered, where normally hardware organizations have dynamic ones and software organizations tend to more and more go to the static ones, especially for agile software development. And as always there are pros and cons when having static or dynamic virtual structures. Dynamic virtual structures have built-in flexibility in setting up new initiatives, but will generate extra work in the portfolio to keep track of the initiatives and the money, and for management where we need to be vigilant with too much focus on resource allocation, which is sub-optimising per se. This is especially valid for big organizations doing waterfall development with waterfall for small initiatives, where the need of T-shaping is a necessity in order to be able to compete with smaller competitors. The administration waste, originating from having small percent of I-shaped employees participating in many different small projects is devastating for the efficiency and also to some degree for the effectiveness. This is also the reason for the difficulty to compete for big organizations making small projects (small systems), where an inefficiency of more than 50% is not unusual, see this blog post for examples and a deep-dive and into Project Administration waste and the theory behind.
Regarding the maintenance within an organization, the domains of software and hardware, normally have different virtual delivery structures for maintenance. For hardware organizations, the maintenance is normally a separate organization that the development organization hands over to directly, or after a year or two after the customer release in order to correct the first faults in the hardware. The main reason is to free development resources, for next complex initiative to handle, leaving the only complicated released hardware with documented information to maintenance. With a high level of requirements traceability including baselines, the development can continue with next hardware variant any time, since it is separated from the current hardware release. Also, the customer connections are important, and they are also separate due to that every hardware shipped to the customer is unique, also combined with many other unique hardware at the different customers. For software organizations, we have normally only one software for multiple customers, and with some variants due to the continuation of next release, having development and maintenance in the same virtual structure, is the most suitable. The drawback with this setup is the risk of losing control over the maintenance cost, due that the stakeholders believe that they can choose more development and not listen to the teams. But high maintenance cost has always a tight connection to bad quality. This in combination with a superstition on refactoring from the teams also for big software systems, gives high risk for running maintenance costs. By pushing the problems in front of us, makes us to become too reactive instead of proactive, giving us more and more problems. The root cause to this is most probably a bad or non-existing systems design from start of the initiative.
It can be added that for smaller companies, there is normally not yet a line hierarchy due to the smallness, and the company itself can be considered as one whole virtual cross-functional delivery structure. The employees in a small company will work with many different functions, giving the right I-competence, T-competence and cross-functional broad competence needed for developing the product(s). Not only the organization itself gives less complexity, but the small products(s) also give less complexity. This means that there is a possibility to have less of documented information, less structure and order from start of an initiative. All this together means that more information can be in the head of the employees and that the architecture and design may be emergent, even though a lot of new functions then can dramatically reduce the life span of the product(s).
The virtual delivery structure needs to be as flat as possible, in order to reduce intermediates and administrative staff, which means mainly to get rid of mirroring the line hierarchies when the organizations are big. Combined with the need of keeping track of the wholeness of our product/system, we need a wholeness level that is tactical/operational within the virtual delivery structure, keeping track of systems design, architecture, systems test etc. on the whole. Remember also that the work with the systems design and architecture to a great extent is domain independent and therefore very different from the work the teams do, that is almost totally domain dependent. In order to take care of virtual delivery structures of some thousand people, we have a team of teams-level as the only level to add, where of course a team as the wholeness level and only teams below as a virtual delivery structure, will be desirable. This structure is often enough, since it can consist of up to 150 people (Dunbar’s number) and therefore can deliver pretty big products/systems.
Since we always need both a line hierarchy and a virtual structure for our way of working in product development, this also mean that we can skip the discussions about “the bad” silos, or “is the silos good or bad?”. But more important is that we can understand that if do not set up these two people structure deliberate, especially the virtual structure that is there for permeability, flexibility and adaptability, the risk is that we will hamper the interactions due to unnecessary boundaries and restrictions. Since our people always want to solve their tasks and deliver, the risk is high for sub-optimizing solutions when these two structures are intertwined, leading to selfish constellations (silos) unfit for purpose, will pop-up. Another risk to mention here, is when we let our teams be more autonomous than they really can be, this indirectly means that we on purpose are making each team to kind of a silo. Because, since they cannot deliver anything themselves, they can be seen as a silo (if they are delivering a whole, they are not a silo per se in an organizational point of view).
To clarify in which contexts, we need virtual structures and why we need them there, some examples are presented.
We start with an example in production, that is clarifying why a virtual delivery structure is not needed. For a clear context, like production, every unit (process), no matter abstraction level, know exactly what to deliver. This means that every unit themself and its manager, can deliver their purpose (WHAT) and over time they can improve its HOW they manufacture the WHAT. The unit shall be seen as a team. which means that they also can be seen as a kind of virtual sub-delivery structure. With our principle of Respect of People, everybody in the team will grow in their learning by multi-I-shaping within the team, since they learn more and more of the steps in the process. The risk for suboptimization on the whole is low, since; the whole is already set, they do not compete with other teams to deliver more WHAT, and do not need to interact for a new solution with other teams. The total virtual structure can be seen as one sub-delivery team per group of people, and the result from each team are aggregated to the whole. The reason why this is possible is because the product development already has guaranteed that the parts are coherent and cohesive, when aggregated to the whole. The money needed for the organization, can also easily be retrieved and divided according to the line hierarchy and its units.
Having the manager responsible for also the team that is possible in production, will be counter-productive in product development, due to that there will be KPIs on the managers as well as sub-optimising measurements that will compare teams. We of course also have the increased risk of Respect for people problem when having different opinions within the team, when the manager and team leader are the same person. This risk is mostly due to the complexity in product development that the manager may not have any knowledge about at all. To even further increase Respect of People in production-like contexts that is not completely repetitive, the team leader and the manager shall be different persons.
With production example above in mind, we continue with an example from product development, which is a complex context. Many organizations nowadays that are implementing agile at scale, have problems doing a proper implementation of the virtual structure, since they are influenced too much with thinking, methods, techniques, etc. from production. This leads to that they get problems with the deliveries to the customers, where the root causes are that the organizational or the product systems design, or both, are not done in a top-down manner. But due to the influences from production, the belief is that these problems (symptoms) instead will be solved if the line hierarchy is changed to fit the different agile team structures, or changes of the line hierarchy, or the virtual structures or responsibilities of the managers, or in combination, i.e., to make the organization more production like. But it does not matter if the sub-optimizing solution is; the manager has the responsibility for one agile team and some other function (architecture, Scrum Master, Product Owner, etc.), or having the responsibility for three agile teams and no other function, etc., it is still sub-optimizing. This will in both cases instead lead to the following consequences; measurement of a part => comparing teams (indirectly manager) =>gaming (in order to look good) => master suppression techniques as the last station.
Sub-optimizing solutions will always be sub-optimizing and it can take long time before the new problems described are seen. For example, the decrease of specialist competence when not having guilds anymore, can take years to understand the negative effect of. In the case with putting the manager as responsible for much more than 15 persons, we dramatically increase the risk for burn-out as well. This is because we violate another organizational principle and then sadly only give ourselves another built-in root cause in our way of working, instead of removing the ones we already had. The fact is that symptoms are impossible to solve, which leads to a sub-optimization that in this case make us go backwards in our attempts to get rid of the sub-optimization. Unfortunately, this leads to that risk that the organization will make multiple reorganizations of the line hierarchy, since every reorganization does not work on the whole, due to it is only sub-optimising. And this leads to an enormous waste of time and energy, which that the employees get extremely tired on reorganizations. The root causes always need to be solved, where the solution always will be to introduce a virtual (cross-functional) delivery structure, if it is mal-functioning or missing, which means that we need to stop the “agile at scale” reorganizations, continually done in organizations today.
Another example that generates a big number of organizational problems, many of them unrecoverable, is when we have divided our organization into already predefined sub-delivery structures. This means that we are not able to develop our product with the needed top-down approach. These predefined organizational parts, for example value streams, do not give the needed flexibility to make something novel, a new product/system and definitely not a new platform. The reason is that functionality is divided into the predetermined value streams, before the new systems design is considered regarding the requirements, especially the non-functional ones, i.e., before we know what components we need in our systems design. The recognition of that value streams is an inflexible approach, will be painfully visible when the organization needs to do new customer solutions, that requires a completely new systems design, leading to a completely new architecture. But, since the functionality is divided down into the value streams as the first step, there is not even possible to re-evaluate the existing architecture. Due to the value stream thinking, the former positions for top-down thinking, systems design and wholeness have also since long been removed. This makes it even harder to fulfil new customer solutions, since it means that the skill, competence and experience has vanished as well. If there also are managers responsible for the teams within the value stream, this means a non-fruitful mix of the line hierarchy and the virtual delivery structure, making it even harder to fix the problem of this built-in inflexibility.
The examples above shows the necessity of having proper virtual delivery structures in all product development, not matter domain. The examples also shows that the necessity of the top-down approach is valid also for the virtual delivery structures, in the same way as for the line hierarchy.
The domain will, as stated earlier, always decide the shape of the virtual delivery structures, and here the organizational knowledge is the most important aspect, when setting up these structures. Changes in the domain, the competence profile of the organization, the purpose of the organization among others will also change the different virtual structures within the organization, and this flexibility is very important. For the set-up of the virtual delivery structure for software development, please see the method TDSD for software.
Next article in this series, is about the need of a portfolio in all product development.