Now it is time for the wrap-up of solving complex problems in software development, and it also summarises what has been brought up on this subject in earlier blog posts.
The first thing to realise is that it is sometimes difficult to understand that the goal is not clear. That means that there is no specification possible, no gap to fill, which means that we have left the ordered domains and instead entered the Complex domain as our start domain. And the longer time it takes for us to realise this, the harder we fall into the Chaotic domain. And if we still do not want to accept the fail and try to plaster our insufficient and incorrect solution, the longer we will stay in the liminal state between the Chaotic and the Complex domains, without gaining new knowledge about our solution.
And the ATO example in this series and our earlier discussion about Consumer Pull, is very similar. Both is about the Unknown Unknowns, a reality we know too little about, and therefore cannot handle. We need to learn more and innovate, with the feedback loops that many times are of different length and interwoven. The ATO team needed to know about the train, on site. The team need to sit with the customer and end user to understand their need. And in the Complex domain we cannot survive without safe-to-fail experimentation, which means multiple parallel experiments, or as in the ATO teams case, extremely short cycles of minutes to gather information, to gain knowledge. Firstly, failures will be the dominant part of the game, in order to gain knowledge, for the possibility of a future success. With a too controlled way of working, where there is no room (or understanding) of complexity, the intrinsic motivation will vanish, along with the innovations. As Dave Snowden puts it [1] [2]:
Intrinsic motivation is key to innovation and KPIs (on all the available scientific evidence), as well as having total transparency leaving no space for experiments (basic 101 cognitive neuro science fact), destroy it.
With intermediaries between the team and reality, the reality will also be dimmed, so it is a must to have as short interaction chains as possible, which is one of our principles. And in many cases Genchi Genbutsu*, with the meaning “go and see”, is the only feasible way of working, as for the ATO team, and definitely the fastest way to find a solution, and most probably also the only.
And the ATO team, a Lean start-up team, or the GUI programmer talking directly to the customer, are all using the same bottom line thinking; the need to sit at the customer, end user, or on site, where the reality is. And sometimes not a single intermediary is possible, since understanding is key and information not enough. For example, our young engineer did in fact not join the first travel to site, and only got reports from management, so he could just not grasp the situation, and really thought that it was just to change the regulator setting. Not until he frequently travelled to site himself, to deeply analyse the data and the corresponding train set behaviour, he slowly realised, got knowledge, understood and finally accepted, that the current solution would never do the work, and adaptive algorithms were born. But, management was reluctant to do too many changes, since they looked to much on the money spent, there KPIs and not the reality**.
In the picture above we have three loop backs of different kind, which all of them fills its purpose, when dealing with complexity:
- The yellow one is the heavy one that really throws us back to the drawing board, either from the ordered domains, or the first time from the Chaotic domain (maybe via the liminal state). In the case of the ATO, it represents the 4 weeks interval when the ATO team were home and after deep analysis are doing bigger changes
- The red loop back from Complicated to the Complex domain represents smaller changes of the parameter setting, or smaller changes in the software, that the ATO team could do in the evenings every day
- The black loops are the iterations needed with as fast feedback as possible to gain new knowledge from the current solution, and represents every ATO ride between two stations, i.e. five minutes iterations.
The above can be seen as a worst-case scenario, where the unknown unknowns were many, interwoven, and where also new ones were added during the ride, which was stated in the earlier blog posts.
The next “chapter” to read, according to the Reading proposal tree is The beauty of negating our principes to become our root causes blog post.
This was all for today, c u soon again.
*Genchi means place and Genbutsu means information. Genba walk is also a commonly used term, where genba means “where value-creating work actually occurs”.
**The management looking at the figures instead of the reality, made the ATO team very reactive instead of proactive in the beginning of travels to the customer, but changed over time, when management realised the huge problems. In the end the management was very happy, since the adaptive algorithm could do the job on any train set at any customer and was used for at least four different projects and train suppliers. And even the first project that started as a catastrophe was a success, with a very happy customer. The first Walk in the Dark, finally ended up in a flourishing park ;-).
References:
[1] Snowden, Dave. Blog post. Link copied 2018-12-14.
https://cognitive-edge.com/blog/knowledge-economy/
[2] Snowden, Dave. “From Agile to Agility”. Link copied 2019-08-04.
https://www.youtube.com/watch?v=xF6qH8PnxAc at 42:45 minutes.