A Wider Vision of Software Development

In this concluding article, I will explore the idea of the ‘Organising Principle’ further, relating it to the patterns work of Christopher Alexander and, given the relevance of perceiving such principles to software development, I suggest how we can improve our perception of such principles.

Be aware that it is possible that trying to explain the idea of an Organising Principle more clearly is a fool’s errand since its reality cannot be fixed. As I mentioned in the previous article, If you fix it: You Haven’t Got It. It is a far more experiential (or phenomenological) concept.

However, Organising Principles have a number of characteristics. They:

I explicitly mention the separation of Whole and Parts here since it is a key aspect of any competent approach to software design. For the programmer this begs the question of how well he or she understands the problem in terms of its wholeness and yet also sees how the parts need to work therein.

Experienced people in a given domain will have a sense of the whole and yet will be able to identify the risk points in the parts. They can simultaneously see the whole picture and know the essence of what needs to happen in the parts. This is what you pay for when you employ an expert. For example, in the design example in the previous article, an expert will know to check your allocation strategies and if they see out of order allocations anywhere will be able to connect this to the failure of the system as a whole.

Alexander & patterns revisited

At this point it is useful to look at some of Christopher Alexander’s ideas [ Wikipedia-a ] about the perception of beauty that links to what I have been saying about the idea of Cognitive Feeling.

Alexander started with defining a Pattern Language [PatternLanguage] to help foster good architectural design – what he called Living Structure. This metamorphosed into his masterwork, The Nature of Order [NatureOfOrder] where he tried to get a better understanding of why we find certain structures beautiful.

In the Nature of Order , Volume 1 Chapter 5, he identified the following 15 properties of Living Structure:

• Levels of scale
• Strong centres
• Boundaries
• Alternating repetitions
• Positive space
• Good shape
• Local symmetries
• Deep interlock and ambiguity
• Contrast
• Gradients
• Roughness
• Echoes
• The void
• Simplicity and inner calm
• Not-separateness

If we look at this solely as a list of items, it can be difficult to understand how they may be useful in design, apart from using them as heuristic guidelines. Though useful, if we consider them in the light of the dynamic concept of the Organising Principle, they make a lot more sense.

A pointer to why this may be so is in Alexander’s use of the word: Living. Livingness implies ambiguity, and therefore these 15 properties can also be seen as Organising Principles. Thus when we try and fix them in order to come to a better understanding, we will only be seeing one way of looking at each principle and by definition will have come away from the actual thing itself.

But can we better relate such principles to software development?

Requirements implicitly contain Organising Principles

Embedded within a set of requirements is something that we need to find and embody in our implementation. As with high-level schematics, there is a high degree of ambiguity surrounding this ‘something’ and I can usually hand wave to my heart’s content about structure and architecture, but it also needs to be connected to the details of an actual implementation as I tried to do in the previous article.

Generally requirements will implicitly, not explicitly, embody dynamic specifications of what needs to happen. The conversations between users and developers, properly managed, are a key activity. Much of my job when I talk to any user is to try and understand:

As a developer, I need to understand what core dynamic principles are embedded in the requirements. By definition if you are doing a useful piece of software the user won’t know what they actually need . They will be able to talk about what they want , but that is most likely going to be based on extrapolations from their present experience. Any conversation between user and developer will generate new knowledge about future experience, so my job as a developer is to help us both map out this area of new knowledge and its attendant Organising Principles.

Architecture references the Organising Principle

If we take the point that an Organising Principle is not a fixed, discrete idea then we can see that it is much like comparing a film of an activity with its reality. The film is just a set of discrete frames, but the reality is continuous.

In the same way, the various possible architectures and implementations are different views of a specific Organising Principle. This is difficult to grasp and needs a far more mobile thinking than we normally use and is the core of why good software development is so difficult. Certainly design patterns have helped but we need to go further and understand that the development of personal perception skills is more useful than coming up with abstract lists. I believe this is why it can take 15 years to develop such competence.

Code is the precipitate of the Organising Principle

If we manage to perceive this dynamic principle, referencing it by designing a software architecture that we embody into a specific code implementation, we are embarking on a process of precipitation. If we don’t do this then – indeed – as the original title of my ACCU2016 talk suggested, we only have a ‘Hopeful Arrangement of Bytes’ without any coherent design.

Sometimes playing around with code from the ‘bottom up’ may be a valid thing to do as a piece of ‘software research’ just to see how the code works out and will be especially relevant if there is a blank sheet, or ‘green field’ site where you have to start somewhere. In this case you need to identify the highest risk requirements and just implement some proof of concept to check your understanding.

A point worth noting here is that it will not be ideal for a novice or journeyman programmer to follow the example of an experienced one. Be warned: It is possible that a master programmer can work at the keyboard and seemingly design as they go, creating from internalized mature ideas. Yet this is something that is definitely not recommended for inexperienced developers. They will need to spend much more time maturing and externalizing their ideas first before those ideas become part of their ‘cognitive muscle memory’ from which they can work direct to the keyboard. It is to be hoped that the experienced developer will realise earlier when they need to step away from the keyboard.

Perceiving Organising Principles

The above comments are all very well but just how can we develop such a ‘living and mobile’ thinking perception? Unfortunately as programmers we are at a disadvantage.

We work in a domain where much of our thinking needs to be fixed into a rule-based structure in order to imagine how a computer program will function. This can have unwanted side effects of making it difficult to think in a mobile, living way. Hence Ted Nelson’s [ Wikipedia-b ] rant in his book Geek’s Bearing Gifts about techies only being able to think in terms of hierarchies.

If we personally want to develop this other way of seeing, we need to engage in some activities that foster such a mode of cognition. Perceiving livingness, almost by definition, requires that we need to handle ambiguity. This is what is required when we are working in the ‘gap’, or whenever we are dealing with human situations. Logical thinking can cope with known and static issues, but as programmers we need to be very aware of the boundaries of our knowledge, more so than the lay person due to the inherent fixity of the domain of computer programming.

My thesis is that in order to develop a mobile dynamic cognition that can better perceive Organising Principles, we need to take up some artistic pursuit in a disciplined and self-aware way since it is the artistic process that can allow us to move outside the boundaries of what we already now.

Disciplined artistic process

In developing a balanced perception though an artistic approach, do whatever appeals to you. For me I find painting and dance work well. An example of how the artistic process parallels software development can be seen in an early experience I had with painting.

The following image is a watercolour painting of my daughter done at an early stage of my painting hobby (so please be gentle with any criticism!). As one of my first forays into the painting world and like the good novice artist I was, I decided to draw the picture first, using a photograph as a reference. It took me 4 hours!

The first effort took 2 hours. The next took 1 hour and the last two each took half an hour. I had intended the final result to be the basis for the final painting. But being the worried novice that I was, all too aware of my lack of experience, I decided to perform a ‘colour check’ painting freehand, away from the drawing, before doing the final version. To my complete surprise this became the final painting I have shown here. I found that afterwards when I tried to paint into the final drawing it did not have the same life as the freehand painting.

This is an example of the difference between the ‘master’ freehand approach as compared to the ‘journeyman’ drawn approach. Of course I do not consider myself to be a master painter, but this example illustrates the perceptual self-developmental dynamic inherent in the artistic process.

We can also see here the need to do the foundational, ‘analytic’ work, in this case the drawing; followed by the ‘gap’ of putting the drawing away and using the developed freehand skill to come up with the ‘solution idea’.

A final, and slightly frivolous, example of an artistic pursuit is that of the improvised dance form that is Argentine Tango, notable for the response it elicited from John Lakos in my original talk where he asked me to teach him some moves! I teach and dance Tango as a hobby and this particular dance form is strongly founded on being far more conscious about the primary human activity of walking, the signature movement in Tango. (For those interested, see my post on dance as True Movement. [ Tolman16 ])

Here there is a need for structure, and a mobile process of interpretation and improvisation, both founded on a disciplined form of the dance. It can take years to learn how to ‘walk’ again but if followed in a disciplined manner can lead to sublime experiences of artistic ‘Living Structure’ as the ‘team’ of two people dance from a common centre of balance.

The author giving John Lakos an impromptu Tango lesson at ACCU2016. Photograph courtesy of Mogens Hansen.

A wider vision

In conclusion, I hope I have conveyed the implicit yet substantial link between art and technology. My wish is that it will enable us to widen our vision of software development, and help us realise that we cannot separate it from perceptual development, with its attendant need for a balanced development of the Self.

References

[NatureOfOrder] https://www.natureoforder.com

[PatternLanguage] https://www.patternlanguage.com

[Tolman16] https://charlestolman.com/2016/07/18/tango-thoughts-true-movement/

[Wikipedia-a] https://en.wikipedia.org/wiki/Christopher_Alexander

[Wikipedia-b] https://en.wikipedia.org/wiki/Ted_Nelson