Performitis

Patterns try to solve problems. Klaus Marquardt looks at one from a medical perspective.

Patterns for software systems are a success story. They provide insights that have been gained over years and decades, and convey them in a conveniently accessible way. Programmers and software architects can use them as instant solutions.

A communication view of patterns

Patterns are not just solutions; they are a key mechanism in passing on the software discipline's heritage. Looking at how knowledge and experience is passed on elsewhere, we can see a spectrum: from highly formal languages in mathematics, through examples in arts and architecture, to some cultural and ethical insight hidden in stories and fairy tales. Patterns about software design and architecture are certainly more on the formal side of this spectrum.

Patterns embrace the value of examples to give credibility to their message. However, story telling is virtually irrelevant to patterns. Not surprisingly, it is not irrelevant to the software people themselves. When a group of software people gathers in a canteen or a bar, patterns play, if anything, a minor part in the conversation. Stories about projects are dominant, often told without references and from merely indirect experiences. Some resemble more the characteristics of urban legends rather than sound research. Still, we love to hear these stories and we learn from them.

So what is missing from patterns that can be found in our traditions and collective knowledge?

The project stories hardly mention fundamental technical problems - in contrast with patterns, which are usually about technical problems and their solutions. (While there are numerous exceptions to this, these happen to be less popular and well known.) This observation triggers two thoughts. First, if projects fail they appear to fail for non-technical reasons only. Second, a technical problem also has a project context. This context offers many non-technical reasons for the technical solution chosen by the developers.

Beyond design: diagnosis patterns

Team culture, organization structure, and software solutions mutually influence each other. Conway's Law [ Conway68 ] is probably the best known coupling force, but there are other less well known relationships, and our stories and anecdotes are attempts to bring some light to them.

Again, a look at other disciplines brings further insights. Medicine is as old as mankind. Medical doctors have learned that beyond injuries and diseases, their patients' sociological situation influences their health. Depending on the individual, scientific western, traditional eastern, or alternative medicine may be most effective. Pragmatists have come to accept that whatever cures is right. And even more pragmatic, a patient need not be cured to feel much better and gain perceived (and possibly objective) health.

We could probably make use of the approaches medical doctors use to help their patients. They have found ways to identify many influences on a disease, and in turn know many different treatments that may address symptoms, or causes, or just help for whatever reason.

This resembles an important aspect of what senior developers, architects, and consultants do. Facing an unfamiliar situation or a project in trouble, which is the rule rather than the exception, their task is problem solving rather than developing a master plan. They observe on many different levels, and they start acting before everything is known.

The idea behind the pattern form of diagnoses and therapies is that these diagnoses collect knowledge from the non-technical regime and combine therapy options from different angles. This is not a contradiction to design and architecture patterns, but a complement acknowledging the actual needs in project dynamics. Finally, diagnoses make use of stories from real or fictitious projects to trigger memories and thoughts.

The doctor will see you now

As an example, the rest of this article describes a disease found in some software systems. Performitis arises from an overly narrow focus on performance during development. While it appears a technical issue at first, closer examination shows that its foundations are people and process issues. Accordingly, Performitis should be treated by team and process therapies as well as technical, and a combination of both is typically most successful.

Medical Apporach

Once we start with some medical terminology, a whole bunch of concepts, vocabulary, and approaches comes to us. The good news is that these concepts have been in use for ages and can safely be considered more mature than patterns. The bad news is that they sound only vaguely familiar. How does a doctor work?

To get to a diagnosis, doctors start with an examination. They watch for symptoms in the broadest sense: physical findings, movements, behaviour, clothing, smell, speech. Unlike many engineers, doctors often start some treatment even without a clear diagnosis. In emergency situations it is essential to keep a patient alive regardless of his injuries. And if you happen to have a fever you'd appreciate paracetamol and a broad-band antibiotic before someone has identified the exact germ.

In case of doubt, and if the consequences would be significant, a differential diagnosis might be required with a more detailed examination. Finally, a number of treatment options are considered and a treatment scheme initiated. As the disease and the cure proceeds, the treatment is adjusted appropriately. Often a therapy is continued while the doctors "proactively wait" and observe.

Diagnosis: Performitis

Every part of the system is directly influenced by local performance tuning measures. There is either no global performance strategy, or it ignores other qualities of the system such as testability and maintainability.

Tuned Trabant 601, GDR, 19kW [ Trab03 ]

With Performitis developers use every opportunity to optimize their code, and are eager to discuss different performance measures with their peers. While each single measure may be perfectly justifiable, check whether you observe the majority of the strategies from the following list:

Usage of C or C++ and its language specific low level features.
Extensive and early usage of language specific constructs that favour local performance gains over other qualities like encapsulation, reusability or portability. Examples would be public attributes, or inlining compromising the design.
Avoidance of indirections, at the expense of tighter coupling, limited extensibility and increased effect of individual changes. Examples for indirection range from virtual functions to entire encapsulation layers.
Keeping control of implementation details by preferring hand written code over advanced language or library features. E.g., extensive usage of pointer arithmetic instead of advanced specific data types (C/C++).
Avoidance of code libraries that are not controlled within the organization, due to the expectation that its performance will be less optimal than when implemented for the specific project.
Responsibility of code modules is clustered according to execution threads rather than to consistent logical responsibilities.
Wide-spread usage of environment specific features, like direct calls to OS, DB triggers and stored procedures, linker directives, or scheduling priorities.
Application specific code contains knowledge of technical issues and their optimization. Examples would be knowledge of network package size and frequencies, or database structure and joins.

These techniques are applied by more than a small minority of the programmers involved, and all over the source code. Trying to improve qualities that have been neglected due to performance tuning would cause expensive changes to large portions of the code.

The most significant observation is that there is no overall evaluation or strategy in place that determines where and when which of the above measures is taken. All decisions for tuning measures are made individually on a local scale.

The project is staffed with experienced developers who are familiar with the domain and the task at hand. This is not the first project the key team members have worked on, and they have learned some important lessons in their previous projects. The key developers all agree that performance is the single property that is hardest to achieve, and that it could cause a project to fail even when it is presumably almost completed. They exhibit a strong desire to get the performance aspects right first, paying merely lip service to other quality aspects.

When new colleagues join the project, or some contractor gets insight into the system, discussions about the way to ensure performance and other intrinsic qualities will arise. These discussions tend to become emotional quickly because they are at the heart of the individual working style and value system - and they could potentially exhibit deficits in the high level systems design. Performitis is only resident in projects that are either closed against outside influences, or have developed mechanisms to terminate discussions that raise inconvenient questions. Look out for the closed project society (if you have the chance).

Performitis infected systems are unstable with respect to technology or requirement changes. Because the structure ignores logical separations, the changes have a large impact and are costly and timely. The effort related to such changes sometimes exceeds the effort required for initial feature development. Look out for fear of changes and explicitly scheduled, often postponed technology updates.

The negative effects of Performitis are also visible to upper management. For all but the most experienced teams, delivery dates can frequently be missed. Performance oriented development neglects qualities like testability, and creating tests is an expensive endeavour. The project typically provides little tests on code level and integrates late in the development cycle. The spread-out local tuning measures make it difficult to fix the problems that arise with testing and integration while preparing the delivery. It is ironic that the measures taken then may dramatically degrade performance. Look out for more than one seemingly surprising schedule slip immediately before releases, or for an extremely high effort in test and integration of limited functionality.

Not all of these symptoms are unique to Performitis, and some only become visible in late states of the disease. Early and sufficient signs are the combination of the spread of local optimizations, the lack of a global strategy, and the team attitude.

The pathogen is the limited experience of key developers. Humans tend to remember their failures better than their successes, and many developers have learned particular painful lessons. Mainly in distributed or embedded systems, projects can miserably fail due to performance problems - which none of the participants will ever forget. The limitation in their minds is like a Pavlovian Reflex that happens especially to developers with extensive experience in a particular domain only. They will not realise that different systems need to balance different qualities.

Symptoms checklist
A number of different practices are used to increase local performance at the expense of other qualities
These practices are not limited to a dedicated portion of the code, but spread all over the project
There is no strategy for which local tuning practices are applied where
Performance is considered by far the most significant system quality
There is no objective measure of what performance is required
The project closes itself against external influence
Changes occupy large parts of the schedule and are frequently postponed
Either a lot of effort is spent in tests and integration, or several milestone dates have not been met with very little warning time

Diagnosis: Performitis ¹ , also known as Performance Bloat.

Figure 1

Prescription

Now that we have a diagnosis, treatment can be applied. Next time we'll introduce some therapeutic patterns and possible application strategies.

Acknowledgements

The first and foremost thanks go to Jens Coldewey, the shepherd of the first version of this paper. The workshop participants at EuroPLoP 2003 gave valuable and encouraging feedback: Frank Buschmann, Arno Haase, Kevlin Henney, Wolfgang Herzner, Michael Kircher, Alan O'Callaghan, Kristian Sörensen, Markus Völter, Nicola Vota, and Tim Wellhausen.

Further thanks to many unnamed colleagues who sometimes unintentional contributed to this diagnosis, its therapies and the examples.

References

[ Conway68] Conway, M.E. 'How do Committees Invent', Datamation , 14 (5): 28-31.

[ Trab03] Picture available at http://www.tuning-scene-droyssig.de/601maik1.jpg

1. A term closer to the medical nomenclature would be 'performance related softwareosis'. Any chronic disease is called an ~osis. Furthermore, it is not the performance that is infected but the system itself. However, 'performitis' is a more popular name.