(or Feeling a Bit Sheepish)
"Software products are nothing like sheep - they're not soft and cuddly" (Pip's Mother)
It is my contention (and following discussions on the subject with my colleagues I use that word advisedly) that the software development process can be compared to sheep farming. You may find this a little hard to swallow at first (unlike the software as pasta metaphor so ably expounded by Pete Goodliffe [ Goodliffe ], which is much more edible), but bear with me and I shall explain all.
My mother's initial reaction (see above) was typical, but the more I thought about it and the more I talked the idea through with others, the more I came to the conclusion that all development processes and the products that emerge from those processes have features in common. I started this exercise more as a joke than anything serious, after all, software products really aren't anything like sheep. However, I've found that many of the techniques, methods, and even truisms that are applicable to software development have very strong parallels in sheep farming - a development process that's been refined over millennia.
Setting the Scene
Last summer I had a job interview. The position I'd applied for was effectively an internal promotion for me, although there were a few external candidates as well. The second interview involved the usual HR-generated "criteria-based" questions, whatever that means, but an equally important part of the interview was for me to give a 30-minute presentation on what I considered the best approach to managing software development within my department. Fair enough, I was applying for the Software Development Manager job.
Now, I've pretty much got the hang of answering interview questions, I can do aptitude tests standing on my head, I always seem to confuse HR people who try to analyse my personality tests (this article may go some way to explain that), and I've even had a job offer resulting from a horrendous interview where they stood me in front of a whiteboard and made me write and explain a C++ program to some highly intelligent nerds. I've also done lots of presentations, ranging from giving papers at conferences to demonstrating prototype software to clients. But for a couple of reasons this particular presentation wasn't anywhere near as straightforward as it should have been. Firstly, I didn't quite know how to pitch it - should I go for "blue sky" and assume no knowledge of the workings of our department, or should I address the problems and issues that I knew (and they knew I knew) existed? Secondly, the interview was two days before the final copy deadline for the ACCU journals (I was the production editor at the time), so my preparation time was drastically reduced, and I didn't even get time to sit down and think about it in any depth until the day before.
So, perhaps understandably, I was a little worried about this presentation. I'd managed to jot down a few ideas in spare moments, when I wasn't busy eating or sleeping or breathing, but with two days to go before the interview I still didn't have much idea of what on earth I'd talk about. I was in a dither as to whether to use my employer's standard Powerpoint template, or whether to even mention that I actually already work with the team I'd hopefully be managing. After a long evening of typesetting Overload I fell into bed and lay awake for a while trying (unsuccessfully) to marshal my ideas into something resembling a coherent framework. And so to sleep. An hour or so later I woke up with a brilliant idea. You know how it is, you think about something, and think, and think, and get absolutely nowhere, and then you wake up in the middle of the night with the Ultimate Solution. Then you go back to sleep again and forget all about it. I was determined that this wasn't going to happen this time, so I lay there in the dark for an hour or two and got it all straight in my head.
What was this idea? Why, that software products are like sheep, and the software development process is just like sheep farming. I have no idea where this came from. Possibly it was a subconscious association with my boss's surname (same as the old bloke who has a farm and sings E-I-E-I-O), or it may be related to my obsession with Lundy (a large lump of granite in the Bristol Channel that's liberally scattered with various kinds of sheep), but it's more likely something to do with my mother, who has a flock of Grey-Faced Dartmoor - a rare and shaggy breed of sheep. Blame the parents, at least that's what I always tell my kids.
Right, enough bleating. It's time to see if I can convince you that software products are indeed soft and cuddly, just like sheep.
Breeds of Software Products
One of the distinguishing features of software products is that they don't actually have many distinguishing features - there are big ones and little ones, simple ones and complex ones, huge monolithic desktop products and straggly internet applications. This is in fact not so different from sheep, of which there are many breeds, each with their own personality. Some have horns, some don't. Some have short wiry hair, some have long shaggy wool. Some are large, some small, some white and fluffy, some brown and scruffy. Some bounce around and play, others simply sit and chew the cud. And as any shepherd will tell you, each individual sheep has its own foibles and eccentricities.
The Ram
Rams don't actually do very much. They're usually bigger than the ewes, and assuming they get past the stage of being lambs (when most rams are slaughtered for meat) their one purpose in life is to father a new generation - for three or four weeks in the autumn they have a great time. So they can be compared to a typical large function-rich desktop software application, whose main purpose is to process and pass on data to other applications and to users. You don't have to do very much with them from year to year, just make sure they're fit for their information-providing purpose.
On a fairly regular basis rams are "rented out" to service other flocks, ensuring widening of the gene pool. This may involve some logistics, such as timing and transport, but in general it's a fairly straightforward process. Similarly, your large data-providing application may be used to process data from other sources, or provide data for a purpose outside its original business specification. This may involve some additional plugins or external data processing to deal with unfamiliar formats, but this tends to be standard operational procedure and as such is relatively trivial.
Incidentally, did you know that when a ram is put with a flock of ewes it has a sachet of dye tied to its midriff so the shepherds can tell which ewes have been serviced by the bright splodges on their backs? This may be stretching the metaphor slightly, but I guess this could be considered similar to data copyright notices that have to be included in reports that are produced using the data.
What can we learn?
Rams are for life, not just for Christmas. Once they're fully grown, they need to be healthy enough to do their job and pull in the stud fees, year in year out, without too much looking after. So it pays to put the effort in to get them right in the first place.
The Old Ewe
This old lady is quite similar to the ram, in that all she really does is to produce one or two healthy lambs every year, thus making a valuable contribution to the productivity of the flock. Again, she's like one of those big desktop products we all know and love, bringing in a steady income without too much attention. She'll probably need more looking after than the ram, but in return she'll provide spin-off products that will in turn go on to contribute to the farmer's income.
What can we learn?
Pretty much the same as the ram. Put the time and resources in up front, and your flock will increase in size without much further input from you.
The Prize Sheep
Farms are quite similar to IT companies in that they need to have a good name among other farmers and farm product consumers. There are a few ways in which a flock can make a name for itself. Lundy has a very successful business selling lamb via the internet - I think this has probably taken off because people who visit Lundy try the lamb while they're on the island and know it's extremely high quality (which, I guess, is another valid comparison - if you see how well a software product performs by trying an evaluation copy you're more likely to buy that product or others from the same company).
Anyway, back to the point. If one or two lambs show all the required qualities of their particular breed they may well be kept for entry into agricultural shows - the ram escapes the chop (and if he's lucky he might be kept for breeding purposes) and the ewe doesn't necessarily have to produce healthy lambs year in year out to be assured of a long and happy life. Rosettes mean the flock is marked as worth considering as a source for butchers or other flocks. How many IT companies can you think of that have based their reputation and their success on one or two flagship products?
What can we learn?
Appearances matter too. Sometimes it's not important for a sheep to bring in much actual revenue - if it is covered in rosettes it will draw attention to the quality of the rest of your flock.
Wild Sheep
There are many flocks of wild sheep in the UK and probably worldwide. These are generally left to look after themselves, possibly with a little judicious tinkering where required. They tend to be hard to control as they're not used to interference by humans or collies, and they often go off in unexpected directions. This is not necessarily a bad thing, as they're hardy and tough and can deal with whatever nature throws at them. Lambing just happens, no need for a shepherd to be up at all hours ready to lend a hand or bottle-feed rejected lambs.
Open source software seems to run along these lines. Applications are developed with an initial idea in mind, and assuming they survive the birthing process and don't fall off a cliff they take on a life of their own, being added to and modified to deal with slightly different problem domains. I guess you could argue that this is due to a high level of external influence, but I imagine that to the original programmer it would seem that their lamb has a life of its own and has taken control of its own destiny.
What can we learn?
Don't ignore a wild sheep because it has no pedigree - it may have evolved attributes that are needed to deal with environments common to all sheep (or a significant subset).
Rare Breeds
The sheep farming community has its equivalent of nerds - those (like my mother) who keep flocks of rare breeds and are actively involved in ensuring these endangered species don't die out. Rare breed shepherds are often also very active in keeping knowledge and use of "outdated" breeding and farming techniques alive along with the sheep. In general these breeds and techniques were once commonplace, but have been superceded by more commercially viable breeds and farming methods.
Rare breeders can be compared to programmers who write ZX Spectrum emulators for XBoxes, or Amstrad PCW emulators for PCs. Often the reason for writing these emulators is to play fondly-remembered games even after the original hardware has fallen to pieces, but a more respectable excuse is to keep old tools and techniques alive just in case they should prove useful in the future. My mother claims this holds true for rare breeds as well - keep the good old standbys around just in case commercial breeding programmes could find a use for the particular traits embodied in breeds that have been around for generations. I'm prepared to give her the benefit of the doubt on this one, especially as I still enjoy playing Hungry Horace on my Psion.
What can we learn?
Don't write off "old" breeds or farming methods, they are tried and tested and can even be better in some situations than their modern equivalents.
Lambs
Lambs are fun. Anyone who tries to tell my mother they're not gets a Hard Stare, and I'm sure many of you secretly number gambolling lambs among the first signs of spring. Newly-born software products are also fun - ask any programmer. You get to try out new ideas, see how high you can make the product jump, see how well it plays with other products, watch it grow - and don't they grow quickly? Coding a new product is like bottlefeeding a lamb - you hang on as tight as you can to the bottle while it gulps the milk down, little tail waggling away madly, then when the bottle's empty, off it goes to skip around and see what mischief it can get into next.
Of course, there is the mortality rate to consider. Mum's flock usually produces around 30 lambs every year, and between one and four of those lambs will not survive. There are those who are stillborn (the sales manager's bright idea for a program that he knows he can sell, which is technically impossible to put into practice), those who die soon after birth (write a prototype, discover the client didn't want it after all, or it's impossible to meet the business requirements), and those who require so much effort and money to keep going that the decision has to be made to let them die. There's a useful shepherd's adage that applies equally to young software products - if a lamb's no good, hit it on the head.
What can we learn?
Let your shepherds and sheepdogs play with the lambs if they want (within reason), it keeps them happy and productive. And be realistic - not all lambs will survive to become useful members of your flock.
Tending the Flock
Right, hopefully by this point I've persuaded you that the metaphor is actually worth exploring. Hey, I've even almost persuaded myself! Anyway, the above is an expansion of the ideas that came to me in the middle of that fateful night. I didn't even start to consider the peripheral issues until later. OK, so sheep are intrinsically like software products. What about their respective development processes?
Stockmen
In general, a flock of sheep is managed by one shepherd on behalf of the farmer. There may be other humans who help manage the flock on a regular basis - farm hands, additional shepherds, the farmer's family. There will certainly be specialists who are called in occasionally (either regularly or in response to specific situations) - vets, shearers, seasonal workers. Many shepherds also keep dogs to help manage the flock - my mother usually enlists the help of my sister, or me and my kids if we're around - which is more or less the same thing. In addition to the farmer, there may be other humans who provide overall direction to the shepherd. There may be a farm shop, whose manager might ask for more lamb, more mutton, more wool, or even more ewe's milk, depending on customer requirements.
There are many (possibly unflattering) parallels to be drawn with software development here. If sheep are the software products, the farmer is the board of directors or CEO of the company, the head shepherd is the software development manager, and the other farm employees (human and canine) are the programmers. Seasonal workers such as shearers would be contractors, and vets and other experts are consultants. The chap who runs the farm shop may be an account manager, product manager, or possibly the marketing manager.
Interestingly, shepherds are as reluctant to spend money on outside experts as software development managers. If there's a problem, every possible home remedy (often passed down from father/mother to daughter/son through the ages) will be tried first before calling in the vet. This is usually OK - these folklore cures have been tried and tested for centuries after all - but it can be counterproductive in the extreme if modern veterinary science has come up with a cheap simple solution to the problem. The mark of a good shepherd is one who keeps up with the times and knows when to call in the experts - qualities also required for managing software development.
What can we learn?
Shepherds do not work alone. Being an expert at dealing with sheep is not enough, a shepherd has to be able to communicate with other humans and canines to get the job done. S/he also has to be able to take in and effectively process and use information relevant to his/her work, adapting to specific and general changes in situations.
Breeding a Flock
There are many ways to develop a flock of sheep. You start with some ewes - possibly just one or two, maybe a whole existing flock, but more likely somewhere in between. You might buy two or three year old ewes that are already proven to be good breeders, or you might buy ewe lambs, who may or may not produce lambs in the years to come. Over the next few years you need to make decisions about which ram (or rams) to use to service your ewes, which lambs to keep and which to sell, whether to buy in new ewes to widen the gene pool, whether to "retire" old ewes that don't produce viable offspring. With the strides made in cloning technology it may even be possible at some point in the future to make copies of your best sheep - although there are currently problems with this process, I believe Dolly the Sheep suffered terribly from arthritis and died at an early age. All the time you have to balance the immediate profitability of your flock against investments for future prosperity - you may know that in five years' time you'll have the best flock in the county if you pay a fortune to have Billy the SuperRam service your ewes this autumn, but that's no use to you if you then can't afford to feed your sheep in the spring.
Each of these options has advantages and disadvantages, which I won't go into here as they're not particularly relevant. What is interesting is that there are also several almost directly comparable ways to develop a portfolio of software products. I'd guess most software development organisations start with one or two ideas that are developed into products. These will inevitably generate offspring - related products which are either add-ons to the original software or modified versions for different markets, or new products for the existing market. As the company expands, it may buy in and take over development of other software products (with all the pain that can entail - no amount of due diligence will expose all the potential time-bombs in someone else's code), and it will certainly produce newer and better versions of much of its existing product set and continue to develop new products. Software that is no longer profitable for whatever reason will be deprecated and eventually no longer supported. As with sheep farming, the bottom line is all-important. Developing a product portfolio is expensive, there must be money coming in from existing products to support the resources required to develop new products.
What can we learn?
Planning is everything. Having a long-term strategy for the flock is essential, but the farmer must also know how he's going to cover the day-to-day expenses while putting that strategy into practice (and if necessary the long-term strategy must be modified if it is not supportable).
Growth and Development
Sheep need grass. They don't actually need too much else - if you lived on a desert island with a flock of sheep and a field of grass you'd probably be OK. Sure, you'd be better off if you had a trusty collie, or access to a vet and modern medications, or bedding and pens, but your flock would survive as long as it could graze. In the real world, if the grass is poor quality you'll need to give your flock supplements - typically salt licks and various trace elements in some form. You may also need to provide more bulk in the form of hay. Conversely if the grass is too rich you get interesting things happening at the other end (the technical term is "daggy sheep") and you'll need to clean their bottoms regularly to avoid all sorts of nasty possibilities.
Software products need a programming language, and not much else. The equivalent of the field is the compiler/linker or interpreter for that language, and the hardware/operating system to run it on. If it's a functionally sparse language you're likely to need to implement elements in other languages or call other services - I vaguely remember that if you were going to do anything worth doing with ZX Spectrum Basic you'd have to write chunks of assembler code. On the other hand, if it's an all-singing all-dancing language like C++, your expert developers will be able to produce fantastically elegant, fast and highly functional programs... but when another (usually junior) developer comes to de-dag the bottom end (in order to fix a bug or extend the software) they've got an awful lot of untangling to do to work out what's going on.
There are many other minor not-quite-requirements that a sheep farmer has to take into account: transporting sheep to market or slaughter (packaging up software for distribution), tagging and numbering sheep so they can be identified (setting up project ID codes and directory structures), obtaining and paying for the services of experts, for example vets and shearers (employing consultants and contractors), providing and maintaining bedding, fences, gates, pens as required (buying and maintaining hardware resources), meeting health and safety regulations for meat produce or meeting ideal standards for shows (QA to ensure software meets user requirements) [ 1 ] .
What can we learn?
Decisions taken about the basic requirements for the development of your flock will have consequences and potential hidden costs - be aware of these as far as possible and make informed decisions. Also, it's not enough just to breed the perfect sheep, you need to manage its growth and keep it healthy if you intend to make any profit from it.
Bugs
Sheep have bugs. Lots of them. A major part of a shepherd's job is to keep an eye on his/her flock for any signs of disease, and to administer prophylactic treatments regularly to stave off the possibility of infestations or infections. Preventative measures such as spraying the entire flock once or twice a year with insecticide and regularly dosing for worms are taken as a matter of course, although they're not much fun to do. It is second nature for a good shepherd to spot symptoms of acute infections early on so that they can be treated before permanent damage is done to the flock (although see comments on calling in the vet in the Stockmen section above). Shepherds are also very conscious of what's going on with the neighbours - infestations such as lice or viruses can spread very easily between flocks in adjacent fields, and if allowed to proliferate can be potentially devastating - look at the short and long term effects of the last foot and mouth outbreak.
As software developers we too have to deal with bugs. (Thankfully this task isn't quite as offensive or messy as the shepherd's version though.) We all write and run unit tests as a matter of course - don't we? - so we spot and eliminate potentially nasty infections before they can do much damage. We have QA processes and (usually) a dedicated testing team to run detailed and exhaustive checks for bugs, and when they find a bug we deal with it according to its severity and impact. We also install firewalls and virus-checkers on our computers to ensure our virtual neighbours can't pass any nasty surprises on to us, and a good software developer will consider security issues during development of any internet product.
What can we learn?
In general bugs are fairly apparent in both sheep and software products, and must be dealt with as a matter of urgency. It is important to spot bugs as early as possible in the development process - it's far cheaper to treat a single sheep for a maggot infestation than to treat a whole flock.
General Cussedness
Sheep are stupid. If one sheep gets it into its head to run in the wrong direction you can guarantee that most of the rest of them will decide it's a good idea and follow it. And there's a good reason why one of the standard tests at sheepdog trials is to split off a given number of sheep from the flock - sheep are stupid. They can also be quite vicious - you might introduce a new ewe to your flock, only to find one of your existing ewes takes a dislike to it and turns on it. I've even seen a ewe push her own lamb into a crevasse to get rid of it.
Software can be stupid and vicious too (well, sort of). If your team has always developed software one way, it is often very difficult to effectively steer the products in a different direction. One example that springs to mind is the development process itself - if you've always used the waterfall method it's remarkably difficult to introduce elements of agile development. Or you might find that one product is developed using extreme programming, and everyone likes it so much that they race off and start using XP for everything, whether or not it's appropriate. Another example might be the technologies used - we've developed this product using ASP and JavaScript, so we've got to develop all our internet products that way, whether or not ASP.NET and web services makes more sense. As for vicious software, I can think of countless examples of projects where it's seemed sensible to save some time by integrating a third party component - it never seems to quite work out that way.
What can we learn?
Don't let your sheep (or your stockmen or collies) be stupid - make decisions on a case by case basis. "We've always done it that way," or "They did it that way with that sheep and it worked brilliantly," are not in themselves good reasons for making decisions. And be very wary of new sheep - they may be incompatible with your existing flock.
Conclusion
Well, what can one possibly conclude from these slightly insane ramblings? Hopefully not that I shouldn't have got the job - I did forbear from expanding on the sheepy metaphor in my presentation, although I couldn't resist mentioning it - my boss was quite keen to hear about it, so I guess I'll have to show him this article and hope he doesn't demote me (or have me sectioned) on the spot.
I could get into a bit of theology here - for example, Christians use the sheep idea a lot. But I don't think I will, there are enough holy wars about code layout and other such important issues in the programming community as it is.
My preferred conclusion is that a shepherd's crook, a couple of well-trained collies, and a quad bike should be required equipment for a software development manager, and should be provided by every enlightened employer.
I guess if I were to be serious for a moment (it does happen occasionally), I'd propose that it's possible to find useful ideas and process elements and rules of thumb in the strangest of places. People have been farming and breeding sheep for thousands of years - the development process that produces sheep that are "fit for purpose" has been refined and improved upon for generations, and although software products really aren't much like sheep the processes of developing those products have sometimes surprising parallels which we can all learn from.
My favourite example has to be, "If a lamb's no good, hit it on the head." A shepherd will do this without a second thought. How many times have you been involved in trying to rescue an infant software product that is obviously too feeble to survive, and wondered why you're bothering?
Acknowledgments
Thanks to my mother for many entertaining and informative conversations about the ins and outs of sheep farming, to several of my colleagues for joining me in thinking out of the sheep pen, and to Mum, Phil Bass and Alan Griffiths for reviewing the initial draft.
Any errors in my representation of the ins and outs of sheep farming (and software development, for that matter) are entirely my own.
References
[Goodliffe] Pete Goodliffe, "Professionalism in Programming #21 - Software Architecture", C Vu 15.4 , August 2003
[ 1 ] Incidentally, a general rule of thumb for the "perfect sheep" is it should be like a table - a flat top with a leg at each corner. If only user requirements for software products were so simple!
