User Stories and BDD – Part 1

This article is taken from the first in a series of blogs that take a look at user stories, what they’re used for, and how they interact with a BDD approach to software development.You could say that this is a story about user stories. And like every other story, it’s important to choose where to begin – because, contrary to the advice given in the Sound of Music, it’s not always a good idea to “start at the very beginning”.

Figure 1

What’s the problem?

Several years ago, I gave a talk at CukenFest called ‘Let Me Tell You A Story’ [Rose14]. It was inspired by a number of feature files that I’d seen online that started with text that looked remarkably like a user story (Figure 2).

Figure 2

This seemed very strange to me, because most features require the delivery of several user stories. So how could you choose just one one to put at the top of your feature file?

My colleague, Matt Wynne, believes the practice was encouraged by the original feature file snippet [Cucumber] that shipped with TextMate [TextMate] many years ago when Cucumber was first gaining popularity.

Similar shortcuts also ship with Sublime Text and Visual Studio Code – and probably many other editors and IDEs (Figure 3).

The feature snippet that ships with Sublime

Figure 3

The cargo cult effect [Wikipedia-1] has meant the majority of new Cucumber users who pick up these tools wrongly assume that putting their user story at the top of the feature file is the right thing to do.

The relationship between user stories and feature files is not well understood. This article is taken from a series of blogs aiming to put that right.

Software development before agile

Let’s take a journey back in time, to the 20th century. How did people develop software back in the dim, distant past, before the agile manifesto? [Agile] The usual answer that people give is: “they used to use waterfall”.

In the waterfall metaphor, software development is seen as a linear progression through various project stages: requirements analysis, architectural design, coding, testing, operations. The assumption is that work flows smoothly downhill – just like water. (See Figure 4.)

Image: [Royce]

Figure 4

The problem with the waterfall approach is that at each stage people would spend weeks (or months) preparing detailed documents before passing them on to the team responsible for the next stage. Building software the same way you’d build a bridge or a skyscraper. It could be years before the first line of code was written, by which time requirements and priorities had often changed.

Most of the requirements that were discussed in detail were not delivered in the way that they had originally been envisaged. In fact, many never got delivered at all.

This was hugely wasteful, both in terms of the cost of delay [Wikipedia-3], and the time spent analysing and specifying features that would never see the light of day.

The risks associated with waterfall development have been known about for a long time. Iterative and incremental approaches – that mitigate those risks – are not a recent development, though it’s only recently that they’ve become mainstream.

What are stories for?

Fast forward a few decades, and we find a handful of experienced programmers determined to tackle the fact that the waterfall approach led to significant waste.

These consultants were breaking the mould, coming up with their own practices and methodologies aimed at minimising the waste inherent in spending time doing detailed design work prematurely. Instead, they suggested working in an incremental and iterative fashion, deferring detailed design work until it was clear that the functionality was about to be delivered. Collectively, these became known as lightweight methodologies [Wikipedia-4].

One of these methodologies was called eXtreme Programming (XP) [XP].

The XP folk created the concept of a story as a “placeholder for a conversation” [Jeffries01]. This allowed them to do high-level decomposition of the work that needed to be done (stories), while accepting that more detailed analysis (conversations) would be needed before implementation started.

Another popular methodology at the time was Scrum (Figure 5). The Scrum Guide [Scrum] doesn’t mention stories, but includes a concept that embraces them, called the Product Backlog Item (PBI). Each PBI represents some functionality that can be prioritised by the Product Owner. Scrum is not prescriptive about what format a PBI should take, although they are often represented as stories.

Image: [Wikimedia]

Figure 5

A story is a way of remembering that a piece of work might need to be done, without committing to actually doing it, or diving into the details too soon.

Putting the ‘user’ into user story

Lightweight methodologies were a big hit, but there were so many of them that it was difficult for anyone to decide which one to use. This led in 2001 to a gathering of 17 men at a ski resort, which in turn led to the agile manifesto [Agile]. It’s been a very influential document, but many of the things that people associate with “agile” aren’t mentioned in it at all. On the subject of stories, the manifesto is entirely silent. However, one of the twelve principles speaks to the problem stories aim to solve:

Simplicity – the art of maximizing the amount of work not done – is essential.

The original description of the Planning Game in Extreme Programming Explained [Beck04] does mention stories – but the book doesn’t use the term user stories anywhere. So, where did the inclusion of the word user come from?

I asked on Twitter [Rose19], and it seems that no one really knows – at least no one who’s telling. Martijn Meijering dug into the internet archive and discovered that in a snapshot from 2002, story and user story were used interchangeably on the C2 wiki [C2] (which is where the XP community discussed things).

However it happened, calling them user stories emphasised that the goal is to “maximise the value of software produced by the team” [Beck04]. As stories began to be used more widely, teams needed a strong steer to ensure they prioritised work that would deliver value to the user, and the term user story did just that.

However, the use of the word user has led to many arguments. Should every story be from the perspective of a user? What users does the system have? Is the system administrator a user? Is the support team? Are the developers? What about another system consuming your API?

It’s called a user story to help people focus on maximising the value of what’s produced – but we must remember that value comes in many shapes and sizes.

A template is born

The challenges teams had writing user stories led to the creation of the Connextra template by Rachel Davies in the early 2000s. She found that it wasn’t enough to just write the story from the user’s perspective. Her teams seemed to have better conversations when the story contained three specific pieces of information:

1. The feature that needs to be discussed
2. The role that will get the benefit from the feature
3. The benefit that is expected

Rachel’s template is now almost universal, in part due to its inclusion by Mike Cohn in his seminal books Agile Estimating and Planning [Cohn05] and User Stories Applied [Cohn04]:

As a <type of user>

I want <capability>

so that <business value>

The template is guidance to help teams capture enough information in the story, so that the conversation they have is valuable.

Figure 6

Like any one-size-fits all system, while the template can be very useful for beginners, blind adherence to it can become problematic. Rachel herself describes some of the problems that treating this template as catechism can cause [Davies06]:

... this format can lead people to focus more on end users interests rather than the perspective of the person putting the business case forward. Also when given a template people can start treating story cards written this way as mini-requirements specs focussing on the written words rather than using stories as tools for driving a conversation. Even worse, stories that don’t fit this form will be battered into shape until they do.

Stories help you ask the right questions about the context and reason for the request. The important part is not about the words on the card but the shared understanding developed in the team.

Back to the future

Fast forward another couple of decades, and we find the user story as an almost universally recognised term in our industry’s vocabulary. However, in practice, we see the understanding of the term varies widely from team to team, from organisation to organisation.

Even though stories were intended to defer detailed analysis until the functionality was about to be developed, this is often not how they are used. There are many teams that use electronic tracking software (e.g. Jira) to build backlogs that contain hundreds of stories, each concealing detailed requirements. And despite the original intent of a story as a placeholder for a conversation, these details have never been discussed with the team.

So, while the term user story is widely used, it can mean very different things depending on who you talk to. This series of posts aims to fix that.

More ...

In this article, I’ve given an overview of the evolution of the user story as it is commonly used today. My blog digs into the BDD practice of Discovery, and the uncomfortable metamorphosis that stories undergo in the process.

References and further resources

[Agile] ‘Manifesto for Agile Software Development’, available at: https://agilemanifesto.org/

[Beck04] Kent Beck (2004) Extreme Programming Explained: Embrace Change, Addison-Wesley Professinal, ISBN: 978-0321278654

[Bell76] T. E. Bell and T. A. Thayer, ‘Software requirements: are they really a problem?’ published by TRW Defense and Space Systems Group, available at https://static.aminer.org/pdf/PDF/000/361/405/software_requirements_are_they_really_a_problem.pdf

[C2] C2 wiki (2002): https://web.archive.org/web/20021105182544/http://c2.com/cgi/wiki?PlanningGame

[Cohn04] Mike Cohn (2004) User Stories Applied for Agile Software Development, Addison-Wesley Professional, ISBN 978-0321205681

[Cohn05] Mike Cohn (2005) Agile Estimating and Planning, Pearson, ISBN: 978-0131479418

[Cucumber] Cucumber Plain Text Feature Completions.tmPreferences, available on github at: https://github.com/bmabey/cucumber-tmbundle/blob/bb89925f54372282e6f7500cc53b746e44dbc31a/Preferences/Cucumber%20Plain%20Text%20Feature%20Completions.tmPreferences

[Davies06] Rachel Davies ‘As a coach I want a story template so that people ask questions’ posted on 1 December 2006, available at https://agilecoach.typepad.com/agile-coaching/2006/12/as-a-coach-i-want-a-story-template-so-that-people-ask-questions.html

[Davies19] Rachel Davies conversation on Twitter: https://twitter.com/rachelcdavies/status/1186313143611469826

[Jeffries01] Ron Jeffries ‘Essential XP: Card, Conversation, Confirmation’ posted 30 August 2001, available at http://www.extremeprogramming.org/

[Larman03] Craig Larman and Victor Basili, ‘Iterative and Incremental Development: A Brief History’, published in Computer June 2003, available at: https://www.craiglarman.com/wiki/downloads/misc/history-of-iterative-larman-and-basili-ieee-computer.pdf

[Rose14] Seb Rose, ‘Let me tell you a story’, a presentation delivered on 3 April 2014 as part of CukeUp! 2014, available from https://skillsmatter.com/skillscasts/5101-let-me-tell-you-a-story

[Rose19] Seb Rose, initiated conversation on Twitter: https://twitter.com/gojkoadzic/status/1186316368121159680

[Royce70] Winston Royce ‘Managing the Development of Large Software Systems’, Proceedings of IEEE WESCON August 1970, available at http://www-scf.usc.edu/~csci201/lectures/Lecture11/royce1970.pdf

[Scrum] The Scrum Guide published by Scrum.org: https://www.scrum.org/resources/scrum-guide

[TextMate] ‘TextMate: The missing editor for OX X’, published 6 August (year unknown) at https://dhh.dk/arc/000270.html

[Wikimedia] Srum diagram (labelled), developed by Mountain Goat Software and available from https://commons.wikimedia.org/wiki/File:Scrum_diagram_(labelled).png under a Creative Commons Attribution 2.5 Generic Licence.

[Wikipedia-1] ‘Cargo cult’: https://en.wikipedia.org/wiki/Cargo_cult

[Wikipedia-2] ‘Waterfall model’: https://en.wikipedia.org/wiki/Waterfall_model

[Wikipedia-3] ‘Cost of delay’: https://en.wikipedia.org/wiki/Cost_of_delay

[Wikipedia-4] ‘Lightweight methodology’: https://en.wikipedia.org/wiki/Lightweight_methodology

[XP] ‘Extreme Programming: A gentle introduction’, available at: http://www.extremeprogramming.org/

Resources

Rebecca J. Wirfs-Brock ‘Responsibility Driven Design’, reprinted from The Smalltalk Report (date unknown) and available at https://www.wirfs-brock.com/PDFs/Responsibility-Driven.pdf

A Laboratory for Teaching Object Oriented Thinking - Kent Beck, Ward Cunningham

This article was published on Seb’s blog on 31 October 2019: https://cucumber.io/blog/bdd/user-stories-are-not-the-same-as-features/

Seb Rose Seb has been a consultant, coach, designer, analyst and developer for over 40 years. Co-author of the BDD Books series Discovery and Formulation (Leanpub), lead author of The Cucumber for Java Book (Pragmatic Programmers), and contributing author to 97 Things Every Programmer Should Know (O’Reilly).