Introduction to Agile Usability: User Experience (UX) Activities on Agile Development Teams

This article presents a coherent strategy for bringing usability practices into agile teams, summarizing Chapter 4 of Maturing Usability: Quality in Software, Interaction, and Value. User experience (UX) and agile practitioners need to learn about and respect each other’s philosophies and techniques, and then actively find ways to work together. This requires both communities to make minor changes to the way that they work, but if they choose to make these changes I suspect that their work products will be much better for it.

With an agile UX approach you will do high-level, very broad modeling at the beginning of the agile initiative to address the majority usability and UI issues facing the team. You don’t need a detailed answer, the details can come later on a JIT basis during development, but you do need a strategy. During development usability practitioners should be embedded within the development team, working on UX tasks when needed but also working with their teammates on non-UX activities. Agile teams are made up of generalizing specialists with the ability to work on a wide range of things, instead of narrowly focused specialists typically found on traditional teams. User testing, including usability testing, is done throughout the lifecycle, it isn’t left to the end of the initiative where it is invariably too late to act.

In The Inmates Are Running The Asylum Alan Cooper indicates that many of today’s software-based products suffer from usability challenges. He believes that the key to solving the problem is to perform what he calls interaction design before the programming effort to design the way that software behaves before it is built. Cooper believes programmers consciously design the code inside programs but only inadvertently design the interaction with humans. Although programmers work hard to make their software easy to use, their frame of reference is themselves and as a result they make it easy for other software engineers, not normal human beings. He argues that programmers have too much influence over the design of the human interface and due to a lack of skills in this area do a poor job of it. In other words, the programming inmates are running the software development asylum.

My experience is similar to Cooper’s, there is a need for improved interaction design skills within the programming community, although I don’t agree with his proposed solution. Reading between the lines of both The Inmates Are Running the Asylum and a discussion between Kent Beck and Alan Cooper, it seems to me that most of Cooper’s experiences are with traditional development teams but not agile teams. Although I believe that his observations about the interaction design skill levels of developers apply to both the traditional and agile software development (ASD) communities, I fear that his advice is difficult for traditionalists to implement and all but impossible for agilists. The goal of this article is to show how to make the advice of the user experience community palatable, or perhaps “usable” is a better term, to the agile community.

Agile Software Development (ASD)
User Experience (UX) Activities
Current state of the art
Addressing some misconceptions
User experience modeling on an agile team
User testing on an agile team
A call to action
Potential Challenges

1. Agile Software Development (ASD)

The vast majority of agile initiatives are teams of less than ten people, are co-located, have direct access to stakeholders, have access to inclusive modeling tools such as whiteboards and corkboards, have their own development machines, and have access to the development tools that they require, including testing tools. Having said that, some agile teams are very large (upwards of several hundred people), some are dispersed geographically, and some do not always have easy access to stakeholders. Although most agile teams take an test-driven development (TDD) approach where they write a unit test before writing just enough production code to fulfill that unit test, they typically do not have access to UI testing tools. Furthermore, they rarely have access to a usability lab, so in this respect agile is little different than traditional development.

Figure 1 depicts my rendition of a generic agile SDLC, which is comprised of six phases: Concept, Inception, Construction, Transition, Production, and Retirement. Although many agile developers may balk at the idea of phases, the fact is that it’s been recognized that processes such as XP and DAD do in fact have phases.

Figure 1. The Agile (Project) Lifecycle (click to learn more).

Let’s consider each phase in turn:

Concept. The initial idea/concept for the product or service is typically identified as part of your organizations product management process.
Sprint 0/Inception. The first week or so of an agile initiative is often referred to as “Iteration 0” or “Sprint 0” (sprint is an alternative term for iteration). The goal during this period is to initiate the team by garnering initial support and funding for the initiative; actively working with stakeholders to initially envision the requirements of the system at a high-level; starting to build the team; modeling an initial architecture for the system; and setting up the environment.
Construction phase. During development iterations agilists incrementally deliver high-quality working software which meets the changing needs of stakeholders.
Transition phase. During the release iterations(s) agile practitioners transition the system into production.
Production. The goal of this phase is to keep systems useful and productive after they have been deployed to the user community. The fundamental goal is to keep the system running and help users to use it.
Retirement. Eventually your solution may be removed from production, perhaps because it supports a line of business your organization has exited or because it has been replaced by a more effective solution.

On the surface, the agile SDLC of Figure 1 looks very much like a traditional SDLC, but when you dive deeper you quickly discover that this isn’t the case. Because the agile SDLC is highly collaborative, iterative, and incremental the roles which people take are much more robust than on traditional teams. In the traditional world a business analyst created a requirements model that is handed off to an architect who creates design models that are handed off to a coder who writes programs which are handed off to a tester and so on. On an agile teams, developers work closely with their stakeholders to understand their needs, they pair together to implement and test their solution, and the solution is shown to the stakeholder for quick feedback. Instead of specialists handing artifacts to one another, and thereby injecting defects at every step along the way, agile developers are generalizing specialists with full lifecycle skills. More importantly, from a user experience point of view, they take a very different approach to modeling and t esting than traditionalists do.

2. User Experience (UX) Activities

Usability is a quality attribute of a system which encompasses learnability, efficiency, memorability, error recovery, and end-user satisfaction (Neilson 1994). User-centered design, also known as UCD although I will use that abbreviation for usage-centered design, is a highly structured, product development process where the focus is on understanding the needs and goals of the user of the product. Interaction Design (ID) is a methodology described by Alan Cooper where the goal is to provide end users with functions that are both desirable and useful. In ID, interaction designers focus on what’s desirable, engineers on what they’re capable of building, and business stakeholders on what’s viable I use the term user experience (UX) to encompass all of these concepts – although there is good reason to distinguish between the various ideas, that isn’t relevant for my current discussion.

An important question to ask is why should agile practitioners consider UX important? Jeff Patton believes that UX addresses several issues which are critical to the success of ASD teams:

UX places emphasis on the usage necessary for roles to meet their goals.
UX helps meet the goal of identifying the behavior the software should have.
UX practices can be applied with varying degrees of formality, thereby making them compatible with agile methodologies.

Other important terminology which I use in this article includes:

System. The product, which often includes software, under development.
User. Also known as an end user, a user is a person who will actually work with the system/product being built.
Developer. An IT professional involved with the creation of the system.
Stakeholder. A stakeholder is anyone who has a stake in the creation or operation of the system. This includes people who are a direct user, indirect user, manager of users, senior manager, developer, operations staff member, support (help desk) staff member, developers working on other systems that integrate or interact with the one under development, or maintenance professionals potentially affected by the development and/or deployment of a software solution. Some agile methodologies, XP in particular, uses the term “customer”.
Acceptance testing. A testing technique, the goal of which is to determine whether a system satisfies its acceptance criteria and to enable the stakeholder(s) to determine whether to accept the system
Usability testing. A method by which users of a system are asked to perform certain tasks in an effort to measure the system’s ease-of-use, task time, and the user’s perception of the experience. Usability testing can be both formal and informal, using dedicated rooms and equipment to simply using physical mock ups of the system.
User testing. Testing activities, including both acceptance and usability testing, where stakeholders are actively involved.

3. Current State of the Art

Table 1 summarizes the differences between the agile and UX philosophies.

Table 1. Comparing agile and UX philosophies.

Agile Philosophies

UX Philosophies

Asks “How can what we have now be improved this iteration?”
You should work closely with your stakeholders/customers to identify their exact needs.
Details behind requirements can be identified on a just-in-time (JIT) basis during development.
Detailed, up-front modeling is a risky endeavor at best.
Interaction design has a role to play from the beginning, as a way to understand what is needed.

Asks “What is the ideal system?”
Defining the behavior of software-based products and services is very difficult and it has to be done from the point of view of understanding and visualizing the behavior of complex systems, not the construction of complex systems.
All behavioral issues need to be addressed before construction begins.

4. Addressing a Few Misconceptions

To help promote effective collaboration between the two communities we need to clear up a few misconceptions that each community may have with the other. There are several that UX practitioners may have about the agile community:

Agilists don’t model. The actual fact is that agile practitioners do in fact model, it’s just that they discourage extensive up-front design work in favor of more agile approaches to modeling.
Agilists are continually deploying software into production. Although some teams do this, it’s not the norm. What is common is to deliver working software on a regular basis, perhaps every few weeks, into an internal environment for system and user testing. Deployment into production may occur every six-to-twelve months, if not longer, based on need and the ability of our end users to accept new releases.
XP is the only game in town. This is a serious misunderstanding because UX practitioners who believe this miss agile methods such as Scrum, Agile Modeling, DAD, and DSDM which are arguably more attuned to their philosophies.
There is no role for UX practitioners. Many agile methods forgo the concept of specific roles in favor of more generic roles such as developer/programming, coach/leader, and customer/stakeholder.
Agilists aren’t specialists. This is partly true because agilists prefer to be “generalizing specialists“.
User interfaces shouldn’t be refactored. Many people fear refactoring the user interface, often because they believe the UX issues will be forgotten or that end users will not be able to deal with continual change. The reality is that UI refactoring results in the slow but safe evolution of the UI, thereby improving its design. To ensure that UX issues aren’t forgotten, people with UX skills should be involved with the development and evolution of the UI. Yes, the UI changes, hopefully for the better, but the only people affected by the changes on a continual basis are those actively involved in user testing. And when you stop to think about it, shouldn’t developers act on the findings of usability testing efforts and thereby improve the UI?

The agile community equally suffers from debilitating misperceptions about the UX community:

All you need is a good set of UI guidelines. That’s a good start, but there is a fair bit more to UX than creating consistent UIs.
Working closely with stakeholders is good enough. That’s also a good start, but Jokela and Abrahamsson (2004) found that even a close and frequent co-operation between developers and stakeholders does not ensure good usability at all.
UX is just about UI design. UI design is clearly a part of UX, but so is understanding how your users will work with your system and what their goals for using the system are so that you can build something that is usable by them. This requires significant modeling and collaboration skills to accomplish.
UX relies on comprehensive up-front modeling. Although some people in the UX community want you to believe that, many others believe different. In many ways getting the UI right is similar to getting your architecture right, it often requires some up front thought although that doesn’t imply that you need to take a serial approach to development.

5 User Experience Modeling on an Agile Team

ASD practitioners are very pragmatic when it comes to modeling. The Agile Modeling (AM) methodology describes in detail how agilists approach both modeling and documentation. Figure 2 overviews the lifecycle of an Agile Model Driven Development (AMDD) approach to ASD, an approach which originally grew out of the XP community but which seems to capture the essence of the modeling approach on agile teams in general. Each box in the diagram represents a development activity. The initial modeling activity during iteration 0 includes two main sub-activities, initial requirements envisioning and initial architecture envisioning which are done iteratively in parallel. The iteration planning/modeling, model storming and implementation activities potentially occur during any iteration, including iteration 0 (yes, the rumors are true, agile practitioners will often implement working software the very first week of an initiative). The time indicated in each box represents the length of an average session: for example, during development you’ll often model storm for a few minutes with a stakeholder to explore a requirement and then code for several hours.

Figure 2. The Agile Model Driven Development (AMDD) lifecycle (click to enlarge).

The initial envisionment effort is typically performed during the first week of an initiative. For short efforts (perhaps several weeks in length) you may do this work in the first few hours and for long efforts (perhaps on the order of twelve or more months) you may decide to invest up to two weeks in this effort. There are two aspects to the initial envisioning effort:

Requirements modeling. You need to identify the high-level requirements as well as the scope of the current release. The goal is to get a good gut feel what needs to be achieved, and to do that you will likely need to create an initial usage model to explore how users will work with your system (e.g. use cases or usage scenarios), an initial domain model which identifies fundamental business entity types and the relationships between then, and optionally other important artifacts exploring technical requirements.
Architectural modeling. The goal of the initial architecture modeling effort is to try to identify an architecture that has a good chance of working. 99% of the time agile practitioners simply gather around a whiteboard and create free-form diagrams as they discuss various architectural strategies. When the UI architecture is an important consideration, agile modelers will create a UI navigation diagram (see Figure 3) which explores initial relationships between important screens/pages and reports (thereby giving you a birds-eye view of the UI, enabling you to ask fundamental usability questions).

Figure 3. A UI navigation model for a university system (hand drawn).

UI navigation diagram

In later iterations your initial models will evolve as you learn more, but during Iteration 0 the goal is to get something that is just barely good enough so that the team can get going. You don’t need to model a lot of detail, and I cannot stress this enough: the goal is to build a shared understanding, it isn’t to write detailed documentation.

During development iterations the vast majority of modeling sessions involve a few people, usually just two or three, who discuss an issue while sketching on paper or a whiteboard. These model storming sessions are “just in time” (JIT) efforts: you identify an issue which you need to resolve, you quickly grab a few team mates who can help you, the group explores the issue, and then everyone continues on as before.

Sometimes modeling storming isn’t enough. Perhaps you need to model complex requirements which require input from someone outside of your immediate team, or perhaps you need to model a legacy asset which can take a significant amount of time. In other words, you may need to model a bit ahead of actually implementing a requirement. This is actually a rare occurrence, regardless of what traditional modelers may hope for, but it does happen every so often.

All of this begs the question “how can UX activities fit in to an agile way of working (WoW)?” The easy answer is that agilists need to adopt usage-oriented requirements artifacts, such as personas, scenarios, or even use cases. The quickly explore user interface requirements, it is common to create abstract/low fidelity prototypes out of paper, such as the one in Figure 4 created from a flip chart sheet and sticky notes. This allows to you quickly explore the UI without doing a lot of up front work, and even enables agile usability testing.

Figure 4. An abstract UI prototype created with paper.

Essential UI prototype

UX practitioners can rail about the need for doing lots of design up front, but that message falls on deaf ears within the agile community. The bottom line is that from an agile perspective, traditional UX techniques aren’t very usable for them, which is rather ironic when you stop and think about it. To make UX techniques usable to agile practitioners, they must reflect the agile lifecycle. Luckily, this can be accomplished by:

Doing some UI modeling up front. You need to establish 3 things: Overall organization for the parts of the UI that fits with the structure of user tasks; a common scheme for navigation among all the parts; a visual and interaction scheme that provides a consistent look-and-feel to support user tasks. Yes, this takes some up-front work, but for the vast majority of systems this could easily be accomplished during Iteration 0. Remember that the amount of modeling that you need to do up front is dependent upon the context — some teams will need to do more than others.
Using modeling tools which reflect agile practices. Good options include sticky notes, index cards, and whiteboards.
Modeling a bit ahead when appropriate. If you need to, explore important aspects of the UI before you implement them. Conducting a user session often involves scheduling specialized facilities that are in high demand, and making appointments with the appropriate stakeholders. Therefore you need to adopt the practice of look-ahead modeling.
Doing UI development on a JIT basis the majority of the time. The UI is important, but then again so are many other aspects of a system (such as the network, the database design, and so on) and therefore the UI should be treated differently only when appropriate. When the goal of design is limited to a single feature or small set of features it is effective to perform UX activities as part of the development of those features during the iteration.
Adopt UX-friendly requirements artifacts. As I pointed out earlier, some agile methods such as DSDM and DAD already do this. This same philosophy could be applied in XP, and frankly many XP teams already capture the need to implement a UI view as a user story.

6 User Testing on an Agile Team

For the sake of this article, user testing encompasses:

Acceptance testing
Usability testing
Usage testing

6.1 Acceptance Testing

The agile community has embraced the importance of acceptance testing. The automated tests will be run often, at least daily if not several times a day. Manual user testing, on the other hand, is typically done an iteration behind. At the end of an iteration, many agile teams will deploy the working system into a QA/testing environment where user and system testing is performed. The team continues on, developing version N+1 of the system while obtaining defect reports pertaining to version N. Defect reports are treated just like any other requirement: they are estimated, prioritized, and put on the requirements stack to be addressed at some point in the future.

6.2 Usability Testing

Usability testing is considered optional, and I have no doubt that many UX practitioners will find that frustrating. Agile teams are little different from traditional teams in this respect, they very likely don’t appreciate the need for usability testing (or other UX practices for that matter). True usability testing requires repeated testing with numbers of users under controlled settings. Just like acceptance testing can be done regularly throughout development, so can usability testing. On agile teams usability testing should occur during the user testing effort after each iteration, assuming of course that someone on the team has those skills.

There is a range to usability testing. At the “agile end” of the spectrum is Jeff Patton‘s agile usability testing. In a workshop at the Agile 2006 conference Patton described a technique where people simulate the system using abstract prototypes (see Figure 4). Using this strategy one developer simulates the system. They’re not allowed to speak, only help the user navigate between “screens” (the paper prototypes). At least one user, although two seems to work well, works through a scenario via the paper prototype. For example, in a university system a scenario might be that they try to enroll in a seminar. The user(s) should narrate/discuss what they’re thinking as they use the system. One or more developers act as observers, taking notes but not fixing the paper prototypes as the users work with the system.

At the formal end of the spectrum is traditional usability testing where users are observed in a usability lab which is typically a room where a user can be observed working with the system. Usability labs often have one-way windows which enable people to watch the user, often a valuable learning experience for developers who mistakenly think that they’ve done a great UI design job. Some labs will even be outfitted with cameras so that you can record the exact interactions, in the hope of replaying the usage and hopefully acting to improve the design to support more effective usage.

There is nothing stopping you from taking an agile approach to usability testing at some points in your initiative and formal usability testing at other points. Nor is there anything stopping you from doing something in between.

6.3 Usage Testing

Usage scenario testing is a technique that can be used to test the logic captured within your design before you invest in implementing it. This is an inclusive technique which stakeholders can be actively involved with which can and should be performed in parallel with your modeling efforts to ensure that your model accurately reflects your business requirements. Using one ore more usage scenarios (a series of steps describing how someone works with your system) you walkthrough your class model and validate that it is able to support the scenario(s). If it does not, you update your model appropriately (or your code, as the case may be). Figure 5 presents usage scenario testing from the point of view of walking through a class model, although the same logic can also be followed to validate your user interface prototype (even if it is an abstract prototype).

Figure 5. The process of usage scenario testing (for a class model).

Usage scenario testing

7. A Call to Action

If the agile and UX communities are going to work together effectively, they need to find a middle ground. I believe that middle ground exists, but that both communities need to adopt several changes in order to succeed. First, agile professionals must:

Learn UX skills. Developers should be trained in, and adopt into their practices, UX techniques. This will enable developers to work more collaboratively and effectively with UX practitioners.
Accept that usability is a critical quality factor. Luckily, agile practitioners are “quality infected” – they understand the importance of doing high-quality work and have a proven track record of adopting techniques such as test-first programming, code refactoring, and database refactoring. Good usability of an end product can be ensured only by systematic usability engineering activities during the development iterations.
Adopt UI and usage style guidelines. Developers must understand that not only should their code follow common guidelines, so should their UIs.

Similarly, UX practitioners must make some changes. They need to:

Go beyond UX. I believe that many of the challenges experienced between programmers and UX practitioners are due to overspecialization of roles and hand-offs between people in those roles. Agilists have mostly abandoning the concept of building teams of specialists and instead favor teams of generalizing specialists. The implication is that although UX practitioners bring a critical skillset to a development team, they still need to learn a wider range of skills to become truly effective. Furthermore, agile practitioners have tightened the feedback loop on software teams, and thereby reduced both risk and cost, by increased collaboration which in turn results in decreased hand-offs.
Become embedded in ASD teams. By embedding UX practitioners on agile teams, not only will this increase the chance UX issues are addressed, it will help to promote UX skills within the agile community because people pick up new skills from one another as they collaborate.
Give ASD approaches a chance. Kent Beck suggested to Alan Cooper that a week be invested at the beginning of an initiative to explore interaction issues, although Cooper believed that wasn’t sufficient. The easiest way to find out who is right is to actually try it in practice.
Start looking beyond Scrum. I’ve said it before and I’ll say it again, there is more to ASD than Scrum. Agile methodologies are flexible, they’re not meant to be used “out of the box” but instead to be tailored to meet the exact situation which the team finds itself in. To address UX concerns you will very likely find that you need to tailor some of the principles and practices of Agile Modeling and/or the techniques of User-Centered Design into your base software process.

8. Potential Challenges

It is very easy to suggest that agile practitioners take the time to learn UX skills, and to adopt appropriate guidelines, but the reality is that these skills are competing for attention along with other equally important skills such as database design and modeling. To make matters worse, few developer-oriented books cover UI/usability issues, and the few that do such as my own The Object Primer rarely seem to devote more than a chapter to it. I fear that many agile practitioners aren’t even aware of the issue.

Similarly, UX practitioners receive mixed signals. Although I’m calling for them to become generalizing specialists, the industry still rewards specialization – UX specialists are paid very well, and most organizations expect them to focus on doing that specific sort of work. Agile practitioners also suffer from this challenge: Why take an introductory UI design course when you can take a Java programming course which leads to certification and more pay?

It’s also easy to say that UX professionals should be embedded into agile teams, but that only works when you have UX professionals available. Few organizations have such people on staff, and worse yet believes that few organizations think in terms of interaction design as part of the requirements planning or work planning process. Therefore, many organizations may not see the need to hire anyone with these skills.

The lack of UI design ownership means that everyone wants to be involved with UI design, regardless of their skill level, which can lead to design by committee. Although a common philosophy throughout the agile community is that agile practitioners should have the humility to know their limits, and the respect of others with the appropriate skills to address a specific issue, it doesn’t always work out that way – apparently agile practitioners are still only human.

It is possible for UX practitioners to make a valuable contribution on agile teams, and I suspect that there is a much greater chance of them succeeding as compared to working with traditional teams. Up until this point the UX community has had little success in their attempts to become an active part of mainstream development teams, so it is time for a new approach. My suggestion is that UX practitioners should work closely with the agile inmates to try to get the prison under reasonable control.

9. Acknowledgements

I’d like to acknowledge the input from Paul Oldfield and Tim Tuxworth which helped me to evolve this article.