Humans are only able to hold so many distinct pieces of information in their head at a time. Past a certain point, the mind naturally starts treating things as bundles, groups of related items, in order to continue working on them. Some people are able to hold these separate items, others are better at transparently grouping things together. But in the quantity of items that are needed for anything but the simplest of projects, every product owner will need to group those user stories together in order to facilitate the creation process.

For example, as I create user stories, I might find or create two that tell the same story, but from a different perspective. Each adds value, and I would want the different perspectives captured and communicated, but ultimately they are the same story. If I were literally using ’story cards’ as in 3×5 cards on a desk, then in that case I would capture that information by stapling the cards together, so they are treated as one.

As the number of story cards increases, I might start making piles of story cards that are closely related, but they are not quite the same. For example, one story card may represent the basic functionality, while another represents a bit of additional functionality that can be added later which one stakeholder or another might be able to leverage. As more stakeholders weigh in, the pile gets more and more cards in it. Once everybody feels like they have defined what they want, I might give that pile a ‘label’ so I understand what is in it, and then I would paper-clip them all together. From that point forward, I will usually treat that pile as a unit, peeking inside only when I need to.

As the number of story cards continues to grow, more and more of these piles are going to exist. These piles will need some way to organize them. I would look them over, find things they have in common, and start making groupings. Once I have a grouping that seems to me to be ‘complete’ I would use a binder clip to hold them all together, with a ’summary’ card on top of them all, so I can treat them all as a single element.

In some cases, I may deliberately create ‘groupings’ and then start creating story cards to go into that grouping. But in many cases, I will be creating groupings after-the-fact. And unfortunately, I may undo and re-do my groupings once or twice before I am satisfied with them. I may move individual cards from group to group, or I may move entire groups to other groups.

While creating the cards, I will periodically re-open my groups one by one to inspect them and see if any additional stories come to mind. If not, I close them up again and move on to the next, until I feel like I have a degree of completeness.

Most likely, the grouping that I will use initially will be functional in nature. Related functionality will be grouped together. But sometimes, it helps to look at things from a different perspective. So I will want to see my stories grouped by role, so I can look at all of the stories told from the perspective of a particular stakeholder. In doing so, I may realize that there are stories that I may have missed. Or, I may take that bundle of story cards to that stakeholder to review.

Later, when assigning business value, I may want to re-group them again, according to the type of value that they provide to the business, such as attracting new customers, retaining existing customers, increasing income from existing customers, reducing expenses, limiting legal risk, and so forth. In this case especially, one card may belong to multiple groups. By reviewing the cards in context with other cards that have a similar goal in mind, it will make it easier to get a perspective on relative business value.

Of course, a story is not complete if I have not defined acceptance criteria. So I will need to go through the stories one by one, starting with the ones that provide the most business value, and write the acceptance criteria.

Finally, the sizing of stories can begin.

A good backlog tool will be at least as flexible as the analog that it seeks to emulate. Creating story cards will be quick and easy. Moving them around and grouping them together should be nearly effortless, from a technical standpoint.

Ideally, it will be even better than the analog. I can create as many groupings as I want, and nest them as deeply as I need to, without running out of ways of holding them together. I can move some aside that I’m not actively working on, sort them according to whatever criteria I want, or find a specific story without having to remember which group I put it in.

The Tools

In looking for a backlog management tool, the first thing that I found was that the majority of tools completely ignore the need for grouping. The concept of ‘epics’ and ‘themes’ are absent from a great many of them. While I did see a tool that really impressed me, for a number of reasons, the absence of that one feature meant that I had to eliminate the tool from consideration.

After reviewing dozens of backlog management tools, I managed to narrow the field to four: Rally, VersionOne, Mingle, and AgileOnDemand. Of course, for completeness, I need to consider using Excel, the ‘recommended’ tool from the literature, and actually using stacks of 3×5 cards. The deciding factor is going to have to be, does the tool let me do what I want, the way that I want to do it?

As we move forward in this analysis process, we will take specific scenarios and see how they compare.

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

The goal of every sprint should be to create something that gives the business a sense of perspective, a frame of reference to use to direct future activities. What you deliver needs to work, at least in some respect or with some limitations, in order to offer the business that sense of perspective. If what you deliver can not be seen by the business, it adds no value.

“But it adds value to the development effort!” you might be thinking. Sure, assuming that your understanding of the direction that you need to go is correct.

Note that you are not simply taking a traditional development project and breaking it into 2-week chunks of work. You are creating a functional piece of software. It is all too easy to fall into the trap of breaking things down functionally, and then trying to map your functional decomposition into a backlog. Too often, you end up with pieces which can not stand alone, which do not deliver value to the business. The business can’t prioritize those kinds of things.

The problem is diving too deep. It’s so easy to do, too. You want to understand the problem, so you divide and clarify, divide and clarify, and at some point you magically and invisibly have gone beyond what you are trying to accomplish and have transitioned into the how of accomplishing it.

Therein is the secret of making sure your backlog includes only items that add value: They only tell you what it must do for the business, it does not specify how it is done. Review every user story, and ask yourself: “Is this telling me what the system needs to do, or is it telling me how the system works?” If it is the latter, change it to be the former. If you can’t, then you probably broke down the functionality too far, and that story needs to be rolled into another one that it supports.

It takes discipline, and maybe even a shift in ways of thinking, but you need to deliver something that “works” every time. It may be a mock-up or a prototype, it may not do everything it needs to do when initially delivered, but it is something that the business can see and wrap their brains around. The purpose of this is to empower the business to give their guidance sooner rather than later, and save everyone the frustration of wasted effort.

“But isn’t a prototype a waste of time?” Not really. Build it right and you can simply refactor it as necessary to make it into what it needs to be. And in the worst case, if you have to throw the whole thing away, you still ended up with valuable understanding of the problem, and that all-important frame of reference for the business to guide forward progress.

So, deliver working software. But deliver it often. Very often. Here’s why:

• If you have a term paper due in six months, what is the likelihood that you will finish it tomorrow? What is the likelihood that you will do the bulk of the work the night before it is due? The longer the sprint length, the more this tendency might rear its ugly head.
• The bigger the sprint backlog, the more overwhelming it can feel, whether there is any reason for it or not. So, the longer the sprint, the bigger the sprint backlog, the harder it is to focus, for some people.
• The shorter the duration, the sooner the business can give its feedback, the more Agile you become.

Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.

There is an innate aversion to wasted effort. Nobody really wants to do something, only to find that they have to start all over again because what they did was not what was really wanted. In response to this aversion, people are naturally inclined to do more up-front planning. “If they just would have asked for what they really wanted, we could have delivered it, and it would have saved so much time!”

It’s true. If the development team were handed a detailed plan, and the plan were perfect, they would avoid a great deal of rework that happens when plans change. We all know it is true. And so, instinctively, we resist change, instead trying to push for “getting it right the first time.”

What we don’t always realize is that you could probably build it 3 times, two a total waste of effort, in the same time that it would take to plan to get it half-right. Yes, that’s right, there is something gained in doing it even if it’s wrong.

“What is gained in doing it wrong?” you ask? The short answer is “perspective.” You want to get from A to Z. A is where you are. Z is where you want to be. You think you know how to get there. Or maybe you don’t, but you know that the first thing you need to do is go from A to B. Once you have gotten to B, your position has changed, and so has your perspective. Now you have something tangible to look at, and that may reveal to you that Z isn’t over here, it’s actually more like over THERE. And now, it is a little more clear that the next step in the progression is C. Suppose it turns out that C didn’t get you any closer to Z. It was a waste, right? No. You have something tangible that you can point to and say “From here, we need to go in THAT direction.” It is the hardest thing in the world to say “I want to go in THAT direction” when there is no frame of reference. Giving yourself a frame of reference will accelerate progress by orders of magnitude more than the time wasted doing it the ‘wrong’ way. Besides, who says you won’t use C later on for something else?

So, a truly agile team, both in form and in mindset, welcomes change. Because when the roadmap is changing, that means that forward progress is being made. It means that previous unknowns are no longer unknown. The mature team realizes that if things aren’t changing, it could well mean that somebody isn’t paying attention to what you are doing. And that doesn’t bode well for anyone.

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

A basic tenet of Agile software is to deliver Value to the business. The challenge is to do this in the face of continuously changing priorities and/or requirements.
Part of what causes the priorities or requirements to change is a basic truth in psychology: it’s easier to see what is in front of you than it is to operate mentally on a figment of your imagination. The more concrete things become, the more the definition seems to change.

What this also means is that the more frequently you deliver something that works, the faster the business gets to see the truth of what they had asked for. This allows them to evaluate it in a concrete way, and thereby be empowered to offer meaningful feedback on it.

But delivering something that ‘works’ is only half of the equation. What works should be something that adds value. Time spent working on something that no one will use is time wasted. Delivering something that costs more to create than it will earn or save for the company is also time wasted. There may come a point in the project where there is still plenty to work on, but none of it adds enough value to be worth the investment. A good product owner will recognize that point, and the project will end. A good team will help identify when that point is reached.

When breaking down the work that is to be done, keep this in mind. If it doesn’t add value by itself, don’t deliver it by itself. If something else adds more value, do that first. But don’t fall into the trap of needing to deliver everything all at once. Really, all ‘features’ are not created equal.

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

Individuals and interactions over processes and tools

Burndown charts are tools. Obviously, they are valuable. But they are only valuable insofar as they help individuals engage one another. A burndown chart reveals where the team is compared to what they committed to. It reveals whether the team over-committed or over-estimated. But ultimately, it is a tool to encourage communication. It is a signal when things are wrong, or when there are opportunities for positive change. But the tool has no value if no one is looking at it. It has no value if no one knows how to interpret it. The tool has value only as a means to an end, that end being the constructive dialogue of the team, and the continuous improvement of the process.

Scrum is a framework, a ‘process’. It is the most popular implementation of Agile software development. Obviously, it is valuable. But what gives it that value? The dialogue between the team members that happens on a daily basis. The dialog that happens between the team and the customer on a regular basis. The way in which it reveals shortcomings in the way the team or the company works. But all of the trappings of Scrum are of none effect if they do not lead to continuous discussion and improvement. Again, the process serves to highlight the need for communication, and creates a context for it to take place. But having the trappings of Scrum without the dialogue and the evolution of the team that it facilitates, and you get very little from it.

Working software over comprehensive documentation

Documentation is valuable as a tool for communication. Its purpose is to help everyone get on the same page. Of course, having everybody together in the same room, having a lively discussion, getting all of the details out, all this yields the same results, in a fraction of the time, and with significantly higher value. The other value of documentation is to convey information to those that could not be there for that meeting. Of course, being able to point to what was built and say “That’s what we built” is just as good as pointing to a document and saying “that tells you about what we built”, in most cases. That’s not to say that documentation has no place. If it is hard to wrap your brain around it, document it, save someone else from having to spend as much time doing so. But if it is self-evident, documenting it is a case of putting the process before the results the process is intended to achieve. Like mindlessly serving a tyrant king.

Besides, at the end of the day, customers don’t use your process documentation. They use the working software. If you have to stop half way through the project, and all you have is documentation, you have nothing of value.

Customer collaboration over contract negotiation

“That’s not what you asked for!” “But that’s what I need!” Contracts are important because they set expectations, and they define roles. But they can also serve as a pair of handcuffs. The customer doesn’t win when they can’t get what they want because they didn’t ask for it up front. And the team doesn’t win when the customer is dissatisfied. In a customer service organization, which ultimately every development organization ultimately is, your success and longevity is determined by how well you can serve the needs of the customer. The contracts that you negotiate should help you do that, not hinder it.

Responding to change over following a plan

Plans are essential. Even in Agile development, you need to plan your work and work your plan. But circumstances change. The plan needs to change, too. It’s a lot of work to create a plan that lays out the next 2 years. Having to go back to the drawing board after six weeks is demoralizing, and a huge waste of effort. Refusing to go back to the drawing board, even though you should, will ruin you. The further out in time you plan, the less detailed your plans should be, so that effort is not wasted as you adapt to the inevitable changes that come your way.

1. quick and well-coordinated in movement; lithe: an agile leap.
2. active; lively: an agile person.
3. marked by an ability to think quickly; mentally acute or aware: She’s 95 and still very agile.

No plan can make it unscathed from concept to implementation.  No matter how meticulous the planning, by the time you can get it half implemented, half of what you built will be irrelevant, and there will be things you never thought of that are more important than anything you planned to build.  Reality in the software development business is the continuous attempt to hit a moving target.

The playing field is never static.  Business needs change.  Technology is continuously evolving.  “Agile” software development is a way of working that recognizes the inevitability of change, and embraces and readily adapts to it.

The difference between Agile and more traditional development methods (such as Waterfall) lies in the speed with which they facilitate adapting to change.  Through deliberately planning out only enough work to get through the next few weeks, and by revisiting the priority of upcoming work on a regular basis, changes to the playing field can be adapted to quickly, and in stride.

It’s simple, in principle.  But old habits die hard.  Truly Agile development is a paradigm shift, a complete change in thinking as well as methodology.  Attempts to adopt the methodology without understanding the reasons behind them will be frustrating at least, and may not reap all the rewards that it promises.  If you are going to adopt the methodology, invest in the people who will be exercising it.  Get them the training they need to be successful.  It will pay for itself before the end of your first Agile project.