Mike is right, of course. It is called a Scrum team for a reason. The team will work together in order to achieve success. If the Product Owner has not been able to gather enough information to be confident of the direction to move forward, the team won’t sit around watching youtube until they do. The team will ask the product owner what they DO know, and will exert their efforts in ways that will help the Product Owner. For example, they may build prototypes that the PO can take to the users, to facilitate more constructive conversations.

One might argue that if the team builds the wrong thing, it is on the head of the product owner. But the team is full of living, breathing, thinking human beings. The developers are not mindless automatons serving the whims of the product owner. Everyone should have the mindset of serving the needs of the customer. If team members think that perhaps the product is going in the wrong direction, they have a responsibility to communicate that. If they do not, their neck deserves to get choked as much as the PO. Now, if you have a product owner who doesn’t take feedback, then by all means, you have your single throat to choke.

One argument leveled against Agile is that it can be used as an excuse for not delivering, citing the need for ‘direction.’ This, of course, is a Waterfall mentality trying to do iterative development, and not Agile. The team that takes responsibility for their success will never have that argument used against them.

Analog (3×5 card) Method
This hands-on approach is definitely a good, low-tech starting point for any story-writing workshop. For what it can do, it is the gold standard in usability. Pick it up, put it down where it goes. It’s easy, intuitive, and hard to screw up. For me, however, what it can NOT do means that I definitely need to combine this with some form of electronic tool.

Excel
It is no wonder that Excel is the recommended tool for backlog management in the literature. It’s easy, ubiquitous, and a decent compromise in features versus usability. If I had no budget for one of the other tools, this is definitely a workable solution, if I were willing to let go of some of the features I want.

AgileOnDemand
This tool is in its infancy, and yet it blows away the competition in everything which this tool is designed to do. With one exception, that being the Merge functionality, this tool is at least on par with writing user stories on 3×5 cards, as far as usability is concerned. In many ways, it is vastly superior. Yes, vastly superior to the gold standard. That’s high praise.

Yet, being as it is such a new tool, it is lacking some of the features sported by some of the other, older, more mature tools. Give it time, I expect great things from these guys.

For me, because of my need for being able to look at cross-sections of the backlog from various perspectives, I can not use this tool, yet. But I’ll be continuing to watch as they implement new features. And I hope to be able to contribute towards turning this promising piece of software into that ‘Ideal Digital Tool’ I kept talking about.

Rally
This is one of the best looking of all of the tools. And there are a few things that it does very well. Unfortunately, one of the things it does poorly is one of the features I need the most: the ability to quickly re-organize my backlog differently than I had previously organized it. But it is a fairly mature tool, so it has many good features, such as test management, something AgileOnDemand has not yet implemented. I will continue to watch these guys, and I will give them my feedback, and we will see what comes of it. With a few small changes, I could easily make use of (and thus recommend) this tool.

VersionOne
VersionOne consistently scored relatively high marks across the board. Unfortunately, the model that they use in their user interface leaves a few things to be desired. Unfortunately, rectifying that probably means a paradigm shift for the user experience guys at VersionOne, so I’m not holding my breath. But it really isn’t THAT bad from a usability standpoint, and it is a highly functional tool. As such, I can imagine finding a way to make it work the way I need it to, at least, for the most part. But, if at all possible, I’d really rather find a tool that suits my needs a little more closely.

Mingle
Mingle stands out from all the rest of the tools I reviewed, because it is not, strictly speaking, an Agile Backlog Tool. Instead, it is a generic tool that can be configured to match the way you work, including Agile. Unfortunately, it has a few serious usability issues, and is missing a few key features. But it also has features that none of the others have, such as an integrated wiki, something I consider to be immensely valuable. As with the other tools, I intend to give them my feedback, and find out which of the things I need most from them are already on their way.

Conclusion

The truth is, all of the tools are decent, a couple of them outstanding, but none of the tools that I surveyed are “good enough” for me in their current incarnation. Some are closer than others, and they are all contenders. If I were willing to compromise on some key points, I would be able to decide between them. But I know what I want, and I’m going to keep looking until I find it, until one of these tools build it for me, or I break down and build it myself!

The ideal tool would allow the user to set a ‘ranking context’. This would be a way of preserving different sets of rankings for different reasons. By default, the tool would come with ranking contexts for Business Value, Technical Risk, Story Size, and Priority. It should also allow the user to create their own custom ranking contexts, so that the users can utilize this metaphor for whatever they like, without having to compromise.

Another thing I would like to see, in addition to having multiple ranking contexts, is the ability to allow one ranking context to be the starting point for another. For example, in the Priority context, if an absolute priority between two stories has not been determined, then it could default to the ranking defined by the Business Value context. If that context does not define the relative priority, then look at Technical risk, favoring the more risky items. And finally, in the absence of a difference in risk, opt for the smaller stories first. All this should be readily configurable.

Of course, the purpose of these ranking contexts is to augment or replace having to provide an absolute ranking value. So, one could tie the context to a numeric field. As one drags stories up and down in the list, the value changes. Or, one can change the value directly, and anchor the ranking to a particular value, allowing it to jump to the approximately correct position within the list. This position is then able to be refined by dragging, which in turn alters the values of the stories involved. This would best serve those who don’t use Planning Poker or the Business Value Game, but could easily be used in conjunction to serve as a kind of sanity check on the values that are coming out of the estimating.

What I would also like to see is for related items (groupings) to be ranked as well, and for those rankings to influence the relative ranking of the items they contain. So, for example, if I rank one feature group higher than another, then the stories within that group get a slight boost so that the average ranking of the one group is higher than the average of the other group. Likewise, if I rank one Epic or Theme higher than another, again the stories in that epic get a boost compared to the others. I could also consider a particular role above others, and a particular business proposition above others. This, alone, might do well to align the relative ranking of stories without having ever adjusted a single story up or down.

Another thing I would love to see is for there to be calculated fields that represent ratios between some of these fields that are set by ranking. So, for example, if a story has both a Business Value and a Technical Risk value, then a calculated field would estimate the risk/reward ratio. If it has both Business Value and Size, then a Return on Investment ratio is calculated. I would even calculate a Risk/Size ratio, and perhaps even an overall weighted value that takes all three into account to give the product owner a suggested priority starting point.

Of course, this all sounds really complicated. But it doesn’t have to be. A simple configuration combined with a simple but powerful user interface, and the tool can help guide even a novice product owner towards delivering maximum value with a minimum of cost.

Analog (3×5 card) method
Put all of the story cards together into one large pile. Move more important cards towards the top of the pile, less important cards towards the back.
Usability: Medium-High Downside: you lose the ability to view the other relationships that exist between cards. You can’t search through the stack very easily.

Ideal Digital Tool
Any time you have a list of story cards, you can drag them up or down in the list in order to set relative order. You can filter the list to include only a subset of the cards (such as for a single feature group, a single theme or epic, a single role, or a single value proposition), and then rank those relative to one another. Prioritizing a small set of closely related stories will be faster and easier than reviewing the entire backlog at once. Repeating this process with different cross-sections of the backlog will allow the product owner to gain perspective about what he or she is trying to prioritize. The ideal tool will be able to correlate all these theoretically disparate and potentially conflicting priorities and putting them together into a coherent whole, a prioritized backlog that the product owner can groom.
Usability: High

Rally
You can move stories within groups, move groups within larger groups, and reorder the groups. You can switch to the backlog view, and sort them there, as well, by dragging up and down.
Usability: High Downside: Does not readily support ranking within the various cross-sections of the backlog.

VersionOne
Add the ‘Order’ field to your view, then sort by it. Then, you can drag rows up and down in order to specify a custom order. As you move from view to view, applying various filters at different times, the overall sort order gets more and more precise, exactly as with the Ideal tool.
Usability: Medium-High Downside: You have to know you can do it, as it is not enabled by default. It tends to be a bit slow, in general, which is made worse by the fact that you can only drag one at a time to prioritize them.

AgileOnDemand
While viewing the backlog, you can create Epics and you can create stories within epics. You cannot control the order of stories within an epic, except by a naming convention. But you can drag epics up or down to prioritize them.
Usability: Medium Downside: some things are harder to do than you would expect them to be.

Excel
Highlight a row. Drag from one of the borders of the row, while holding the shift key. Where you let go, that’s where it goes to.
Usability: Medium If you don’t hold down the Shift key, you may overwrite an existing backlog item, which would be very bad. The mouse target for initiating the drag is very small, meaning you need dexterity. Though you can also do it via the keyboard pretty easily.

Mingle
From the Grid view, you can enable ranking. This lets you drag and drop cards into the desired order. However, the ordering is left to right, not top to bottom. And you can’t do this from the more compact Hierarchy or List views, which is a serious shortcoming in my opinion.
Usability: Medium-Low Downside: really doesn’t meet expectations, that is, forces me to do things their way, rather than adapting to me and the way I think or want to work.

Usability Comparison Summary on a scale of 1 to 10

It is important that you be able to associate more than one value proposition to a story. What attracts new customers may also encourage existing customers to upgrade their service. In that regard, those tools that do not facilitate the use of Tags, multiple values in a single field, or many-to-many relationships, are at a strong disadvantage when it comes to being able to implement this feature.

From a technical standpoint, implementing this behavior in the various tools is possible in roughly the same way as Role based grouping. You can use Tags, where tags are offered, or you can define a field/property with a set list of values. Their respective usability is the same as that from the previous segment on grouping by Role.

One notable difference is that the user interface of the Ideal Digital Tool can’t cheat and extract it from the user story text, you have to set it yourself. This may require a more sophisticated user interface, though what works for this should work equally well for managing Role assignments.

Usability Comparison Summary on a scale of 1 to 10

Hopefully, the above makes the case adequately enough: It would add value to the backlog creation process to be able to view the backlog from the perspective of a particular role, in addition to the perspective of a particular feature group. So, how can I do that in the various tools?

Analog (3×5 cards) method
You can’t. A card’s place is represented by its physical presence. The only way you could do this is to make sure you make at least two copies of every user story (maybe in different colors) whenever you create one, and you put one in the feature group pile, and one in the appropriate role pile.
Usability: Very Low Downside: having two of every card is tedious, and means things will get out of sync. Don’t go there!

The Ideal Digital Tool
As I create the user story itself, when I type “as a ” I am then prompted with a pop-up list of all of the other roles I have ever created. I can select one, or I can make a new one. When I finish writing the story text, it will transparently pull the Role information out of the text, and use it to populate a ‘role’ field, a multi-select value field, or a relationship record, that allows me to find this story again by the role. If I don’t use the ‘as a’ template, I can still manage the role relationship directly, though the fact that I need to will encourage me to use the template. Or the tool will let you define a search pattern to extract the role so you can use your own story template. There may be a place in the application where I can define the relationships between roles. I will be able to filter any view by Role, or by multiple roles, quickly and easily. Ideally, I will be able to specify a full filter (where non-matching records are hidden) or a highlight filter, where matching records stand out from the rest, visually.

Excel
If you have broken the user story into three columns, one of which is the Role, then role based filtering/grouping is a snap.
Usability: High Downside: hard to have one record tell multiple stories (merged)

Mingle
When creating stories, you can add tags. So, for example, you might add a tag for role:Marketing to indicate that this user story is told from the perspective of the marketing stakeholder. Then, in any place where you can filter, you can filter to include only those items that have that tag.
Usability: Medium-High Downside: you have to remember to add the tag, and it does not support Highlight filtering, which can have undesirable side-effects

Rally
Like Mingle, you can add tags. Unfortunately, you can’t filter by those tags, so I’m not sure how good they are.
Usability: Medium Downside: Have to add the tag, doesn’t support any filtering on tags (that I can find)

VersionOne
You could co-opt the ‘Goal’ feature, and create a Goal for every Role in your backlog, and then associate them. This will allow you to have the many-to-many relationship that is needed, but then you couldn’t filter on it (I don’t think), you have to go to the Goals management screen, which is not ideal. And it is a hack. Or, you could define a ‘Role’ column, and pre-populate it with all the appropriate roles.
Usability: Medium Downside: If you forgot one, you need to have the administrator add it. And you can’t multi-select, so you can’t have multiple user stories in one backlog item, so you can’t merge them.

AgileOnDemand
Much like VersionOne, you can add a ‘Role’ field, and pre-populate it with options.
Usability: Low Downside: unfortunately, this doesn’t add the field to the WorkItem detail screen, so there is no easy way to set it. You can add it to the grid view, but then editing it is a bit tedious. Moreover, there is no clear way to filter by it.

Usability Comparison Summary on a scale of 1 to 10

So, in terms of breaking down, or more accurately, building up, your backlog, what is an epic? It’s more than just the result of functional decomposition of your intended application. It is a usage scenario, a process flow if you will.

One Epic Story will span many user stories, telling the adventure that a particular piece of data will go through as it is created, lives its life in the system, and finds its way to the end of its lifecycle after being handled by many people. One such Epic will tell the story of the “Happy Path,” the one that 80% of the time is the one that is taken. Other Epics will tell the story of the less common paths. It is the common theme of these Epic stories, what they have in common, that makes up a Theme.

Because Epics are a kind of process flow, multiple Epics may contain the same story. Which ever Epic is selected for implementation first will result in the stories that make it up being implemented. Later epics that are implemented will not (necessarily) need to do anything for some of the stories that make up that epic, but will involve implementing only those stories that do not yet exist in the system. This means that Stories and Epics have a many-to-many relationship.

Wait… many-to-many? How many of the tools actually support creating Epics in a many-to-many relationship? NONE. Yes, that’s right, NONE that I have evaluated, not even the Analog (3×5 card) method. How can that be?!

Chances are, it’s because the Analog can’t do it. Why can’t the analog do it? Because you have a physical card that can only be in one place at a time. The focus on automating the analog process has blinded the developers of the various tools to the advantage that a digital tool can have, namely that one ‘card’ can be in many places at the same time.

To make matters worse, the users don’t know how to express the fact that they need this to the tool vendors. They build documents OUTSIDE of their backlog management tool, to make up for the fact that the backlog management tool is not facilitating the true modeling of their system in this way. It doesn’t occur to them that EVERYBODY is doing this, for any project of any respectable size, and that this is an opportunity for the tools to help the product owners do a better job of defining what units of work really define “business value.”

You see, a feature, by itself, may not add business value. It’s only when several individual features are delivered together that the users can actually get WORK done. So in order to get business value, a collection of features need to be created in concert.

Problem is, the tools that allow grouping stories only allow those stories to belong to one group at a time. This precludes the creation of this kind of Epic story, that re-uses individual story cards time and time again. The end result? The tools actually encourage people to do functional decomposition. And there is NOTHING in the tool to allow the product owners to do the mapping between low-level features and high-level processes. The really good product owner will have to do it all OUTSIDE the tool, and map the end result into the tool. Less skilled product owners struggle to lead their teams to deliver value, because the tool has encouraged them to break down the work into something other than units of value-providing functionality.

There is something wrong, here.

You COULD define your user stories at a higher level. They could be the epic adventures, already. Sure. But what would you do with the user story that addresses one small piece of functionality, providing an important requirement of the system? You NEED to be able to capture those fine-grained requirements, but you also NEED to be able to capture and express the process flows. Agile tools, today, don’t really meet the product owner’s needs in that regard.

If Only…

If only there was a tool that understood this need. What would that tool be like?

Well, first of all, it would allow the user to create a hierarchical grouping that maps to stories in a 1:N relationship, and it would ship with one such grouping already defined, and called a ‘Feature Group’. It would also allow hierarchical groupings that map to stories in an N:M relationship, where a story could belong to multiple groups. It would ship with at least one such grouping: Epics, or Process Flows, or Use Cases, and maybe another, called ‘Theme’ that allows one Epic to be a part of multiple Themes. Two other groupings that would exist would be one for grouping stories by the Role, the person whose perspective the story is told from, and another that lets stories (or epics) be categorized by their impact on the bottom-line of the business. I’ll talk more about these other types of groupings, and how existing tools can or can not allow you to accomplish them, in a future installment of this series.

But here’s the important part: The ideal tool will understand the difference between a 1:N hierarchy and an N:M hierarchy, and will let the user create the one that the user needs. It will not impose upon the user by only allowing one such hierarchy to exist. After all, if you have created the code to allow such data relationships, why limit the user’s creativity by only letting them use that ability in one narrowly defined way?

Okay, sure, if you try to be all things to all people, chances are that you will suck equally to everybody. The reason that happens is because you reflect that generality in your user interface. So, you don’t provide the guidance and focus people need to be successful with your tool. So don’t do that!

Instead of building a single user interface for relating objects, you create a user interface that is optimized to the use case of creating feature groups in a 1:N relationship. Then you let the user rename the relationship type to whatever they want it to be called, AND you let them create ANOTHER relationship type, called whatever they want, that also describes a 1:N relationship. You let them define what the components are called, and how they aggregate, and all that fun stuff.

Then, instead of reusing the 1:N user interface for the N:M user interface, build a custom one, one that is more applicable to that use case. One that lets the user view the candidate objects (story cards or something else) in their 1:N hierarchy, or in some other N:M relationship, and drag and drop (for example) onto the hierarchy to assign membership. Then, again, you come pre-configured for certain useful N:M relationship types, but you give the user the ability to create their own.

Of all of the tools I evaluated, the only tool that comes close to this vision is Mingle. It is a totally generic, but very powerful tool that lets you define your own card types, and your own relationships between them. Unfortunately, they do not support the ability to define relationships where a particular card type (like a Feature Group) can have a hierarchical relationship, such as a feature-group containing other feature-groups. And it lacks the ability to make N:M relationships. Add to that the fact that they are a ‘generic’ tool, and the fact that the generic nature of the tool comes out in its user interface, and you find that Mingle still has a ways to go before it meets my idea of the ‘perfect’ tool.

In the 3×5 card world, you might paper-clip some together, binder-clip groups of paper-clipped groups together, you might place some groups physically near one another on purpose, or in clusters or groups withing confined areas of the desk or wall or floor. You may draw boundary lines in tape, and put stories or groups into one boundary or another. The Analog method, which all tools are meant to emulate, allows you at least three or four different grouping levels, possibly more if you get creative.

Some of the tools let you do as many levels of grouping as you like. Others do not. Some let you browse them hierarchically, others do not. Let’s take a look.

Analog (3×5 card) Method
Paper-clips, Binder-Clips, marked or defined areas, clusters within areas.
Usability: Medium Downside: hard to get the big picture without taking extra steps like summary cards or labels, and there is no (easy) search mechanism, hard to ‘browse’ the details if they are in piles or clipped together.

Ideal Digital Tool
Stories can be put into an effectively infinite level of nested groupings. Can expand/collapse groupings as needed to see as much or as little as you want.
Usability: High

Rally
Allows at least 5 levels of grouping, there is probably no practical limit. Can expand/collapse groupings as needed to see as much or as little as you want.
Usability: High Downside: difficulty managing the tree

VersionOne
Allows at least 5 levels of grouping, there is probably no practical limit.
Usability: Medium-High Downside: Can’t browse the hierarchy and look at the stories that are assigned to each grouping. Can filter by feature group, and display all stories at that level or lower, but you can’t quickly browse. Also, can’t view ONLY the stories in a particular group if there are stories in sub-groups.

Mingle
Requires each level of a hierarchy to have a defined card type. Scrum template offers Theme, Epic, and Story, giving two levels of grouping. Has a hierarchy view, and a tree view, for browsing.
Usability: Medium Downside: Can not create a ‘folder’ that can contain other ‘folders’. Also, the generic approach creates some confusing things, such as your themes and epics and stories and tasks all showing up as equals (in some views) and the default columns not making it so you can tell what is what.

Excel
To create one level of grouping, create one group column. To create two levels, create a second column. You MIGHT be able to do some custom programming to create a ‘group’ column and a ‘parent’ column and then navigate the tree with a custom form, but I’ve never tried.
Usability: Medium-Low No easy way to navigate, prone to errors or corruption

AgileOnDemand
Defines ONE level of grouping: Epic. Theme is a text field in the epic. You can’t put an epic inside an epic.
Usability: Low Tool imposes greatly on how you can organize your backlog

Usability Comparison Summary on a scale of 1 to 10

Analog (3×5 cards) method
First I fan out the cards in one group. I see one that I want to move. I pick it up, find the group it belongs in, and I set it there. Or something equally simple.
Usability: Medium-High Downside: may benefit from having more than two hands, and a search feature

Ideal Digtital Tool
While viewing the cards in one group, if I select one or more of them and drag them, I begin either a re-prioritize operation (if I drop them within the same group), or a reparent operation if I move to another group. While dragging, if I hover over a closed grouping, it will auto-open so I can see if that’s really where I want it to go. Moreover, before I drop it, I can drop it in the right position within the group, for ranking purposes. If I drop in a group that is auto-opened, it stays open. If I decide against dropping in that group, it auto-closes when I move off of it, but without disrupting my attempt to find the ideal home for them. NOTE: For accessibility and ease of use reasons, this process can be carried out via the keyboard, as well. Another option for those that have a hard time dragging: Right-click, Reparent (or move to another group), which shows a split screen, the items to be moved, and the group hierarchy to move them to. A single-click on the parent item selects it, click OK to move them.
Usability: High

AgileOnDemand
Find the stories you want to move. One by one, drag them to the new parent object and drop them.
Usability: Medium-High Downside: can only move one at a time. Can’t open a closed epic as you hover.

Excel
Find the name of the group you want to move the cards into. Copy the group name to the clipboard. Find the stories you want to move into that group. Paste the group name into the Group column.
Usability: Medium-High Downside: harder to find the right ones than with a hierarchical view

VersionOne
Check the checkbox next to the stories you want to move. Select the ‘Move to Feature Group’ menu item. Select the target, and click OK.
Usability: Medium-High Downside: can’t look at existing stories before putting the stories into the group

Mingle
Switch to the tree view. One at a time, drag-and-drop the stories to where they need to go.
Usability: Medium-High Downside: can only do it from one, not very compact, view, and can only do it one at a time

Rally
One-by-one, edit the Parent field of each item to be moved.
Usability: Low Downside: Exceedingly tedious, to the point of discouraging experimentation and change

Usability Comparison Summary on a scale of 1 to 10

Analog (3×5 cards)
Write the story on a card. Set the card on the appropriate pile.
Usability: High Downside: it may be hard to find the right pile if you have too many of them (no search feature)

Ideal Digital Tool
Find the group and select it. Create a new story card. By default, the selected group is the parent object for the new story, though I can always change it. OR, create a new story card without selecting a group, can select a group from the creation screen.
Usability: Very High

AgileOnDemand
Select the Epic or any story within the epic. Add a new story. The new story is automatically in that epic.
Usability: High Downside: there is actually no way to add it OUTSIDE of an epic unless you first select a story that is not in an epic, which is not self-evident.

Rally
Locate the object that will be the parent, and click the Add Child link. Save the new story. Or, if you click the New User Story action, then you need to select the Parent field before saving.
Usability: Medium-High Downside: no concept of ‘context’ for the Actions menu, and no right-click menu, must click the correct button on the far right.

Excel
Jump to the bottom. Create a new record. Make sure you populate the Group field correctly.
Usability: Medium-High Downside: Gotta spell the group correctly, though filtering before adding makes that easy

VersionOne
Add a new Story. In the Feature Group field, select the parent grouping. Save.
Usability: Medium Downside: no concept of ‘context’. In fact, the Feature Group tree is seldom viewable as a tree, and never with inline stories.

Mingle
From the Tree view, click the Add Child link for the parent object. In the popup, enter the summary for the story. Then, find the newly created card, and click it to display a popup. From there, click the Open link (yes, you can’t double-click it or otherwise edit in one operation), and from there you can enter the user story text. OR, from the List view, if you Add With Details, you can find the relationship field (it’s not obvious, it’s marked: “Feature – Epic Story”) and click the (not set) link, click ‘Select’, then use the filtering tools to find the story to make the parent.
Usability: Medium-Low Downside: too many clicks caused by the fact that this is a generalized tool – powerful, but not optimized

Usability Comparison Summary on a scale of 1 to 10