Category Archives: Requirements

How much for that Gorilla in the window? – Assigning value to requirements

Posted on by
Reply

Hogarth shoveled another handful of popcorn into his mouth (Yes, gorillas like popcorn – read the right sidebar, go figure). He was sitting at the end of the table, size huge feet propped up on the table and looked as if he were thoroughly enjoying himself.

Which couldn’t be said for me. I turned back to the discussion at the table, wishing a crack in the earth would open and swallow me up.

“Every one of them is a P Zero feature.” said Tully, the junior product manager. Though I swear I could see Bob’s lips moving as Tully spoke.

We were in the Icarus project’s concept phase exit meeting. Product management had just presented their list of requirements for Icarus. The entire list was up on the projector screen, two columns wide and in ten point font. I had surreptitiously taken a photo with my phone and was zooming in just so I could read the list.

“But which one is the highest priority?” Jake asked. This meeting was where engineering saw the requirements for the first time and started looking at what could be done. Feeling like I was in a tennis match, I let my eyes swing back toward the two product managers.

“All 100 features are must ship, they are the same priority.”

I was starting to feel like I was witnessing the start of a train wreck and there wasn’t a damn thing I could do about it.

“Look, even if we could do all these features in the release, which we can’t, you still would need to prioritize them so you can flesh out the top of the backlog. None of these are good enough to start developing on.” Jake was keeping his voice surprisingly calm.

Bob had apparently gotten tired of the ventriloquist act and spoke up. “Are you saying you’re only going to estimate the top items?”

Jake sighed, shaking his head. “No, we can give you a relativistic estimate on everything, with this data. But none of these features is defined enough for the team to start work. I’ve got no idea what to start on in the first sprint.” He laid his hands on the table, “Look, if you can’t prioritize them then you have to do full user stories on all of them and I’ll just pick which one to do first. That’s fine by me.”

“But we can’t do that, it would take weeks to flesh out all these features, and besides the requirements might change.”\

Jake shrugged. “Then we need to find a way to rank order them. I can do effort estimates on all of them and just start on the easiest ones first.”

Oh boy, Jake was playing hardball now. Bob leaned over his laptop with a pained expression. “The easy stuff isn’t what the customer wants right now.”

Jake leaned back. “Well at least we’re getting somewhere.”

At this rate, this was going to be a long meeting and it wasn’t going to solve anything. Against my better judgment, I turned to cast an imploring look at Hogarth. Wiping his popcorn encrusted mouth with the back of a furry arm, he reached into the popcorn bucket and pulled out a deck of Fibonacci cards.

“It works for story points…” He offered.

 

What do we do first?

Today we talk about one of the fundamental problems with product requirements. Last week’s“Garbage In, Gorilla Out”focused just on the need for good requirements. This is important, vitally so. A well executed project, built on bad requirements is a bad product (Can you say HP Web Tablet? I knew you could)

The problem is, then what?

If I have one hundred perfectly defined features what do I do with them now? I can hand them off to the development team so that they can give me a level of effort (Story Points in Agile). That’s great, but now I have one hundred perfect features with story points. I still don’t know which one to do first.

Business value: What is the value of this benefit to the customer and/orthe business? This can also be asked in reverse, “what will it cost us to not implement this?” (Kanban tends to look at it from this view point). If you don’t know what the value of the feature/user story then you don’t know if it is more or less important than the next feature. You then end up in the model some old style engineering managers love. “Just give me the list of what you want, I’ll figure out what I have time to build.”

Technically perfect and customer flawed.

Apple has constantly focused on the value. Products they delivered did exactly what you wanted them to do (sometimes even doing things you didn’t know you wanted, but now can’t imagine how you would ever live without). Being a PC guy, this has sometimes been a hard pill to swallow. I’ve seen the exact opposite with PC products. “Do I really need 7mm instead of a 9.5mm hard drive? What I would really like is it if it included Bluetooth so I didn’t have to have a separate set of headphones just for my computer.” PCs have been really great at offering up new features that had no real infrastructure to use right away while lagging on basic,s like user interface.

Great, I’m sold already, how?

Ah and there in lies the rub. How do you figure out business value? Raise your hand if you’ve seen a product management forecast you believed in. Yeah, thought so. So if we have trouble figuring out how much we’ll sell of the whole product, then how do we decide the value of a single feature/user story? Some companies have put together very elaborate procedures for this that practically call for a Ph.D. in math. Reportedly Intel has eighteen value “levers” used to evaluate not just projects but features in a project. Yikes.

Reaching into the agile pool isn’t a lot of help either. While a whole blog of its own, the core issue with the most common estimating technique,Planning poker, is that it focuses on one feature at a time. And the problem with that is the human mind. Say I’m playing planning poker on a home improvement project. The user story is “As the home owner I want a blue exterior because it will make my house blend in with the skyline.” Okay, painting the house. No matter how hard I try I’m going to be thinking about how many hours I’m going to need. When I flip over the 13 I’ve just mentally associated three days with 13. So of course a one day project is a 5, right?

When you evaluate each feature on its own, it is very hard for the human mind to not associate time with it. 

So don’t…

Steve Bockman developed an estimating process called “The Team Estimating Game.” He and Agile Learning Labs have been using the game for years, teaching teams and companies this new way of estimating story points. Last year, Agile Learning Lab’s maven Laura Powers realized that it could be used for assigning Business Value. 

And?…

And with permission of Agile Learning Labs, I’m sharing their excellent game. After attending a Product Owner Training, I wrote down what I learned and have been using it ever since.
The Exercise:Take the defined user stories. These can be at any level of detail, but obviously it is better if they have some basic story decomposition done and they are clearly understood by the product owner and other business stakeholders. It is best to start with no more than twenty stories. After completing the game with these stories, future stories are then played against this initial set of stories. This makes subsequent evaluation go quicker and allows for estimating the value of hundreds of stories.
Phase 1:The first step is to estimate their value relative to each other:
  1. Place Story Cards in pile on table.
  1. First player places top card on playing surface.
  1. The second player places the top card on playing surface relative to first card. They decide if the card is worth more business value or less than the original story.
  1. The third player then has several choices. They can either:
* Play top card from pile, placing it on the field where they think it fits (moving other stories to fit between stories as needed), or
* Move a card on the playing surface from one location to another, or
* Pass
Note: When placing or moving a card the person conducting the action typically gives a short explanation. So long as it does not derail the conversation, team discussion is  encouraged. The Agile Coach/Facilitator monitors this and if a conversation starts to deep end they ask “So whose turn is it now.”
  1. Repeat Step 4 until
a) no more cards remain in pile, and
b) no player wishes to move a card
At the end of the first phase of the game you have a single, linear line of user story cards. The cards to the left are deemed the least valuable and the cards to the right are the most. No overlap of cards occurs at this time.
Phase 2:Having ranked each story to create a single ordered list, it is now time to assign a value rating.
Start with a standard deck Fibonacci numbers. Typically up to 144 is more than sufficient. Add to this the (-) card, to represent stories that have no business value. Note: Business value may not represent the only value in a project. You may have regulatory concerns, infrastructure concerns, etc. Whenever possible try to assign a business value to the story. This usually requires looking at the operational This may require asking the “what do we lose if this is not here?” Zero business value should be truly something that will not be missed in anyway if it is not in the product (“No one really wants a wood stove in their car”).
  1. Place the (-) card to the left of the left most story. Anything that moves left of this card will be considered of no business value.
  1. Place the “1” Card over the left most story. This story now represents your baseline that all other stories are measured against.
  1. Hand the deck to the first player. The player takes the “2” card from the deck and then looks at the stories to the right of the “1” card. They place the card at the story they think is twice as valuable as the first story.
  1. The third player, then has several choices. They can either:
* Place the “3” card to the right of the “2” card where they think a story is either three times more valuable than the first story or is one and a half times more valuable than the story with the “2” over it.
* Move the “2” card left or right to where they think the user story is equal to twice the value of story one.
* Pass
Note: Fibonacci Cards are placed on the table in order. You do not place the “21” card until the “13” card has been placed.
Note: Any stories between Fibonacci cards are assumed to belong to the Fibonacci number to its left. So if there are three stories between the “5” and the “8” then it is assumed those three cards are considered a value of “5.”
  1. Repeat Step 4 until
    1. All stories have an assigned value (not all Fibonacci cards must be played), and
    1. All players pass on adjusting any of the Fibonacci cards.
Concluding:At this point any “orphan” cards, those between Fibonacci cards, are then moved into columns below the Fibonacci Number to their left. You are now left with a number of columns of cards (there is no right number).
The exercise is done. Capture the value for each story in the appropriate manner.
Important Note:The Coach/Facilitator should watch for ranking based on cost. This exercise should be conducted with the “infinite budget” concept. “Don’t worry about how hard it is to build right now, just worry about how valuable it will be.”
Important Note:If you are planning to have these stories estimated for cost (time to develop), it is recommended that the business value not be shared with the team doing the cost estimating. This prevents the cost estimating being skewed by the Business Value.

Garbage In, Gorilla Out- Know "What" before "Do"

Posted on by
1

“What do you mean he said it’s crap?!?” Jake’s voice thundered. I’m pretty sure there was steam coming out of his ears. I could feel my own kettle on a rapid boil myself so Jack must have been at nuclear meltdown stage.  I turned and cast a sympathetic eye down the table. Poor, Bob, our hapless product manager looked like he’d just been served up as the main course at a great white shark feeding frenzy.

I’ll give him credit though; Bob didn’t cower in the face of our rampaging development manager. I mean after all, it wasn’t his fault, he was just the messenger. Bob nodded his head gravely, “he did. He went on to say it was lifeless, had no vision, was poorly put together and had no business being on the store shelf.”  Ouch… Fiat Armburger was the man. If he gave your product a pass, you could name your price with the channel. But if he panned you, might as well start selling the office furniture now.

Jake shook his head, a look of complete disbelief washing over him. I don’t think I’d ever seen him look so defeated. “We executed perfectly. We even shipped three months early. Hell! We delivered everything you asked for in.” His eyes turned back to Bob at the last words.

“I know, I know,” said Bob. “What can I say, we missed the mark. We’ll just have to try harder the next time.”

I jumped to my feet. “Well now,” I interjected, because if I didn’t Jake was going to levitate across the room and kill Bob with his eyeballs alone. “We have a stack of bug reports here. Bad review or not, we’ve got to fix these issues before GA. After all, we still have to ship, the schedule is committed.”

It worked, everyone calmed down and we got back to making the product ready to ship. I don’t know about anyone else, but all I could really do is wonder whether anyone would even buy it.

“Nope, it’ll be the worst launch since Ishtar.”

Great, just what I needed. A failing product and now a gorilla who thinks he’s a movie buff. Why me?

“Well cause it’s your fault.” Hogarth offered up.

I spun my chair to face my personal gorilla. He was lounging on the wall contemplating his naval (yes gorillas have belly buttons, Hogarth’s was large enough to misplace loose change in). “How on earth is it my fault?”

Hogarth looked up, producing a quarter from his naval. “Well technically it’s Bob’s fault, but come on doesn’t it get boring always blaming  the product manager?”

“Hogarth, explain to me how this is my Bob’s fault much less mine?” Though the idea of it being Bob’s fault was certainly more appealing to me.

Hogarth smiled, “G.I.G.O.” 

“I… what… ” I stammered. Then my mind realized what he meant. “Oh… my george…”

 

Garbage In, Garbage Out:

GIGO is a term that reaches back to the dawn of the commercial computer age. It’s a fundamentally simple fact that states  “computers  will unquestioningly process the most nonsensical of input data (“garbage in”) and produce nonsensical output (“garbage out”).” The overall concept goes back even father to early “computer” inventor Charles Babbage (now seeing a resurgence in fame thanks to the Steam Punk phenomenon) .

It’s hard to argue with the concept, in fact the software programmers I know wouldn’t even try. To them it’s like trying to say “Brick+Music=42”. It just doesn’t work. You have to use good inputs if you ever want to have good outputs.

So then, why the heck do we think the magic solution is in the development soup?

For the sake of this discussion, we’re going to take a scrum team. This problem applies to any kind of development process, from BDUF to XP. I’m using scrum because let’s face it, some of us have held it up as the golden ticket to solve all ills. So let’s see what happens when the golden ticket uses lead to develop.

The official Scrum Guide is seventeen pages long. Of those seventeen pages, about one page is devoted to the concept of a Product Backlog. This page covers that the product backlog is the Product Owner’s responsibility, that it is important, that higher stories have more details, that it should be rank ordered and that it should be regularly groomed.

You can groom a Canadian Hairless cat all you want, he’s still not going to win any beauty contests. A scrum team needs a good product backlog (requirements) to work from. If they don’t have a good backlog, then  they can ship the project on time, on budget and it will still fail because it was a bad idea to start with. New Coke failed because there wasn’t a need for it, everyone was happy with the old coke.

GIGO– If you don’t take the time to figure out what you want, then you won’t get it.

But, but that’s not agile! You’re saying I have to do big planning up front! See I knew agile was a crock.

Yeah, no…

You don’t have to document the Library of Congress, before you start a project, but you need to make sure you at least understand what the customer wants and have an end vision in mind. You then have to make sure that when the developers are getting ready to start coding, they understand EXACTLY what is wanted for that functionality. You can absolutely be agile in your product planning process. In fact, you can just be agile in planning and I think you’ll get a better project, even if development still builds in “waterfall.”

The Gorilla Planning Outline:

I found much of my inspiration in Jim Highsmith’s Agile Project Management: Creating Innovative Products. I heartily recommend the book for anyone involved in project planning or project management. Another good bit of inspiration came from Agile Learning Labs, a really great agile coaching and training shop in the San Francisco Bay Area. If I have stood farther, it is because I have stood on the shoulders of giants (Isaac Newton).

Note: This is an outline only. Within each stage I promote agile based exercises to get to the end goal.

The Cast:

Product Manager/Owner

Product Sponsor

Customer Voice/Representative

Architect*- Optional

Agile Coach/ Facilitator

*- An architect level technical person can provide valuable insight to this stage of the project. It is vital though to ensure this person(s) can separate business from technical constraints. An architect who is constantly focused on the how and how long will not add value to the process during this stage. The exception is in Stage 6, Estimate the Cost.

Stage 1: Defining the Product

Purpose: Create the “product box.”

Time Needed: Half Day

Description: Ensuring everyone thinks that blue is really blue is a fundamental starting point. If the product owner things you are building a Ford Pickup and the Sponsor is envisioning a Land Rover, you have a disconnect. This stage of product planning focuses on the overall corporate goals and customer “box” experience. The “box” experience is the level of experience you get when you pick a product up off the shelf and look it over. What compels the customer to buy? The core of these concepts comes from Jim Highsmiths’ chapter on Envisioning from Agile Project Management.

Stage 2: Layout the Product Story Outline

Goal: Create a timeline of the product’s use and identify missing gaps.

Time Needed: Half day to full day.

Description: “How did we forget that feature?” has been a long asked question, typically so far into the development process as to require derailing the project to get the feature in. Story Mapping is a process that discovers the users, the end to end experience and then allows you to drive into the highest level of user stories but doing vertical break downs.

Stage 3: Create the Chapters (User Stories)

Goal: To create clear and concise descriptions of user experience with the product.

Time Needed: Oneweek offline work, after Stage 2 and then half day.

Description: A poorly written user story can lead to the implementation of something that is technically perfect and a complete failure as it doesn’t meet the needs of the user, as envisioned by the product owner. A critical component of this stage is that even the highest level stories need a basic acceptance test to validate against.

Stage 4: Decompose the User Stories

Goal: Break large user stories down into iteration obtainable goals.

Time Needed: Half day (can be done with Stage 3 to save some time overall)

Description: One of the largest challenges in the creation of a backlog is to go just far enough with the details of a user’s experience and not to far that you’re creating a functional feature spec. Product Managers often struggle to not provide so much data that they are directing the engineers to a specific implementation – which can both stifle a creative new idea and can upset the engineers who feel they are being dictated to. Engineers who get too vague a story are faced with many challenges, not the least being “what exactly are we building,” “If I’d know that originally, I would have estimated this as a much bigger story,” and “This will take at least six weeks, I can’t do it in a four week sprint.”

By breaking user stories down to iteration manageable levels the team is able to better estimate value and cost (time to complete), removes confusions that can result in a feature that doesn’t match the vision of the product owner, allows for shorter iterations – which itself improves the overall development process by allowing more chances to apply the “inspect and adapt” philosophy of agile.

Stage 5: Value the User Stories

Goal: To create a relative ranking of all user stories on a project.

Time Needed: One to two hours

Description: P0, P1, P2 ultimately breaks down. Fifteen years ago I never even heard of P0 and you might actually see a P3 in a product. Today you see projects where everything is a P0 and engineering says it will take two years.

Doing force ranking, feature weighting and other traditional techniques rarely leads to the level of consensus required to create a final ranked list. The realistic question to ask teams, at this stage, is how often estimating against real world artifacts has worked. When was the last time a product forecast was accurate? Using agile values opens up the process to creating a system that creates agreement not conflict.

NOTE: This is one of the secret sauces of this process. In my next blog I document an excellent exercise for how to define business value for an entire release in just a couple of hours.

Stage 6: Estimate the Effort (Cost)

Goal: Rank the user stories in relative order of effort (cost to produce).

Time Needed: One to two hours

Description: Classic estimating of user stories. It needs to be done on even the grossest level stories for the product owner to be able to properly rank order the backlog. Otherwise a backlog can be skewed towards value, costs or even worse, gut judgment. Poker Planning and Delphi models have an inherent flaw of focusing on the story itself which leads to exact effort and not rank ordering estimates.

The Cast: This stage does not involve the product owner/stakeholder cast. Effort estimating is done by the engineers doing the actual work. It is important to not just have management at this stage but engineers doing the functional work.

Note: The developers should not have knowledge of the business value rating at this stage. They are just assessing the complexity of the future. If they know a particular feature is the absolute silver bullet, they may over or under estimate it.

Stage 7: Order the Requirements List

Goal: One backlog to rule them all.

Description: At this point the Product Owner has both a business value and a cost. These are both placed on the user stories. From here the product owner then uses their judgment and skill to create the product backlog. They weigh the value and cost of a story (feature) to create the backlog.

Note: In agile development, best practice is to groom the backlog once iteration with the product team. This is where cost can be reevaluated, better definitions of stories can be done and the product backlog reordered as a result.