“10 Days” Eric said.
I sighed. “Eric, you’re supposed to use the planning cards and put down a value.”
As he flipped through his stack of cards I could see him counting to himself. “Okay, fine. I think this is an 89 point task”
Sigh… “Eric, you just counted out ten cards and put that card down.”
I leaned forward and tried to use my calmest voice. “Eric, we’re not trying to figure out how many days right now, we’re trying to do an abstract estimate on the relative size of each story in the whole project. “
He waved his hand, “What’s the point? If we’re painting a house and you ask me to estimate how long it will take to paint the garage door I can put down a 2 or just tell you two hours. You’re asking me to use an abstract number when I already know how long it will take.”
I sat back in my chair perplexed. The problem was that I could see his point. We were tackling each task one at a time and no matter what I did, I couldn’t break the team out of thinking about how many days something would take. And I knew this was a slippery slope that led back to padding, swags and a whole slew of inefficient planning practices.
A long meeting later I dragged myself into my office. Only to find both my chair and the guest chair occupied. One of the occupants I expected to be there, a post meeting debrief on this last meeting was much to much for my personal gorilla to miss. Seeing the black swan flipping playing cards onto my desk was quite another thing.
Flip, flip, flip “144!” Hogarth said, triumphantly. Looking at the black feathered swan he grinned. “Hah, that beats your 21.”
“Hogarth!?” You’d think I would get tired of yelling his name. Or maybe I would just get used to his ever present level of restrained chaos. You’d think… “What are you doing with my Planning Poker decks?”
“Oh, hey there,” Hogarth turned his cheerful smile my way. “Wanda and I were just playing war.”
Tossing her last card, a 2, down Wanda slid off the chair. “I have to get back to the server room anyway, one of the HVAC’s is about to short out because a gopher climbed on the roof and ate through the power cable.”
Trying not to think of electrocuted gophers and overheating servers, I ignored the departing black swan and turned my attention back on Hogarth. “Why are you playing war with my Fibonacci cards?”
“Oh, that. Well I figured you might want them and then we just kind of got bored waiting for you to be done not estimating the release.”
I blinked at him. “Hogarth, I know you were eavesdropping on the meeting. You know perfectly well the team thinks planning poker is useless.” I tossed myself into the vacated guest chair with a sigh. “And I’m not sure I don’t disagree with them.”
“Since when have you believed there’s only been one way to use a tool?”
Blink, blink… Why did he always have to be right?
Story Point Estimating
This is one ofthe foundational principles of Scrum software development (and Agile in general). Done well, story estimating can create an incredible level of predictability and transparency. A high performing team can look at a set of requirements and provide a high confidence assessment of when they will done.
But this isn’t about the value or use of estimating. There is plenty written on this and I’m not here to rehash it. I believe in estimating, just as I believe in the principles of agile and good management.
What I’m here to say is I think Planning Poker is the wrong way….
What? Come on, I’d be a lousy gorilla talker if I wasn’t willing to tackle tough subjects head on. I do absolutely think story point estimating is the right thing. I just think planning poker is not the best way to do it.
Naturally this begs the question of “why?” I have two reason for my concerns on the use of planning poker:
- Each user story (feature, tasks, item) is evaluated on its own. When you do planning poker you are just looking at how long it will take to paint the garage door. You are hard pressed to look at it in relation to putting a new latch on the back gate. I think the mind is hard pressed not to assign a physical value to the estimate. “I think this will take two hours, that’s about a 3.”
- At the end of the day, the planning poker process occurs in an individuals head. Yes, there is a discussion among the team, but only after each person has drawn a line in the sand. Once the human mind draws a line, it finds it hard to move that line. We’ve invested ourselves in that line.
So what the heck do you do?
Well if you’ve been following my recent agile related blogs, you’ll know I’m a very big fan of something called the “Team Estimation Game” developed by Steve Bockman. I personally like to call it “Team Planning Solitaire.” The style of the game reminds me of classic Klondike solitaire and the interaction of the team makes it anything but a solo activity (what can I say, I like irony). I will also note that while I learned this from Steve Bockman and Agile Learning Labs , I have seen some similar exercises in the last few months. Whether they are parallel development or evolutions of Bockman’s game, I don’t know.
I detailed how this game worked in “How much is that Gorilla in the window.”
At the highest level, the process starts with the story cards in a pile. Each person takes a turn laying out a new card or moving an existing card on the table. At the end, you have a line of cards in rank order. Only then do you break out the planning poker cards. Your estimates end up being based on the relation to the other stories, not to how long one story will take.
Why is the Team Estimating Game so good?
- Stories are estimated in relation to one another and not in a vacuum. “Is painting the garage door more or less effort than replacing the latch on the back gate? All right, is painting more or less effort than rehanging the front door.”
- It’s a team activity. Nothing happens in anyone’s head. It is all out on the table and very straight forward. You aren’t sitting there arguing that the Database re-architecture will take a week or two weeks. You are just trying to decide if it is more or less effort that localizing the user interface to Japanese.
At the end of the day I think Team Estimating is much more effective than planning poker. Leave the poker cards for a nice game of Klondike or Texas Hold ‘Em.
Who’s ante is it?
Hogarth shoveled another handful of popcorn into his mouth (Yes, gorillas like popcorn – read the right sidebar, go figure). He was sitting at the end of the table, size huge feet propped up on the table and looked as if he were thoroughly enjoying himself.
Which couldn’t be said for me. I turned back to the discussion at the table, wishing a crack in the earth would open and swallow me up.
“Every one of them is a P Zero feature.” said Tully, the junior product manager. Though I swear I could see Bob’s lips moving as Tully spoke.
We were in the Icarus project’s concept phase exit meeting. Product management had just presented their list of requirements for Icarus. The entire list was up on the projector screen, two columns wide and in ten point font. I had surreptitiously taken a photo with my phone and was zooming in just so I could read the list.
“But which one is the highest priority?” Jake asked. This meeting was where engineering saw the requirements for the first time and started looking at what could be done. Feeling like I was in a tennis match, I let my eyes swing back toward the two product managers.
“All 100 features are must ship, they are the same priority.”
I was starting to feel like I was witnessing the start of a train wreck and there wasn’t a damn thing I could do about it.
“Look, even if we could do all these features in the release, which we can’t, you still would need to prioritize them so you can flesh out the top of the backlog. None of these are good enough to start developing on.” Jake was keeping his voice surprisingly calm.
Bob had apparently gotten tired of the ventriloquist act and spoke up. “Are you saying you’re only going to estimate the top items?”
Jake sighed, shaking his head. “No, we can give you a relativistic estimate on everything, with this data. But none of these features is defined enough for the team to start work. I’ve got no idea what to start on in the first sprint.” He laid his hands on the table, “Look, if you can’t prioritize them then you have to do full user stories on all of them and I’ll just pick which one to do first. That’s fine by me.”
“But we can’t do that, it would take weeks to flesh out all these features, and besides the requirements might change.”\
Jake shrugged. “Then we need to find a way to rank order them. I can do effort estimates on all of them and just start on the easiest ones first.”
Oh boy, Jake was playing hardball now. Bob leaned over his laptop with a pained expression. “The easy stuff isn’t what the customer wants right now.”
Jake leaned back. “Well at least we’re getting somewhere.”
At this rate, this was going to be a long meeting and it wasn’t going to solve anything. Against my better judgment, I turned to cast an imploring look at Hogarth. Wiping his popcorn encrusted mouth with the back of a furry arm, he reached into the popcorn bucket and pulled out a deck of Fibonacci cards.
“It works for story points…” He offered.
What do we do first?
Today we talk about one of the fundamental problems with product requirements. Last week’s“Garbage In, Gorilla Out”focused just on the need for good requirements. This is important, vitally so. A well executed project, built on bad requirements is a bad product (Can you say HP Web Tablet? I knew you could)
The problem is, then what?
If I have one hundred perfectly defined features what do I do with them now? I can hand them off to the development team so that they can give me a level of effort (Story Points in Agile). That’s great, but now I have one hundred perfect features with story points. I still don’t know which one to do first.
Business value: What is the value of this benefit to the customer and/orthe business? This can also be asked in reverse, “what will it cost us to not implement this?” (Kanban tends to look at it from this view point). If you don’t know what the value of the feature/user story then you don’t know if it is more or less important than the next feature. You then end up in the model some old style engineering managers love. “Just give me the list of what you want, I’ll figure out what I have time to build.”
Technically perfect and customer flawed.
Apple has constantly focused on the value. Products they delivered did exactly what you wanted them to do (sometimes even doing things you didn’t know you wanted, but now can’t imagine how you would ever live without). Being a PC guy, this has sometimes been a hard pill to swallow. I’ve seen the exact opposite with PC products. “Do I really need 7mm instead of a 9.5mm hard drive? What I would really like is it if it included Bluetooth so I didn’t have to have a separate set of headphones just for my computer.” PCs have been really great at offering up new features that had no real infrastructure to use right away while lagging on basic,s like user interface.
Great, I’m sold already, how?
Ah and there in lies the rub. How do you figure out business value? Raise your hand if you’ve seen a product management forecast you believed in. Yeah, thought so. So if we have trouble figuring out how much we’ll sell of the whole product, then how do we decide the value of a single feature/user story? Some companies have put together very elaborate procedures for this that practically call for a Ph.D. in math. Reportedly Intel has eighteen value “levers” used to evaluate not just projects but features in a project. Yikes.
Reaching into the agile pool isn’t a lot of help either. While a whole blog of its own, the core issue with the most common estimating technique,Planning poker, is that it focuses on one feature at a time. And the problem with that is the human mind. Say I’m playing planning poker on a home improvement project. The user story is “As the home owner I want a blue exterior because it will make my house blend in with the skyline.” Okay, painting the house. No matter how hard I try I’m going to be thinking about how many hours I’m going to need. When I flip over the 13 I’ve just mentally associated three days with 13. So of course a one day project is a 5, right?
When you evaluate each feature on its own, it is very hard for the human mind to not associate time with it.
Steve Bockman developed an estimating process called “The Team Estimating Game.” He and Agile Learning Labs have been using the game for years, teaching teams and companies this new way of estimating story points. Last year, Agile Learning Lab’s maven Laura Powers realized that it could be used for assigning Business Value.
- Place Story Cards in pile on table.
- First player places top card on playing surface.
- The second player places the top card on playing surface relative to first card. They decide if the card is worth more business value or less than the original story.
- The third player then has several choices. They can either:
- Repeat Step 4 until
- Place the (-) card to the left of the left most story. Anything that moves left of this card will be considered of no business value.
- Place the “1” Card over the left most story. This story now represents your baseline that all other stories are measured against.
- Hand the deck to the first player. The player takes the “2” card from the deck and then looks at the stories to the right of the “1” card. They place the card at the story they think is twice as valuable as the first story.
- The third player, then has several choices. They can either:
- Repeat Step 4 until
- All stories have an assigned value (not all Fibonacci cards must be played), and
- All players pass on adjusting any of the Fibonacci cards.
“What do you mean he said it’s crap?!?” Jake’s voice thundered. I’m pretty sure there was steam coming out of his ears. I could feel my own kettle on a rapid boil myself so Jack must have been at nuclear meltdown stage. I turned and cast a sympathetic eye down the table. Poor, Bob, our hapless product manager looked like he’d just been served up as the main course at a great white shark feeding frenzy.
I’ll give him credit though; Bob didn’t cower in the face of our rampaging development manager. I mean after all, it wasn’t his fault, he was just the messenger. Bob nodded his head gravely, “he did. He went on to say it was lifeless, had no vision, was poorly put together and had no business being on the store shelf.” Ouch… Fiat Armburger was the man. If he gave your product a pass, you could name your price with the channel. But if he panned you, might as well start selling the office furniture now.
Jake shook his head, a look of complete disbelief washing over him. I don’t think I’d ever seen him look so defeated. “We executed perfectly. We even shipped three months early. Hell! We delivered everything you asked for in.” His eyes turned back to Bob at the last words.
“I know, I know,” said Bob. “What can I say, we missed the mark. We’ll just have to try harder the next time.”
I jumped to my feet. “Well now,” I interjected, because if I didn’t Jake was going to levitate across the room and kill Bob with his eyeballs alone. “We have a stack of bug reports here. Bad review or not, we’ve got to fix these issues before GA. After all, we still have to ship, the schedule is committed.”
It worked, everyone calmed down and we got back to making the product ready to ship. I don’t know about anyone else, but all I could really do is wonder whether anyone would even buy it.
“Nope, it’ll be the worst launch since Ishtar.”
Great, just what I needed. A failing product and now a gorilla who thinks he’s a movie buff. Why me?
“Well cause it’s your fault.” Hogarth offered up.
I spun my chair to face my personal gorilla. He was lounging on the wall contemplating his naval (yes gorillas have belly buttons, Hogarth’s was large enough to misplace loose change in). “How on earth is it my fault?”
Hogarth looked up, producing a quarter from his naval. “Well technically it’s Bob’s fault, but come on doesn’t it get boring always blaming the product manager?”
“Hogarth, explain to me how this is my Bob’s fault much less mine?” Though the idea of it being Bob’s fault was certainly more appealing to me.
Hogarth smiled, “G.I.G.O.”
“I… what… ” I stammered. Then my mind realized what he meant. “Oh… my george…”
Garbage In, Garbage Out:
GIGO is a term that reaches back to the dawn of the commercial computer age. It’s a fundamentally simple fact that states “computers will unquestioningly process the most nonsensical of input data (“garbage in”) and produce nonsensical output (“garbage out”).” The overall concept goes back even father to early “computer” inventor Charles Babbage (now seeing a resurgence in fame thanks to the Steam Punk phenomenon) .
It’s hard to argue with the concept, in fact the software programmers I know wouldn’t even try. To them it’s like trying to say “Brick+Music=42”. It just doesn’t work. You have to use good inputs if you ever want to have good outputs.
So then, why the heck do we think the magic solution is in the development soup?
For the sake of this discussion, we’re going to take a scrum team. This problem applies to any kind of development process, from BDUF to XP. I’m using scrum because let’s face it, some of us have held it up as the golden ticket to solve all ills. So let’s see what happens when the golden ticket uses lead to develop.
The official Scrum Guide is seventeen pages long. Of those seventeen pages, about one page is devoted to the concept of a Product Backlog. This page covers that the product backlog is the Product Owner’s responsibility, that it is important, that higher stories have more details, that it should be rank ordered and that it should be regularly groomed.
You can groom a Canadian Hairless cat all you want, he’s still not going to win any beauty contests. A scrum team needs a good product backlog (requirements) to work from. If they don’t have a good backlog, then they can ship the project on time, on budget and it will still fail because it was a bad idea to start with. New Coke failed because there wasn’t a need for it, everyone was happy with the old coke.
GIGO– If you don’t take the time to figure out what you want, then you won’t get it.
But, but that’s not agile! You’re saying I have to do big planning up front! See I knew agile was a crock.
You don’t have to document the Library of Congress, before you start a project, but you need to make sure you at least understand what the customer wants and have an end vision in mind. You then have to make sure that when the developers are getting ready to start coding, they understand EXACTLY what is wanted for that functionality. You can absolutely be agile in your product planning process. In fact, you can just be agile in planning and I think you’ll get a better project, even if development still builds in “waterfall.”
The Gorilla Planning Outline:
I found much of my inspiration in Jim Highsmith’s Agile Project Management: Creating Innovative Products. I heartily recommend the book for anyone involved in project planning or project management. Another good bit of inspiration came from Agile Learning Labs, a really great agile coaching and training shop in the San Francisco Bay Area. If I have stood farther, it is because I have stood on the shoulders of giants (Isaac Newton).
Note: This is an outline only. Within each stage I promote agile based exercises to get to the end goal.
Agile Coach/ Facilitator
*- An architect level technical person can provide valuable insight to this stage of the project. It is vital though to ensure this person(s) can separate business from technical constraints. An architect who is constantly focused on the how and how long will not add value to the process during this stage. The exception is in Stage 6, Estimate the Cost.
Stage 1: Defining the Product
Purpose: Create the “product box.”
Time Needed: Half Day
Description: Ensuring everyone thinks that blue is really blue is a fundamental starting point. If the product owner things you are building a Ford Pickup and the Sponsor is envisioning a Land Rover, you have a disconnect. This stage of product planning focuses on the overall corporate goals and customer “box” experience. The “box” experience is the level of experience you get when you pick a product up off the shelf and look it over. What compels the customer to buy? The core of these concepts comes from Jim Highsmiths’ chapter on Envisioning from Agile Project Management.
Stage 2: Layout the Product Story Outline
Goal: Create a timeline of the product’s use and identify missing gaps.
Time Needed: Half day to full day.
Description: “How did we forget that feature?” has been a long asked question, typically so far into the development process as to require derailing the project to get the feature in. Story Mapping is a process that discovers the users, the end to end experience and then allows you to drive into the highest level of user stories but doing vertical break downs.
Stage 3: Create the Chapters (User Stories)
Goal: To create clear and concise descriptions of user experience with the product.
Time Needed: Oneweek offline work, after Stage 2 and then half day.
Description: A poorly written user story can lead to the implementation of something that is technically perfect and a complete failure as it doesn’t meet the needs of the user, as envisioned by the product owner. A critical component of this stage is that even the highest level stories need a basic acceptance test to validate against.
Stage 4: Decompose the User Stories
Goal: Break large user stories down into iteration obtainable goals.
Time Needed: Half day (can be done with Stage 3 to save some time overall)
Description: One of the largest challenges in the creation of a backlog is to go just far enough with the details of a user’s experience and not to far that you’re creating a functional feature spec. Product Managers often struggle to not provide so much data that they are directing the engineers to a specific implementation – which can both stifle a creative new idea and can upset the engineers who feel they are being dictated to. Engineers who get too vague a story are faced with many challenges, not the least being “what exactly are we building,” “If I’d know that originally, I would have estimated this as a much bigger story,” and “This will take at least six weeks, I can’t do it in a four week sprint.”
By breaking user stories down to iteration manageable levels the team is able to better estimate value and cost (time to complete), removes confusions that can result in a feature that doesn’t match the vision of the product owner, allows for shorter iterations – which itself improves the overall development process by allowing more chances to apply the “inspect and adapt” philosophy of agile.
Stage 5: Value the User Stories
Goal: To create a relative ranking of all user stories on a project.
Time Needed: One to two hours
Description: P0, P1, P2 ultimately breaks down. Fifteen years ago I never even heard of P0 and you might actually see a P3 in a product. Today you see projects where everything is a P0 and engineering says it will take two years.
Doing force ranking, feature weighting and other traditional techniques rarely leads to the level of consensus required to create a final ranked list. The realistic question to ask teams, at this stage, is how often estimating against real world artifacts has worked. When was the last time a product forecast was accurate? Using agile values opens up the process to creating a system that creates agreement not conflict.
NOTE: This is one of the secret sauces of this process. In my next blog I document an excellent exercise for how to define business value for an entire release in just a couple of hours.
Stage 6: Estimate the Effort (Cost)
Goal: Rank the user stories in relative order of effort (cost to produce).
Time Needed: One to two hours
Description: Classic estimating of user stories. It needs to be done on even the grossest level stories for the product owner to be able to properly rank order the backlog. Otherwise a backlog can be skewed towards value, costs or even worse, gut judgment. Poker Planning and Delphi models have an inherent flaw of focusing on the story itself which leads to exact effort and not rank ordering estimates.
The Cast: This stage does not involve the product owner/stakeholder cast. Effort estimating is done by the engineers doing the actual work. It is important to not just have management at this stage but engineers doing the functional work.
Note: The developers should not have knowledge of the business value rating at this stage. They are just assessing the complexity of the future. If they know a particular feature is the absolute silver bullet, they may over or under estimate it.
Stage 7: Order the Requirements List
Goal: One backlog to rule them all.
Description: At this point the Product Owner has both a business value and a cost. These are both placed on the user stories. From here the product owner then uses their judgment and skill to create the product backlog. They weigh the value and cost of a story (feature) to create the backlog.
Note: In agile development, best practice is to groom the backlog once iteration with the product team. This is where cost can be reevaluated, better definitions of stories can be done and the product backlog reordered as a result.
[Thanks to Phillip Chen, for the inspiration for this blog. His discussion thread on “Do you tailor PMBOK or other PM methodologies for your projects?” in the PMP LinkedIn group, sparked a reply by myself that spawned this blog. ]
Things are really looking up. Just got transferred to the newly formed product incubation group. The company is finally putting innovation up as one of the top three business goals for the coming year and you’re on the bottom floor of the new team.
You were worried for a while there, market share was slipping and it just didn’t seem like anyone was willing to break the cycle. No one wanted to point out the Emperor had no clothes and winter was coming. But the board did. Now you’ve got a new CEO and he’s shaking things up. What could go wrong?
“That is not how we do it.”
“But this is totally new product line, it’s nothing like our existing products. If we follow the same process we won’t get to market for two years.”
“No exceptions, we have a process, it works and you will follow it.”
Welcome to the PIG:
And wham, you run right into the Process Inflexibility Gorilla. Hogarth and I have talked about his cousin, PIG, on many occasions. As Hogarth oft recalls “He makes the immoveable object look like a hockey puck in the Stanley Cup.”
PIG generally hangs out in larger, more established companies. He’s at home in long standing businesses that have managed to keep doing what they do, with the minimal amount of change. It seems that sometimes that success is in spite of themselves. As often happens, the process inflexibility gorilla is so firmly entrenched, that he is all but invisible to those around him. He is not just ignored but not even seen. It takes a business change, or a new set of eyes to see him. The challenge that then comes, is how to get those entrenched with him, to actually see him.
A company hired a Director of QA. They had previously practiced a developer QA model, with the philosophy that the best person to test code was the person who wrote it. This director, we’ll call him Saul (I just made it up on the spot folks), was given a broad charter, promises of support and let loose on the engineering organization. Now Saul was an effective executive. He knew he couldn’t just sweep in and “lay down the law” no matter how much air cover he might (or might not) have. Saul took his time, he asked lots of questions, observed, got to know people and laid out his plan. He made some minor wins and changes, but for the most part he spent the first few months collecting data. All in preparation for laying out a whole new QA methodology, just like the charter he’d been given said to do.
Only when it came time to roll it out he ran into a massive wall, one that made China’s Great Wall look like one of those Irish rock walls in a sheep pasture. So powerful was the institution, to the way things were ‘supposed’ to work, that even his powerful executive sponsors backed down. So entrenched was the “way its done”, that no one was willing to consider that the business had changed, or needed to change for it to continue to compete effectively. In the end Saul left the company, unwilling to spend the rest of his career trying to get people to see the invisible gorilla.
So, how do you deal with such a pervasive and hard to see gorilla? It’s not easy and it may not even be possible, but there are some things you can try.
Now one thing you may of noticed in my style, is I like analogies. I’ve found if you can break something out of the now and use something totally unrelated to explain it. When this topic came up in the PMP LinkedIn discussion forums I used the ‘screwdriver story’.
Of course many entrenched people will argue that everything is identical. Yeah that may be their point of view, but I just keep on with my analogy.
“We both have a budget of $200. I’ll take the money and buy a nice , simple toolbox with all the normal tools in it, you know hammer, phillips, flat head, wrench, etc. You can use your $200 to buy a super whiz bang phillips head screw driver that is exactly perfect for the currently defined job.” “I’ll do this because when you get stuck in a room (project) that has nothing but lug bolts, your fancy screw driver is just a pointy stick.”
Something you have to go back to, in times like these, is the concept of innovation. If you have inflexible process you probably have one of two things. You have inflexible products, which will be unable to compete in the continuing market place. Or you have products that are not being managed efficiently because they are square pegs being shoved in round roles and shaving off parts to fit. If your company is not flexible, how long will it continue to survive?
All right, while a very satisfying conversation it won’t sway every listener. The people who are inflexible in process are often not going to want to consider there might be anything but phillips screws in their company. These kind of people are going to bristle when you imply the company might fail through lack of change. “That’s the way we’ve always done it.” So what do you do? Tough question and no easy answer.
One thing you can try, is to work around the road block. If you have good cross organization and seniority relationships, you might be able to push for change from another direction. You have to be careful though, as this steps into the touchy ‘going over/around someone’ politics.
When it comes down to addressing the Process Inflexibility Gorilla (PIG), we once again come back to relationship and influence. While Saul failed in his endeavor, more often than not a strong and broad set of relationships should allow you to get the value benefit of making process change across to people who can affect the change.
Don’t cry “The emperor has no clothes” or in this case “Look an invisible gorilla!” Instead steer people so they can’t help but run into the invisible gorilla. Once you run into an 800 pound gorilla it doesn’t really matter if you can’t see it, you know its there.
Talking with gorillas, I’m Joel BC