estimating

I saw Steve McConnell of Code Complete fame who is also the CEO of Construx speak on estimating, the hardest challenge I've faced in technology, at AgileAustin on Wednesday night. Code Complete is an industry-revered treatise on software development not unlike Fred Brooks' The Mythical Man-Month, which I do not mention randomly. Mr. McConnell suggested that Brooks' Law of "adding manpower to a late software project makes it later" is not a reality in disciplined agile projects. This law is detailed in The Mythical Man-Month. My mother, Susan Jaeschke, worked for IBM for 33 years from 1969 to 2002 and she read Brooks' book, which I haven't, when it was brand new as Mr. Brooks worked at big blue like her at the same time. She has told me that one of the analogies in the book is that while it takes a woman nine months to have a baby it is a mistake to believe that one may put nine women together on a team to make a baby in one month. Naturally this speaks to the hero knowledge problem and how someone new to a team, not holding the hero knowledge, struggles to stay relevant. I suppose Fred Brooks saw a waste of time in ramping up others who are not already the hero. I've worked on two different Agile teams with hero knowledge problems. I get it, yet Steve McConnell envisions a light at the end of the tunnel for this pain point. The burn down chart shown with him in the photo I took here showed off his vision. He saw actual rate of process pulling away from desired rate of progress in a performance-decaying, worsening way initially upon adding new team members and then, ultimately, a correction in which the actual rate of progress bends back to come closer to the desired, impossibly-consistent rate of progress which is based on a fixed, estimated number of story points to be completed per sprint and represented by a diagonal line across time denoting how much work is left to be done and when it will be done. (Actually development is represented by a draw-as-you-go line which starts out at the same point as the diagonal line and then typically strays to the left of it as less gets done in a sprint than expected.) The word "late" in Brooks' Law was where Mr. Connell saw a problem. He felt that the projects Fred Brooks saw as almost done were anything but and thus Mr. Brooks was disappointed in what became of adding team members, but not because the team members were not performing. On the other hand, if a project was almost done, then adding more team members does not really make sense in Mr. Connell's model either where there is going to be some initial disorientation. I cannot help but read into Fred Brooks from Mr. Connell's perspective and believe that the projects were not almost done but instead that was just what management wanted to believe. If there had really been a light at the end of the tunnel there would not have been a reactionary demand for more resources. As complexity grows it often grows exponentially harder to make progress (especially so if you are fighting an anti-pattern) and thus a team near a finish line may really be merely halfway through an undertaking. There is a lot more to speak to in estimations for an Agile team than this resources quirk however. Estimating is not non-Agile and when teams see estimating from a we-hate-it-as-we-are-Agile perspective then they are apt to alienate management and make management hate Agile. Early successes of Agile in small projects have reinforced the we-should-not-do-estimates thing when the reality is that bigger projects have bigger demands.

Businesses do have finite plans and demand estimates for: Cost Schedule Functionality

There are three possible sources to collect estimate data from: Your Industry Your Organization Your Project

Clearly the third data source is better than the second which is itself much better than the first. As you start a project you will not have project data to draw from and as you build from Waterfall requirements (and this is the way Steve McConnell recommended to go) initially you will estimate correctly perhaps thirty percent of the time. Mr. McConnell asserted that in one to two years a team may be estimating correctly ninety to one hundred percent of the time however. It helps to practice and it helps to have the project data that you will eventually accumulate. Do not try to estimate a velocity until you have done a couple of sprints as then you may really see what your velocity actually is based upon what has gotten done. You do need to estimate however. Iterative development maybe a tremendous benefit but iterative requirements are a negative. A business would rather you were wrong than vague. If you plant a stake in the ground and then ultimately have to move the stake, well that is better than "I dunno" -flavored apathy.

The slide above shows a funnel of progressively dying ambiguity and waste by way of bad estimates (the yellow) which Mr. Connell envisions a successful project to have. You have to be a third of the way into the funnel before you are really hitting your stride in estimating the stories you are going to do (the green) and the stories you are not going to do (the red) in the name of bringing your burn down chart’s draw-as-you-go line back towards its diagonal line. (Those are my terms for the lines. :P) You have to be a third of the way into the funnel before you may use project data for estimates. Before you may get to that point, the following five legs of the project must be successfully bridged in chronological order:

1. Initial Product Definition
2. Approved Product Definition
3. Requirements Specification
4. Product Design Specification
5. Detailed Design Specification

Wow! Wow! Waterfall! You do not want just anyone writing specifications. You need trained requirements workers. Software Requirements by Karl Wiegers was recommended by Steve McConnell as the best book on this topic.