The Estimation Game

Recently, there has been various articles and blog posts about project or work estimation and whether it is worth doing.

What are the reasons for doing estimation?

• Know the size of work.
• Know the cost of work.
• Know the staffing.
• Know when you are done.

What can you estimate?

• Known requirements/features.

Challenges?

• Changing/unclear/missing Requirements.
• Knowledge of the Domain.
• Skill of staff.
• Changing Staff.
• Feedback/communication.

The reason for doing anything is to get reasonable value in return. The value that you expect typically is predictability. No cost overruns, work completed on time, and completed with the features as expected in working condition.

This sounds simple enough but it really depends. The problem really comes in with the size of the work.

Bigger projects have so much variability and unknowns that even the best attempts to estimate work up front will likely fail. It is hard to know what in fact will happen and the outcomes depend on the flexibility of the overall plan and the attitude of the team and the stake holders. The real problem here is that estimates become an unchangeable plan, and then they also become time commitments. When you fix these two things you have nowhere to go when surprises occur and new work is invariably discovered. On the other hand, if you are flexible, large swaths of the project plan might change on the way and budgets and time lines might actually have less relevance in the overall process. Also, the human component is so dominant in long running projects that it is very hard to guess what the output of the team is, especially if the team keeps changing along the way.

Smaller and often unrelated work items can be estimated quite easily and there are far fewer surprises. However, this kind of workflow often has so much flexibility that the work plans keep changing from week to week. Many of the items that are estimated will never be worked on and new items are inserted to the work flow. It might be that there is a lot of churn and time spent on estimates but at the same time it is quite well known how soon work is completed when it is started anyway. If you put a work item on top of the priority queue you can expect it to be done within the iteration, as will many other items reasonably close to the top of the list. Further down in the list the item is, less you really care about it, and the stake holders might never really care about it at all after couple of iterations.

So, where is the value? I think the only real value is the ability use living estimates as a decision making tool.

In the long running projects, you only care what the estimate is NOW. It could be different from what it was a month ago, and it will be different in a month from now. You will gain experience what the team can do over time. That will give projections for cost and completion times. With good enough management of priorities and the value of the feature set it is possible to approach something attainable. Also, there must be openness to react and change the plan. Staffing changes will provide additional challenges but on the long run it is possible to gauge how the project is doing over how the team is doing in the project.

There might not be so much value at all doing estimates for variable short term work where experience will quickly tell what the 'event horizon' is because you only really care about the immediate results anyway.

This is still somewhat simplistic analysis because it does not fully appricient the complexities of understanding work, producing right results, dealing with feedback and inspection. Building the right thing is still the most important goal and measuring that is very esoteric and based on things that are hard to measure. Many teams can call things done and can appear to make progress but what exactly are they producing and how? Sometimes you see those high profile failures that leaves you puzzled - obviously it was considered done, and all the rest, but it just did not work. Obviously the estimates were wrong, even when budget goals or time lines were met.