Table of contents for Project Estimation Model

1. Project Estimation Model
2. Project Estimation Model Example
3. Project Estimation using Scenarios

Defining estimation scenarios

The first stage in the steps to estimating a project was defined to be the definition of the scenarios to be supported in the solution.  The scenarios should be defined at as low a level of detail as possible, with the following diagram showing the example scenario definition.

Scenario definiton

Definition of the scenarios sizing

In the spreadsheet the light blue fields are those that you would normally fill in. The light purple fields are intended to fill in automatically based on the calibration parameters. If you prefer, however, it is suggested that you replace the calculated value with your own value where a better value is available. In this case I would suggest changing the colour of the cell so that you know this has been done later. I use bright highlighter yellow for this.The purple values are filled in from the “Calibration” sheet based on the complexity selected in the drop-down within those cells. The values represent typical operation counts for scenarios of the defined complexity.Scenario sizing calibrationThe scenario sizing information is drawn from the following part of the calibration sheet:Scenario calibration

As for all the calibration, the provided values are likely to be useful but the optimal values will depend on the business context of the estimator. It is suggested that the calibration should be adjusted from project to project until the most appropriate values are found. For a more rigorous approach a function-point counting style could be adopted on a specification of some suitable scenarios. The idea is to count the number of entities (c.f. database tables) that need to be managed by each scenario.

In the scenario definition sheet these filled in values are then summed at the bottom of the page, and each operation type is multiplied by a weighting factor to produce a total size. This can be seen at the bottom of the scenario definition sheet in the example:

Scenario sizing

The scenario sizing calculation

As can be seen, a standard “Operation Weight” is applied to each of the operation types within the columns of the estimation spreadsheet.  These are drawn from this section within the calibration sheet:

Scenario operation weight

Operation weighting calibration

This process is similar in principle to function point analysis, but the constants have not been adjusted to provide a function point count. It would be possible to complete this and hence to provide an estimate very close to the true function point count of the solution. This, however, is left as an exercise for the reader since I don’t have access to a library of function point analysis data which would make it worthwhile.

Project Assurance is, as the name suggests, assurance of the way that the project is being run. Relevant experience is around project management rather than the implementation solution space. The concept od Solution Assurance is more related to confirming that solution being delivered will be fit for purpose once in place. In PRINCE2 terms the project assurance role will be most closely aligned to the project executive, whereas the the Solution Assurance will be aligned with the senior suplier. (Arguably the senior user could also be looking to take on solution assurance, but in PRINCE terms it would be the senior supplier.)

The idea of a turnaround specialist would be slightly different again. To require “turnaround” there must be an identified issue to address. The assurance role is required whether or not there is a problem, and its presence would hopefully avoid problems. The turnaround specialists task is more one of a troubleshooter – identifying the symptoms, analysing the cause and leading the solution. This might be in Project or Solution terms.

Let me provide some examples:

• In my last role I was employed as a Solutions Architect, and my primary focus was on assuring that the solution to be delivered met its functional and non-functional requirements. The skills I employed for this were primarily technical, and example questions were around whether the particular solution would fully deliver the functional and non-functional requirements.
• At the same time I also performed a level of project assurance for the programme manager for projects I was less directly involved in. In this case the experience I was drawing on was one of a project manager with years of delivering solutions under my belt. The examples of questions I was asking here were around the potential issues to do with processes being used for risk and issue management and whether they were appropriate.
• In a turnaround role I would employ analysis based on *both* of the other assurance roles, but would go further to address an identified issue. I would use my own creative skills to provide potential approaches, and would use an engagement approach that would get buy-in and participation from the delivery team. I would then propose an exception plan, in PRINCE terms, in conjunction with the project or programme manager. The skills being used are more active and leadership based.

In the description above I have assumed a project/programme context for turenaround. The approach, however, is as likely to be applied in an operational setting, and in this case the set of techniques will differ somewhat. The result of business level turnaround application is likely to be one or more change programmes.

Table of contents for Project Estimation Model

1. Project Estimation Model
2. Project Estimation Model Example
3. Project Estimation using Scenarios

As a means for discussing the model I will follow through an example of the model being used. To do this the needs to be an example usage scenario. In this case I have chosen to use the following:

“A banking web site has retail and corporate clients, and well as a set of automated processes that must be completed overnight in the bank’s overnight batch window. The site has a standard Java web architecture, with the batch processes being initiated using a batch process at the application server. The purpose behind the model is to examine the capacity required in the major system components, and to make ensure the ongoing capability of the host systems for the site.”

If you have followed through the performance model series (here), you will note that this is exactly the same scenario. I have chosen this in order to be able to compare and contrast the application of the performance and project estimation models. There are clear differences in approach that need to be considered. For those that like to skip to the end I have provided a copy of the final estimation example filled out here:

Example project estimate

The key to deciding is to consider the level of risk that performance problems will pose to the business. There seem to be relatively few organisations that use a blend of testing and future projection based on a risk-driven approach. It is this approach that is going to produce the most cost effective projects, since the degree of testing and proof provided will reflect the business risk that is inherent in the system. Spend sufficient to offset the cost of risk that a system carries if the performance is insufficient in the long term. Remember that it is likely to be much more expensive to solve a performance problem a number of years after initial delivery than when the project is in full flow. However important the initial delivery may be, a project is not complete until it has been proven to be capable of being operational for the long term.

Suggestion:

Use a simple set of standard criteria for a project representing “average” performance risk for the business:

1. A soak test to prove that the system will run for an extended period, say 12 hours, at average loading without performance degradation.
2. A load test to demonstrate that the system can process peak demand volumes for a predefined period, such as an hour, without degrading.
3. A volume test, to prove that the system can operate with production volumes at a defined point in the future – say 2 years hence.
4. Within any performance tests use a small set of key business functions to test that the system performs adequately.

If the risk related to poor system performance, or failure, is especially high then these performance criteria would be made more stringent. The degree of performance proof required, however, should be kept proportional to the level of risk in the system failing to perform well. If the system was business critical, for example, it may be considered appropriate to require that the system be run using a much wider set of business functions, for 72 hours continuously, and at 5 times the expected peak volumes for the system for the next 20 years. Unsurprisingly, as the proof requirements become more stringent the cost and time needed to prove them will increase significantly.