The CMMi has been developed and integrated with other approaches and become the CMMi. This process has increased its apparent complexity, as well as its flexibility. The information on the SEI web site (link) has split into either detailed manuals of CMMi and presentations that primarily seem to say “Why you need to buy in our consultants…” The CMMi has been adopted by large players and so there is a strong consultancy market surrounding it, including training and appraisal of your organisational level. The manuals are reference material and not intended to be a thrilling read. The presentations do well at selling CMMi benefits but don’t offer much in the way of a “how to guide”.

The best links that I found to describe the CMMi to allow a quick “Is this for me assessment were:

Tutorials point:

http://www.tutorialspoint.com/cmmi/cmmi-process-areas.htm

Wikipedia:

http://en.wikipedia.org/wiki/Process_area_(CMMI)

These sites provide an overview of CMMi at a level more suitable for someone who needs simply to understand the core nuggets of wisdom contained in the CMMi. The still are quite long though, so here is a quick summary of the summary…

CMMi is structured to allow consideration of how an organisation does things, for simplicity I will assume it is software development that is being performed. The CMMi assumes that the road to improvement means ending up with a standardised approach for doing this which can be measured, optimised and customised to the situation in a controlled fashion. It has this in common with many “Quality” processes including the international quality standards. It asserts that there 5 levels of maturity:

• Level 0 – Incomplete – We can get there, we’re sure we can. Not sure how though, and sometimes it don’t quite work right.
• Level 1 – Performed – We can do it as long as it is about the same as the last time.
• Level 2 – Managed – We can do it, and we can manage the process.
• Level 3 – Defined – We can do it, and have a definition of how.
• Level 4 – Quantitatively Managed – We can do it, and can provide numbers about how well. After all what you don’t measure you can’t manage!
• Level 5 – Optimising – We can use the numbers as a basis for continual improvement of how well we do it.

The idea of CMMi is to move up between the levels, and the way to do this is to develop various process areas in your organisation at each level. The areas form a structure, so it is very hard to fully benefit from a higher level process until the lower level ones have been addressed.

The processes organised by level are:

Level 2:

• Requirements Management
• Project Planning
• Project Monitoring and Control
• Supplier Agreement Management
• Measurement and Analysis
• Process and Product Quality Assurance
• Configuration Management

Level 3:

• Requirements Development
• Technical Solution
• Product Integration
• Verification
• Validation
• Organizational Process Focus
• Organizational Process Definition
• Organizational Training
• Integrated Project Management
• Risk Management
• Decision Analysis and Resolution

Level 4:

• Organizational Process Performance
• Quantitative Project Management

Level 5:

• Organizational Innovation and Deployment
• Causal Analysis and Resolution

It is likely that if you are having quality or delivery problems that by looking at these process areas in turn and considering whether you “Do that” you can start to gradually improve. The CMMi manuals have a lot to say that can support you, or you can start looking on Google etc. for ideas that might help you improve.  If you need help then seek outside help – it can be value for money. If you need to be certified to a certain level then the game is very different and outside input becomes a necessity.

• The most obvious approach to assure performance is to have sufficient capacity from day 1 to allow the system to perform in the long term. This means that you don’t, in theory, need to monitor the system carefully. The problem is that you then buy considerable capacity up front – which may be an expensive approach given the falling cost of hardware and the reducing time value of money. This approach is also liable to involve extensive up-front testing, which further extends any project pay back period.
• The other extreme is to have hardware, software and procedures in place to only install capacity on a “just in time” basis. This means that you only invest in capacity when it is strictly necessary. In turn that means there is more spend in monitoring, projection and control. It also leaves a question over the objectives during volume and performance testing, and exposes the organisation to a level of additional risk.
• There is an intermediate point where capacity is installed that is believed sufficient to cover a period (such as a year). This is coupled with an end of period review process that examines a need for extra capacity in time for the end of the period. This needs monitoring and projection, coupled with some purchase of excess capacity.

Thus the answer to the original question depends on the strategy adopted for capacity management. Unfortunately, the most appropriate strategy inherently depends on the individual situation. If the cost of the hardware needed in the long term is low then up front installation of the projected capacity requirement makes sense. If the organisation already has the necessary tools and procedures to meet just in time capacity management then that would make sense. What is most critical, however, is that this is a management decision made with a through understanding of the considerations and implications.

If you would like advice, or to discuss how to decide on the most appropriate approach in your circumstances, then please feel free to contact me at dh@sarquol.com, or call on +44 7887 536083.

If you take the first path then good luck to you, and I would be very interested in knowing how it turns out for you. The issue is that a business user, and in fact most technical people, just do not wish to work with an “average” user and then work out variations. A business is structure as departments with functions. If the business has measurements at all, and thankfully most do, it will be in terms of departmental sizes and throughputs. There are also likely to monitor the volume
of business that is done, and if you are lucky have documented processes for how this business is performed. Thankfully the tools needed to convert these figures into what you need are already built into a performance model. The following discussion, therefore, assume that what you will have been given is user levels and business volume projections./p>

1. Start with the year-on-year departmental sizes, with each department modelled as one or more sections. If you only have past figures then provide a simple basis for projecting this forward.
2. Fill in the likely levels of variation in system usage, based on either you own knowledge or on discussion with people who know the likely behaviour of the business.
3. Fill in the data volumes generated by the different functions, and there needs to be a relationship between the data or function volumes and the figures supplied by the business.
4. Once the model calculates, this will provide a projection of the numbers of functions and items of data that will be produced year-on-year. Compare this to the volumes that you have been provided by the business. If there is an approximate correlation then adjust the figures to improve the match and use this as the behaviour profile.
5. If there is a large mismatch then you may have a misunderstanding about the figures that you have been provided with, or it is possible the business estimates need refinement. This will need to be discussed with the people who own the business figures. The process can in itself add value for the business. If they are trying to do levels of business that there people and business processes can’t support then this may manifest as complaints about system performance. (“I can’t meet my targets because the system is too slow.” may actually be “I can’t meet my targets because they are unrealistic.”)

The process of generating performance model figures from business or systems figures is one of approximation. The target is not a precise and accurate model, but one which is of the correct order and believable. The idea with this sort of modelling is to 1% accuracy, but order of magnitude estimation. If you need help in starting a performance modelling exercise then call me on +44 (0)7887 53608, or e-mail me. I will be happy to have a talk about the most appropriate approach for finding a usable set of figures, and the levels of accuracy needed.

This is a model for a new system implementation that is played out, with a few variations, for the majority of new system implementations. I have left out the blood, sweat and tears part of the story for brevity. I have also left out the successful implementation memos and party. I have left this part out because I would suggest that whether the implementation was successful won’t really be known for quite a while.

The rest of the life of the system will generally be complex, and involve change requests, and support calls. From a performance perspective, however, it is all too common for a system to get gradually, or dramatically, slower during its lifetime. Initially this may not be noticeable, but by a year or two into a system’s lifetime complaints start that the system is slowing down to the point that it is impacting the business. The way that the initial project performed, approached and managed the system’s performance and the monitoring and management techniques used dictate how likely this scenario is.

In the “Principles of Capacity Management” document I have indicated that the expected volumes of usage will need to be examined over time. As systems are used they will tend to grow both in data volume and levels of usage. This means that a system’s capacity requirements will expand over time, as will the stress on the underlying algorithms of the system. I will skip an outline of the Order of algorithms (e-mail us if you would like this included in a future bulletin), but depending on the way the system is produced this additional volume can cause complete failure. To avoid this it is important to make sure that the likely levels of future usage and data volume are estimated and built into the performance testing. Thus separate “Volume Tests” are needed to check that the system can cope with the predicted data levels for the future.

The “Generic Performance Model” supports the analysis of a 10 year time window. In the case of a new system, therefore, this can be used to project the likely performance over a 10 year lifetime. In the case of an existing system it is probably more effective to model between 2 and 3 years of past and current behaviour, with the remainder extending into the future. This model uses as its inputs the expected number of users per year and their behaviour profile. From the changes in volume figures it is then possible to calibrate the model so as to estimate the increase in resource utilisation over time. To this is added the expected architecture in order to estimate the likely response times from the system.

If you need help in working out how to approach analysing the likely future performance then e-mail me or phone me on +44 (0)7887 536083. It can be difficult at first to understand how best to approach this sort of modelling and the approaches needed to calibrate and validate the modelling. The long term benefits of this approach, however, are substantial and so well worth the initial effort.

The key to deciding is to consider the level of risk that performance problems will pose to the business. There seem to be relatively few organisations that use a blend of testing and future projection based on a risk-driven approach. It is this approach that is going to produce the most cost effective projects, since the degree of testing and proof provided will reflect the business risk that is inherent in the system. Spend sufficient to offset the cost of risk that a system carries if the performance is insufficient in the long term. Remember that it is likely to be much more expensive to solve a performance problem a number of years after initial delivery than when the project is in full flow. However important the initial delivery may be, a project is not complete until it has been proven to be capable of being operational for the long term.

Suggestion:

Use a simple set of standard criteria for a project representing “average” performance risk for the business:

1. A soak test to prove that the system will run for an extended period, say 12 hours, at average loading without performance degradation.
2. A load test to demonstrate that the system can process peak demand volumes for a predefined period, such as an hour, without degrading.
3. A volume test, to prove that the system can operate with production volumes at a defined point in the future – say 2 years hence.
4. Within any performance tests use a small set of key business functions to test that the system performs adequately.

If the risk related to poor system performance, or failure, is especially high then these performance criteria would be made more stringent. The degree of performance proof required, however, should be kept proportional to the level of risk in the system failing to perform well. If the system was business critical, for example, it may be considered appropriate to require that the system be run using a much wider set of business functions, for 72 hours continuously, and at 5 times the expected peak volumes for the system for the next 20 years. Unsurprisingly, as the proof requirements become more stringent the cost and time needed to prove them will increase significantly.