The field of business strategy has looked at this issue of an uncertain future and come up with a number of approaches for managing considering it. One of these is called “Scenario Planning”, and basically consists of building a set of different future scenarios which are considered plausible. They are not attempts at predicting the future, only of painting plausible futures. It is then possible to consider how these scenarios may play out according to the decisions you are taking now. This allows a form of sensitivity analysis on the decisions that you are making.

A process for approaching this would be to:

1. State the architecture decisions that need to be made.

2. Identify the major environmental forces that impact on the architecture.

3. Build four scenarios based on the principal forces.

4. With the scenarios in hand, identify architecture opportunities within each scenario.

5. Examine the implications of the decisions across the range of scenarios.

As an example, lets assume that you need to decide between two architecture design patterns, A and B. For the example “A” provides a highly resilient and fault tolerant solution but involves significant additional hardware, operational administration and development costs over “B”. The decision that needs to be made is whether the extra capability of “A” is justified. If this is a new system then it may not initially require large processing volumes, and may be able to accept a system outage. The scenarios, however, could be used to examine under what circumstances the additional capability would be justified. If the system catches on, for example, how long might it take for the system to become business critical? Would you have time to re-engineer it? This may lead you to decide the additional capability is justified now, or would allow you to understand what changes in the environment might lead to the additional capability being needed.

In terms of the Architecture Roadmap then, the Scenarios developed and decisions taken may then be included in the roadmap. With this in mind a Planning Scenario is chosen. This defines the assumption set on which the roadmap has been built. In doing so, however, the other scenarios are not discounted but used as alternatives to which to architecture should be resilient. Where the architecture isn’t resilient to a scenario, then the roadmap will be able to indicate symptoms that will indicate that the architecture needs to be revisited. These then become business risks that must be managed based on the selected architecture.

All of this seems relatively complicated, so why bother? The answer is that architecture decisions are taken on a regular basis under the in the knowledge that the requirements of the system are known and understood. This may be true of the explicit requirements elicited from users, but these are based on assumptions about the future. In reality the future is generally much more surprising than we would like to believe, and so taking the time to consider what this might mean for the decisions you are taking can be worthwhile. An exercise like this can be considered surprisingly quickly, and may significantly help decision making.

The most valuable way to validate a performance model is to monitor a production environment and then to check that all of the predictions of the model are reflected by the production system within a reasonable tolerance. This allows all of the assumptions and mathematics to be checked in a single stage. The practice should be a matter of routine after the implementation of systems, and acts as an important operational performance assurance activity.

In the time before the initial implementation, however, it is only practical to check the technical aspects of the model using a set of performance tests. Thus, it is possible to test using a defined level of user load, and to make sure that the predictions of the model match the performance profile experienced to within an acceptable tolerance. If not, the reason for the differential needs to be investigated and resolved. It will usually be necessary to run a series of tests until the model’s accuracy is sufficient for the model’s use.

It is important to stress, however, that this form of validation still leaves a significant level of uncertainty about the level of demand to be placed on the system. It is worthwhile, therefore, using the model to predict the likely performance under a range of demand-profile assumptions. At least the impact of the level of unknown for the demand can then be investigated.

This is a model for a new system implementation that is played out, with a few variations, for the majority of new system implementations. I have left out the blood, sweat and tears part of the story for brevity. I have also left out the successful implementation memos and party. I have left this part out because I would suggest that whether the implementation was successful won’t really be known for quite a while.

The rest of the life of the system will generally be complex, and involve change requests, and support calls. From a performance perspective, however, it is all too common for a system to get gradually, or dramatically, slower during its lifetime. Initially this may not be noticeable, but by a year or two into a system’s lifetime complaints start that the system is slowing down to the point that it is impacting the business. The way that the initial project performed, approached and managed the system’s performance and the monitoring and management techniques used dictate how likely this scenario is.

In the “Principles of Capacity Management” document I have indicated that the expected volumes of usage will need to be examined over time. As systems are used they will tend to grow both in data volume and levels of usage. This means that a system’s capacity requirements will expand over time, as will the stress on the underlying algorithms of the system. I will skip an outline of the Order of algorithms (e-mail us if you would like this included in a future bulletin), but depending on the way the system is produced this additional volume can cause complete failure. To avoid this it is important to make sure that the likely levels of future usage and data volume are estimated and built into the performance testing. Thus separate “Volume Tests” are needed to check that the system can cope with the predicted data levels for the future.

The “Generic Performance Model” supports the analysis of a 10 year time window. In the case of a new system, therefore, this can be used to project the likely performance over a 10 year lifetime. In the case of an existing system it is probably more effective to model between 2 and 3 years of past and current behaviour, with the remainder extending into the future. This model uses as its inputs the expected number of users per year and their behaviour profile. From the changes in volume figures it is then possible to calibrate the model so as to estimate the increase in resource utilisation over time. To this is added the expected architecture in order to estimate the likely response times from the system.

If you need help in working out how to approach analysing the likely future performance then e-mail me or phone me on +44 (0)7887 536083. It can be difficult at first to understand how best to approach this sort of modelling and the approaches needed to calibrate and validate the modelling. The long term benefits of this approach, however, are substantial and so well worth the initial effort.