Can you measure the effectiveness of your risk management process?
Published: 28 November 2014
Martin Hopkinson asks: What’s the ‘Goldilocks measure’ for your organization’s risk management strategy? How do you know whether you’re doing too much, not enough or just right?
Risk management has become firmly established as an important project management process. Most large organizations have invested in the development of their process and put significant resources into sustaining it on their projects. But do they know if their risk management process is as effective as it could be? Could their process be improved? Or do some projects implement the process to much better effect than others? To answer these questions, you need the means to measure project risk management capability.
You can make a start by asking key questions about each project’s risk management process. The process implemented on different projects can then be compared. The organisation can also judge whether or not there are some questions that identify issues of more general concern. As an example, consider the following ten questions:
- Do risks that impact on projects tend to have been identified at the time of project approval, or do new risks often arise during project delivery?
- Is the process capable of supporting risk-based decision-making during the earliest stages of a project, thus optimising its solution before the main project delivery phase?
- Is there a sound understanding of which sources of risk merit the most management attention?
- When contracts with other organisations are involved, does contract design ensure that risks are shared where appropriate or owned by the party best able to manage them?
- Do projects produce realistic forecasts for the effects of risk on the project schedule and budget?
- Are risk estimates made by people with appropriate experience and in an environment that is sufficiently free from pressures that tend to produce bias?
- Are risk response plans implemented as intended, or do other activities tend to take precedence?
- Are new sources of risk identified and escalated in a timely fashion to the appropriate level of management?
- Is each project’s risk management process well-aligned with its objectives and other related processes such as planning, cost forecasting and contract management?
- Are all reports produced by the process useful to the people for whom they are produced?
However, while considering questions such as these can be helpful, additional structure is required to produce a satisfactory measurement tool. For example, there needs to be evidence that the question set interrogates all aspects of the process, including both technical and behavioural issues. Structure is also required to reflect the fact that certain aspects of the process are fundamentally important to its overall capability. For example, even if risks are identified appropriately and their effects modelled to a high standard, failure to act on the implications would result in a fundamental failure of the overall process. As I once heard a corporate risk manager admit: “In the past, my company has been very good at risk identification, risk assessment and risk filing!”
Finally, a process capability measurement tool must identify the means by which performance can be scored objectively. Sliding scales such as 0–10 or poor–excellent are open to interpretation and would not form a reliable basis for comparing projects or benchmarking the organisation. Instead, the tool needs to include objective criteria that articulate the characteristics typical of different levels of process capability.