Guide to Using the Project Complexity and Risk Assessment Tool
5. Using and Scoring the Assessment
5.1 Defining Risk
TBS's Framework for the Management of Risk encompasses best practices on risk, including those supported by the International Organization for Standardization (ISO) and the Software Engineering Institute (SEI). This framework provides a standard for Government of Canada application:
Risk refers to the effect of uncertainty on objectives. It is the expression of the likelihood and impact of an event with the potential to affect the achievement of an organization's objectives4.
There are three key characteristics of a risk:
- It looks ahead into the future;
- It has an element of uncertainty: a condition or a situation exists that might cause a problem for the project in the future; and
- It has an outcome. Although it is acknowledged that risk, if managed properly, can lead to opportunity, the definition of risk adopted for the purposes of this assessment focuses on adverse outcomes5.
The risks that are present before the development or application of any mitigating strategies are called inherent risks. These are the risks that could have a negative impact on the project if no action is taken. Once mitigation strategies have been put in place, the remaining risks are called residual risks. The PCRA focuses on inherent risks.
5.2 Defining Complexity
Complexity is, fittingly, a much more difficult concept to define. SEI provides a definition of complexity:
(Apparent) the degree to which a system or component has a design or implementation that is difficult to understand and verify.6
(Inherent) the degree of complication of a system or system component, determined by such factors as the number and intricacy of interfaces, the number and intricacy of conditional branches, the degree of nesting, and the types of data structures.7
The Standard for Project Complexity and Risk states that project complexity is based on the number of business rules, the technology employed and the project's size. It further indicates that complexity is a major component of project risk and that complexity should be determined at the start of all large projects, and when changes occur, so that appropriate action can be taken to manage risk.
5.3 Structure of the Assessment
The assessment is a series of questions organized into seven sections. The following table demonstrates the maximum score allowable per section and the number of questions per section. Moreover, the weighting attributed to each section is indicated, which demonstrates the overall impact of the section on the final score.
| Category | No. of Questions | Maximum Score |
|---|---|---|
| Total | 64 | 320 |
| Project Characteristics | 18 | 90 |
| Strategic Management | 6 | 30 |
| Procurement | 9 | 45 |
| Human Resources | 5 | 25 |
| Business | 5 | 25 |
| Project Management Integration | 6 | 30 |
| Project Requirements | 15 | 75 |
5.4 Complexity and Risk Classifications
The following table outlines the PCRA scoring ranges that relate to corresponding complexity and risk levels. The PCRA score reflects a very wide range of inherent risks and it is unlikely that a single project would be susceptible to all of the potential risks. For the purposes of the assessment, the worst-case scenario is that up to 70 percent of the inherent risks would be present in a project. Therefore, the baseline score (i.e., total possible maximum score) is multiplied by 70 per cent to reflect the worst case. The total raw score (i.e., the cumulative score of all question responses for a given project) is then divided over this adjusted (multiplied by 0.7) baseline score to produce a percentage score that corresponds to a complexity and risk level shown in the table below.
| Complexity and Risk Level | Definition | Score |
|---|---|---|
| 1. Sustaining |
Project has low risk and complexity. The project outcome impacts only a specific service or specific program, and risk mitigations for general project risks are in place. The project does not consume a significant percentage of departmental or agency resources. |
Less than 45 |
| 2. Tactical |
A project rated at this level affects multiple services within a program and may involve more significant procurement activities. It may involve some information management or information technology (IM/IT) or engineering activities. The project risk profile may indicate that some risks could have serious impacts, requiring carefully planned responses. The scope of a tactical project is operational in nature and delivers new capabilities within limits. |
45 to 63 |
| 3. Evolutionary |
As indicated by the name, projects within this level of complexity and risk introduce change and new capabilities and may have a fairly extensive scope. Disciplined skills are required to successfully manage evolutionary projects. Scope frequently spans programs and may impact one or two other departments or agencies. There may be substantial change to business process, internal staff, external clients and technology infrastructure. IM/IT components represent a significant proportion of total project activity. |
64 to 82 |
| 4. Transformational |
At this level, projects require extensive capabilities and may have a dramatic impact on the organization and potentially other organizations. Horizontal (i.e. multi-departmental, multi-agency or multi-jurisdictional) projects are transformational in nature. Risks associated with these projects often have serious consequences, such as organizational restructuring, change in senior management, or loss of public reputation. |
More than 82 |
Footnotes
4 Framework for the Management of Risk, Government of Canada, 2010
5 This definition is consistent with the Software Engineering Institute's Continuous Risk Management discipline supported by TBS's Enhanced Management Framework, and conforms to the risk management concepts described in the Framework for the Management of Risk, Government of Canada, 2010.
6 Institute of Electrical and Electronics Engineers. IEEE Standard Computer Dictionary: A Compilation of IEEE Standard Computer Glossaries. New York, NY: 1990.
7 Evans, Michael W. and Marciniak, John. Software Quality Assurance and Management. New York, NY: John Wiley & Sons, Inc., 1987.
- Date modified: