Lessons Learned: Implementation of the Policies on Investment Planning and the Management of Projects by Four Pilot Departments

Introduction

In June 2007, Treasury Board approved the Policy on Investment Planning—Assets and Acquired Services and the Policy on the Management of Projects. The policies provide a comprehensive and integrated approach for the planning of investments and the management of projects. Both policies came into effect on June 7, 2007; however, they represent a significant change in how investments are planned and project approval limits are determined, so a phased implementation strategy was devised. Implementation will occur in four waves, with groups of departments transitioning to the new policies over time. The transition period will end April 1, 2012.

The first phase of implementation was the pilot phase, which involved the transition of the following four departments to the new policies: National Defence, Royal Canadian Mounted Policy, Canada Border Services Agency, and Environment Canada. A key objective of the pilot was for the Treasury Board of Canada Secretariat (TBS) to obtain important feedback from the pilot departments to improve guidance, tools, training, and communications activities that will facilitate government-wide implementation of the new policies. As one of the deliverables from the pilot phase, this report describes key lessons learned by the pilot departments in implementing the new policies in their respective organizations. This report is intended for the successive waves of government organizations implementing these new policies.

The lessons learned detailed in this report were compiled from the following sources: two surveys of key stakeholders in the pilot departments and TBS program sector analysts, a review of voluntary report documentation provided by pilot departments, and a review of relevant notes from the Policy Pilot Interdepartmental Working Group.

Lessons Learned

The lessons learned presented here have been grouped into categories based on the four phases of implementation described in the Implementation Strategy for the Policy on Investment Planning—Assets and Acquired Services and the Policy on the Management of Projects: learning, planning, challenge, and Treasury Board submission. The first category assembles the lessons learned from the learning and planning phases of implementation, while the second category presents the lessons learned from the challenge and Treasury Board submission phases. What appears below is not exhaustive, but highlights some of the key observations stemming from the pilot. Following each finding is a description of the pilot departments' implementation approach or practices. TBS guidance is also included to highlight specific good practices or provide additional direction on policy expectations.

Learning and Planning Phases

1. Establish an appropriate governance structure for review and approval of key deliverables (e.g. Investment Plan, Organizational Project Management Capacity Assessment, Project Complexity and Risk Assessments).

Pilot Practices

In establishing the appropriate governance structure, a full range of options should be examined. Whether creating new committees or expanding the mandate of existing ones, there should be a documented and approved governance structure in place to guide transition and support compliance with policy requirements.

One department undertook a review of its governance structures within the first few months of initiating the pilot and identified key points within the existing processes to support the review and approval of departmental deliverables (i.e. specific committees).

Another pilot department leveraged existing governance structures to establish a distinct governance process for approving deliverables associated with the new policy requirements. A director general-level steering committee was also established to provide senior management guidance and oversight to the departmental pilot project team.

Good practice: In considering changes to governance structures, departments should examine a full range of available options to best meet their needs.

2. Engage senior management throughout.

Pilot Practices

All four pilot departments considered senior management engagement in the implementation process to be a critical component of success.

One of the pilot departments regularly engaged senior management through a number of presentations. By treating implementation of the new policies as a formal project, it also sought approval of the Project Charter and Project Implementation Plan.

Another pilot department developed a Policy Pilot Implementation Plan, which summarized the key deliverables and timelines and was endorsed by senior management. Its implementation team also carried out ADM-level interviews to solicit their views on asset investments and planned projects. During these interviews, policy requirements, asset requirements throughout their life cycles, and future Treasury Board submissions were discussed in addition to historical trends in acquired services expenditures, which would form the basis for funding allocation decisions for planned future investments.

3. Identify a project team to support departmental implementation of the policies.

Pilot Practices

Each of the pilot departments established multidisciplinary teams or working groups, composed of representatives from across the organization, to support implementation of the policies. Some had dedicated staff for the initiative. The pilot departments noted that a project team might include the following:

• Representatives from all branches that manage capital programs;
• Representatives from all branches that procure significant services;
• Representatives from the department's corporate services branch;
• Representatives from a corporate planning/ investment planning function;
• Communications advisers;
• Persons responsible for writing or revising departmental policies;
• Strategic planners;
• Project leaders and sponsors; and
• Risk management advisers.

Before launching their investment planning cycle, one of the pilot departments first established an Investment Plan Framework, developed through its existing governance structures. This approach helped to identify the relevant responsibility areas that would be involved in the larger planning effort. The department established an investment plan working group, supported by various sub-working groups (e.g. human resources, capital equipment, acquired services) responsible for integrating their particular investment demands within the limits of the allocated resource envelopes. This approach facilitated engagement across the department in support of planning activities and resource allocation decisions. Coordination across the sub-working groups was overseen by the investment plan working group, providing a forum through which an enterprise-wide view of the investment component elements was considered, balanced and consolidated with the resource limitations of the Departmental Investment Plan.

Good practice: A key component of success is to systematically identify and involve the right people and areas of responsibility early in the planning process. In addition to the representatives listed above that might be on the project team, officials in the enabler functions (e.g. HR, IM/IT) may also offer important contributions.

4. Allocate responsibility for specific implementation activities and results to specific individuals or positions.

Pilot Practices

All four of the pilot departments assigned responsibility for coordinating implementation activities to specific individuals or groups of individuals.

One of the pilot departments developed a work breakdown structure and performed a crosswalk between functional areas within the organization and responsibilities for the individual work components. Leads at the director general level were then assigned accountability for each work package, and timelines for the completion of each work component were developed to satisfy internal review and approval processes and TBS timelines.

Another pilot department treated the implementation of the new policies as a project, establishing a Project Charter that was signed by key stakeholders and a Project Implementation Plan that clearly identified and documented the work required to produce defined deliverables as well as the individuals responsible for specific tasks and results.

Good practice: Once the appropriate responsibility centres with a role in implementation have been identified, it is useful to develop a framework that not only assigns accountabilities and defines timelines, activities, milestones, and deliverables, but also integrates the governance structure and approval processes.

5. Adopt a department-wide approach, which involves functional experts from across the department, for completing the Organizational Project Management Capacity Assessment

Pilot Practices

A coordinated, enterprise-wide approach is essential when establishing the department or agency's capacity to manage projects. The investment planning cycle will provide for the identification of the different project responsibility centres and an understanding of the project management practices within the department, which should result in the relevant funding allocation decisions being made. It is key to note that all projects need not be subject to a single and identical set of management practices, but merely that the management practices in effect be appropriate for the group of projects being managed by the responsibility centre. Once the number of planned projects has been established and the responsibility centres identified, the Organizational Project Management Capacity Assessment (OPMCA) weighting methodology can be applied and relevant evidence for each of the assessment criteria can be documented.

An example of an effective approach employed during the pilot was for the department to establish a formal, multidisciplinary working group to complete the departmental OPMCA. The working group included representatives from the main project implementation areas as well as its corporate and policy groups. Working group members first completed the OPMCA questionnaire on an individual basis; they then met as a group to discuss and agree on evidence to support departmental scoring. While a strict numerical weighting, based on the precise number of projects to be managed over the planning horizon, was not applied in the initial iterations of the department-wide assessment, areas of higher project activity were given a higher weighting when determining the scores for individual questions. Once the investment planning process or cycle had been completed, the department was then able to accurately assess and justify its capacity class at the enterprise level.

Having determined that the vast majority of departmental projects are managed within two key areas of responsibility, an organization chose to prepare two separate OPMCAs, which were shared with TBS for discussion at a very early stage. These OPMCAs referenced and also included relevant supporting evidence that was provided by the respective responsibility centres. The assessments and the supporting evidence were shared with TBS through one departmental coordination point. Once a thorough review was completed, the department then aggregated the separate OPMCAs to determine the single capacity class for the department. The ratings for individual questions were merged using the appropriate methodology, based on a weighting of the number of projects undertaken by each of the two responsibility centres. The assessment then proceeded through the departmental validation process to receive deputy head endorsement.

Good practice: While a department may find it challenging early on to adopt an enterprise-wide approach for completing the OPMCA, all departments are required to complete one single enterprise-wide assessment and capacity class for Treasury Board consideration. In instances where a department's governance structure may cause challenges in producing a single initial assessment, the department may wish to explore options with TBS for weighting multiple assessments from the various responsibility centres. A department's ability to consider those options depends heavily on the maturity of its planning processes. Departments are encouraged to explore all options for ensuring an enterprise-wide approach from the outset and for developing a consistent and accurate measure of departmental capacity.

TBS notes that the Organizational Project Management Capacity Assessment Tool was developed to accommodate a weighting for each assessment criteria that reflects documented evidence. This weighting methodology is based on the precise number of projects, not dollar value. For a department to determine that an appropriate level of capacity is in place to effectively manage the planned projects, the number of planned projects must first be established.

Also, it is important to note that in many instances, OPMCA scores of 5 for individual assessment criteria require an extensive and thoroughly documented validation process. This level of rigour is not beneficial in most instances. A score of 4, which is a more realistic score to justify, indicates that there is sufficient evidence to confirm that the criteria are met in up to 90 per cent of all departmental projects.

6. Establish a consistent departmental interpretation of the Project Complexity and Risk Assessment requirements and criteria for supporting evidence.

Pilot Practices

Both the OPMCA and Project Complexity and Risk Assessments (PCRA) must be based on documented evidence. Some of the pilot departments added a comments column to the PCRA and OPMCA Excel tools to reference evidence in support of individual responses and document its justification. Documented evidence is important to support both internal validation and audit requirements as well as to facilitate discussions with TBS¹.

One of the pilot departments gathered a group of select employees, including representatives from regions, for an in-depth session to apply the draft assessment tool to several projects of different sizes and types and to ensure a consistent understanding of the questions.

Another pilot department provided presentations on the PCRA requirements to working groups, branch planners, and project managers. It also provided an internal directive concerning PCRA implementation and included a column with guidance on and interpretation of the questions in the Excel version of the PCRA tool. In addition, a frequently asked questions document was developed to record feedback on specific questions, and it provided names of contacts from each major branch of the department to assist with the tool. These branch representatives were expected to communicate on a regular basis to discuss consistency of assessments across the department.

Good practice: As a first step, departments should determine which projects will be assessed using the PCRA and discuss their proposed approach with TBS. They may also wish to establish guidance as to the stage at which a PCRA should be conducted (e.g. after initial planning, but before project definition). It is imperative that the process for completing and approving PCRAs be fully defined and supported by TBS.

TBS notes that PCRAs are to be applied to all “types” of projects, regardless of funding source (capital, operating), except those funded through grants or contributions.

It is also important that departments identify an appropriate responsibility area to control and manage the online application.

7. Review data sources early on or undertake an early, initial OPMCA to identify key weaknesses and gaps in organizational capacity.

Pilot Practices

It is understood that departments are already engaged in planning and managing projects. The new policies do not require departments to establish completely new planning processes or project delivery environments as part of their transition efforts. The expectation is that current processes and practices will be defined and any action plans for improvement will be identified.

Early in the process, one of the pilot departments assessed its existing data sources for assets, acquired services, and projects and identified certain information gaps. The team responsible then reviewed the gaps that were identified and developed an approach for gathering necessary information and filling those gaps; however, recognizing that only some issues could be addressed within the immediate time frame, the team established targets for the work that will continue after Treasury Board's consideration of the deliverables.

Another department's approach was to complete an initial OPMCA and, based on the results, undertake to develop a work plan to strengthen key areas of weakness.

Good practice: Departments should establish an implementation plan early on and outline what is achievable within their transition timelines. It is recommended that early discussions about this plan be held with TBS to arrive at some common set of expectations to be met within the transition timelines.

TBS recognizes that addressing all areas for improvement may not be desirable or cost effective for a department and implementing all potential changes and demonstrating results may take more time than is provided for in the initial transition timelines.

8. Engage TBS early to determine which acquired services investments to profile in the Investment Plan.

Pilot Practices

As with all investments planned over a 5-year horizon, TBS engagement is key to determining which investments in acquired services to profile in the planning document. The acquisition of services in many instances is a critical component of departmental operations, as is the acquisition, maintenance, use, and disposal of capital assets. Given the importance of these investments, their consideration must be included in any planning activities and subsequent funding allocation decisions. Once this information is available and contextualized in the plan for the deputy head and Treasury Board ministers, high-risk investments or those with the greatest magnitude or sensitivity can and should be identified and specifically profiled in the plan in sufficient detail.

During the pilot, each of the departments struggled with identifying acquired services investments. In most departments, there is no central point of coordination for these investments and many are planned and executed at delegated levels in the department, without central monitoring and/or reporting. While it is key to note that this may not necessarily be a significant risk depending on the governance structure in place and the low number and dollar value of these service contracts, planning for these investments is clearly critical for demonstrating the affordability of the department's overall investment plan.

One pilot department created a separate section in its investment plan dedicated to acquired services investments. In this section of the plan, the department contextualized its planning processes with respect to acquired services, explained its total planned investments in acquired services, and highlighted the key groupings of those investments. It also detailed some of the key individual acquired services investments that are planned over the next five years.

Good practice: The requirement that acquired services be included in the investment plan is meant to ensure departments provide evidence of the affordability of all of their planned investments over their entire life cycle, from planning to disposal. It is also expected to provide deputy heads and Treasury Board ministers with more and better information at the planning stage of all significant and potentially sensitive investments, including the acquisition of services.

Although departments are required to demonstrate the affordability of their planned investments, there is some flexibility regarding how to identify, classify, categorize, and profile their planned acquired services. Each department's approach, however, should reflect its operational reality and be able to withstand external scrutiny.

Challenge and Treasury Board Submission Phases

1. Ensure the Investment Plan is targeted to a ministerial-level audience.

Pilot Practices

Both the Investment Plan and the proposed Organizational Project Management Capacity Class should be appropriately contextualized for Treasury Board ministers. The submission should set expectations of an organization's appropriate capacity class that are clear to a ministerial audience. Typically, the capacity class would be expected to be commensurate with the level of risk and complexity of most of the projects the organization plans to undertake.

To make the Investment Plan relevant to a ministerial audience and also a functional departmental planning document, one of the pilot departments created a higher targeted, more strategic-level Executive Summary.

Good practice: The submission should include a list of projects and an appropriate amount of detail so that Treasury Board ministers can make informed decisions on which additional projects may require subsequent Treasury Board approval.

2. Ensure that project submissions identify key project risks and/or complexity factors.

Good practice: Project submissions should clearly explain what makes a specific project risky or complex and identify what measures the department proposes to effectively manage the risk and ensure appropriate oversight, such as implementing the project in phases or conducting independent reviews. Consideration should also be given to how the key project risks link to the organization's capacity assessment (e.g. does it relate to an area where the department has a particular strength or weakness).

Lessons Learned—Going Forward

As described earlier, these lessons learned were collected over the pilot implementation phase of the new Policy on Investment Planning—Assets and Acquired Services and the Policy on the Management of Projects. Over the remaining three years of the transition period for the new policies, TBS will continue to monitor and document progress in these pilot and other departments. As a result of this growing knowledge base and departmental experience, this document will be regularly updated over the coming years with new findings to help guide departments in the transition to the new policies and the development of meaningful investment plans and accurate OPMCAs and PCRAs.

Footnotes

¹ Note that the online application and Excel workbook include a text box for each assessment criteria to support the documentation of evidence and its rationale. [ Return ]