ARCHIVED - Summative Evaluation of the Pilot Project on Non-Lapsing Appropriations for Capital Asset Management

• Previous Page
• Table of Contents
• Next Page

This page has been archived.

Acknowledgements

Goss Gilroy Inc. would like to thank all the individuals who contributed to this evaluation—interviewees, survey respondents, and the members of the Evaluation Advisory Committee for the Pilot Project on Non-Lapsing Appropriations for Capital Asset Management for their valuable input and support. Goss Gilroy Inc. would also like to acknowledge the role of the Internal Audit and Evaluation Bureau at the Treasury Board of Canada Secretariat who collaborated in finalizing the report.

Abbreviations

Abbreviations	Description
AAFC	Agriculture and Agri-Food Canada
ARLU	Annual Reference Level Update
CAR	Capital Asset Review
CCG	Canadian Coast Guard
CRA	Canada Revenue Agency
DFAIT	Foreign Affairs and International Trade Canada
DFO	Fisheries and Oceans Canada
DPR	Departmental Performance Report
MAF	Management Accountability Framework
MOU	Memoranda of Understanding
NLA	Non-lapsing appropriation
OAG	Office of the Auditor General of Canada
PWGSC	Public Works and Government Services Canada
RCMP	Royal Canadian Mounted Police
RPMPD	Real Property and Materiel Policy Division
Secretariat	Treasury Board of Canada Secretariat
SOP	Standard Operating Procedure
SPA	Special Purpose Allotment

Executive Summary

Evaluation Objectives and Governance

The evaluation of the Pilot Project on Non-Lapsing Appropriations (NLAs) for Capital Asset Managementwas undertaken as a Treasury Board commitment and part of the Secretariat's Five-Year Evaluation Plan. The evaluation was conducted in 2009–10 with the following objectives:

• Assess the relevance of two-year non-lapsing appropriations;¹
• Assess the overall implementation and designof thepilot project;
• Determine the extent to which the pilot project has achieved immediate outcomes and has made progress toward intermediate outcomes (i.e., its success); and
• Document best practices and lessons learned and identify possible alternatives.

Using multiple lines of evidence, 45 documents were examined, 33 key informant interviews were conducted and all related administrative data were analyzed. Due to inadequate administrative data sets, the evaluation was limited by its reliance on qualitative data.

The evaluation's governance structure included an interdepartmental advisory committee, the pilot project working group, the program office and the Secretariat's Internal Audit and Evaluation Bureau. This evaluation was also guided and reviewed by the Departmental Evaluation Committee, which recommended it for approval by the Secretary.

Background

The 2004 Capital Asset Review (CAR), named as one of the government operational reviews, was undertaken by the Secretariat with the aim of identifying "best practices, problem areas and solutions" in the management of capital assets. This was achieved by examining overarching framework issues, such as asset management policy, asset information, funding for assets and historical expenditures—all in the context of best practices.

The CAR noted several issues including the lack of funding for adequate maintenance of capital assets, which is widely believed to increase costs nearly fivefold. This lack of funding led to addressing pressures in the short-term at the expense of the long-term stewardship of assets. The CAR recommended providing "non-lapsing asset funding over a 3 to 5 year time frame, contingent upon the quality of the long-term asset plan and department performance."  This recommendation would allow departments to "carry unexpended asset funds forward from one fiscal year to the next and [would] give managers a more stable funding base for planning their operations without fear that their resources will be cut off." During this period, a report from the Office of the Auditor General of Canada (OAG) also recommended that capital assets be managed using an accrual method of accounting (i.e., departments must improve and use financial information for their daily decision-making, management and reporting practices).

Prior to the launch of the NLA pilot project, only two other carry-forward mechanisms existed: reprofiling and the 5% carry-forward. The 5% capital carry-forward allows eligible departments and agencies to carry forward capital budget amounts from one fiscal year to the next, up to 5% of their capital vote to a maximum of $75 million. Reprofiling, on the other hand, has no explicit limit, but requests must meet eligibility equirements and are subject to certain conditions. Neither mechanism, however, supports long-term planning and management needs as stated in the CAR.

In response to the CAR recommendations, the Secretariat embarked on a pilot project involving four federal organizations: Agriculture and Agri-food Canada (AAFC), Foreign Affairs and International Trade Canada (DFAIT), the Royal Canadian Mounted Police (RCMP), and Fisheries and Oceans Canada (DFO). The pilot's objectives were as follows:

• Improved overall effectiveness of capital spending (including improved management decisions and enhanced value for money);
• A move toward a longer-term, strategic investment approach consistent with the nature of the asset; and
• An ability to respond to recommendations from the OAG regarding accrual accounting (i.e., the OAG recommended that departments improve and use accrual financial information for daily decision making).

Evaluation Findings and Conclusions

Relevance

The NLA pilot project responds to the recommendations from the CAR and the OAG. Its objective, namely, improved overall effectiveness of capital spending, shows clear alignment with federal government priorities for delivering results to Canadians while maintaining fiscal discipline.

The evaluation of the pilot project demonstrates that departments see a need for a mechanism such as NLA to facilitate the management of federal government capital programs while taking into account the uncertainties faced in managing them. Both participating and non-participating departments believe that no other carry-forward mechanism provides more certainty and advantageous timing than NLA to enable overall effective project management.

Performance

The pilot project produced a strong base of qualitative evidence that shows progress on immediate outcomes including improved financial management, project management and risk management. Similarly, the evaluation found progress on intermediate outcomes, such as enhanced value for money, improved management decisions and optimal allocation of resources in investment plans.

The overall evidence, ranging from project-level benefits to increased rigour in governing and managing departmental capital asset investments, links better capital asset decisions to the pilot's design. These improvements were overwhelmingly attributed to the flexibility, timing and predictability provided by the pilot's non-lapsing mechanism, as follows:

1. Timing—Most key informants provided positive feedback on the timing for identifying carry-forward amounts. The pilot allowed participants to identify funds after the March 31 year-end, which contrasts with existing mechanisms that require departments to predict their lapses up to six months in advance.
   
   
2. Predictability—Because departments make the decision themselves to carry forward, the availability of funds is not contingent on Treasury Board approval and the related uncertainty. By providing an alternative to lapsing funds, the pilot altered the decision-making environment, allowing managers to:
   • plan over a longer duration;
   • apply more disciplined focus on project management; and
   • make higher-quality decisions.
3. Flexibility—Flexibility was identified as a key advantage in determining the amount of funding to carry forward and in adjusting project schedules and activities to changing circumstances. The evaluation found that departments with the most flexibility under the NLA pilot were also the most successful at minimizing net lapses.

Although results were not quantified, the evaluation did find that participating departments demonstrated behaviour that would generally be considered cost-effective, such as the following:

• Ensuring that funding availability corresponds to the optimal construction season;
• Targeting certain times of the year to minimize costs related to construction cycles, availability of materials and contractors;
• Maintaining focus on project-related timelines rather than on corporate reporting timelines;
• Maintaining original project scope without adversely impacting other operations or projects; and
• Delaying project start dates to ensure adequate internal capacity.

Options

The following four options are proposed for discussion as a follow-up to the evaluation:

1. Extend the pilot project temporarily;
2. Extend NLA to qualifying departments;
3. Discontinue the NLA mechanism, enhance existing mechanisms and make available to qualifying departments; and
4. Expand the NLA mechanism to be available to all custodial departments.

Since the evaluation demonstrates that the pilot was successful, the option of discontinuing the NLA mechanism without considering an alternative was not included.

Recommendations

1. Given that the Secretariat had various views on NLA, it would be beneficial for the department to develop a position on the NLA pilot project in light of the evaluation results and in relation to the other mechanisms. This process would facilitate a discussion and enable a decision regarding the proposed options. The discussion would address key questions including the following:
   • To what extent can the 5% carry-forward and reprofiling mechanisms be modified to build in the flexibility, timing, and predictability that resulted in the improved decision making when NLA was used? What are the benefits and risks of this?
   • Would enhancing existing mechanisms still result in improved decision making, financial management and project management?
   • What criteria should be used to qualify a department to participate in a new mechanism or enhancements to an existing mechanism?

   It is recommended that the Secretariat review existing mechanisms to determine whether they can be modified to build in the flexibility, timing and predictability that resulted in the improved decision making under the NLA pilot. Based on the results of the evaluation and the review, the Secretariat would then determine which of the options (2, 3 or 4) should be implemented.

   If the NLA pilot is continued or an existing alternative mechanism is enhanced (options 1 to 4), the following recommendations are made:

2. It is recommended that a performance measurement strategy be put in place with clear reporting requirements for participating departments.
   
   
3. Clear criteria for the selection of departments and design of projects for NLA should be developed and communicated. Criteria should be linked to the department's investment plan.
   
   
4. Roles and responsibilities between and within the Secretariat, the Department of Finance Canada and departments should be clearly defined, articulated and communicated.
   
   
5. SOPs regarding the functioning of the mechanism, rules and restrictions, and reporting requirements should be developed, articulated and communicated.
   
   
6. Departments should be reassessed for qualification whenever submitting a revised investment plan. Their capacity and track record (e.g., ability to carry out the projects in the investment plan, net and gross lapsing, MAF ratings, audit results, and results against indicators in the performance measurement strategy) should be carefully reviewed for this reassessment.

• Previous Page
• Table of Contents
• Next Page