Horizontal Internal Audit of Compliance with the Policy on Management, Resources and Results Structures

March 2013
Office of the Comptroller General

Contents

Executive Summary

The objective of the audit was to assess compliance with the Policy on Management, Resources and Results Structures (Policy on MRRSFootnote 1).

Why This Is Important

Effective performance information enables departmentsFootnote 2 to ensure accountability, value for money, and overall effectiveness of operations. Performance information underpins management decision making at all levels of government and supports more effective resource allocation decisions when results are integrated into future business plans.

The Policy on MRRSFootnote 3 is designed to ensure that the government and Parliament have integrated financial and non-financial program performance information to support allocation and reallocation decisions in individual departments and across the government. The Policy on MRRS enables the establishment of a structure and an inventory of all Government of Canada programs to which expected results and performance indicators are linked to facilitate performance management.

The Policy on MRRS reinforces the government’s commitment to strengthen public sector management and accountability by providing a standard basis for reporting to citizens and Parliament on the alignment of resources, program activities and results. The Policy on MRRS also provides for linking interdepartmental financial and non-financial information to facilitate government-wide decision making.

Although the Policy on MRRS does not apply directly to horizontal initiativesFootnote 4, its underlying principles are particularly relevant to horizontal initiatives, given the risks related to initiatives involving multiple departments; specifically risks to the clarity of accountability, the rigour of governance, and the manner in which outcomes associated with the initiatives are aligned with the Government of Canada’s performance objectives.

Key Findings

Departments have established their Strategic Outcomes and most departments have developed accountability structures to support these. The majority of departments have Strategic Outcomes and accountability structures that are clear and well defined.

In order to improve the understanding of policy requirements, the Treasury Board of Canada Secretariat (the Secretariat) has delivered various outreach and leadership activities to departments. However greater central agency guidance is required to support departments in the implementation of the Policy on MRRS, mainly in the areas of performance measurement, the definition and allocation of internal services costs, and the demonstration of horizontal linkages of Strategic Outcomes.

Specifically, the Policy on MRRS states that the Program Activity Architecture (PAA) should serve as the basis for departmental resource allocations however, given other expenditure management requirements, including the current Parliamentary vote structure, the PAA is currently not the sole tool being used for resource allocation within departments and across government. Clarification from the Secretariat regarding the requirements of the Policy on MRRS with respect to managing departmental resources and the use of the PAA as the basis for allocations would therefore be beneficial.

Departments have created Performance Measurement Frameworks (PMFs) and are collecting performance data, but departmental performance measurement processes do not provide the information needed to assess how programs are achieving their expected results. Incomplete performance information does not support effective and accurate planning for decision makers.

Although the Policy on MRRS does not directly apply to Horizontal Initiatives, departments have for the most part applied the principles of the Policy on MRRS to horizontal initiatives. Departments have adequately designed accountability structures and have defined their roles and responsibilities for their participation in horizontal initiatives. Opportunities exist to improve the accountability structures of horizontal initiatives. Additional guidance from the Secretariat for the implementation and management of horizontal initiatives would be beneficial. While most departments are monitoring their departmental horizontal initiative activities, they are not publicly reporting on the results of the overall initiative.

Conclusion

Both large and small departments are generally compliant with the requirements of the Policy on MRRS. Departments would benefit from additional guidance in the areas of performance measurement, the definition and allocation of internal services and program linkages across departments. Clarification from the Secretariat of the Policy on MRRS with respect to the management of departmental resources and the use of the PAA as the basis for allocations is needed. Although departments have created PMFs, performance measures could be improved to further support departmental planning and decision making.

Overall, the principles of the Policy on MRRS are being applied to horizontal initiatives; however, explicit guidance from the Secretariat on implementing and managing these initiatives would be beneficial. Departments should make improvements in implementing performance measures for horizontal initiatives and ensuring that performance results are publicly accessible.

Conformance With Professional Standards

This engagement was conducted in accordance with the Internal Auditing Standards for the Government of Canada. A practice inspection has not been conductedFootnote 5.

Anthea English, CA
Assistant Comptroller General
Internal Audit Sector, Office of the Comptroller General

Background

In response to the Government of Canada's goal of improving information on program performance for decision making and accountability, the Expenditure Management System (EMS) of the Government of Canada was introduced in 1995. The objective of the EMS is to deliver quality programs and services, described in departmental service standards, within the resources that Canada can afford.

The EMS introduced two new annual reporting requirements as part of the Estimates process:  the Report on Plans and Priorities (RPP) and the Departmental Performance Report (DPR). These documents are key in setting an expectation for departmental performance measurement. The RPP is tabled in the spring and establishes performance expectations and outlines the general direction the minister wants the department to take during the estimates year and the next two fiscal years. The DPR is tabled in the fall and provides information on results actually achieved in serving Canadians both as beneficiaries/users of specific services and as taxpayers.

To further advance performance measurement across government, the Policy on Management, Resources and Results Structures (Policy on MRRS) came into force in 2005, replacing the 1996 Policy on Planning, Reporting and Accountability Structure (PRAS). The Policy on PRAS focused on measuring high-level business lines, whereas the Policy on MRRS was intended to provide more comprehensive measures. The Policy on MRRS has been updated several times, most recently in 2012. It supports the development of a common, government-wide approach to the collection, management and reporting of financial and non-financial information on program objectives, performance and results. All federal organizations must plan and manage their operations and report their performance against Strategic Outcomes that represent enduring benefits to Canadians.

In , the EMS was renewed to ensure that program results and resources were aligned at all levels. The renewed EMS is based on three pillars. The first pillar, “managing for results” focuses on providing a framework for capturing, assessing and reporting financial and non-financial performance information. The second pillar, “up-front discipline” is intended to enhance the information content of the Memorandum to Cabinet in order to better support the decision-making process by ensuring that new expenditures are aligned with the results of existing expenditures. The third pillar, “ongoing assessment” is intended to ensure that all departmental programs are reviewed periodically for their relevance and effectiveness. Departmental MRRS form the basis of these reviews.

The establishment of MRRS in each department is a key element of the EMS because it provides a common framework within which financial and non-financial information is linked and managed within departments and across government. The Policy on MRRS also reinforces the government's commitment to strengthen public sector management and accountability, consistent with the Management Accountability Framework, by providing a standard basis for reporting to citizens and Parliament on the alignment of resources, program activities and results.

The deputy head of each department is accountable to his or her minister and to the Treasury Board for the establishment of an appropriate MRRS, for the management of the department and for public performance reporting. The Treasury Board of Canada Secretariat (the Secretariat) is working with stakeholders to develop a common framework to address key performance measurement requirements across the federal government.

As part of the establishment of MRRS, departments are expected to establish Strategic Outcomes, which stem from a department's mandate and vision; a Program Activity Architecture (PAA), which is an inventory of all the programs undertaken by a department, depicted in their logical relationship to each other and to the Strategic Outcomes to which they contribute; and a Performance Management Framework (PMF) that describes the performance data to be collected and the expected results to be achieved by its programs.

As a companion document to the MRRS requirement for public performance reporting, Canada's Performance is an annual report to Parliament on the federal government's contribution to Canada's performance as a nation. This report provides an overview of how the performance of individual departments contributes to broader, government-wide outcomes. Departments are required to indicate in their RPP and DPR the alignment of their program activities to Government of Canada outcome areas.

Horizontal Initiatives

Many of the objectives or results that the Government of Canada aims to achieve require the contribution of two or more departments (including Crown corporations), jurisdictions or non-governmental organizations. The ability to build alliances, form partnerships, and effectively manage horizontal initiatives is, in many cases, key to delivering services to Canadians.

Horizontal initiatives are established through existing expenditure management processes. These include the development of a Memorandum to Cabinet and the Treasury Board submission processes that are led by the Privy Council Office and the Secretariat respectively. Departments inform the Secretariat of their funding and operational requirements, and the Secretariat provides feedback and guidance to help departments receive approval from the Treasury Board. Once the Treasury Board ministers approve the implementation of the initiative, departments are bound by the terms of the submission.

Managing a horizontal initiative involves entering into an agreement with partners where there is shared authority and responsibility, joint investment of resources (for example, time, funding and expertise), shared risks, mutual benefits and common results. One department is designated as the “lead” department, which may entail providing secretariat support to an interdepartmental governance committee, allocating funds to participating departments, and reporting on the overall progress of the initiative.

Audit Objective and Scope

The objective of the audit was to assess compliance with the Policy on MRRS.

The audit examined the departmental MRRS, including, but not limited to, the design and clarity of Strategic Outcomes, the PAA and governance structures that were in place to manage program performance within departments. Although the Policy on MRRS does not specifically refer to horizontal initiatives, the audit examined how the policy's principles apply to horizontal initiatives.

The primary period of the review was the 2011–12 fiscal year. Plans and processes implemented for 2012–13 planning were also taken into consideration to supplement evidence, as required.

Fourteen large departments and seven small departments were reviewed during the audit. In addition, the Secretariat was also included because of its role in providing policy and functional guidance to departments.

Appendix A provides a list of the departments and the horizontal initiatives examined.

Appendix B provides a list of the audit criteria used to assess the audit objective.

Detailed Findings and Recommendations

Finding 1: Central Agency Guidance

Greater central agency guidance is required to support departments in the implementation of some aspects of the Policy on MRRS.

We examined how the Secretariat works with departments to ensure that the spirit and intent of the Policy on MRRS is understood and fully implemented. We also assessed how the Secretariat fulfills its role to provide leadership, advice and guidance in the use and advancement of integrated financial and non-financial program performance information across the government. Finally, we examined written guidance and outreach activities provided by the Secretariat to departments when establishing a MRRS, including when horizontal linkages are established between departments.

Central agency advice and guidance are required to ensure that departments are implementing the Policy on MRRS as intended and that the requirements are applied consistently across the government. Central agency leadership is key to ensuring that departments keep abreast of performance measurement techniques, and are in a position to share best practices. Although the Policy on MRRS encourages alignment of program activities within the department, the establishment of interdepartmental linkages presents challenges, especially from a reporting perspective. As a result, guidance in this area is important.

The Secretariat has delivered various outreach and leadership activities to departments as needed.

In accordance with the Policy on MRRS, the Secretariat provides leadership, advice, and guidance for departmental implementation of the Policy on MRRS. For example, the Secretariat has an outreach function that periodically holds information sessions for program managers within departments to communicate policy developments or to provide additional guidance in areas highlighted by departments. The Secretariat also interacts with all departments through an annual call letter process, whereby departments propose updates to the departmental MRRS (if needed and based on departmental monitoring of compliance to the Policy on MRRS). The Secretariat reviews submissions from the call letter process and changes that meet the established criteria are approved in accordance with the Policy on MRRS. Departments are required to reflect the approved changes in their reporting to Parliament the following fiscal year.

Performance measurement guidance is under development.

In addition to those requirements established in the Policy on MRRS, there are several policies with performance measurement requirements, including the Policy on Evaluation and the Policy on Transfer Payments. Each of these policies uses different terminology to define performance measurement requirements. This difference in terminology leads to confusion within departments as to what needs to be measured and reported, and how. As a result, performance reporting is not done uniformly and does not provide a complete picture of departmental performance for decision makers. The Secretariat has instituted a working group to develop integrated guidance for program managers on MRRS performance measurement requirements. The working group is developing a guidance document that will detail the linkages between the performance measurement definitions included in the various policies.

Internal services have been defined by the Secretariat, but the definitions are not being applied consistently across government.

For the purposes of the Policy on MRRS, internal services are defined as those that support the department and cannot be attributed to a specific program. The Instructions to Departments for Developing a Management, Resources and Results Structure identify eleven different departmental activities as internal services, including (but not limited to) human resources, finance and legal services. Although most departments use these services as an internal support function for their operations, some departments, such as tribunals, employ legal services directly in the delivery of their mandate. For example, one small department examined had allocated its legal costs between three program activities, but the other small departments included all legal costs as part of internal services. The definition and allocation of internal services costs varies across departments. Departments require additional guidance in implementing this element of the Policy on MRRS.

The Secretariat has established an interdepartmental working group to develop standardized performance measures for the eleven categories of internal services. Departments require additional instructions on what should and should not be recorded as a program cost. Without such guidance, program costs versus internal services costs are not being consistently presented in PAAs and reported to Parliament.

Linkages of Strategic Outcomes between departments are not evident.

The Strategic Outcomes of most large departments do not provide the basis for linking programs across government. Linkages between departments occur at operational levels that are not depicted within a departmental MRRS owing to the size and scope of departmental involvement. Although the Policy on MRRS refers to the Management, Resources and Results Structure as the basis for the establishment of interdepartmental linkages, the term “horizontal linkages’’ is not defined and instructions have not been provided to departments. As a result, linkages between Strategic Outcomes cannot be concretely demonstrated across government. More guidance from the Secretariat is required in this area.

Recommendations

  1. The Treasury Board of Canada Secretariat should review and clarify the Instructions to Departments for Developing a Management, Resources and Results Structure in areas including, performance measurement, the definition and allocation of internal services, and the establishment of program linkages across departments.

Finding 2: Strategic Direction and Accountability Structures

Strategic directions are established in departments, and accountability structures have been developed in most departments to support them.

We assessed whether departmental Strategic Outcomes reflect the mandate and vision of each department and provide the basis for establishing program and outcome linkages between departments. We examined the basis on which parliamentary reporting and estimates are presented. Finally, we reviewed the process by which departments hold senior managers accountable for MRRS outcomes.

Clearly defined Strategic Outcomes aligned with the mandate of the organization and to government-wide priorities increase the likelihood that government objectives will be achieved. Established linkages allow departments to show how collaboratively delivered programs affect their established Strategic Outcomes. Accountability structures are required to ensure that all stakeholders are clear on how decisions are made and who is responsible for those decisions.

Departments have Strategic Outcomes that are clear and well defined.

Both large and small departments have developed processes to establish and update Strategic Outcomes. Deputy heads have been involved in the approval of, and updates to Strategic Outcomes. Established Strategic Outcomes reflect the mandate and vision of departments. For most of the small departments assessed, their Strategic Outcomes directly reflect the legislation that created them.

Departments have adequately designed accountability structures.

Departments have established accountability structures, such as steering committees and performance agreements, to ensure program governance. Responsibility is formally assigned to departmental executives through established departmental accountability structures. Reporting to Parliament through the RPP and DPR processes is based on departmental PAA structures.

Senior managers are held accountable for agreed outcomes.

Large and small departments have implemented multiple approaches to hold senior managers accountable for outcomes. Although most departments use the Performance Management Agreement process to formalize accountability expectations, some rely on signed attestations or a balanced scorecard approach. We noted as a good practice one instance where a large department has instituted Strategic Outcome Committees. These committees are established for each Strategic Outcome to monitor performance, and they report results to both the Deputy Minister’s Policy Committee and the Departmental Management Board.

Finding 3: Resource Allocation

The PAA is not the sole tool for resource reallocation within departments and across government.

We assessed the process for identifying, grouping and linking program activities to the Strategic Outcomes they support. We examined the process for approval of the PAA, and we assessed whether departments are allocating and reallocating resources based on the PAA structure.

Departmental program activities should support Strategic Outcomes to ensure that all activities support the organization’s mandate and government priorities, with the PAA being the basis for resource allocations. Approval of the PAA ensures that resource allocation supports government priorities. Linking resource allocations and reallocations to the PAA allows departments to report on results based on expected performance.

PAAs identify and group related activities in small departments; however, some large departments are still encountering challenges.

All small departments examined had PAAs that were clear and complete groupings of all of the programs the departments manage. The programs in the PAAs of most small departments were drawn directly from their enabling legislation. As an example of a good practice, one small department has created a logic model that provides a visual overview of how all the components of the MRRS and other strategic measures work together, ensuring that all programs and projects support the organization’s MRRS.

Some large departments are experiencing difficulties in identifying all related activities in their PAA. The size, scope and diversity of programs within some large departments create difficulties in explicitly aligning and defining all program activities. This may present a challenge in ensuring that all activities within large departments are clearly tied to their mandate.

The PAA is one of the tools used to manage departmental resources.

The Policy on MRRS states that the PAA should reflect how a department allocates and manages its resources and should serve as the basis for allocations by Parliament, the Treasury Board and departmental management. Most small departments have linked their initial budget allocations and management of their departmental budget to their PAA structure. The allocation and management of departmental budgets according to the PAA is facilitated in part by the size and focused mandate of most small departments.

All large departments link their initial budget allocations and year-end reporting to their PAA; however, because of other expenditure management requirements, including authorities set by votes and Parliamentary allocations, departments primarily manage their resources throughout the fiscal year according to departmental organizational structures, which are not always aligned with their PAAs. Although the PAA represents only one of the tools used by large departments to manage resources, the Policy on MRRS does not acknowledge other expenditure management requirements for managing departmental resources. This causes confusion as to the extent to which the PAA should serve as the basis for departmental allocations. There is no guidance related to how departments are expected to perform a mapping between organizational units and PAA elements beyond what is done annually. As a good practice, some departments can perform this mapping at any time in the fiscal year through automated systems.

We noted that as part of the Strategic Review process, departments were required to use the PAA as the basis for reallocation proposals. We encourage the Secretariat to examine opportunities to further utilize the PAA as a basis for ensuring sound resource allocation and reallocation decisions within departments and across government.

Recommendations

  1. The Treasury Board of Canada Secretariat should clarify the requirements of the Policy on Management, Resources and Results Structures with respect to the management of departmental resources and the use of the PAA as the basis for allocations.

Finding 4: Performance Monitoring

Departmental frameworks to measure performance do not always provide the results information needed to assess how programs are achieving their expected results.

We examined whether departments had developed PMFs and were using performance measures to track the performance of departmental programs against expected results.

An objective and measurable PMF provides assurance that intended results are achieved and, if necessary, facilitates proactive corrective measures. The timely collection and analysis of performance information enables management to make reallocation decisions and continuous improvements to business operations.

Departments have created Performance Measurement Frameworks and are collecting performance data, but this information does not support planning and decision making.

Most large and small departments have developed PMFs and have integrated monitoring of performance into their business processes. In some large departments examined, performance indicators specified in the PMF did not measure the real impact of the program on the lives of Canadians; in other large departments, performance measures had not been specified for all elements of the PAA; and in one large department, client survey information was planned to be reported in the DPR, but the survey was not carried out because of a lack of resources. In this instance, the department monitored the information to be provided by the survey via formal and informal meetings with clients. Incomplete performance information does not support effective and accurate planning for decision makers.

As an exemplary practice, one small department has established a scorecard for each program activity, which highlights expected results and targets. Senior managers present their scorecard to the Senior Management Committee quarterly. Another small department has included its performance measures in its Integrated Plan and logic model. This department has included the timeframe within which these measures are expected to be achieved.

Recommendations

  1. As part of their Performance Measurement Frameworks, departments should improve Management, Resources and Results Structures performance measures, in support of planning and decision-making processes.

Finding 5: Horizontal Initiatives

Central agency guidance is required to support departments in the management of horizontal initiatives.

We examined the guidance on horizontal initiatives that the Secretariat provides to departments. In light of the principles of the Policy on MRRS as a framework for management, we examined the roles and responsibilities of the departments participating in a number of horizontal initiatives and the accountability requirements. Finally, we examined whether departments had developed performance measures and tracked the performance of horizontal initiatives against expected results, as well as how overall performance of the initiatives are reported publicly. See Appendix A for a list of Horizontal Initiatives examined.

Central agency advice and guidance is important to ensure that departments are implementing key management structures that will enable sound performance management of horizontal initiatives. Given that horizontal initiatives include reporting lines outside a department, they present a challenge for reporting on an initiative as a whole. Without a common set of strategic objectives and expected results, horizontal initiatives may be delivered by departments in a fractured manner without alignment. Accountability structures are required to manage the day-to-day delivery of horizontal initiatives and to support the principles of the Policy on MRRS. Individual roles and responsibilities must be defined for participating departments to ensure horizontal accountability. An established accountability structure ensures that all stakeholders are clear on how decisions are made and who is responsible for those decisions. An objective and measurable PMF provides assurance that intended results are achieved and, if necessary, facilitates proactive corrective measures. A framework for measuring performance ensures that activities are linked to the initiative’s objectives; that performance information is collected and analyzed in a timely manner, enabling management to make allocation decisions, as required, and continuous improvements to business operations; and that the overall results of the initiative are reported on collectively and are accessible to Parliament and the public.

Departments have defined their roles and responsibilities for their participation in horizontal initiatives.

Roles and responsibilities are documented and agreed upon by all participating departments. Departments have established interdepartmental committees to set overall accountability for the initiatives; departments have also defined their roles and responsibilities for the initiatives. Expectations, including provisions for evaluation, are formally established for departmental roles in the initiatives. A good practice was noted in some initiatives, in which the expectations were elaborated in the Treasury Board submissions, which explained in detail the roles of the departments before funding for the initiative was approved.

Additional guidance is required for the implementation and management of horizontal initiatives.

Other than the RPP and DPR reporting requirements related to horizontal initiatives, there is no specific guidance supporting horizontal initiatives. In the absence of requirements or guidance, the principles of the Policy on MRRS serve as a framework for the management of horizontal initiatives. Although the Secretariat performs outreach to departments in implementing the Policy on MRRS, this outreach has focused on department-specific guidance rather than on issues unique to horizontal initiatives. The Secretariat’s support to departments in implementing horizontal initiatives is provided mainly through advice during the Treasury Board submission process. The absence of structured guidance has resulted in accountability requirements that vary across horizontal initiatives, and inconsistency in how horizontal initiatives are represented in departmental activities and reported on horizontally.

Opportunities exist to improve the accountability structures for horizontal initiatives

For the governance of horizontal initiatives, accountability structures such as steering committees or performance agreements exist. However, efficiencies in the delivery of some horizontal initiative activities are being compromised when approval processes in some departments take longer than in others. For example, four departments partnered to deliver an initiative and all were required to obtain internal approval before proceeding, even though funding was provided by the initiative. When one department encountered delays in obtaining internal approval, it was not possible for the other three departments to proceed with the delivery of the project.

Departmental executives are formally assigned responsibility through established departmental accountability structures, and departments place greater emphasis on vertical (that is, within their departments) accountability over horizontal accountability. Departments are able to monitor accountability for their horizontal initiative activities, but overall horizontal collaboration is being hindered in some departments.

Most departments are monitoring their horizontal initiative activities.

Consistent with the Policy on MRRS, departments are reporting vertically on the programs linked to horizontal initiatives that departments lead or contribute to. Departments have incorporated horizontal initiative activities into their common reporting processes, and are in a position to determine the effectiveness of their internal activities related to horizontal initiatives.

Departments have developed performance measures for their horizontal initiatives, but not all measures are fully implemented. The need to implement PMFs is established in Treasury Board submissions; however, such frameworks can be developed at a later date without necessarily having a defined timeline. Horizontal initiatives have frameworks available to measure the effectiveness of their activities, but these frameworks require full implementation in order to accurately measure performance.

Most departments are not reporting publicly on the overall results of horizontal initiatives.

Lead departments for horizontal initiatives are collecting performance information from participating departments in order to develop an annual report on overall performance results; however, the form and availability of the annual reporting varies by initiative. Treasury Board submissions outline the need for annual reporting but do not explicitly state in what form or how it will be made available.

One initiative has taken steps to actively monitor performance. The lead department publishes an annual report on their website detailing the activities of all participating departments and includes this information in its DPR; other initiatives do not publish performance information. The Secretariat currently has a database of government horizontal initiatives on its website that is populated by information provided by lead departments; however, the database is incomplete and is not kept up-to-date. Without a specified reporting mechanism, Parliament and the public are not in a position to obtain performance information on horizontal initiatives.

Recommendations

  1. The Treasury Board of Canada Secretariat should assess the need to develop guidance on horizontal initiatives, including, Treasury Board submission requirements specific to horizontal initiatives.
  2. Departments should finalize and implement frameworks for monitoring the performance of horizontal initiatives.
  3. The Treasury Board of Canada Secretariat should develop and implement a process to ensure that the website providing information on horizontal initiatives is kept up-to-date.

Management Response

The findings and recommendations of this audit were presented to each of the fourteen large departments and seven small departments included in the scope of the audit as well as to the Secretariat, because of its role as a central agency.

Management has agreed with the findings included in this report and will take action to address all applicable recommendations.

Appendix A: Departments Included in the Audit

The following table lists the departments and the horizontal initiatives examined in the audit.Footnote 6 A description of the three horizontal initiatives (the Genomics Research and Development Program [Genomics], the International Police Peacekeeping and Peace Operations Program [IPPPOP], and the Strategic Partnerships Initiative [SPI]) follows the table. Departments that did not participate in any of these three initiatives and had only their departmental Management, Resources and Results Structure (MRRS) examined are identified in the column titled “Departmental MRRS only.”

Department Name and Abbreviation Departmental MRRS and Horizontal Initiative Departmental MRRS only
Genomics IPPPOP SPI

Table 1 Notes

A: Applicable
N/A: Not Applicable

Table 1 Note 1

CAS was considered an LD for purposes of this audit. The Policy on Internal Audit was updated on , and CAS is now considered an SD.

Return to table 1 note * referrer

Table 1 Note 2

IRB was considered an LD for purposes of this audit. The Policy on Internal Audit was updated on , and IRB is now considered an SD.

Return to table 1 note ** referrer

Large Departments        
1. Agriculture and Agri-Food Canada (AAFC) A N/A A N/A
2. Aboriginal Affairs and Northern Development Canada (AANDC) N/A N/A A N/A
3. Atlantic Canada Opportunities Agency (ACOA) N/A N/A N/A A
4. Courts Administration Service (CAS)table 1 note * N/A N/A N/A A
5. Fisheries and Oceans Canada (DFO) A N/A A N/A
6. Foreign Affairs and International Trade Canada (DFAIT) N/A A N/A N/A
7. Human Resources and Skills Development Canada (HRSDC) N/A N/A A N/A
8. Immigration and Refugee Board of Canada (IRB)table 1 note ** N/A N/A N/A A
9. Industry Canada (IC) A N/A A N/A
10. National Research Council Canada (NRC) A N/A N/A N/A
11. Parks Canada (PC) N/A N/A A N/A
12. Public Health Agency of Canada (PHAC) A N/A N/A N/A
13. Public Safety Canada (PS) N/A A N/A N/A
14. Royal Canadian Mounted Police (RCMP) N/A A N/A N/A
Small Departments        
15. Canadian Radio-television and Telecommunications Commission (CRTC) N/A N/A N/A A
16. Copyright Board Canada (CB) N/A N/A N/A A
17. Federal Economic Development Agency for Southern Ontario (FedDev) N/A N/A A N/A
18. Financial Consumer Agency of Canada (FCAC) N/A N/A N/A A
19. Hazardous Materials Information Review Commission Canada (HMIRC) N/A N/A N/A A
20. Public Service Staffing Tribunal (PSST) N/A N/A N/A A
21. Transportation Appeal Tribunal of Canada (TATC) N/A N/A N/A A

Horizontal Initiatives Examined in the Audit

Genomics Research and Development Program (Genomics)

This initiative was established to build and maintain genomics research capacity in government and to deliver value-added applications for responding to national priorities and government mandates as well as for building wealth for Canadians.

International Police Peacekeeping and Peace Operations Program (IPPPOP)

This program was created to support the worldwide deployment of Canadian police officers and to develop a pool of pre-trained candidates, enabling Canada to be more responsive to foreign requests for police assistance.

Strategic Partnerships Initiative (SPI)

This initiative aims to increase economic development opportunities for Aboriginal entrepreneurs and communities by promoting partnerships between federal and non-federal partners in priority sectors of the economy, including forestry, fisheries, mining, energy and agriculture.

Appendix B: Audit Objective and Related Criteria

The objective of the audit was to assess compliance with the Policy on Management, Resources, and Results Structures (Policy on MRRS).

  1. Strategic Outcomes: With guidance and leadership from deputy heads, departments have established clear and measurable Strategic Outcomes that provide a basis for horizontal linkages.
    • Departmental MRRS have clearly defined and measurable Strategic Outcomes that reflect the department's mandate and vision as well as being linked to the government's priorities. (Policy on MRRS, Section 6.1.1.1)
    • Departmental Strategic Outcomes provide the basis for establishing horizontal linkages between departments. (Policy on MRRS, Section 6.1.1.1)
    • Deputy heads provide overall leadership in developing, in consultation with key stakeholders in the department, the changes required to the Strategic Outcomes and to the Program Activity Architecture (PAA). (Policy on MRRS, Section 6.1.4)
    • Deputy heads approve the departmental MRRS and ensure regular and timely reviews and updates take place to ensure its appropriateness. (Policy on MRRS, Section 6.1.2)
  2. Program Activity Architecture: Departments have explained their PAA in sufficient detail to reflect how resources are allocated and managed to achieve the intended results.
    • Related program activities are identified, grouped and linked to the Strategic Outcomes they support. (Policy on MRRS, Section 6.1.1.2)
    • Planned resource allocations, expected results, and performance measures are linked to program activities against which actual results are reported, and are used in monitoring performance and for decision making. (Policy on MRRS, Section 6.1.1.2)
    • The PAA is the basis for resource allocation at all levels of government. (Policy on MRRS, Section 6.1.1.2)
    • The Treasury Board of Canada Secretariat (the Secretariat) approves the PAA level, which allocates and controls resources. (Policy on MRRS, Section 8.1.1).
  3. Governance Structure: Departments outline responsibilities and accountabilities for decision-making mechanisms, monitoring and reporting.
    • The departmental MRRS outlines the decision-making mechanisms, responsibilities and accountabilities of the department. (Policy on MRRS, Section 6.1.1.3)
    • Departmental information systems, performance measurement strategies, and reporting and governance structures support the MRRS. (Policy on MRRS, Section 6.1.5).
    • Senior executives are accountable for outputs and outcomes set out in the MRRS. (Policy on MRRS, Section 6.1.6)
    • Departments monitor their compliance with the Policy on MRRS and notify the Secretariat of any changes they intend to make. (Policy on MRRS, Section 6.2.1)
    • Parliamentary reporting and estimates display are based on the departmental PAA. (Policy on MRRS, Section 6.1.1.2).
    • The terms and conditions of a horizontal initiative outline roles and responsibilities related to funding allocations, horizontal collaboration, and adjustments and improvements to the management of the horizontal initiative. (Horizontal Initiatives audit only)
    • Performance monitoring of horizontal initiatives is conducted on a regular and timely basis. (Horizontal Initiatives audit only)
    • Horizontal initiatives include provision for evaluation and adjustment. (Horizontal Initiatives audit only)
    • A framework for horizontal initiatives is used to report to Parliament through the Estimates documents and any other parliamentary reporting documents in the manner and form defined by Treasury Board or its Secretariat. (Horizontal Initiatives audit only)
  4. Role of The Secretariat: The Secretariat provides leadership, advice and guidance on the implementation of the Policy on MRRS
    • The Secretariat works closely with departments to ensure that the spirit and intent of the Policy on MRRS is understood and fully implemented. (Policy on MRRS, Section 7.1)
    • Changes contemplated to the Strategic Outcome and PAA levels are made only with further Treasury Board approval.(Policy on MRRS, Section 6.1.3)
    • The Secretariat provides leadership, advice and guidance in the use and advancement of integrated financial and non-financial program performance information across the government. (Policy on MRRS, Section 8.2).

Appendix C: References

Policies and Directives

  1. Policy on Management, Resources and Results Structures
  2. Policy on Evaluation
  3. Policy on Transfer Payments
  4. Cabinet Directive on Streamlining Information

Background Material

  1. Governing Responsibly: A Guide for Ministers and Ministers of State
  2. Expenditure Management System of the Government of Canada
  3. Better Practice Guide: Better Practice in Annual Performance Reporting (Australian National Audit Office)
  4. Canada's Performance
  5. Whole-of-Government Framework

Appendix D: Recommendations by Department

The full names of the departments are provided in Appendix A. For each recommendation, the letter “A” denotes that the recommendation applies to the department and will require a management action plan, and “N/A” denotes that the recommendation does not apply. The applicability of the recommendations to each department is subject to variability; some departments may not have begun to address these recommendations.

Recommendation Large Departments
Secretariat AAFC AANDC ACOA CAS DFO DFAIT HRSDC IRB IC NRC PC PHAC PS RCMP
1. The Treasury Board of Canada Secretariat should review and clarify the Instructions to Departments for Developing a Management, Resources and Results Structure in areas including, but not limited to, performance measurement, the definition and allocation of internal services, and the establishment of program linkages across departments. A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A
2. The Treasury Board of Canada Secretariat should clarify the requirements of the Policy on Management, Resources and Results Structures with respect to the management of departmental resources and the use of the PAA as the basis for allocations. A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A
3. As part of their Performance Measurement Frameworks, departments should improve Management, Resources and Results Structures performance measures, in support of planning and decision-making processes. N/A N/A N/A A A N/A A N/A N/A N/A N/A N/A A N/A A
4. The Treasury Board of Canada Secretariat should assess the need to develop guidance on horizontal initiatives, including, but not limited to, Treasury Board submission requirements specific to horizontal initiatives. A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A
5. Departments should finalize and implement frameworks for monitoring the performance of horizontal initiatives. N/A N/A A N/A N/A N/A A N/A N/A N/A N/A N/A N/A A A
6. The Treasury Board of Canada Secretariat should develop and implement a process to ensure that the website providing information on horizontal initiatives is kept up-to-date. A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A
Recommendation Small Departments
CRTC CB FedDev FCAC HMIRC PSST TATC
1. The Treasury Board of Canada Secretariat should review and clarify the Instructions to Departments for Developing a Management, Resources and Results Structure in areas including, but not limited to, performance measurement, the definition and allocation of internal services, and the establishment of program linkages across departments. N/A N/A N/A N/A N/A N/A N/A
2. The Treasury Board of Canada Secretariat should clarify the requirements of the Policy on Management, Resources and Results Structures with respect to the management of departmental resources and the use of the PAA as the basis for allocations. N/A N/A N/A N/A N/A N/A N/A
3. As part of their Performance Measurement Frameworks, departments should improve Management, Resources and Results Structures performance measures, in support of planning and decision-making processes. N/A N/A N/A N/A N/A N/A N/A
4. The Treasury Board of Canada Secretariat should assess the need to develop guidance on horizontal initiatives, including, but not limited to, Treasury Board submission requirements specific to horizontal initiatives. N/A N/A N/A N/A N/A N/A N/A
5. Departments should finalize and implement frameworks for monitoring the performance of horizontal initiatives. N/A N/A N/A N/A N/A N/A N/A
6. The Treasury Board of Canada Secretariat should develop and implement a process to ensure that the website providing information on horizontal initiatives is kept up-to-date. N/A N/A N/A N/A N/A N/A N/A

Appendix E: Risk Ranking of Recommendations

The following table presents the recommendations and assigns risk rankings of high, medium or low. The determination of risk rankings was based on the relative priorities of the recommendations and the extent to which the recommendations indicate non-compliance with Treasury Board policies.

Recommendations Priority
1. The Treasury Board of Canada Secretariat should review and clarify the Instructions to Departments for Developing a Management, Resources and Results Structure in areas including, but not limited to, performance measurement, the definition and allocation of internal services, and the establishment of program linkages across departments. High
2. The Treasury Board of Canada Secretariat should clarify the requirements of the Policy on Management, Resources and Results Structures with respect to the management of departmental resources and the use of the PAA as the basis for allocations. High
3. As part of their Performance Measurement Frameworks, departments should improve Management, Resources and Results Structures performance measures, in support of planning and decision-making processes. Medium
4. The Treasury Board of Canada Secretariat should assess the need to develop guidance on horizontal initiatives, including, but not limited to, Treasury Board submission requirements specific to horizontal initiatives. High
5. Departments should finalize and implement frameworks for monitoring the performance of horizontal initiatives. Medium
6. The Treasury Board of Canada Secretariat should develop and implement a process to ensure that the website providing information on horizontal initiatives is kept up-to-date. High

© Her Majesty the Queen in Right of Canada, represented by the President of the Treasury Board, [2013],
[ISBN: 978-0-660-25654-2]

Page details

Date modified: