This page has been archived.
Information identified as archived on the Web is for reference, research or recordkeeping purposes. It has not been altered or updated after the date of archiving. Web pages that are archived on the Web are not subject to the Government of Canada Web Standards. As per the Communications Policy of the Government of Canada, you can request alternate formats on the "Contact Us" page.
The successful monitoring of Treasury Board (TB) policies is critical in today's environment of rapid change and challenges in addressing policy compliance and effectiveness. It is essential that effective policy monitoring practices be in place at the Treasury Board Secretariat (TBS) to ensure the success of TB policies. This review was requested by the TBS Internal Audit and Evaluation Committee to provide TBS management with an independent study of leading practices in the area of policy monitoring. The review was designed to assess the current status of policy monitoring within TBS and provide practical examples of leading practices. The recommendations made in this report will provide guidance for the Policy Review and Reporting Project as it proceeds to streamline TB policies and reporting requirements.
Leading Practices Research
The review has researched leading practices for policy monitoring as documented in literature. Our research indicates that the critical success factors for implementing effective monitoring and reporting include the following:[1]
Proposed TBS Framework for Policy Monitoring (the "Framework")
The review also studied policy monitoring practices at TBS by reviewing seventeen TB policies. Based on our findings, leading practices have been presented in a framework to provide a tool for TBS to effectively analyze policy outcomes and monitoring activities in a consistent and systematic manner.
There are four levels of focus in the Framework:
1. Level of Risk – Risk is the overriding factor that impacts all elements of policy monitoring. Policies that have greater risk need to be emphasized to ensure they have effective monitoring procedures in place. The continued effectiveness of these procedures should be evaluated on an on-going basis.
2. Policy Objectives and Outcomes – Policies are designed to support the achievement of an organization's objectives and the management of risks. The following elements are fundamental requirements for an effective policy to achieve its desired outcomes:
3. Process Enablers – Policy monitoring "process enablers" are mechanisms that assist TBS and departments with information sharing and reporting in order to achieve TBS policy objectives. Policy monitoring is effective where process enablers are systematically applied to develop and implement policy and monitor outcomes.
The Framework defines seven process enablers. The one that appears to be the most effectively applied is TBS and departmental working groups and committees. The process enabler that appears to be the least effectively applied at TBS is committed funding, due to insufficient resources.
4. Monitoring and Reporting Activities – Monitoring and reporting requires multiple approaches and activities for information gathering and follow-up. Information assists TBS and departments in setting priorities and compiling outcome gap analysis. Structured reporting relationships are required between TBS and departments to report and analyze outcomes on a timely and systematic basis.
The Framework defines ten policy monitoring and reporting activities. The one that appears to be the most effectively applied is TBS' ongoing advice to departments. The activities that have been under-utilized or are the least effectively applied are the use of key performance indicators, ongoing reporting, analysis and follow-up by TBS, and dependence on internal audit and evaluation reports.
The Framework has been presented to and discussed with the representatives of the TBS policy centres, and other TBS staff we interviewed in the study. The Framework was well received and there was a general consensus that adoption of the Framework will lead to improved policy monitoring.
Summary of Findings Related to Selected TB Policies
Each element of the Framework was compared to the results from the sample of policies studied. We grouped the sample of policies by level of risk, and charted the level of monitoring identified in comparison to a desired level. From this exercise, we made three observations.
Areas for Improvement
Our study has revealed that certain elements described in the Framework have been adopted within TBS. We also observed that policy areas generally understand the level of risk, a fundamental component of the Framework. However there is an inconsistent application of the monitoring elements throughout TBS and certain practices are more effective as TBS monitoring tools.
The report recommends that TBS should adopt the Policy Monitoring Framework as a tool for a systematic approach to defining policy monitoring and reporting for policies that are developed in the future. A portfolio of the Framework's process enablers and policy monitoring and reporting activities should be selected and tailored to each policy area based on the level of risk.
Recommendations are also made for improvement in the use of nine of the elements of the Framework. Of these, TBS should initially focus on the following four areas.
1. Key Performance Indicators
We recommend that TBS continue to establish key performance indicators (KPIs) as a key requirement for policy development and monitoring. KPIs should be practical and cost-effective to measure.
2. TBS Committed Funding
We recommend a more effective use of TBS resources in conducting monitoring of policy outcomes. Steps would include:
3. Ongoing Reporting and Analysis
We recommend that TBS adopt a structured reporting approach to policy monitoring including:
TBS providing more meaningful and value added feedback to departments comprising government wide trends and benchmarks as well as departmental comparisons.
4. Actionable Items
We recommend that consideration be given to improving mechanisms for information sharing within TBS and to convert information into actionable items for correction including:
Management response
Due to the nature of the report, management has chosen to provide an overall response to the findings and recommendations. See the management response.