Guideline on Performance Measurement Strategy under the Policy on Transfer Payments

Hierarchy

More information

Directive: 

Print-friendly XML

1 Introduction

The purpose of this guideline is to assist deputy heads and departmental managers in meeting their responsibilities under the Policy on Transfer Payments and the Directive on Transfer Payments as they relate to performance measurement strategies (see the Appendix for relevant excerpts from both the Policy on Transfer Payments and the Directive on Transfer Payments).

A performance measurement strategy (PM strategy) in the context of a transfer payment program is the selection, development and ongoing use of performance measures for decision-making in the context of program management. A PM strategy will assist in ensuring that there is adequate and relevant information to support deputy heads and departmental managers in:

  • the continuous monitoring and assessment of the results of the transfer payment program including its relevance and effectiveness as well as the efficiency of the management of the program;
  • making informed decisions and taking appropriate timely action with respect to the transfer payment program, including the amendment or termination of terms and conditions and/or funding agreements, where appropriate;
  • supporting statutory requirements under section 42.1 of the Financial Administration Act[1] and ensuring that the information gathered will support an evaluation or review of relevance and effectiveness of the transfer payment program and inform decisions on the continuation, amendment or termination of the terms and conditions; and
  • providing effective departmental reporting on the relevance and effectiveness of the transfer payment program.

The Policy on Transfer Payments requires a deputy head to establish a PM strategy for a transfer payment program at the time of program design, and ensure that it is maintained and updated throughout the life cycle of the program. This requirement is consistent with the Policy on Management Resources and Results Structures which sets out the expectation that departments collect and use sound financial and non-financial program performance information as articulated in the department's Program Alignment Architecture.

This guideline, which focuses specifically on performance measurement strategies for transfer payment programs, is a companion document to Supporting Effective Evaluations – A Guide to Developing Performance Measurement Strategies which provides a step-by-step process for identifying and planning for the collection of performance information to effectively support program evaluation.[2] In preparing a PM strategy for a transfer payment program, consultation with the departmental evaluation function and/or departmental performance measurement advisors in its development is strongly encouraged. These units have subject matter expertise in the selection of performance measures or indicators and in data collection. They can also offer assurance that the PM strategy will effectively support an evaluation of the transfer payment program. Consultation with other departmental functions, such as the corporate planning function, is encouraged as well.

2 Elements of a PM strategy for transfer payment programs

A PM strategy for transfer payment programs should include, as a minimum, the elements identified in the following figure.

Figure 1: Elements of a PM Strategy for Transfer Payment Programs

  • Select performance measures or indicators
  • Identify data requirements to support each of the performance measures or indicators
  • Identify sources of data
  • Identify targets and timelines for achievement of performance measures or indicators
  • Identify means and methods to be used to collect data
  • Establish responsibilities for collection
  • Determine frequency of collection
  • Determine responsibilities for data analysis and synthesis, and reporting

2.1 Performance measures or indicators

  • Select performance measures or indicators

The PM strategy should identify the performance measures or indicators to be used. The key measures and indicators must be included in the terms and conditions of the transfer payment program and should be selected to:

  • measure the progress of the transfer payment program toward its desired results and its achievement of departmental and government objectives;
  • be useful in the measurement of the relevance and effectiveness of the transfer payment program;
  • include the measurement of efficiency of the transfer payment program;
  • meet any defined reporting requirements and effectively support evaluation;
  • be valid and reliable;
  • be sufficient in numbers bearing in mind the availability of data and the cost of collection of the data that would be required to support them.

Where a transfer payment program is shown as a distinct program within a department's Program Alignment Architecture (PAA) and performance measures have been identified in the department's supporting Performance Measurement Framework, the PM strategy should reflect these performance measures. In cases where a transfer payment exists as a component within a program in the PAA, the Performance Measurement Framework should also be reviewed for any relevant, pre-existing performance measures that could be referenced in the PM strategy.

2.2 Required data

  • Identify data requirements to support each of the performance measures or indicators
  • Identify sources of data
  • Identify targets and timelines for achievement of performance indicators

The PM strategy should identify the data that will be required to support each of the selected performance measures or indicators and identify the sources of this data. There may be some repetition between this step and step 2.1 above. As the availability and cost of collection of data is explored, it may prove necessary or desirable to redefine performance measures or indicators.

Sources of data may include:

  • Administrative records and internal databases relating to the management of the transfer payment program;
  • Reports from recipients: it is important to identify the data that will be collected from recipients. This has to be described in the terms and conditions of the transfer payment program, and be identified in funding agreements;
  • Specialized data collection exercises such as focus groups, expert panels or surveys;
  • Data collected for other purposes such as national statistics, e.g. Statistics Canada.

Specific targets should also be established for each performance measure or indicator identified as they are important for effective monitoring and for assessing the progress of the transfer payment program.

2.3 Data collection

  • Identify means or methods to be used to collect data
  • Establish responsibilities for collection
  • Determine frequency of collection

The PM strategy should set out the means or methods to be used to collect data, and clearly identify the responsibilities for collection and when it is to be done. Not all data needs to be or can be collected with the same frequency. The key determinants are the availability of the data, the proposed use of the data, and reporting requirements. Ongoing collection will ensure that data will be available for timely reporting and at the time of evaluation. Baseline data may need to be established for measurement purposes.

2.4  Reporting

  • Determine responsibilities for data analysis and synthesis, and reporting

To be useful, the data collected under the PM strategy has to be converted into relevant information and used for decision making and reporting. The PM strategy should identify the responsibilities for data analysis and data synthesis, and for internal and external reporting in relation to the transfer payment program.

Reporting content and frequency should support ongoing tracking of the performance of the transfer payment program and support timely decisions and actions where necessary to improve program performance or program delivery.

The Policy on Transfer Payments requires deputy heads to ensure there are cost-effective performance measurement and reporting systems in place to support the management of transfer payments.

3 Maintenance of PM strategy

The Policy on Transfer Payments requires that a deputy head ensure that the PM strategy for a transfer payment program is maintained and updated throughout the life cycle of the program. The PM strategy is to be an evergreen document that is changed as required to ensure that it remains relevant and effective in meeting its purposes. Consultation with the departmental evaluation function is encouraged, especially as program managers are responsible for implementing and updating the PM strategy to ensure that it is relevant to the program and that credible and reliable performance data are being collected to effectively support evaluation.[3] Consultation with other stakeholders is also encouraged.

Updating of the strategy may be required for reasons such as:

  • The measures or indicators chosen are proving not to be valid, reliable and credible;
  • Better data sources have become available;
  • Data collection is proving to be difficult or expensive;
  • Program design and terms and conditions are modified.

The PM strategy is a departmental document for internal use. Although this document does not need to be approved by or sent to the Treasury Board Secretariat, it may be shared for developing common understanding and knowledge of the transfer payments programs.


Appendix

Excerpts from the Policy on Transfer Payments

6.5 Deputy Heads are responsible for:

6.5.2 Ensuring that a performance measurement strategy is established at the time of program design, and that it is maintained and updated throughout its life cycle, to effectively support an evaluation or review of relevance and effectiveness of each transfer payment program.

6.5.3 Ensuring that an evaluation or review of relevance and effectiveness of each transfer payment program is undertaken at least every five years as required under the Financial Administration Act and that appropriate and timely action is taken to respond to the results of the evaluations or reviews.

6.5.5 Ensuring that cost-effective oversight, internal control, performance measurement and reporting systems are in place to support the management of transfer payments.

Appendix A: Definitions

Evaluation
Is, in the Government of Canada, the systematic collection and analysis of information about the outcomes of programs. The information is used to make judgments about a program's relevance, its results and cost-effectiveness as well as find alternative ways to deliver the program or to achieve the same results. Section 42.1 of the Financial Administration Act requires that every department conduct, every five years, a review of the relevance and effectiveness of each on-going non-statutory program of grants and contributions. This review is a form of evaluation which would be conducted in a manner that would be appropriate for the risk, complexity and materiality of the program.

Excerpts from the Directive on Transfer Payments

Appendix B: Core design elements

Departmental managers with responsibility for the design or redesign of a transfer payment program are expected to assess the following elements and document evidence of their consideration.

7. The performance measurement strategy for ongoing performance management of the transfer payment program, including the performance measures and indicators and the supporting data requirements and data collection strategy.

Appendix D: Terms and conditions for grants

Departmental managers responsible for preparing terms and conditions are to ensure that the following mandatory elements are addressed in terms and conditions for a transfer payment program using grants.

2. The purpose and objectives of the transfer payment program with clear linkage to departmental and government objectives, and an identification of expected results with performance measures and indicators for monitoring and reporting.

Departmental managers are also to ensure that the following elements are addressed where they are relevant:

13. A description of any information that may be requested from a recipient on results achieved through the use of the grant to support the program's performance measurement strategy and departmental reporting.

Appendix E: Terms and conditions for contributions

Departmental managers responsible for preparing terms and conditions are to ensure that the following mandatory elements are addressed in terms and conditions for a transfer payment program using contributions.

2. The purpose and objectives of the transfer payment program with clear linkage to departmental and government objectives, and an identification of expected results with performance measures and indicators for monitoring and reporting.

11. A general description of the information that will be requested from recipients in financial and performance reporting, including information on results achieved to support the program's performance measurement strategy and departmental reporting.

This guideline will be updated periodically as required. Suggestions are welcome.

Enquiries concerning this guideline should be directed to:

Assistant Comptroller General
Financial Management and Analysis Sector
Office of the Comptroller General
Treasury Board of Canada, Secretariat
Ottawa, ON
K1A 0R5

E-mail: fin-www@tbs-sct.gc.ca
Telephone: 613-957-7233
Fax: 613-952-9613