ARCHIVED - Deputy Head Consultations on the Evaluation Function - Summary Report

• Previous Page
• Table of Contents
• Next Page

This page has been archived.

1. Introduction

This report provides a summary of consultations held with a small sampling of Deputy Heads of federal organizations about the Evaluation function. The objective was to provide information to serve as input to the Treasury Board Secretariat’s (TBS) first annual report to Treasury Board on the ‘Health of the Evaluation Function’.

The TBS Centre of Excellence for Evaluation (CEE) engaged Robert Lahey, President of REL Solutions Inc. to conduct interviews with nine Deputy Heads over September 2010, addressing four broad lines of enquiry:

1. How evaluation is used by the department/agency;
2. How the Expenditure Management System Renewal (e.g. Strategic Reviews) and the 2009 Policy on Evaluation has impacted the conduct, resourcing and planning for evaluation;
3. How TBS could best support the organization relative to Evaluation and the new requirements of the 2009 Policy;
4. How well Evaluation and ongoing performance monitoring are providing a comprehensive picture of ‘performance’ of the organization.

A list of the Deputy Heads interviewed is provided in Appendix 1.

This report provides a summary overview of feedback from the Deputy Head consultations, reporting against each of the four issues identified above.

2. Background

The annual Report to the Treasury Board on the Health of the Evaluation Function is a new requirement under the 2009 Policy on Evaluation. It is intended to support the Treasury Board and Secretariat by providing an overview on how implementation of the 2009 Policy is progressing across departments. Operationally, such a report is expected to help both TBS in its functional leadership role and departments, by identifying needed improvements in the Evaluation function.

While the Deputy Head consultation is only one input to the Report on the Health of the Evaluation Function, the consultation with Deputy Heads was expected to provide a rich information base of how deputies see and use Evaluation within the management of their organizations.

Eighteen months into the transitional implementation of the 2009 Policy on Evaluation, it was expected that organizations would have a sufficient amount of experience with the new Policy. Moreover, key issues, should they have arisen, would have had time to come to the attention of the most senior official in the organization.

3. Methodology

The number of organizations to consult (9) and the identification of which specific organizations was determined by TBS officials.

Though the total sample of nine is quite small, and should therefore not be considered as representative of the full universe, the selection of organizations covers a broad cross-section of departments and agencies. Listed in Annex 1, the nine Deputy Heads cover a broad range of government business and circumstances for Evaluation, including: a department with a large Grants and Contributions (G&C) component; a ‘policy’ department; a regional agency; the department with the largest Evaluation unit; a diversity of program types including social programs, economic programs, regulatory programs and international programs; and, organizations where Evaluation was/was not co-located with Internal Audit.

In setting up meetings, each Deputy Head was formally contacted by the TBS Secretary (See Appendix 2 for the information forwarded by TBS in advance of the meetings with Deputy Heads). The request was made for a 30 to 60 minute interview and it was suggested that no advance preparation was required on the part of the Deputy, as the consultation would be seeking ”top of mind” reactions to questions about your evaluation function”. Only the four broad lines of enquiry were identified[¹]

Deputy Heads made themselves available for interviews that ranged in duration from 45 to 60 minutes. In all cases, the Deputy Heads were more than willing to devote their attention to the discussion on Evaluation and its use.

This summary report is a synthesis of findings as they relate to the four broad lines of enquiry. Given the small sample size, the analysis is more qualitative, but does point out those areas or instances where a majority of Deputy Heads take a similar position.

• Previous Page
• Table of Contents
• Next Page