ARCHIVED - Deputy Head Consultations on the Evaluation Function - Summary Report

• Previous Page
• Table of Contents
• Next Page

This page has been archived.

5. Conclusions

This report summarizes the views of nine Deputy Heads consulted in September 2010 on four broad issues related to the use of Evaluation and the 2009 Policy on Evaluation. Such a sample is certainly too small to draw general conclusions for the universe of Deputy Heads of federal organizations. Nevertheless, it does provide a snapshot of opinions from a diverse set of organizations.

On the basis of the nine Deputy Heads consulted, it would seem that Deputy Heads are paying more attention to Evaluation than might have been the case in the past. For some, the recognition that Evaluation can play a useful role in the oversight and management of the organization is certainly not a new phenomenon. They have used Evaluation in the past in a variety of ways within their organization. In general though, where the perception and use of Evaluation in the past was probably ‘uneven’ across federal organizations, it has likely risen somewhat in profile and stature across the system.

The 2009 Policy on Evaluation, with its requirement for 100 per cent coverage over a five-year cycle and a pre-determined set of core issues, has created a mixed set of reactions. Many have come to recognize that the departmental Evaluation function can play a useful role in assisting the Strategic Review exercise, made more meaningful with the current period of frozen budgets. But most feel that system-wide challenges (particularly the insufficiency of the pool of skilled Evaluators) will stand in the way of most organizations meeting the requirements of the Policy. And, the perceived rigidities within the Policy and its application are expected to raise the overall cost of carrying out Evaluation work. Deputies’ perception of a lack of flexibility also raises for them a tension around whether Evaluation can be conducted in a way that serves all the needs of the department/agency.

In many respects, the requirements of the 2009 Policy on Evaluation for 100 per cent coverage, coupled with the MAF Annual Assessment of each organization on its ‘quality and use of Evaluation’[⁷], seems to be prompting Deputies to take a harder look at the prospect of evaluating all of their programs in a more systematic fashion.

Deputies have raised a number of concerns however with how the Policy on Evaluation is being rolled out and have offered some suggestions to TBS as feedback. Since the 2009 Policy on Evaluation is less than halfway into its ‘transition’ period, this is an opportunity to ‘learn and adjust’ as necessary. That said, as some deputies noted, it is probably still too early to appreciate the full implications of the Policy and its impact on departments and agencies until they have experienced a full cycle.

An important area in going forward that was noted by several Deputy Heads calls for TBS to play a higher profile leadership role for the Evaluation function. Comparisons were made with the Office of the Comptroller General (OCG) where, in 2006, with the introduction of a new Internal Audit Policy, departments and agencies were faced with a comprehensive set of new accountability requirements, and at a time when the professional Audit community was also in need of significant support. Deputies noted that the high profile of the OCG helped ensure that federal organizations responded to the new IA Policy and also worked to support the capacity building needs of the community. The view is that TBS has not responded with that same high-level support for the Evaluation function. More visibility for Evaluation and a high level TBS champion for the function were cited as important factors that ought to be addressed.

Additionally, with its focus on oversight, the view is that TBS is directing too little effort towards supporting community development for the Evaluation function. All deputies had something to say about human resource (HR) issues facing the Evaluation function. There is a wide-held view that Evaluation capacity across the system is challenged by too few skilled Evaluators and that TBS needs to play a role in addressing this. A variety of suggestions were advanced.

Comments from deputies also suggest that the Evaluation function is facing some challenges in today’s environment that were not there when the IA Policy was being introduced. Unlike 2006 when funding was less of an issue and there was a general appetite for more accountability in the system, the Evaluation function faces challenges on both counts. Some deputies seem to feel that there are currently too many ‘oversight’ and accountability mechanisms; several deputies wondered how well aligned are the requirements of the various TBS policies and initiatives, suggesting that TBS needs to do a better job in communicating this. Coming at a time when budgets are frozen raises the added challenge of the 2009 Policy on Evaluation for Deputy Heads – where best to spend their marginal dollars; i.e. on more evaluation/oversight or on program and service delivery?

Finally, based on feedback from most Deputy Heads, there is an apparent communication/information gap surrounding the requirements of the 2009 Policy on Evaluation. Several deputies raised concerns about a lack of flexibility in the way that they could apply the 2009 Policy on Evaluation to their organization. This is at odds however with feedback from TBS/CEE which indicates that organizations do indeed have flexibility in their application of the Policy. The fact that deputies did advance a number of suggestions can provide TBS with a useful way to address this and focus on any issues of misperception and clarify, where needed, practical implementation issues related to the Policy on Evaluation.

This of course speaks to the broader view of many of the deputies that TBS needs to be more visible insofar as the Evaluation function is concerned, not only providing the needed guidance, but also giving more profile to the Evaluation function.

• Previous Page
• Table of Contents
• Next Page