ARCHIVED - MAF Assessment: Treasury Board of Canada Secretariat - 2008

1.1 Leadership: Strong

• Executive leaders engage employees and stakeholders on an ongoing basis in ethical discussions and openly address organizational ethical issues through public statements and internal messaging.

1.2 Infrastructure: Strong

• Organization monitors risks in regard to possible breaches of public service values and ethics, and risk management is integrated into decision making.
• Values and ethics plans or strategies are tailored to an organization's work, span several years, and measure results and are used to inform senior management on the state of the organization's values and ethics.

1.3 Culture: Strong

• Organization, on an ongoing basis, uses employee feedback from across the entire organization to measure its values and ethics culture and employee engagement.

2.1 PAA Consistency: Acceptable

• The Strategic Outcome(s) is/are measurable and represents an end-state.

2.2 Measurability: Acceptable

• An adequate Program Activity Architecture has been developed with some issues to be resolved.

2.3 Quality:

• The organization has developed a weak performance measurement framework.
• The performance indicators are not clear and cannot be used for data collection to provide reliable insight into program effectiveness.
• Expected results are not clear and distinct, and are not appropriate to their respective program descriptions.
• Targets are not stated appropriately as a performance level of the indicator.

3.1 Business Plan: Acceptable

• Corporate business plan generally aligns resources and accountabilities to priorities.
• Sector or branch business plans are generally aligned with the corporate business plan.

3.2 Governance Structure: Strong

• Management oversight of the organization's program activities is clearly evident.
• Organization's corporate governance structure is fully aligned to the organization's PAA.
• Senior corporate management structure or subordinate governance structure (e.g., committees) meet regularly.
• Terms of reference are generally current and complete.

4.1 Leadership of Priority Initiatives: Strong

• The organization has established a highly effective management structure for each initiative, including clear roles and responsibilities, outcomes and an engagement strategy.
• The organization shows strength in leadership of its priority interdepartmental initiatives.

4.2 Participation in Priority Initiatives: Acceptable

• An internal management structure has been established to manage the organization’s participation in Public Service Renewal.
• The organization contributes effectively to Public Service Renewal.

4.3 Portfolio Coordination: Strong

• The Department’s portfolio coordination displays deliberate and effective leadership.

Treasury Board (Secretariat) has been assessed on its leadership of the Web of Rules (Strong) and the Horizontal Strategic Review of Human Resources Agencies (Acceptable) initiatives, and its participation in Public Service Renewal (Acceptable), and with regard to its portfolio coordination responsibilities.

5.1 Supporting Information: Acceptable

• Adequate information is submitted for business cases.
• Organization has established a capacity to assemble usually accurate, reliable and complete supporting information in TB submissions.
• Response to TBS comments is excellent.
• Submission contains an adequate level of detail.

5.2 Analysis: Acceptable

• Established capacity for analysis on implementation is evident.
• Established capacity for options analysis is demonstrated.
• Established capacity to analyze value for money, effectiveness and efficiency is evident.
• Sustained capacity for appropriate responses to TBS comments is demonstrated.

5.3 Consultations: Acceptable

• Established capacity to initiate consultations with TBS with sufficient lead time is evident.
• Submissions are usually on time (six weeks before TB meetings).

5.4 Quality control: Acceptable

• All important information is usually included in the first draft.
• Established capacity for consistency of information throughout documents is evident.
• Generally rigorous and effective quality control process is in place and is usually followed for TB submissions.
• TBS feedback is usually fully addressed.

6.1 Quality: Acceptable

• The majority of evaluations submitted to TBS consistently address questions of program relevance, success and effectiveness.
• The majority of evaluations submitted to TBS use multiple lines of evidence. Evaluations reflect the diversity and perspectives of multiple program stakeholders.

6.2 Neutrality: Acceptable

• Head of Evaluation has explicit authority to submit evaluation reports directly to the deputy head. Head of Evaluation has access to the deputy head, as required.
• Senior management committee is in place to support, oversee and monitor the evaluation function and management accountabilities arising from evaluations and evaluation related products. The committee is chaired by the deputy head or senior level designate. The committee meets regularly during the year.
• Some resources dedicated to evaluations are directed by the Head of Evaluation, some resources are directed by the program base.

6.3 Coverage: Acceptable

• The organization has shown evidence of moving towards full coverage of all ongoing programs of grants and contributions over a five year cycle as per 42.1 of the FAA. Year to year percent coverage indicates organization is on track to achieve 100% coverage.
• Option 1: The organization has committed to moving toward full evaluation coverage of their program base (e.g. over a five-year cycle). However, relatively low number of evaluations are completed each year and they cover less than 10%.
• The organization has shared its risk-based evaluation plan with TBS. The evaluation plan has information on evaluations planned, completed and carried over. It also includes links to the organization's PAA.

6.4 Usage: Opportunity for Improvement

• Active, systematic and regular tracking of management action plans arising from evaluation recommendations is in place.
• Evaluation commitments, plans and requirements are delivered on time or extensions are due to circumstances beyond the department's control. Organization usually requests extension from TBS.
• Less than 40% of RMAFs are implemented.

During the last year, TBS made progress in terms of strengthening its evaluation function, including the introduction of new mechanisms designed to improve planning and neutrality. If TBS is able to deliver on the commitments outlined in its Departmental Evaluation Plan (DEP), while remaining vigilant with regard to quality assurance and governance, there is potential for significant gain in the overall quality of the TBS evaluation function.

7.1 MRRS Basis: Acceptable

• Good links between performance and plans are present.
• Linkages between resources and results are adequately demonstrated in the reports.
• RPP and DPR present a clear PAA (with crosswalks as necessary).

7.2 Credible information: Acceptable

• It is difficult for the reader to determine what the source of data and information reported in the DPR is and what the quality of the underlying data is.
• The DPR is concise. It contains an adequate amount of information relevant to the estimates process.

7.3 Context: Acceptable

• DPR is for the most part balanced. It presents both positive and negative aspects of performance, and substantiation or explanation is consistently provided.
• Reports adequately present the strategic context and operating environment information including challenges, risks, opportunities and capacities.

8.1 Change plan: Strong

• Comprehensive organizational change plan exists and matches the scope of change that has been identified.
• The organization has the capacity to evaluate whether or not change is required.

8.2 Engagement: Strong

• A learning culture exists within the organization.
• Change management related training programs are available to some components of the organizations.
• Employees and stakeholders are actively engaged at all phases and are committed to advancing strategies and initiatives.

8.3 Assessment: Acceptable

• Assessment plans exist and are broad in scope and detail.
• Change plans and strategies are included in Performance Management Agreements of Senior Executives.
• Course adjustments and improvements to strategies are made regularly.
• Results are not apparent.

9.1 Engagement: Acceptable

• Accountability for key risks is assigned to senior management and performance is assessed.
• Senior management encourages effective Risk Management and a risk-smart culture.
• Senior management ensures that the organization’s Risk Management approach is tailored to the specific needs of the organization.
• Senior management has reviewed/approved the Corporate Risk Profile within the past year.
• Senior management reviews the organization’s Risk Management approach within the current three-year planning cycle.
• The organization has a common risk assessment approach and it has been approved by senior management.

9.2 Implementation: Acceptable

• Risk Management guidance and tools that enable the organization’s risk management approach are inconsistently made available to staff.
• The Corporate Risk Profile is systematically (horizontally and vertically) implemented into most operational levels across the organization.
• The organization’s Risk Management approach is inconsistently communicated to staff and stakeholders.

9.3 Integration: Acceptable

• Operational level risks are prioritized into key risks.
• Risk information and Risk Management principles are adequately captured in senior management reporting.
• Risk information and Risk Management principles influence planning and resource allocation decisions.
• Risk information is inconsistently considered and consulted for senior management decision-making.
• The organization makes adequate course corrections based on Risk Management performance and new information.

9.4 Continuous Improvement: Acceptable

• Comprehensive risk information was extensively gathered from internal sources of the organization for preparing the CRP.
• Corporate risks are consistently linked to the organization’s strategic outcomes.
• Most relevant external sources are consulted during the development of the organization’s CRP.
• The CRP provides a reliable assessment of the quality of risk information used.
• The organization adequately builds on past experience, better practice, and adjusts to fit any changes in management structures, priorities or strategic direction.
• The organization has implemented most recommendations provided during its last MAF assessment.

Treasury Board of Canada Secretariat should be commended for its efforts to improve the integration of risk management into its day-to-day business, and respond to recommendations in its MAF Round V assessment.

In addition to explicitly considering risks in corporate and operational business planning and tracking, TBS has engaged staff through its Risk Coordinators Working Group, and conducted an ethical risk assessment.

10.1 Fair: Acceptable

• Evidence shows that labour relations matters are consistently and appropriately managed/addressed.
• Evidence shows that the organization is in compliance with Labour Relations and Compensation Operations direction (terms and condition of employment, collective agreements and/or applicable legislation).
• Evidence shows that the organization meets standards of timeliness in payments to employees.
• The organization is undertaking action to improve the classification program, which is at medium-high to high risk; however, there is evidence that the monitoring program is not yet fully effective.

10.2 Enabling: Acceptable

• Organization demonstrates the necessary linguistic capacity to provide personal and central services and supervision in both official languages.
• Organization is representative of all four employment equity designated groups.
• Promotions among employment equity groups are equal or less than previous year's performance.
• Separations among employment equity groups are equal or greater than previous year's performance.
• Work instruments, electronic systems and communications with employees are always or nearly always available in both official languages.

10.3 Healthy and safe: Acceptable

• Evidence shows that the organization has an inadequately managed program to protect employees' occupational health and safety.
• The majority of employees feel recognized for positive performance.

11.1 Productive: Strong

• A significant number of employees indicate their organization supports their career development and learning needs.

11.2 Principled: Opportunity for Improvement

• Adequate linguistic capacity is generally in place as shown by the majority of incumbents of bilingual positions who meet the language requirements of their position.
• Communications with and services to the public in both official languages are always or nearly always available.
• Employees consider that they generally can communicate in the official language of their choice within their organization and work instruments, electronic systems and communications in both official languages are generally available.
• Progress against the previous year's performance on recruitment, promotion and separation for employment equity groups is less than the organization's average for all employees.
• Promotions among employment equity groups are less than representation for at least one group.

11.3 Sustainable: Acceptable

• Evidence indicates human resources planning integrated with business planning is generally in place and governance/organizational infrastructure generally exists to support it.

11.4 Adaptable: Acceptable

• A sufficient number of employees indicate their organization encourages continuous learning, improvement and innovation.

12.1 Governance: Acceptable

• IM requirements are somewhat integrated as a part of the approval, development, implementation, evaluation, and reporting of departmental policies, programs, services, or projects.
• IM is represented in the corporate-wide governance structure and/or in the corporate-wide governance or approval committee(s).
• Participation is evident in GC-wide approaches and initiatives related to developing, implementing, sharing, and leveraging IM policies and practices.

12.2 Strategy: Acceptable

• A current and active IM strategy identifies support to business priorities and operations, information needs and accountabilities, IM policy considerations and is partially integrated with other corporate strategies, plans and planning cycles.
• An IM strategy implementation plan, including some timelines and resources, is underway and some achievements to date are identified.
• IM awareness activities are underway in the department to help staff and executives understand their IM roles, responsibilities and accountabilities.

12.3 Privacy Act: Acceptable

• Most of the organization’s collections of personal information are described in registered Personal Information Banks and/or Classes of Personal Information in accordance with the requirements of the Privacy Act.

12.4 Access to Information Act: Acceptable

• A significant number of institution-specific Classes of Records do not meet Treasury Board Secretariat requirements.
• Most of the organization's functions, programs, activities and related information holdings have been appropriately identified and described in its 2008 Chapter of Info Source: Sources of Federal Government Information.

13.1 Leadership: Opportunity for Improvement

• Senior official for information technology has only a partial scope of responsibility and accountability for information technology.
• Some participation in setting government-wide directions for information technology is evident.

13.2 Planning: Acceptable

• Acceptable information technology plan is in place that aligns with the government-wide directions for information technology and departmental business needs.
• Organization has aligned corporate and information technology governance structures and has an integrated planning process.

13.3 Value: Strong

• Organization analyzes and plans for the appropriate use of information technology shared services to an optimal extent.
• Organization devotes adequate management attention to service costing, asset management, performance measurement and reporting to ensure value delivery.

14.1 Investment Planning: Acceptable

• The investment planning process integrates investments decisions across all asset classes.
• The organization’s investment planning document has not been approved by the proper authority.
• The organization’s investment planning process considers investments over multiple years.

14.3 Materiel Management: Acceptable

• All elements of a materiel management framework are evident.
• Comprehensive internal policies are documented and disseminated.
• Governance structures, approval processes and authority limits are documented and disseminated.

14.3 Materiel Management TBS has a reliable materiel management information system.  Though unclear if this system is fully integrated, a new fully integrated system is scheduled for implementation in April 2009.

15.1 Governance and Oversight: Acceptable

• Business cases, which define expected outcomes, are required to support proposals for major projects.
• There is evidence of formal project governance and oversight mechanisms and that approved projects are generally linked with the strategic plans and priorities of the organization through established organization-wide procedures. Approval and corrective action decisions are documented.
• There is no evidence that the organization has exceeded Treasury Board approval limits.

15.2 Effective Management of Project Resources: Opportunity for Improvement

• Adequate processes/procedures exist to ensure that planned projects have the required resources to achieve expected outcomes.
• The funding models used for projects support the achievement of expected project outcomes and cost estimates are generated at the work package level and consider historical data and/or industry benchmarks.
• There is no evidence that the organization has failed to meet TB conditions regarding projects.
• While project management related training is made available by the organization for employees, there are no processes to ensure that employees with project management responsibilities are encouraged to complete relevant training and the number of qualified project managers is unknown.

15.3 Effective Management of Project Results: Acceptable

• There is evidence of organization-wide procedures and processes which communicate project monitoring and performance information to project managers and project oversight mechanisms.
• While there is evidence of project monitoring and reporting activities, there is no evidence that these activities are ongoing or that the information is used to support corrective action.
• While there is evidence that project milestones, deliverables and outcomes are documented for some projects, it is not a requirement across the organization.

16.1 Governance and Oversight: Acceptable

• Effective and accountable procurement management processes and controls are in place (e.g., contract review mechanisms, documented decision making, guidance documents, appropriate delegation instruments or proper use of delegated authorities).
• Clear links have been established between procurement activities and the organization-wide program plans, priorities and long-term investments.
• Some procurement planning.

16.2 Meeting Operational Requirements: Acceptable

• Informed decision making and oversight exist.
• Mandatory training underway.
• Consistent procurement training is evident.
• Links to human resources planning are inconsistent (e.g., insufficient succession planning and recruitment strategies for procurement staff).
• Qualified procurement human resources exist.
• Some results or reviews are used to adjust current procurement management activities and future procurement plans.
• Timely and accurate procurement financial and non-financial reports have been submitted.

Clear links exist between procurement and business planning. Contract review process with records of decisions evident. Operational requirements met using qualified staff, online tools, efficient processes and timely and accurate information systems. Some effort towards continuous improvement. Effective and efficient procurement activities contribute to achievement of program outcomes and government-wide priorities.

17.1 Authorities and Policies: Acceptable

• Audit report results show evidence of deficiencies that are of serious concerns.
• Departmental procedures, tools, training and support for those individuals delegated with Section 34 authority show evidence of solid financial management practices.
• Departmental processes for classification of moneys, internal controls for receiving and recording money and depositing money show evidence of solid financial management practices.
• Departmental processes for informing those delegated with Section 33 authority of their responsibilities and dealing with requests for payments that are problematic show evidence of good financial management practices.
• Departmental processes to provide individuals delegated Section 33 authority with the information necessary to assess and approve specific transactions and to assess the adequacy of Section 34 account verification show evidence of solid financial management practices.
• The reporting of external user fee information meets or nearly meets the requirements of the reporting guidelines.

17.2 Public Accounts Reporting: Acceptable

• Minimal Central Financial Management Reporting System (CFMRS) coding errors.
• Ninety to 96% (Grade A) of Public Accounts reporting plates submitted on time.
• No significant errors found during the course of the OAG Public Accounts audit.

17.3 Management Capacity: Acceptable

• A reasonable proportion of FIs and management team members in the financial management organization have current, approved learning plans.
• A significant amount of training is provided for the financial management organization.
• Many processes in support of a sound succession plan for key positions are in place.
• Positions, the duties of which are being performed by an individual indeterminately appointed to that position, comprise a low proportion of the FI segment of the financial management organization.
• Positions, the duties of which are being performed by an individual indeterminately appointed to that position, comprise a reasonable proportion of the positions on the management team of the financial management organization.
• There is a position (or positions) established in the financial management organization that is dedicated to community management and development.

17.4 Financial Statements: Acceptable

• All concerns identified in the audit readiness assessment are addressed in a detailed action plan.
• Several known financial internal control weaknesses remain unremedied.
• The Financial Statements are compliant with Treasury Board Accounting Standard 1.2 – Departmental and Agency Financial Statements and reporting deadlines were met.

17.5 Internal Reporting: Strong

• The internal financial reporting package is accompanied by a comprehensive discussion and analysis.
• The internal financial reporting package is presented to senior management less than 15 calendar days after period end.
• The internal financial reporting package is presented to senior management ten or more times per year.
• The process for reviewing information before it is presented to senior management to ensure no material errors or omissions is well established.
• The scope of the internal financial reporting package is comprehensive.

17.6 Other Initiatives: Acceptable

• Evidence of some initial measures taken towards implementing the Guide to Costing.

While TBS maintained an overall rating of Acceptable, there were improvements in the quality of its organizational financial statements and progress towards realizing audited financial statements along with the quality of its internal financial reporting.

18.1 Internal Audit governance: Strong

• There is an approved Internal Audit Charter in line with the 2006 Policy on Internal Audit.
• The Implementation Plan covers all of the required policy elements.
• Ongoing monitoring of, and progress in implementing, key elements of the plan are ahead of schedule.
• Chief Audit Executive reports solely and exclusively to the Deputy Head.
• An independent Departmental Audit Committee is in place.
• There is an approved Departmental Audit Committee Charter in line with the 2006 Policy on Internal Audit.
• There is an approved Departmental Audit Committee Annual Plan for fiscal year 2008-2009.
• The Departmental Audit Committee has met at least four times over the past twelve months.
• There is a written statement indicating that a Departmental Audit Committee Annual Report will be produced for fiscal year 2008-2009 and future years.

18.2 Internal Audit Professional Practices: Acceptable

• The Risk-Based Audit Plan was approved by the Deputy Head and sent to the Office of the Comptroller General in a timely manner.
• Annual Risk-Based Audit Plan methodology is evident and applied.
• There is evidence of preparation to provide for holistic assurance.
• All post-engagement follow-up activities are clearly identified.
• There is partial information on the planned use of all audit function resources.
• Majority of planned work is on audit assurance versus other types of activities.
• Continuity of previous years work is identified with status or rationale.
• Approved assurance products are consistent with policy and internal audit standards requirements.
• High completion rate of assurance products (number of assurance audit reports) against 2007-2008 Risk-Based Audit Plan.
• An Internal Quality Assurance and Improvement Program has been drafted.
• Assurance products (reports) are produced in a reasonably timely manner.
• Post-engagement follow-up process is well documented, and all recommendations are followed up using a risk-based approach.
• The department or agency provides notification to the Treasury Board Secretariat on issues of importance on an ad hoc basis or is aware of this requirement.

18.3 Administration of the Internal Audit Function: Strong

• The Internal Audit Human Resources Plan is linked to the business plan, and includes an elaborated recruitment action plan and an external resourcing strategy that are being actioned in a formal manner.
• Investment in Certified Internal Auditor certification, learning and training represents a minimum of 5% of FTE salaries.
• Planned spending, *, was given to the Office of the Comptroller General. When comparing current spending of 2008-2009 with planned financial resources of 2007-2008, resource levels identified maintain the resource levels identified in 2007.
• Planned FTEs dedicated to internal audit have been maintained comparatively to 2007-2008. They meet the resource level identified in the planned internal audit function’s budget for 2008-2009.

18.4 Internal Audit Performance: Strong

• A comprehensive Chief Audit Executive Annual Report for 2007-2008 was presented to the Departmental Audit Committee and the Deputy Head and submitted to the Office of the Comptroller General.
• Regular periodic reporting on the follow-up of Management Action Plans is evident.

The secretariat has continued to maintain momentum towards the achievement of the fundamental requirements of the 2006 Policy on Internal Audit. Progress has been made in the areas of Internal Audit Governance, Professional Practices and Reporting on Audit Performance.

19.1 Departmental Security Program: Attention Required

• Organization has limited or no evidence of leadership, administration, or implementation of a security program that includes key policy elements.
• Limited or no evidence that a security organization and governance structure have been put in place to effectively manage the organization's security program.
• Limited or no evidence that the organization's security program ensures the coordination of all security policy functions.
• Several deficiencies in meeting key policy requirements for the departmental security program.

19.2 Management of IT Security (MITS): Opportunity for Improvement

• Organization has achieved the three priority objectives that form the foundation for Management of Information Technology Security (MITS), but does not fully comply with MITS requirements.
• Some deficiencies in meeting key MITS requirements.

19.3 Business Continuity Planning (BCP): Opportunity for Improvement

• Organization has partially developed measures to provide for the continuity of critical business operations and services.
• Several deficiencies in meeting key BCP program requirements.
• Business Continuity Planning (BCP) program governance has not been fully established.
• Business Impact Analysis (BIA) has been completed to identify and prioritize the organization's critical services and assets.
• Significant deficiencies in establishing business continuity plans and arrangements.
• Maintenance cycle has been partially put in place to review, test and audit business continuity plans.
• Completed and approved plans are in place for Pandemic and Information Management / Information Technology emergency preparedness.

20.1 Management Engagement – Service and CLF: Opportunity for Improvement

• There is little monitoring by senior management to ensure that the requirements of CLF 2.0 are being met institution-wide; there is limited information on which to make decisions and course correction.

20.2 Public/client views: Acceptable

• Evidence of incorporating feedback in the implementation of its services, programs, policies or initiatives.
• Little or no evidence of making consultation results available to the public.
• Many strong tools are used to obtain views from clients.
• There are plans to obtain views from clients.
• There is a clearly identified target clientele for public consultations.

20.3 Official Languages: Strong

• Analysis of the Annual Review on OL shows the institution is fully meeting its obligations.
• No complaint or minimal number of founded complaints exits.
• The institution has the necessary linguistic capacity to serve the public in both OL.