ARCHIVED - MAF Assessment: Economic Development Agency of Canada for the Regions of Quebec - 2008

1.1 Leadership: Acceptable

• Executive leadership regularly communicates and encourages ongoing dialogue on public service values and ethics among employees.

1.2 Infrastructure: Strong

• Risks in regard to possible breaches in public service values and ethics are regularly assessed, documented and followed up with mitigation plans and action by management.
• Values and ethics plans or strategies are tailored to an organization's work, span several years, and measure results and are used to inform senior management on the state of the organization's values and ethics.

1.3 Culture: Acceptable

• Managers and staff are encouraged to engage in values and ethics dialogue.
• Organization has a good understanding of the current state of public service values and ethics as evidenced by qualitative or quantitative information.
• Public service values and ethics are generally understood.

2.2 Measurability: Opportunity for Improvement

• A Program Activity Architecture has been developed with major issues to be resolved.
• An inventory of programs has been developed and most of the listed programs meet the definition of a program.

2.3 Quality:

• Expected results are not clear and distinct, and are not appropriate to their respective program descriptions.
• The performance indicators are not clear and cannot be used for data collection to provide reliable insight into program effectiveness.

3.1 Business Plan: Strong

• Human resources, IM/IT, communications and other key corporate plans are well integrated and communicated internally.
• Organization's corporate business plan is well aligned to corporate priorities.
• Sector priorities, accountabilities, business plans and resource allocations are fully aligned.

3.2 Governance Structure: Strong

• Management decisions and interventions (includes resource reallocation) are consistently proactive and timely.
• Organization's corporate governance structure is fully aligned to the organization's PAA.
• Senior corporate management structure or subordinate governance structure (e.g., committees) meet regularly.

4.2 Participation in Priority Initiatives: Strong

• Senior management is engaged in all the initiatives.
• The organization contributes effectively to Public Service Renewal.
• The organization shows strength in its participation in priority interdepartmental initiatives.
• The organization's commitments to the initiative are clear, measurable, and consistent with its role, and an internal management structure has been established.

TBS has assessed Economic Development Agency of Canada for the Regions of Quebec with regard to its participation in Public Service Renewal - Acceptable, the Web of Rules Initiative - Strong, and the 400th Anniversary of Quebec Initiative - Strong.

5.1 Supporting Information: Acceptable

• Organization has established a capacity to assemble usually accurate, reliable and complete supporting information in TB submissions.
• Organization has the capacity to respond effectively to most TBS feedback.
• Policy and budget authorities are usually identified.

5.2 Analysis: Acceptable

• Business cases may have comprehensive information and demonstrate good analysis.
• Established capacity for analysis on implementation is evident.

5.3 Consultations: Acceptable

• Established capacity to initiate consultations with TBS with sufficient lead time is evident.
• Submissions are usually on time (six weeks before TB meetings).

5.4 Quality control: Acceptable

• Generally rigorous and effective quality control process is in place and is usually followed for TB submissions.
• Submissions usually have SFO or Head of Evaluation sign offs when appropriate.

6.1 Quality: Acceptable

• All evaluations submitted to TBS employ appropriate methodologies to gather data and inform the analysis.
• Evaluations submitted to TBS usually present findings, conclusions and recommendations that are supported by the evidence found in the evaluation report.
• The majority of evaluations submitted to TBS consistently address questions of program relevance, success and effectiveness.
• The majority of evaluations submitted to TBS include analysis of the limitations of the methodology and data sources used.
• The majority of evaluations submitted to TBS use multiple lines of evidence. Evaluations reflect the diversity and perspectives of multiple program stakeholders.

6.2 Neutrality: Acceptable

• Evaluation function resourcing is commensurate with the organizational evaluation plan.
• Head of Evaluation has no explicit authority to submit evaluation reports directly to the Deputy Head and occasionally has access to the deputy head.
• Senior management committee is in place to support, oversee and monitor the evaluation function and management accountabilities arising from evaluations and evaluation related products. The committee is chaired by the deputy head or senior level designate. The committee meets regularly during the year and is attended by designated committee members.
• The majority of resources dedicated to evaluations are directed by the Head of Evaluation.

6.3 Coverage: Acceptable

• The organization has shown evidence of moving towards full coverage of all ongoing programs of grants and contributions over a five year cycle as per 42.1 of the FAA. Year to year percent coverage indicates organization is on track to achieve 100% coverage.
• Option 2: The organization is working according to its evaluation plan. Evaluation coverage of direct program expenditures fluctuate highly from year to year, however, organization appears on track to achieve 100% coverage over a set year cycle (e.g. over a five-year cycle).
• The organization has shared its multi-year, risk-based evaluation plan with TBS. The evaluation plan has information on evaluations planned, completed and carried over. It also includes links to the organization's PAA. As well, there is evidence to indicate that the organization has calibrated its evaluation efforts and/or approaches according to program context.

6.4 Usage: Acceptable

• Active, systematic and regular tracking of management action plans arising from evaluation recommendations is in place.
• Evaluation commitments, plans and requirements are delivered on time or extensions are due to circumstances beyond the department's control. Organization occasionally requests extension from TBS.
• Majority of evaluations submitted to TBS incorporate data from a performance measurement system to support the evaluation. Submitted evaluations sometimes cite data availability and/or quality as constraints.
• The results of evaluations are almost always brought for consideration in TB submissions, Memorandum to Cabinet, RPPs, DPR and Strategic Reviews.

7.1 MRRS Basis: Acceptable

• Clear performance expectations in the RPP are all tracked and rigorously reported on in the DPR.

7.2 Credible information: Acceptable

• The DPR is concise. It contains a high proportion of estimates-related information and is drafted in a concise reporting style.

7.3 Context: Acceptable

• DPR is generally balanced – the report presents both positive and negative aspects of performance and substantiation or explanation is generally provided.
• There is adequate discussion of lessons learned.

8.1 Change plan: Strong

• Comprehensive organizational change plan exists and matches the scope of change that has been identified.

8.2 Engagement: Strong

• Employees and stakeholders are actively engaged at all phases and are committed to advancing strategies and initiatives.

8.3 Assessment: Strong

• Assessment plans exist and are comprehensive in scope and detail.
• Change plans and strategies are priorities for the whole organization.

9.1 Engagement: Acceptable

• Senior management encourages effective Risk Management and a risk-smart culture.
• Senior management ensures that the organization’s Risk Management approach is tailored to the specific needs of the organization and is adjusted as required.
• Senior management has reviewed/approved the Corporate Risk Profile within the past year.
• Senior management reviews the organization’s Risk Management approach within the current three-year planning cycle.
• The organization has a common risk assessment approach and it has been approved by senior management.

9.2 Implementation: Acceptable

• Risk Management guidance and tools that enable the organization’s risk management approach are made available to staff.
• The Corporate Risk Profile is systematically (horizontally and vertically) implemented into most operational levels across the organization.
• The organization’s Risk Management approach is regularly communicated to staff and stakeholders.

9.3 Integration: Acceptable

• Operational level risks are prioritized into key risks.
• Risk information and Risk Management principles are adequately captured in senior management reporting.
• Risk information and Risk Management principles influence planning and resource allocation decisions.
• Risk information is adequately consulted for senior management decision-making.
• The organization makes adequate course corrections based on Risk Management performance and new information.

9.4 Continuous Improvement: Acceptable

• Key risk information was adequately gathered from internal sources of the organization for preparing the CRP.
• Most relevant external sources are consulted during the development of the organization’s CRP.
• The CRP provides a reliable assessment of the quality of risk information used.
• The organization explicitly builds on past experience, better practice, and adjusts to fit any changes in management structures, priorities or strategic direction.
• The organization has implemented most recommendations provided during its last MAF assessment.

Consistent with commitments in the 2007-08 and 2008-09 RPP, progress has been made toward formalizing and improving the means of managing risks in the organization and these efforts should be recognized.

The organization should continue to act on the recommendations of the 2008 internal audit of risk management functions to ensure better integration of risk into decision making and viable risk management practice.

10.1 Fair: Acceptable

• Organization is undertaking action to improve the classification program in accordance with its level of risk.
• Evidence shows that labour relations matters are consistently and appropriately managed/addressed.
• Evidence shows that the organization exceeds standards of timeliness in payments to employees.
• Evidence shows that the organization is in compliance with Labour Relations and Compensation Operations direction (terms and condition of employment, collective agreements and/or applicable legislation).

10.2 Enabling: Strong

• Organization is representative of all four employment equity designated groups.
• Organization progress remains unchanged from the previous year in representation, recruitment, promotions and separations of the four employment equity groups.
• Promotions among employment equity groups are equal or less than previous year's performance.
• Separations among employment equity groups are equal or greater than previous year's performance.

10.3 Healthy and safe: Opportunity for Improvement

• A considerable number of employees do not feel recognized for positive performance.
• Evidence shows that the organization has an inadequately managed program to protect employees' occupational health and safety.

11.1 Productive: Acceptable

• A sufficient number of employees indicate their organization supports their career development and learning needs.

11.2 Principled: Strong

• Organization is representative of all four employment equity designated groups.
• Progress against the previous year's performance on recruitment, promotion and separation for employment equity groups exceed the organization's average for all employees.
• Promotions among employment equity groups are less than representation for at least one group.

11.3 Sustainable: Acceptable

• Evidence indicates human resources planning integrated with business planning is generally in place and governance/organizational infrastructure generally exists to support it.

11.4 Adaptable: Acceptable

• A sufficient number of employees indicate their organization encourages continuous learning, improvement and innovation.

12.1 Governance: Strong

• IM accountabilities, roles, and responsibilities are formally established throughout the organization.
• IM requirements are fully integrated as a part of the approval, development, implementation, evaluation, and reporting of departmental policies, programs, services, and projects and mechanism are in place to continuously evaluate and modify the requirements.
• IM is fully represented in the corporate-wide governance structure and in the corporate-wide governance or approval committee(s).
• Participation is evident in GC-wide approaches and initiatives related to developing, implementing, sharing, and leveraging IM policies and practices.

12.2 Strategy: Strong

• A current and active IM strategy identifies support to business priorities and operations, information needs and accountabilities, IM policy considerations and is partially integrated with other corporate strategies, plans and planning cycles.
• An IM strategy implementation plan, including some timelines and resources, is underway and some achievements to date are identified.
• IM awareness activities are underway in the department to help staff and executives understand their IM roles, responsibilities and accountabilities.

12.3 Privacy Act: Opportunity for Improvement

• A significant number of institution-specific descriptions of personal information under the control of the organization do not meet Treasury Board Secretariat requirements.
• Significant collections of personal information under the control of the organization have not been appropriately identified or described in accordance with the Privacy Act.

12.4 Access to Information Act: Opportunity for Improvement

• A significant number of institution-specific Classes of Records do not meet Treasury Board Secretariat requirements.
• A significant number of the organization's functions, programs, activities and related information holdings have not been appropriately identified or described in its 2008 Chapter of Info Source: Sources of Federal Government Information. This information is a requirement of the Access to Information Act to facilitate public access to federal government information.

13.1 Leadership: Opportunity for Improvement

• The senior official has responsibility and accountability for the full scope of information technology responsibilities and ensures that information technology supports organizational outcomes.
• Some participation in setting government-wide directions for information technology is evident.

13.2 Planning: Acceptable

• Acceptable information technology plan is in place that aligns with the government-wide directions for information technology and departmental business needs.
• Organization has aligned corporate and information technology governance structures and has an integrated planning process.

13.3 Value: Acceptable

• Organization is making efforts to appropriately use and plan for further use of information technology shared services.
• Organization devotes adequate management attention to service costing, asset management, performance measurement and reporting to ensure value delivery.

14.1 Investment Planning: Opportunity for Improvement

• The organization does not have a current investment planning document.
• The organization’s investment planning process does not consider investments over multiple years.

14.3 Materiel Management: Acceptable

• All elements of a materiel management framework are evident.
• Governance structures, approval processes and authority limits are documented and disseminated.
• Comprehensive internal policies are documented and disseminated.
• Reliable and sufficiently integrated information systems are in place.
• Some indicators of materiel performance are monitored.

15.1 Governance and Oversight: Acceptable

• Business cases, which define expected outcomes, are required to support proposals for major projects.
• There is evidence of formal project governance and oversight mechanisms and that approved projects are generally linked with the strategic plans and priorities of the organization through established organization-wide procedures. Approval and corrective action decisions are documented.
• There is no evidence that the organization has exceeded Treasury Board project approval limits, or failed to notify TB/TBS when it did.

15.2 Effective Management of Project Resources: Acceptable

• Adequate processes/procedures exist to ensure that planned projects have the required resources to achieve expected outcomes.
• The funding models used for projects support the achievement of expected project outcomes and cost estimates are generated at the work package level and consider historical data and/or industry benchmarks.
• The organization recognizes project management as a discipline and most employees with project management responsibilities have completed relevant project management training.
• There is evidence that most project managers prepare a staffing plan to secure authorization for necessary resources prior to project execution.
• There is no evidence that the organization has failed to meet TB conditions regarding projects.

15.3 Effective Management of Project Results: Acceptable

• The organization requires that outcomes are clearly defined for projects in business case documentation and most projects are subject to a review.
• The organization requires that project milestones, deliverables and outcomes are documented for major projects.
• There is a clear link between the review process and project management governance and oversight mechanisms.
• There is evidence of organization-wide procedures and processes which communicate project monitoring and performance information to project managers and project oversight mechanisms.
• There is evidence that the organization monitors project performance and uses this information to support corrective action.

16.1 Governance and Oversight: Opportunity for Improvement

• Inconsistent links exist between procurement activities and the organization-wide program plans, priorities and long-term investments.
• Some procurement management processes and controls are in place (e.g., contract review mechanisms, documented decision making, guidance documents, delegation instruments or proper use of delegated authorities).
• Some procurement planning.

16.2 Meeting Operational Requirements: Opportunity for Improvement

• Clear links to human resources planning are established (e.g., succession planning and recruitment strategies for procurement staff).
• Consistent procurement training and certification programs exist.
• Efficient and integrated procurement information systems and processes are in place.
• Little or no results and reviews are used to adjust current procurement management activities and future procurement plans.
• Qualified procurement human resources exist.

17.1 Authorities and Policies: Acceptable

• Audit report results show evidence of deficiencies that are of serious concerns.
• Departmental procedures, tools, training and support for those individuals delegated with Section 34 authority show evidence of solid financial management practices.
• Departmental processes for classification of moneys, internal controls for receiving and recording money and depositing money show evidence of good financial management practices.
• Departmental processes for informing those delegated with Section 33 authority of their responsibilities and dealing with requests for payments that are problematic show evidence of deficiencies that are of some concern.
• Departmental processes to provide individuals delegated Section 33 authority with the information necessary to assess and approve specific transactions and to assess the adequacy of Section 34 account verification show evidence of solid financial management practices.
• The reporting of external user fee information shows only a few omissions in relation to reporting guidelines.

17.2 Public Accounts Reporting: Strong

• Few Central Financial Management Reporting System (CFMRS) coding errors.
• Greater than 97% (Grade A) of Public Accounts plates completed on time.

17.3 Management Capacity: Acceptable

• A low proportion of FIs or management team members in the financial management organization have current, approved learning plans.
• A reasonable amount of training is provided for the financial management organization.
• Positions, the duties of which are being performed by an individual indeterminately appointed to that position, comprise a reasonable proportion of the positions on the management team of the financial management organization.
• Positions, the duties of which are being performed by an individual indeterminately appointed to that position, comprise all, or almost all, of the FI segment of the financial management organization.
• Some processes in support of a sound succession plan for key positions are in place.

17.4 Financial Statements: Acceptable

• "There is evidence of some work undertaken to assess and/or monitor and/or improve internal control over financial reporting."
• Minor areas of non-compliance with Treasury Board Accounting Standard 1.2 – Departmental and Agency Financial Statements and/or some non-compliance with reporting deadlines related to the submission of annual departmental financial statements.

17.5 Internal Reporting: Acceptable

• The internal financial reporting package is accompanied by a good discussion and analysis.
• The internal financial reporting package is presented to senior management less than 15 calendar days after period end.
• The internal financial reporting package is presented to senior management ten or more times per year.
• The process for reviewing information before it is presented to senior management to ensure no material errors or omissions is established.
• The scope of the internal financial reporting package is reasonable.

While the agency maintained its overall rating of Acceptable, the quality, timeliness and accuracy of the trial balance and Public Accounts plates submitted for purposes of government-wide reporting was Strong.

18.1 Internal Audit governance: Strong

• There is an approved Internal Audit Charter in line with the 2006 Policy on Internal Audit.
• The Implementation Plan covers all of the required policy elements.
• There is limited ongoing monitoring in implementing key elements and progress is not on track with planned timelines.
• Chief Audit Executive reports solely and exclusively to the Deputy Head.
• An independent Departmental Audit Committee is in place.
• There is an approved Departmental Audit Committee Charter in line with the 2006 Policy on Internal Audit.
• There is an approved Departmental Audit Committee Annual Plan for fiscal year 2008-2009.
• The Departmental Audit Committee has met at least four times over the past twelve months.
• There is a written statement indicating that a Departmental Audit Committee Annual Report will be produced for fiscal year 2008-2009 and future years.

18.2 Internal Audit Professional Practices: Acceptable

• The Risk-Based Audit Plan was approved by the Deputy Head and sent to the Office of the Comptroller General in a timely manner.
• Annual Risk-Based Audit Plan methodology is evident and applied.
• There is evidence of preparation to provide for holistic assurance.
• There is partial information on the planned use of all audit function resources.
• Vast majority of planned work is on audit assurance versus other types of activities.
• Continuity of previous years work is identified with status or rationale.
• Approved assurance products are consistent with policy and internal audit standards requirements.
• Reasonable completion rate of assurance products (number of assurance audit reports) against 2007-2008 Risk-Based Audit Plan.
• Internal Quality Assurance and Improvement Program is documented and is in the process of being implemented.
• Assurance products (reports) are produced in a somewhat timely manner.
• Approved assurance products are made accessible to the public in a reasonably timely manner.
• Post-engagement follow-up process is well documented, and all recommendations are followed up using a risk-based approach.
• The department or agency provides notification to the Treasury Board Secretariat on issues of importance on an ad hoc basis or is aware of this requirement.
• The department or agency provides limited notification to the Treasury Board Secretariat on the posting of reports.

18.3 Administration of the Internal Audit Function: Acceptable

• Some elements of a comprehensive Human Resources Plan have been documented, and evidence of recruitment and external resourcing activity exists.
• Investment in Certified Internal Auditor certification, learning and training exceeds 10% of FTE salaries.
• Planned spending, *, was given to the Office of the Comptroller General. When comparing current spending of 2008-2009 with planned financial resources of 2007-2008, resource levels identified maintain the resource levels identified in 2007.
• Planned FTEs dedicated to internal audit have been maintained comparatively to 2007-2008. They meet the resource level identified in the planned internal audit function’s budget for 2008-2009.

18.4 Internal Audit Performance: Acceptable

• A Chief Audit Executive Annual Report for 2007-2008 was presented to the Departmental Audit Committee and the Deputy Head and submitted to the Office of the Comptroller General.
• Limited periodic reporting on the follow-up of Management Action Plans.

The agency has made further progress in the areas of Internal Audit (IA) Governance, Professional Practices and Reporting on IA Performance. In particular, the Departmental Audit Committee was established and is chaired by an external member.

19.1 Departmental Security Program: Opportunity for Improvement

• Organization has a partially developed security program that contains some of the required policy elements.
• Several deficiencies in meeting key policy requirements for the departmental security program.

19.2 Management of IT Security (MITS): Acceptable

• Organization has achieved the three priority objectives that form the foundation for Management of Information Technology Security (MITS) and complies with most MITS requirements.
• No significant deficiencies in meeting key MITS requirements.

19.3 Business Continuity Planning (BCP):

• Organization has conducted a Business Impact Analysis (BIA) and has determined that it does not provide critical services to the public and private sectors.

NB: The evaluation methodology for Line of Evidence 19.3 will be revised for Round VII of the MAF. Please consult the evaluation for details.

20.1 Management Engagement – Service and CLF: Acceptable

• There are expectations set by senior management for an institutional focus on meeting the needs of clients, specifically with respect to service standards and client satisfaction measurement.
• There are priorities and goals for service, but not always at the institutional level; these limited priorities and goals are set by senior management based on the use of limited performance evidence.
• There is a committee at the institutional level, composed of senior management accountable for service, which has a documented and communicated responsibility for making decisions about the overall management of service.
• There is limited monitoring of progress by senior management towards the achievement of the goals for service, making course correction difficult.
• There is monitoring by senior management to ensure that the requirements of CLF 2.0 are being met institution-wide; this information is generally used to make timely and proactive decisions and course correction.

20.2 Public/client views: Opportunity for Improvement

• Few tools used to obtain views from clients.
• Little evidence of incorporating feedback in the implementation of its services, programs, policies and initiatives.
• Little evidence of making consultation results available to the public.
• Little or no evidence of plans to obtain views from clients.
• There is a clearly identified target clientele for public consultations.

20.3 Official Languages: Strong

• Analysis of the Annual Review on OL shows the institution is fully meeting its obligations.
• Audits reveal few shortcomings in active offer and service delivery in both OL.
• No complaint or minimal number of founded complaints exits.
• The institution has the necessary linguistic capacity to serve the public in both OL.