ARCHIVED - MAF Assessment: Atlantic Canada Opportunities Agency - 2008

1.1 Leadership: Opportunity for Improvement

• Executive leadership sporadically communicates public service values and ethics on a top-down basis with employees.

1.2 Infrastructure: Opportunity for Improvement

• Risks in regard to possible breaches in public service values and ethics are regularly assessed, documented and followed up with mitigation plans and action by management.
• The organization's values and ethics plan is limited to a few activities over the next 12 months. 1.3 Culture: Opportunity for Improvement
• Organization has a limited understanding of the current state of public service values and ethics as a result of informal and non-validated information.

2.1 PAA Consistency: Acceptable

• The Strategic Outcome(s) is/are (a) clear outcome statement(s) that can be understood within and outside the department as a benefit to Canadians.

2.2 Measurability: Acceptable

• An adequate Program Activity Architecture has been developed with some issues to be resolved.

2.3 Quality:

• The performance indicators are not clear and cannot be used for data collection to provide reliable insight into program effectiveness.
• Targets are not stated appropriately as a performance level of the indicator.
• Outputs are not clearly identified as products/services and are seldom aligned with their expected results.

3.1 Business Plan: Acceptable

• Corporate business plan generally aligns resources and accountabilities to priorities.
• Corporate business plan generally integrates human resources, IM/IT, communications or other key corporate plans.

3.2 Governance Structure: Strong

• Management oversight of the organization's program activities is clearly evident.
• Organization's corporate governance structure is fully aligned to the organization's PAA.
• Senior corporate management structure or subordinate governance structure (e.g., committees) meet regularly.

4.2 Participation in Priority Initiatives: Strong

• Senior management is engaged in all the initiatives.
• The organization shows strength in its participation in priority interdepartmental initiatives.
• The organization shows strength in its participation in Public Service Renewal.

4.3 Portfolio Coordination: Strong

• Substantial attention is paid to the responsibility for portfolio affairs.
• The Department's portfolio coordination displays deliberate and effective leadership.

TBS has assessed ACOA with regard to its participation in Public Service Renewal - Strong, the Web of Rules initiative - Acceptable, and the Atlantic Gateway initiative - Strong, as well as its portfolio responsibilities.

5.1 Supporting Information: Strong

• Supporting information in TB submissions is always very accurate, reliable and complete.
• Policy and budget authorities are always identified.
• The organization is highly responsive to TBS feedback.

5.2 Analysis: Strong

• Business cases have comprehensive information and demonstrate robust analysis.
• Sustained capacity for appropriate responses to TBS comments is demonstrated.

5.3 Consultations: Strong

• Consultations are always on time (6 weeks or earlier before TB meetings).
• Consultations with central agencies are planned and conducted in a timely manner with sufficient lead time.

5.4 Quality control: Strong

• All important information is always included in the first draft.

6.1 Quality: Acceptable

• Evaluations submitted to TBS usually present findings, conclusions and recommendations that are supported by the evidence found in the evaluation report.
• The majority of evaluations submitted to TBS consistently address questions of program relevance, success and effectiveness.
• The majority of evaluations submitted to TBS consistently employ appropriate methodologies to gather data and inform the analysis.
• The majority of evaluations submitted to TBS include a management response and an action plan detailing timelines and management accountabilities.
• The majority of evaluations submitted to TBS include analysis of the limitations of the methodology and data sources used.

6.2 Neutrality: Acceptable

• Senior management committee is in place to support, oversee and monitor the evaluation function and management accountabilities arising from evaluations and evaluation related products. The committee is chaired by the deputy head or senior level designate. The committee meets regularly during the year.
• The majority of resources dedicated to evaluations are directed by the Head of Evaluation.

6.3 Coverage: Acceptable

• Option 1: The organization has committed to moving toward full evaluation coverage of their program base (e.g. over a five-year cycle). However, relatively low number of evaluations are completed each year and they cover less than 10%.
• The organization has shared its risk-based evaluation plan with TBS. The evaluation plan has information on evaluations planned, completed and carried over. It also includes links to the organization's PAA.

6.4 Usage: Acceptable

• Active, systematic and regular tracking of management action plans arising from evaluation recommendations is in place.
• The results of evaluations are usually brought for consideration in TB submissions, Memorandum to Cabinet, RPPs, DPR and Strategic Reviews.

7.1 MRRS Basis: Strong

• Clear performance expectations in the RPP are all tracked and rigorously reported on in the DPR.
• Strong links between performance and plans are present. The DPR consistently discuses any changes in plans and how they affected performance.

7.2 Credible information: Strong

• Informative financial tables are included and explained with text, as necessary.
• The DPR is concise. It contains a high proportion of estimates-related information and is drafted in a concise reporting style.

7.3 Context: Strong

• DPR effectively discusses lessons learned.
• DPR is for the most part balanced. It presents both positive and negative aspects of performance, and substantiation or explanation is consistently provided.

8.1 Change plan: Acceptable

• The organization has the capacity to evaluate whether or not change is required.
• An organizational change plan does not exist.

8.2 Engagement: Acceptable

• Individual and organization-wide change-related training programs are available.
• Employees are engaged in the strategy development phase.
• A learning culture exists within the organization.

8.3 Assessment: Acceptable

• Change plans and strategies are included in Performance Management Agreements of Senior Executives.

• Accountability for key risks is assigned to senior management and performance is assessed.
• Senior management encourages effective Risk Management and a risk-smart culture.
• Senior management ensures that the organization's Risk Management approach is tailored to the specific needs of the organization.
• Senior management has reviewed/approved the Corporate Risk Profile within the past year. Senior management reviews the organization's Risk Management approach within the current three-year planning cycle.
• The organization has a common risk assessment approach that is adjusted and approved as required by senior management.

9.2 Implementation: Acceptable

• Risk Management guidance and tools that enable the organization's risk management approach are made available to staff.
• The Corporate Risk Profile is systematically (horizontally and vertically) implemented into most operational levels across the organization.
• The organization's Risk Management approach is regularly communicated to staff and stakeholders.

9.3 Integration: Acceptable

• Operational level risks are prioritized into key risks.
• Risk information and Risk Management principles are adequately captured in senior management reporting.
• Risk information and Risk Management principles influence planning and resource allocation decisions.
• Risk information is adequately consulted for senior management decision-making.
• The organization makes adequate course corrections based on Risk Management performance and new information.

9.4 Continuous Improvement: Acceptable

• Comprehensive risk information was extensively gathered from internal sources of the organization for preparing the CRP.
• Corporate risks are consistently linked to the organization's strategic outcomes.
• Many relevant external sources are consulted during the development of the organization's CRP.
• The CRP provides a reliable assessment of the quality of risk information used.
• The organization adequately builds on past experience, better practice, and adjusts to fit any changes in management structures, priorities or strategic direction.
• The organization has implemented most recommendations provided during its last MAF assessment.

Senior management at the Atlantic Canada Opportunities Agency should be recognized for efforts to concretely integrate risk management into annual planning, audit and evaluation functions. ACOA has developed detailed risk management guides and mechanisms including mandatory instruments. In particular, its clear articulation of risk tolerance is commendable and will support the achievement of its strategic outcomes.

10.1 Fair: Acceptable

• Organization is undertaking action to improve the classification program in accordance with its level of risk.
• Evidence shows that labour relations matters are consistently and appropriately managed/addressed.
• Evidence shows that the organization exceeds standards of timeliness in payments to employees.
• Evidence shows that the organization is in compliance with Labour Relations and Compensation Operations direction (terms and condition of employment, collective agreements and/or applicable legislation).

10.2 Enabling: Acceptable

• Organization is under-representative in one or more of the four employment equity designated groups.
• Organization progress remains unchanged from the previous year in representation, recruitment, promotions and separations of the four employment equity groups.
• Promotions among employment equity groups are equal or less than previous year's performance.
• Separations among employment equity groups are equal or greater than previous year's performance.

10.3 Healthy and safe: Acceptable

• Evidence shows that the organization has an inadequately managed program to protect employees' occupational health and safety.
• The majority of employees feel recognized for positive performance.

11.1 Productive: Strong

• A significant number of employees indicate their organization supports their career development and learning needs.

11.2 Principled: Acceptable

• Organization is under-represented in one or more of the four employment equity designated groups.
• Progress against the previous year's performance on recruitment, promotion and separation for employment equity groups is less than the organization's average for all employees.
• Promotions among employment equity groups are less than representation for at least one group.

11.3 Sustainable: Acceptable

• Evidence indicates human resources planning integrated with business planning is generally in place and governance/organizational infrastructure generally exists to support it.

11.4 Adaptable: Strong

• A significant number of employees indicate their organization encourages continuous learning, improvement and innovation.

12.1 Governance: Strong

• IM is fully represented in the corporate-wide governance structure and in the corporate-wide governance or approval committee(s).
• Responsibilities are identified for IM policy development and implementation is wholly consistent with the GC IM Strategy and policy instruments.

12.2 Strategy: Acceptable

• A current and active IM strategy identifies support to business priorities and operations, information needs and accountabilities, IM policy considerations and is partially integrated with other corporate strategies, plans and planning cycles.
• An IM strategy implementation plan, including some timelines and resources, is underway and some achievements to date are identified.
• An IM awareness campaign or strategy is underway and all staff and executives are informed of their IM roles, responsibilities and accountabilities, and most have attended awareness/training sessions.

12.3 Privacy Act: Opportunity for Improvement

• Significant collections of personal information under the control of the organization have not been appropriately identified or described in accordance with the Privacy Act.

12.4 Access to Information Act: Opportunity for Improvement

• A significant number of the organization's functions, programs, activities and related information holdings have not been appropriately identified or described in its 2008 Chapter of Info Source: Sources of Federal Government Information. This information is a requirement of the Access to Information Act to facilitate public access to federal government information.

13.1 Leadership: Strong

• The senior official has responsibility and accountability for the full scope of information technology responsibilities and ensures that information technology supports organizational outcomes.
• Organization actively participates and demonstrates leadership in setting government-wide directions for information technology.

13.2 Planning: Strong

• A comprehensive information technology plan is in place and it aligns with the government-wide directions for information technology and with departmental business needs.
• Information technology management position is held by a highly engaged senior official designated within the corporate governance structure and related planning processes.

13.3 Value: Acceptable

• Organization is making efforts to appropriately use and plan for further use of information technology shared services.
• Organization devotes adequate management attention to service costing, asset management, performance measurement and reporting to ensure value delivery.

14.1 Investment Planning: Acceptable

• Organizational priorities and areas of highest risk are identified and guide investment decisions.
• The investment planning process integrates investments decisions across all asset classes.
• The organization's investment planning process considers investments over multiple years.

14.3 Materiel Management: Acceptable

• All elements of a materiel management framework are evident.
• Governance structures, approval processes and authority limits are documented and disseminated.
• Comprehensive internal policies are documented and disseminated.
• Reliable and sufficiently integrated information systems are in place.
• Some indicators of materiel performance are monitored.

15.1 Governance and Oversight: Acceptable

• Business cases are not required and there is no evidence that they are used to support project proposals.
• There is evidence of formal project governance and oversight mechanisms and that approved projects are generally linked with the strategic plans and priorities of the organization through established organization-wide procedures. Approval and corrective action decisions are documented.
• There is no evidence that the organization has exceeded Treasury Board approval limits.

15.2 Effective Management of Project Resources: Acceptable

• Adequate processes/procedures exist to ensure that planned projects have the required resources to achieve expected outcomes.
• Cost estimates are generated at the work package level only and do not use historical data or industry benchmarks.
• Little or no project management related training is available to employees.
• There is no evidence of project managers creating staffing plans and authorization for necessary resources is not secured before project execution.
• There is no evidence that the organization has failed to meet TB conditions regarding projects.

15.3 Effective Management of Project Results: Acceptable

• The organization requires that project milestones, deliverables and outcomes are documented for major projects.
• There is evidence of organization-wide procedures and processes which communicate project monitoring and performance information to project managers and project oversight mechanisms.
• There is no evidence that lessons learned are used to improve project management governance and oversight.

16.1 Governance and Oversight: Strong

• Effective, accountable and integrated procurement management processes and controls have been established and incorporated in the organization's business cycle. (e.g., contract review mechanisms, documented decision making or proper use of delegated authorities).

16.2 Meeting Operational Requirements: Acceptable

• Competent (highly qualified or certified) procurement human resources are in place.
• Consistent procurement training and certification programs exist.
• Efficient, effective and integrated procurement information systems and processes are evident.
• Integrated, timely and accurate procurement financial and non-financial reporting exists.
• Proactive and informed decision making and oversight are evident.
• Procurement processes that contribute to cost savings and value for money are fully utilized.

17.1 Authorities and Policies: Opportunity for Improvement

• Audit report results show evidence of deficiencies that are of some concern.
• Departmental procedures, tools, training and support for those individuals delegated with delegated with Section 34 authority show evidence of deficiencies that are of some concern.
• Departmental processes for classification of moneys, internal controls for receiving and recording money and depositing money show evidence of good financial management practices.
• Departmental processes for informing those delegated with Section 33 authority of their responsibilities and dealing with requests for payments that are problematic show evidence of good financial management practices.
• Departmental processes to provide individuals delegated Section 33 authority with the information necessary to assess and approve transactions and to assess the adequacy of Section 34 account verification show evidence of good financial management practices.

17.2 Public Accounts Reporting: Strong

• Greater than 97% (Grade A) of Public Accounts plates completed on time.
• Minimal Central Financial Management Reporting System (CFMRS) coding errors.

17.3 Management Capacity: Strong

• A significant amount of training is provided for the financial management organization.
• All, or almost all, FIs and management team members in the financial management organization have current, approved learning plans.
• Positions, the duties of which are being performed by an individual indeterminately appointed to that position, comprise all, or almost all, of the FI segment of the financial management organization.
• Positions, the duties of which are being performed by an individual indeterminately appointed to that position, comprise all, or almost all, of the positions on the management team of the financial management organization.
• Some processes in support of a sound succession plan for key positions are in place.
• There is an acceptable functional relationship between the CFO/SFO and FI positions that exist outside the financial management organization.*This row is only applicable where the department or agency indicates there are FI positions outside the financial management organization.

17.4 Financial Statements: Acceptable

• "There is evidence of some work undertaken to assess and/or monitor and/or improve internal control over financial reporting "
• The Financial Statements are compliant with Treasury Board Accounting Standard 1.2 - Departmental and Agency Financial Statements and reporting deadlines were met.

17.5 Internal Reporting: Acceptable

• The internal financial reporting package is accompanied by a weak discussion and analysis.
• The internal financial reporting package is presented to senior management less than 15 calendar days after period end.
• The internal financial reporting package is presented to senior management ten or more times per year.
• The process for reviewing information before it is presented to senior management to ensure no material errors or omissions is established.
• The scope of the internal financial reporting package is comprehensive. While the department maintained its overall rating, LOE 17.1 decreased on account of poor audit reviews and poor answers to the questionnaire. On the other hand, its financial management capacity increased from an Acceptable to a Strong rating.

18.1 Internal Audit governance: Strong

• There is an approved Internal Audit Charter in line with the 2006 Policy on Internal Audit.
• The Implementation Plan covers all of the required policy elements.
• Ongoing monitoring of, and progress in implementing, key elements of the plan are on track with planned timelines.
• Chief Audit Executive reports solely and substantively to the Deputy Head.
• An independent Departmental Audit Committee has either recently been established or is scheduled to be in place and is on track with planned timelines.
• There is a Departmental Audit Committee Charter in line with the 2006 Policy on Internal Audit.

18.2 Internal Audit Professional Practices: Acceptable

• The Risk-Based Audit Plan was approved by the Deputy Head and sent to the Office of the Comptroller General in an untimely manner.
• Annual Risk-Based Audit Plan methodology is, for the most part, evident and applied.
• There is evidence of preparation to provide for holistic assurance.
• There is limited information of the planned use of all audit function resources.
• Most post-engagement follow-up activities are identified.
• Vast majority of planned work is on audit assurance versus other types of activities.
• Continuity of previous years work is clearly identified with status and rationale.
• Approved assurance products are not consistent with policy and internal audit standards requirements, containing one key methodological deficiency.
• Very low completion rate or no assurance products (number of assurance audit reports) against 2007-2008 Risk-Based Audit Plan.
• An Internal Quality Assurance and Improvement Program has been drafted.
• Post-engagement follow-up process is well documented, and recommendations are followed up using a risk-based approach.
• The department or agency is proactive in providing notification to the Treasury Board Secretariat on issues of importance.

18.3 Administration of the Internal Audit Function: Acceptable

• Some elements of a comprehensive Human Resources Plan have been documented, and evidence of recruitment and external resourcing activity exists.
• Investment in Certified Internal Auditor certification, learning and training exceeds 10% of FTE salaries.
• Planned spending, *, was given to the Office of the Comptroller General. When comparing current spending of 2008-2009 with planned financial resources of 2007-2008, resource levels identified maintain the resource levels identified in 2007.
• Planned FTEs dedicated to internal audit have been maintained comparatively to 2007-2008. They meet the resource level identified in the planned internal audit function's budget for 2008-2009.

18.4 Internal Audit Performance: Acceptable

• A Chief Audit Executive Annual Report for 2007-2008 was presented to the Departmental Audit Committee and the Deputy Head and submitted to the Office of the Comptroller General.
• Regular periodic reporting on the follow-up of Management Action Plans is evident.

The agency has made ongoing process in the areas of Internal Audit Governance and Professional Practices. In particular, the Chief Audit Executive (CAE) reporting relationship to the Deputy Head (DH) and the conduct of monthly bilateral meetings on substantive internal audit issues.

19.1 Departmental Security Program: Acceptable

• Organization has in place a fully developed security program that comprises key policy elements and is administered by an appointed Departmental Security Officer (DSO) who is positioned to provide strategic advice and guidance to senior management.
• No significant deficiencies in meeting key policy requirements for the departmental security program.

19.2 Management of IT Security (MITS): Acceptable

• Organization has achieved the three priority objectives that form the foundation for Management of Information Technology Security (MITS) and complies with most MITS requirements.
• No significant deficiencies in meeting key MITS requirements.

19.3 Business Continuity Planning (BCP):

• Organization has conducted a Business Impact Analysis (BIA) and has determined that it does not provide critical services to the public and private sectors.

Note: The assessment methodology for Line of Evidence 19.3 will be revised for MAF Round VII. Please refer to the assessment for details.

20.1 Management Engagement - Service and CLF: Opportunity for Improvement

• There are limited expectations set by senior management for an institutional focus on meeting the needs of clients, specifically with respect to service standards and client satisfaction measurement.
• There are priorities and goals for service, but not always at the institutional level; these limited priorities and goals are set by senior management based on the use of limited performance evidence.
• There is a committee at the institutional level, composed of senior management accountable for service, which has a documented and communicated responsibility for making decisions about the overall management of service.
• There is limited monitoring of progress by senior management towards the achievement of the goals for service, making course correction difficult.
• There is monitoring by senior management to ensure that the requirements of CLF 2.0 are being met institution-wide; this information is generally used to make timely and proactive decisions and course correction.

20.2 Public/client views: Acceptable

• Evidence of incorporating feedback in the implementation of its services, programs, policies or initiatives.
• Little or no evidence of making consultation results available to the public.
• Many tools are used to obtain views from some clients.
• There are plans to obtain views from clients.
• There is a clearly identified target clientele for public consultations.

20.3 Official Languages: Acceptable

• Analysis of the Annual Review on OL shows the institution is fully meeting its obligations.
• Audits reveal few shortcomings in active offer and service delivery in both OL.
• No complaint or minimal number of founded complaints exits.
• The institution has the necessary linguistic capacity to serve the public in both OL.