4. Conclusion

You will note that many of the observations and recommendations read like a textbook on project management. For every project there are key critical success factors. We used these factors as our audit criteria and we found that in most cases these factors were either absent or not completely implemented. This was not just a "paper exercise" where we simply looked for the existence of specific documents and checked a box if we found them and wrote an observation if we didn't. We examined all aspects of the project and the project management processes. 

The lack of a formal, structured approach to managing this project raises concerns for its ultimate success. We are not suggesting that simply producing specific reports or documents will be sufficient to turning around the project. There has to be a sweeping change to the way this project is run and this includes the tone from the top. The ad hoc "Get to September" approach is not sustainable and needs to be replaced. 

The project needs clear accountabilities and authorities; there needs to be formal structures and processes, and the people have to have the right mix of ability and experience. The project needs to be handled in a textbook fashion since it cannot afford to fail.

Management Response

The EMIS team has accepted the TBS Enhanced Management Framework as its textbook for project management. Further, it is encouraging TBS business owners to adopt CIOB's BTEP and GSRM to support their business transformation efforts. Also, consideration of CIOB's work on an outcomes management practice will support and help guide the development of the roadmap and detailed project plans.

EMIS has instituted RUP as its development methodology for EPA Phase I; staff are being trained, and analytical tools acquired, to support these two approaches (EMF and RUP). The Secretary has approved the new EMIS governance model, which clarifies roles and accountabilities.

An overall action plan for planned project changes has been developed, with experienced professional support, to help guide the project sponsor and project director in a process of significant refocusing of the project to meet the recommendations set out in this report, and to further the aims of EMIS in delivering support to the Treasury Board's transformational agenda.

Appendix 1:Audit Criteria

• Project Management
  • Authorities and Resources
    • Project Sponsor is responsible for ensuring that the department understands the value and importance of realizing benefits predicted.
    • From project inception through to ongoing maintenance, sponsoring departments should delegate authorities and allocate adequate resources appropriate to the scope, complexity and risk of the project. 
    • Roles and responsibilities are well defined throughout the life cycle of the project.
    • Project should have mechanisms in place to manage the necessary legislative, regulatory and policy requirements.
  • Project Scope
    • Project leaders are accountable for the full definition of the scope and approval of all scope changes for all projects including the wider interests of the government. 
    • A scope change control system outlines the process for changing the project scope, documentation requirements, tracking systems and approval requirements.
    • Projects, particularly long ones, should have scheduled checkpoints or gates when the project is reviewed and management can decide on the future of the project and any appropriate corrective action. 
    • Gates should be chosen to coincide with logical project review points. Critical path and milestone management is in place and is effective for planning and monitoring scope.
  • Project Risk, Complexity and Economy
    • Project leaders should ensure that project managers perform adequate project planning that addresses the size, scope, complexity, risk, visibility and administrative needs of specific projects. 
    • Project leaders should ensure that the proposed project management framework and allocation of project management resources are based and optimized on the complexity and the assessed risk for the individual phases of a project. 
    • Project management should develop a risk management plan that describes how risk identification, qualitative and quantitative analysis, response planning, monitoring, and control will be structured and performed during the project life cycle. 
    • Senior management should have feedback mechanisms in place to assess the effectiveness of the project management.
  • Project Monitoring
    • The project is performing its intended purpose(s):
      • Monitoring should be focused on continuous improvement and meeting both internal and external performance needs.
      • Regular project performance reports (status reports, quality deliverables/ outcomes) should be prepared by project management and communicated with key stakeholders. 
      • The performance review process should include: status reporting, progress reporting, and forecasting.
  • Business Arrangements
    • Dependence on third parties (e.g. PWGSC).
    • Contractual arrangements (e.g. software/consultants) should be monitored against service levels.
  • Communications
    • Identifying the communication needs of stakeholders, and determining a suitable method for meeting those needs is an important factor for project success. The project management team should determine the project's communication requirements in consultation with the project stakeholders:
      • who needs what information,
      • when they will need it,
      • how it will be given to them and by whom (what decisions/ results).
    • Communications within the project team are clear and understood.
• Business Transformation and Organizational Readiness
  • Multiple Stakeholders and Business Process Change
    • Culture changes across multiple stakeholders (e.g. sharing of information) and business process changes are addressed through organizational readiness, ongoing communication, education and awareness, and deployment strategies (e.g. training).
  • Business Case
    • The business case should be based on the full cost of the system from initiation through development, implementation and estimated annual cost of operation. 
    • It should be reviewed and revalidated at each scheduled gate and whenever there is a significant change to the project or the business function. 
    • If the business-case of a project changes it should be re-approved through the departmental planning and approval process.
  • Accountability of Projects
    • Clients of systems should be fully involved in all phases of projects, including project gates, to ensure that systems meet their business requirements in the best manner possible and the expected benefits are obtained. 
  • Accountabilities of Multiple Stakeholders
    • The joint commitment and level of involvement of other federal government departments or agencies should be documented in an appropriate interdepartmental agreement, Memoranda of Understanding or equivalent documentation signed by a senior official of each department. 
    • All stakeholders should agree on objectives, roles, deliverables and levels of participation, and should be signed by key senior officials. Interdependencies, both within TBS and with outside departments, need to be well defined (e.g. PWGSC CFMRS).
• Technology
  • Common Technology, Tools and Methodology
    • In conducting IT projects, departments should conform with approved common infrastructure technology and application and data architectures and infrastructure elements provided by the CIOB of TBS and Government Technology and Information Service of PWGSC, unless there are over-riding operational requirements.
    • Adoption of common IT methodologies and tools used across government should be considered for performing risk assessment and control.  
  • IT Security Resources for Projects
    • In planning new programs, services or major upgrades to existing programs or services, the managers responsible should, at the earliest stage of the funding and approval process, determine the IT security requirements for these programs, services or upgrades and include resource requirements in funding requests.

Appendix 2: List of Interviewees

Ruth Dantzer, Associate Secretary

Kevin Page, Executive Director / Project Sponsor EMIS

Mary Jane Jackson, former EMIS Project Co-Sponsor

James Oickle, Project Leader EMIS

Mike Bennett, Project Consultant EMIS

Jim Libbey, Executive Director, Strategic Systems Infrastructure Directorate, Office of the Comptroller General

Helen McDonald, Acting Chief Information Officer

Mike Joyce, Assistant Secretary, Expenditure and Management Strategies Sector

Jim Alexander, Executive Director, IT/IM Stewardship, Chief Information Officer Branch

Daphne Meredith, Assistant Secretary, Economic Sector

Susan Cartwright, Assistant Secretary, Government Operations Sector

Dennis Kam, Acting Assistant Deputy Minister, Corporate Services Branch

Steve Silcox, Acting Executive Director, Finance and Administration, Corporate Services Branch

Marilyn MacPherson, Executive Director, Human Resources Division, Corporate Services Branch

Robert Brodeur, Executive Director, Information Management and Technology Directorate, Corporate Services Branch

Neil Henriksen, Procurement and Risk Management

Mike Delorme, EMIS Project Control Consultant

Arvind Srivastava, Team Lead - Data and Information Management Team

James Gervais, Team Lead, EMIS Administration Management

Duncan Bailey, Senior Research Officer

Paul Joly, Team Lead, EMIS Development Team

Ian Tait, Deloitte Consulting Managing Partner

Arman Mirchandani, Deloitte Consulting EMIS Project Lead

Carlo Beaudoin, Team Lead Business Requirements

Mark Guyan, Team Lead EMIS Operations – Network Services

Josée Desjardins, Team Lead EMIS Communications Team

Bruno Bernier, Director, Central and Public Accounts Reporting, Public Works and Government Services Canada

Richard Patenaude, Director, Central Accounting Systems Directorate, Public Works and Government Services Canada