This page has been archived.
Information identified as archived on the Web is for reference, research or recordkeeping purposes. It has not been altered or updated after the date of archiving. Web pages that are archived on the Web are not subject to the Government of Canada Web Standards. As per the Communications Policy of the Government of Canada, you can request alternate formats on the "Contact Us" page.
No. 1984-03
Date: 10-10-84
Subject: Pre-implementation Audit
Question concerning this notice should be directed to:
Policy and Special Projects,
Centre of Excellence for Internal Audit
Comptrollership Branch, TBS
(613) 957-2270
Purpose and Scope
Modern internal auditing, as defined in the Standards1, has two main thrusts, namely: (l) helping managers achieve improved productivity ( auditing economy, efficiency and effectiveness) of their ongoing operations; and (2) advising management on the development of economic, efficient and effective major, new, or changed infrastructure (pre-implementation auditing).
The purpose of this Policy Interpretation Notice (PIN) is to provide the internal audit community with guidance related to pre-implementation audit. It supplements standard No. 2, dealing with "scope" and the associated discussions in Chapter Two, Scope and Frequency, and Appendix A, Auditing Computer-Based Systems.[1]
Issues
1. What is a pre-implementation audit?
A pre-implementation audit is an audit carried out on departmental/agency systems during the design/development and installation process rather than after the system has been turned over to the client for operation.
2. What is the scope of a pre-implementation audit?
To be consistent with the Standards, the scope of pre-implementation audits should include all major new systems under development, including: new legislation, policies and procedures, information systems (both EDP and non-EDP) and production processes.
Pre-implementation audit consists of two possible components:
3. Why do a pre-implementation audit?
The rationale for initiating pre-implementation auditing is that it is more cost-effective to correct weaknesses in the control framework during the design/ development and installation process than after implementation, when large quantities of resources have been expended and strong commitment to the entity under design has been generated.
This does not eliminate the need for post-implementation audits as there is no assurance that what was designed and installed was maintained or operated as intended, and that the original requirements continue to hold true.
4. How far can the auditor go without impairing independence?
Auditors always have the problem of possibly compromising their objectivity through their efforts to gain an understanding of, and empathy with, the auditee's situation. This is particularly true for pre-implementation auditing. For example, there is a real danger that the auditor will get co-opted into participation in actual design of controls rather than playing the "advice" role. Even in playing this more restricted role the auditor may become too committed to the resultant systems configuration to be totally unbiased in later, post-implementation audits of that same system.
It is considered however, that the benefits of such auditing far outweigh the risk incurred and that, in any case, it is possible to minimize the danger of loss of objectivity by judicious control of the nature and scope of the auditor's involvement, and by adopting an appropriate assignment strategy.
Interpretation Notice Position
The position of this PIN is that: Pre-implementation audits should be undertaken for all major systems under development in departments and agencies; they should be reflected in the departmental/agency internal audit policies and plans; and, the potential loss of auditors' objectivity can be minimized through appropriate terms of reference and assignment strategy.
Disposition
It is our intention to adopt one or more of the following actions, based on the nature of the feedback received from the internal audit community:
The internal audit community is invited to provide comments to IA&SSD on the attached Discussion Paper with the view to influencing the content and location of future guidance on this important subject.
The purpose of this Paper is to explore various aspects of participation by auditors in systems under development. This type of participation will be termed "pre-implementation audit".
The subject of pre-implementation audit will be discussed from the point of view of its background, definition, scope, purpose, effect on other aspects of the audit function and possible means of implementation.
There are several factors which have the effect of increasing pressure on internal auditors to participate in systems development projects: systems development projects are notorious for cost/time overruns; implemented systems are equally notorious for not meeting all user requirements; systems, particularly EDP systems, often have under-designed control frameworks; and, recent cost-cutting programs, prompted by a recessionary economy and large deficits, have focussed increased attention on improving the productivity/efficiency of all processes. This puts the spotlight particularly on the systems development process because of the costly down-stream effects of inadequate design and implementation.
Audit literature has recognized the potentially useful role of pre-implementation audit for some time, however, it has been almost exclusively centred on EDP information systems. This view of systems has two limitations: not all systems are "information" systems and EDP is only one possible vehicle for implementing systems. In what follows, we will employ a broader view of systems1.
Our recognition of the desirability of pre-implementation audit is reflected in the Standards2 in several ways:
As indicated above, we will term the auditor's participation in systems under development "pre-implementation audit" and will adopt the broadest definition of the term "system", a definition which encompasses all major infrastructural mechanisms in an organization, including: legislation, policies and procedures; information systems (EDP or other); production systems/processes; agreements and contracts; and organization structures. See Figure 1 for an illustration of typical infrastructural hierarchy and the relationships.
The scope of a pre-implementation audit has two components. The first has to do with the entities subject to pre-implementation audit - these have already been defined above as all major new systems (i.e. infrastructural hierarchy) and should include major re-designs of existing systems.
The second component of scope has to do with the type of auditor participation that could be contemplated. These are:
The first type of audit is aimed at assuring management that the development process adheres to prescribed (e.g. central agency) or generally accepted policies and practices, which ensure that sound systems are conceived, developed and implemented and that the development process is economic, efficient and effective. The second type of audit assures management that adequate operating and management controls are being conceived, developed and implemented, which in turn will ensure that the operators, managers and users of a system will have a means of determining whether the system is performing as intended.
For both types of audit the rationale employed is twofold. The first is that it is easier and less costly to ensure adequate design at its conception than to rely on remedial changes after implementation. The second is that systems that will be used repeatedly must be more carefully designed and managed at the front end because of the potentially negative effects on operating efficiency and effectiveness that could result from a poor design/development process.
A derivative benefit from pre-implementation audits is the resultant heightened awareness of the role of control accruing to all participants (managers, users, systems designers, etc.).
There are two aspects to consider. One is the effect on other types of audit and the other is the effect on the audit function's independence.
A pre-implementation audit should reduce the number of systems-oriented findings that the audit group is likely to identify in future periods, however, it is unlikely to eliminate them. There are three reasons for this: It is unlikely that any design will be perfect to start with; even if a design is perfect, it is unlikely that it will be implemented, maintained and operated exactly as designed; and finally, it is unlikely that the original environmental conditions or requirements will continue to hold true over the entire life of the system.
Independence is a difficult aspect to deal with as it is very subjective. Various auditors and managers will have differing views as to what independence is and when it is jeopardized. Guidelines which may prove useful in this context follow:
Although the potential for loss of objectivity in performing audits cannot be eliminated completely, it can be limited. Judicious control of the degree of auditor involvement in pre-implementation audits can minimize both the existence and appearance of such loss of objectivity.
To start with, it is assumed that all departmental/agency internal audit policies and plans appropriately reflect provisions for pre-implementation audit in accordance with the Standards.
Other prerequisites to successful implementation of such audits would include: a requirement (in departmental/agency policies or directives) that managers notify the head of internal audit of all major systems development projects (this does not preclude the internal audit group performing its own surveillance through review of plans and other relevant documentation and through personal contact with managers); a requirement that managers/project leaders invite the internal audit group to participate in all major systems development activity (including participation in systems development teams, systems development Steering Committees, etc.); active support of senior management for such participation, and, the acquisition of a sufficient number of appropriately qualified senior auditors who would lend credibility to the systems and to the internal audit function by their participation (contracting is an alternative, where resource budgets permit).
Although a comprehensive guide to pre-implementation auditing is beyond the scope of this Paper, some suggestions are presented for your consideration:
For example, for the audit of the systems development process the relevant criteria would include those provided for project management, contracting and EDP systems in the Administrative Policy Manual and in the Guide on Financial Administration for Departments and Agencies. Also, relevant IACIA Guides (e.g. EDP Audit) are equally usable for pre-implementation auditing.
In the case of pre-implementation audit of the control framework being designed into or around the system under development, the starting point would be a pre-determined, normative control model, as it is for any audit, except that in this case there would be only a paper representation of an actual control framework to compare it with, rather than the physical one that would exist in the case of a post-implementation audit of the system.
In general, when an auditor is involved in a pre-implementation audit at an intermediate level in the hierarchy, it is important for that auditor to become familiar with the role of adjacent (higher, lower and peer level) elements in the structure in order to be able to judge consistency and continuity.
Pre-implementation audit is a valuable managerial aid. This has been recognized in the literature on auditing and reflected in the Standards.
This type of auditing applies to all major infrastructural mechanisms (systems) under development. It encompasses two distinct activities, i.e. audit of the development process and audit of the control framework being designed.
Pre-implementation auditing methods and techniques do not differ radically from normal, post-implementation auditing. However, the skill of the auditor applying those methods and techniques must be of the highest order because of the downstream impact of such an activity.
1. See Standards for Internal Audit in the Government of Canada.
2. See the Administrative Policy Manual, Chapter
440.