ARCHIVED - Treasury Board Submission Process Audit - September 2003

• Previous Page
• Table of Contents
• Next Page

This page has been archived.

4.0 Audit Findings

Overall we found that there is an established process in place at the Secretariat for review and approval of Submissions. We found that Secretariat staff are dedicated and committed to the preparation of information for Treasury Board ministers that permits them to understand the submissions put forward to them and to question or challenge them as needed. The analysts work hard with departments to ensure Submissions are ready for presentation to Treasury Board.

The observations and recommendations below reflect a lack of formalization and inconsistencies in the process used by the Secretariat for Treasury Board Submissions. They also relate to the need for improved documentation standards.

4.1 Observations and Recommendations

The following are areas where we have identified recommendations for improvements to the Treasury Board Submission process.

Observations Rated High Risk

4.1.1 Treasury Board Submission Policies and Procedures

The audit identified several aspects of the Treasury Board Submission process where formally documented policies and procedures have not been developed. This may create inefficiencies and inconsistencies in the process for reviewing Submissions.

Our interviews indicated that, among the branches, there are inconsistencies in the understanding of the roles and responsibilities of the Program Analyst, Policy Analyst, Directors and Assistant Secretaries.

Below is a list of some of the areas that have been identified where policies and procedures should be developed:

Method of Transferring a Submission from the Government Department to the Secretariat, i.e. via e-mail or Secured Courier

It is common practice to send draft submissions to and from the Secretariat using unsecured email or fax. The analysts interviewed for the audit were uncertain as to whether the draft Submissions should be handled similarly to the final Submissions, which are "Confidence of the Queen's Privy Council" requiring properly secured methods of communication.

We recommend policies and procedures for handling the communication of draft Treasury Board Submissions should be developed to clarify the level of security required.

Management Response

Agreed. TBS will review current practices for communicating draft TB Submissions to and from departments, using the government policy on Confidences of the Queen's Privy Council for Canada as a general guide and will develop a new procedure that will clarify for staff the level of security required for draft TB submissions by November 2003. However, it should also be noted that the Privy Council Office is in the process of revising guidelines for the use of Cabinet Confidence level of security.

The Secretary will communicate this new procedure to departments and all TBS staff as well

as ensure that an increased number of secure facsimile machines and secure courier services are available for use by TBS staff.

Policy Expert Review and Sign-0ff

There are no formal policies or procedures in place regarding the review of Submissions by policy experts and whether their sign-off is required prior to the Submission being presented to Treasury Board. In our discussions with analysts, it was evident that there is no common understanding of whether and when policy sign-offs are required and what such sign-offs would signify. This has led to inconsistencies in the way analysts handle policy reviews for submissions. In our test sample of Submissions, 62% were missing at least one policy sign-off.

We recommend that policies and procedures be developed to clearly define what is required for policy expert review and what this sign-off in fact constitutes. The following areas should be addressed:

• Required policy expert review and sign-off should be completed prior to the Submission being presented to Treasury Board to ensure that all submissions are free from policy issues.
• In some cases, there may be instances where a review by a policy expert is not required. In these cases, there should be a requirement for the rationale for this decision to be documented.
• Procedures for filing and archiving the evidence of sign-offs to ensure records for all Submissions are complete, consistent and properly filed should be developed.
• Ensure that the program analyst and SCDC understand and concur with the procedure for the Pink Jacket. The Pink Jacket's original intent was to ensure the Policy Sectors could review the final submission so that all policy matters have been addressed. Occasionally not all Policy Sectors receive the Pink Jacket, as the program analysts do not always note on the Jacket the Policy Sector that they consulted. This could potentially result in final Submissions going to the Board with outstanding policy matters.
• The Secretariat should establish procedures and guidelines defining the responsibility for review and follow-up when sign-offs are outstanding.

Management Response

Agreed. Current TBS practice is for Official Languages, the Expenditure Strategies Division, the Expenditure Operations Division as well as the Program Division review and sign-off all TB Submissions. Current procedure is to record these sign-offs on the pink jacket covering every submission that goes to Treasury Board.

In addition, there are many other TBS policy experts who review and sign-off submissions depending on the issues raised. In all cases, the sign-off of the Program Director sponsoring the submission, on the pink jacket, indicates that appropriate policy input has been sought and taken into account in TBS's advice to the Treasury Board.

In addition, TBS has developed a formal internal policy sign-off protocol for TB submissions related to Grant and Contribution Programs. TBS will review this model to assess whether it could be developed for use more broadly as a guideline or tool for TBS Analysts to clearly define what is required for policy expert review including procedures for: review and follow-up when sign-offs are outstanding, and filing and archiving the evidence of sign-offs to ensure records for all submissions are complete.

Director Level Review

There are no procedures that define the responsibility of Directors for reviewing the final précis prior to it being presented to Treasury Board. In our sample of 69 Submissions, the Director or delegate had not signed three of the précis. Thus, it is not possible to confirm that the Director or delegate had reviewed and approved the précis.

We recommend the development of a procedure that outlines the responsibility of Directors to review the précis and Submission as well as the areas that they should ensure are completed prior to signing-off on the précis.

Management Response

Agreed. The Submission and Cabinet Document Centre will not accept a précis unless it has been signed-off by a Director.

In addition, TBS will review its internal procedure for sign-off of the précis of a Submission generated by a branch or sector within TBS to clarify responsibilities.

Decision Letters

There is a requirement that decision letters are issued to the government department within two weeks of the Treasury Board decision. In our test sample, 40% of the decision letters did not meet this deadline.

We recommend procedures be developed to ensure that decision letters are issued within the required time frame to allow departments to proceed with their activities using the accurate and formal documentation of the Treasury Board decision.

Management Response

Agreed. The Secretary will remind TBS management and staff of TBS's performance target to issue decision letters within two weeks of the Treasury Board decision.

The Program Sectors will further streamline existing manual procedures to produce Decision Letters to improve TBS's performance in achieving this target.

In 2004, the Expenditure Management Information System (EMIS) Project Team and the Submission and Cabinet Document Centre will automate the production of Decision Letters as a key feature of a new submission tracking and information module of EMIS.

Definition of Classification

A classification system is used to guide Treasury Board Ministers in the review a particular Submission. The Analyst handling the Submission initially assigns the classification. There is guidance available through discussions with other analysts and unofficial documents such as "A Program Analyst Survival Guide". There is no formal readily available definition of the criteria for these classifications.

The lack of a formal readily available definition of criteria for these classifications could lead to inconsistencies or inappropriate classifications of Submissions. This is mitigated to some extent since the classification is reviewed by the analyst's director and at executive meetings prior to the Submission being presented to Treasury Board. Without formal definitions, the process could be subjective.

We recommend a formal definition for each of the classifications should be widely and continuously disseminated to help ensure that Submissions are properly and consistently classified.

Management Response

Agreed. TBS will review and update as necessary definitions of the criteria used to structure agendas for Treasury Board meetings.

The Submission and Cabinet Document Centre and the Learning and Knowledge Management Team within the Program Sectors will communicate and promote more widely a common understanding of the updated definitions.

4.1.2 Analyst Turnover

The role of the analyst is critical to ensure that the précis and, to the extent possible, Submissions are properly prepared before they are presented to Treasury Board. Based on our interviews, there has been a high level of turnover of analysts in recent years. This high level of turnover means that there are many inexperienced analysts, which increases the risk that appropriate review may not be performed on Submissions. As well inconsistent training and a lack of formal guidance material make it more difficult for new analysts to understand what they are required to do. While there is material available to assist them to understand their role, many of the analysts we interviewed were not aware of what was available or were not referencing it. Some of the departments interviewed indicated that the TBS analysts they deal with are not always adequately trained.

Although there is training available for new analysts, referred to as Boot Camp, as well as a workshop covering the information they need to know for reviewing Submissions and preparing précis, it was evident from our interviews that not all new analysts attended the training prior to being required to assist with submissions.

We recommend that formal procedures be developed to ensure that new analysts receive initial training in advance of being required to assist with Submissions. For an introductory period, analysts should be required to review the available guidance and material prior to being asked to review any Submissions. If it is not feasible for new analysts to attend the training, they should be required to review the virtual versions of the training on-line. New analysts should also be required to review other guidance material available through the Intranet prior to reviewing Submissions.

We recommend that a mentoring system be formalized for the entire Secretariat to ensure that new analysts have access to experienced analysts to provide guidance and answer questions as needed. While there are mentoring programs in place within the Sectors, they are not necessarily formalized and may therefore be inconsistent and not as effective as possible. During our interviews with new analysts, they indicated that the more experienced analysts were often too busy to be able to provide them with the mentoring they required.

We recommend that the Secretariat develop a process anticipating that there will be turnover of analysts and implement a knowledge transfer process to ensure that information from departing analysts is retained and accessible.

We recommend that the Secretariat improve information systems and documentation of Submissions as discussed in this document, to help facilitate the transfer of knowledge as analysts change.

We recommend that the Secretariat consider measures to encourage retention of analysts for a longer period of time.

Management Response

Agreed. New TBS analysts receive initial training and orientation in advance of assisting with the analysis of a TB Submission.

In addition, the Learning and Knowledge Management Unit within the Program Sectors will continue to offer "Program Analyst Boot Camps" to orient all new program analysts to their role as well as a variety of internal workshops, sessions and courses on how to analyse TB Submissions and Memoranda to Cabinet, develop their negotiating skills, as well as technical instructions on all phases of the Government's Expenditure Management System.

Over the past two years, TBS has worked hard to develop programs to support recruitment and retention of TBS analysts including a mentoring system and succession planning for the entire Secretariat. Throughout 2003, these programs have become embedded and formalized across the Secretariat.

Through the implementation of RDIMS, TBS, and in particular the Program Sectors, have been able to standardize and better coordinate the electronic repositories of documentation related to TB submissions to help facilitate the transfer of knowledge to new analysts as analysts change. More fundamentally, TBS is developing a new Expenditure Management Information System (EMIS) that will provide improved information management and analytical support to TBS Analysts.

Observations Rated Medium Risk

4.1.3 Treasury Board Policies

Treasury Board has numerous policies (300+) that may be appropriate to consider when preparing and reviewing a Submission. There is a key group of policies that analysts usually consider when they are preparing or reviewing Submissions. While some of the key policies are listed in the Treasury Board Submission guide, this is not a comprehensive resource for assessing which policies should be considered for a particular Submission. Guidance on policies is also available on the Secretariat's Intranet site that can be accessed by analysts. Based on our interviews, the analysts are not consistently using this information and are not always aware that it exists. Due to the number of different policies, analysts can have difficulties in determining the specific policy instruments that are particularly applicable to an individual Submission.

The Treasury Board Secretariat has launched a major review of its total policy suite with a view to reducing and enhancing the relevance of policy instruments and reports, improving the accessibility of policy instruments and targeting them to appropriate audiences. This review may well result in a significant reduction in the number of policy instruments and a greater clarification of their objectives.

Until such a time as the review is complete, we recommend consideration be given to reviewing why analysts may not be using available policy information and taking the necessary steps to ensure that there is a practical guide or road map available on the Treasury Board policies.

Management Response

Agreed. TBS will ensure that all future TBS orientation programs, Program Analyst Boot Camps and various other training and information sessions for TBS analysts who deal with TB submissions provide an overview of the TB Policy suite including a walk through of the TBS intranet site where more detailed information on all TB policies can be found.

In addition, as work on the TB Policy Suite progresses and new policy instruments come on line, TBS will develop and deliver a calendar of learning and information sessions to both TBS analysts and horizontal communities most affected by the policy change to ensure a deeper knowledge and understanding of the objectives of these policies.

4.1.4 Risk Matrix

The Secretariat has implemented a new risk matrix to assist Treasury Board ministers in determining the relevant risk associated with a Submission. According to some of the analysts interviewed, the criteria for the risk matrix are judgemental, not well defined and not a useful tool to assist them when determining the rating for the Submissions. As the Board members are placing reliance on this rating, the Secretariat has to ensure that the assessment of risk is applied consistently and is a meaningful tool to define the level of risk.

We recommend that the criteria to assess the risk for the matrix be revised to be more objective and useful to assist analysts when determining the level of risk.

Management Response

Agreed. Throughout the fall of 2003, TBS will identify the "lessons learned" after more than one year of experience with its Risk Matrix and will fine-tune and revise the criteria and ensure they are objectively and consistently applied to all TB submissions.

4.1.5 Storage of Paper Copies of Submissions

The original copies of the Submissions are filed in the Submission and Cabinet Document Centre (SCDC). During our initial visit to the SCDC, the door was unlocked and once inside, there was no other security in place, although SCDC staff was present and would have noticed if someone entered the filing room. Since our visit, the recently installed card access system has been repaired thus only people with building access can enter the SCDC area.

Treasury Board Submissions are classified as "Confidence of the Queen's Privy Council" and as such require appropriate security.

We recommend that access to SCDC at all times be restricted to people with a Secretariat Building pass. As well, we recommend that access to the paper copies of Submissions be further restricted to authorized personnel only.

Management Response

Agreed. Access to Submission and Cabinet Document Centre is restricted to people with a Secretariat building pass.

Access to the paper copies of Submissions has been further restricted to authorized personnel only.

4.1.6 Submission Time Line

Currently the deadline for final Submissions to reach SCDC from the departments is noon on the Monday eight working days prior to the Treasury Board meeting when it is to be considered. The deadline for the final translated précis to reach SCDC is noon the following Friday, 4 days prior to the Treasury Board meeting. Since the department may make changes to the Submission, the analyst cannot finalize the précis until the final Submission is received. This results in a short time line for the analysts to finalize the précis and have it translated. As a result, the Friday noon deadline is frequently missed for a Submission that is on the Treasury Board agenda. SCDC then has to courier additional information to the Ministers after the binders with the Submissions have been issued but prior to the Treasury Board meeting.

We recommend the time line for receiving final Submissions be revised to allow adequate time to complete the necessary procedures in advance of the Submissions being sent to the Treasury Board Ministers.

Management Response

TBS revised its deadlines for receipt of the final Treasury Board submissions in 2001 to reflect a balance between meeting reasonable scheduling demands of departments and allowing adequate time for analysts within the Secretariat to perform the due diligence necessary to advise TB Ministers.

It is also understood that it is not always feasible for departments to respect existing submission timelines when the submission is driven by unplanned, urgent government or Cabinet priorities and the strict deadlines imposed by the government's supply calendar. TBS philosophy has and will continue to be, to accommodate these urgent situations when it is reasonable to do so.

TBS believes that the current approach is achieving a reasonable balance and that an additional year's experience would be useful before undertaking a further review of its deadlines for final submission in 2004-05.

Observations Rated Low Risk

4.1.7 Support Available to Departments to Assist with Preparation of Submissions

Based on our interviews, it is apparent that some departments are not familiar with the requirements for preparing Submissions and require extensive help from Secretariat analysts before their Submission is ready to be presented to Treasury Board. While there are resources available to assist departments develop expertise in Submission preparation, they do not appear to be extensively used.

The Secretariat has prepared a Guide to Preparing Treasury Board Submissions, which is available to all government departments to assist them with the preparation of Submissions. Our discussions with representatives from a sample of five departments indicated that this guide was not being extensively used.

There is also training available through Training and Development Canada called "Preparing Treasury Board Submission and Cabinet Documents". The departmental analysts we interviewed did not indicate that they were aware of the availability of training. As well, we obtained feedback from TBS analysts that many departmental analysts are not familiar with the requirements for preparing Submissions indicating that they likely have not attended the available training.

We recommend the Secretariat consider conducting a survey of departments to ascertain why the guide is not being used extensively and why departments are not taking advantage of the available training. Consideration should be given to making changes based on the survey to improve the level of expertise within departments for preparing Submissions.

Management Response

Agreed. The Secretary will remind TBS management and staff to actively communicate and promote the "Guide to Preparing Treasury Board Submissions" and the training available through Training and Development Canada as supports available to departments to prepare Treasury Board Submissions.

TBS will also consult with departments to determine the best approach to assist departments in the preparation of Treasury Board Submissions in the context of broader consultation initiatives with departments.

4.1.8 Communication of Analyst Comments

The current process does not have a formal cut-off point for the Secretariat to send comments to departments on their draft Submission. Since departments are not certain when they have received all comments, there is a risk that a department may forward a Submission to its Minister for signature and then have to recall the signed Submission to insert additional information to reflect the most recent comments.

We recommend the Submission process timeline reflect a cut-off date by which time any Secretariat comments on the Submission are to be provided to the department. This will allow a department to know all comments have been received prior to forwarding a Submission to their Minister for signature. To achieve this, the Secretariat should establish a process to ensure all review procedures for the draft Submission have been completed prior to the cut-off date. With a clearly defined timeline for the review process and then communication with the departments, all parties will have a clear expectation of the process thus minimizing the risk of error.

Management Response

Consistent with the TBS management response to recommendation 4.1.6, TBS will include a review of the need for a cut off date by which time any Secretariat comments on the Submission are to be provided to the department in its review of deadlines for final submissions in 2004-05.

4.1.9 Electronic Tracking and Storage of Submission Information

Currently, there are several systems used by the Secretariat to track and record information about Submissions including the Submission Tracking System, Expenditure Status Report, RDIMS and TARS. Information management is inefficient due to the systems not being linked requiring duplication of information in each system and the lack of one system containing all of the information about a Submission. This creates extra work updating each system as well as making it difficult to find out information about a particular submission, which may require access to several different sources. Nor is there one system that contains all the information about a Submission, which would provide a complete record for each Submission. An additional area of concern relates to the adequacy of the security of the systems being used to store Submission information. Only the Submission Tracking System has an appropriate level of security for information, which is "Confidence of the Queen's Privy Council". It is not clear whether Submission information should be entered into the other systems since they may not be properly secured.

We recommend TBS consider developing a system that can be used for all aspects of tracking and recording Submission information. All of the information relating to a Submission would be stored on this system ensuring that there is a complete record for all Submissions. The system would be designed to generate the various Submission documentation including the précis and decision letter reducing the need for duplicate entry. This system would need to have the appropriate level of security for Submission information.

Alternatively, we recommend the Secretariat consider developing a process to ensure that all systems and documents are updated as required. This may involve requiring analysts to make the necessary updates or using administrative support to make the updates. Policies and procedures would need to be developed to ensure that it is clear what information is required to be entered into which system, by whom and when. As well, the issue of the security of the information needs to be addressed.

Management Response

TBS will ensure that over the medium term all aspects of TB submission information will be tracked, recorded and linked to the new Expenditure Management Information System (EMIS) discussed in our response to recommendation 4.1.2.

4.1.10 Archived Submissions

The final Submission file consisting of the Submission, decision letter/turnaround, précis, and sign-offs are archived after three years. As the sign-offs are typically not sent to SCDC, they are not being archived with the final Submission. This results in an incomplete Submission file. As past Submissions are sometimes retrieved for additional review or information, if the file is incomplete, it may be difficult to follow and may not have all the necessary information.

We recommend that the documents noted above be sent to SCDC when the Submissions are finalized so as to ensure that all archived files are complete. A formal policy should be developed which outlines the requirements for archiving and what documents should be included in the archived file.

Management Response

Throughout the fall of 2003, TBS will review its procedures for filing Treasury Board Submissions and revise as necessary to ensure that all archived files are complete

4.1.11 Volume of Submissions

The large volume of Submissions presented to Treasury Board for approval creates an enormous workload on program analysts. There was some speculation amongst analysts that some of this workload was self-inflicted due to new policies that call for Treasury Board approval when a bureaucratic solution may have been appropriate; by attaching conditions, which require subsequent approval by Ministers; and by existing requirements for Treasury Board approval that may be outdated. We recommend that the Secretariat study the reason for the high volume with the aim of reducing the number of Submissions.

Management Response

The volume of TB submissions expected in the future will be significantly affected by changes to the Treasury Board Policies as a result of the review that is currently underway as well as new requirements for Treasury Board approval emerging from its new responsibility for conducting a 5-year cycle of Expenditure and Management Reviews across Government, announced in Budget 2003.

The Stewardship business line of TBS has established a "Workload Reduction Team" mandated to provide advice on immediate, medium and long term and ongoing measures to reduce workload at the Treasury Board Secretariat including identifying opportunities to reduce the number of TB submissions required. Their report will be presented to Stewardship and TBS management for consideration in the fall of 2003.

4.1.12 Availability of Human Resources

One of the objectives of this report was "assessing whether resources appear to be adequate and effective ...". This objective involved assessing many elements of resources including adequacy of staff training, availability of information resources and appropriateness of tools available to support the Submission process. One element of the adequacy and effectiveness of resources is the availability of sufficient human resources to meet the volume of work within the Submission process.

Through the interviews conducted for this audit, it was evident that many people involved with the Treasury Board Submission process are periodically working long hours to ensure that the Submissions are ready for presentation to Treasury Board as scheduled. Many of the recommendations included in this report may improve the efficiency of the Submission process potentially reducing the workload of the analysts.

Once changes have been implemented to improve the efficiency, we recommend that the Secretariat consider undertaking a study of requirements of the Submission process given the volume of Submissions to determine whether the resources available are appropriate given the level of activity.

Management Response

TBS will reconsider the need to review the TB Submission process in 2005-06 to determine whether the resources available are appropriate for the volume and nature of TB Submission activity at the time. That is; once all of the changes discussed above to improve efficiency of the process have been implemented.

 

• Previous Page
• Table of Contents
• Next Page