Report tabled and approved by the Audit Committee and the Evaluation Committee
Results of the evaluation by senior audit and evaluation staff and program managers of the elements in the MAF Universe are shown in Appendix 4 tables 11 and 12. In Table 11 results are ordered in ascending order of the average rating per respondent (i.e., from –2 implying that all raters strongly disagreed that the system was effectively designed, consistently applied, effective and monitored and improved to +2 implying that all raters strongly agreed that these controls were in place, effective and monitored and improved). High, medium and low priority areas in 2007 are color coded as red, yellow and green respectively. Table 12 shows a visual display of the 2008 results in the original MAF ordering of the elements.
In general both the 2007 and 2008 procedures yielded consistent areas of potential audit focus with some exceptions. Areas where the two approaches suggested a need for audit attention included:
A few areas were notably different between 2007 and 2008. Ratings related to Performance Tracking, Reporting, Responding were markedly more positive in 2008 compared to 2007 perhaps because of substantial work done to modify the Agency's PAA structure and associated performance expectations and targets during the 2007-2008 fiscal year. Ratings related to Classification, Recruitment, Hiring and Promotion were more positive in 2008 compared to 2007. Other areas that were more positive in 2008 compared to 2007 included Revenue From Fees and Occupational Health and Safety. The latter area was not rated by the original group of managers in 2007 but was subsequently added to the universe and assigned a rating by the Agency's audit and evaluation committee.
As a result of this and other planning considerations some changes where made to the audit plan for 2008-2009 compared to what was proposed in 2007-2008. Scheduled work related to the MAF element Polices and Programs was dropped and work on values and ethics was added (i.e., largely to support the new audit committee in its responsibilities). Planned work on occupational health and safety was dropped from the year 2009-2010 schedule as a result of the new information gathered this year.
Table 4 shows the modified MAF universe, the OCG draft list of core controls and, the proposed areas for audit assurance work (i.e., highlighted in green) and for descriptions and surveys that serve as background for planning audits (i.e., in yellow). The 2009-2010 fiscal year is not fully committed. Table 5 provides more detail on the 2008-2009 internal audit project schedule.
A key driver of evaluation planning is the requirement in the draft TB Evaluation Policy to provide comprehensive coverage of direct program spending (i.e., the program activities, sub activities and sub sub-activities in the PAA) over a five-year period. For planning purposes therefore the question is not if an element in the universe will be covered but rather when it will be covered and with what type of evaluation activity.
Scheduling considerations for evaluations include the materiality of various programs, past coverage, senior mangers inputs on priorities to support future strategic reviews of the Agency programming and the feedback from participants in the assessment of PAA elements (see Appendix 4, tables 13 and 14).
Participants' input in both the 2007 and 2008 planning processes consistently suggested two sub activities, national parks conservation (PA-2) and through highways management (PA-5) as potential areas for evaluation work. These areas represented approximately 33% of the Agency's spending in 2006-2007. Two sub activities emphasized in 2007, National Historic Sites Conservation (PA-2) and marketing and promotion (PA-4) did not appear in the top four in 2008 (i.e., about 11% of the Agency's spending). Instead, the sub activity National Marine Conservation Areas Sustainability (PA-2) and the sub sub-activity the National Historic Sites Cost-Sharing Program (i.e., PA-2 but not rated in 2007) were identified as areas of potential work (i.e., less than 1% of the spending). Sub-activities that were consistently identified as low priorities for evaluation in both 2007 and 2008 included National Historic Sites Designation (PA 1) And Townsite Management (PA-5) (i.e., about 3% of spending). With the exception of first two areas noted above, most of the identified sub activities do not cover signficant portions of Agency spending. It should be noted that the sub sub-activity National Historic Sites Cost-Sharing Program was the subject of evaluation work in 2007-2008.
Based on this input, commitments in existing RMAFs and consultations with senior management the five-year evaluation plan shown in Table 6 is proposed with evaluation projects highlighted in green and program descriptions and frameworks to support future evaluations highlighted in yellow. Table 7 shows more detail on the projects to be completed in 2008-2009.
Proposed internal audit coverage of the elements of the MAF universe is sufficient to cover several areas of higher risk identified based on managers' input as well as providing some, but not complete coverage, of the draft list of core management controls identified by the Office of the Comptroller General. However, in several cases coverage is limited to completing descriptions and surveys of the elements of the universe rather than assurance audits.
Proposed audit assurance work for 2008-2009 will only cover areas such as core controls related to values and ethics, management of vehicles and vessels, and financial and administrative controls related to revenue from fees and fire management emergency funds. Major areas such as collaborative arrangements and managing with others, sustainable HR policies and planning, information management and information technology and systems will have their governance, controls and risk management processes described and documented to prepare for future audit work. Several of the core management controls related to governance, policies and programs, people, risk management and accountability are not addressed in the plan.
The proposed evaluation coverage the Agency's programs and sub activities is reasonably comprehensive (i.e., about 88% of the Agency's direct program expenditures over five years) although some of the coverage is in the form of descriptions and frameworks designed to set the basis for future evaluations. The focus is on evaluation of major elements of program spending, supporting management information for strategic review and covering off commitments in RMAFs.
It is estimated that an additional four internal auditors would be necessary to be able to conduct sufficient audit work to provide an unqualified opinion on the state of governance, risk management and controls and four evaluators to provide coverage of all program expenditures over five years as envisaged in TBS draft Evaluation Policy.