Parks Canada
Symbol of the Government of Canada

Common menu bar links

Internal Audit and Evaluation Documents

Parks Canada 2009-2010 Evaluation Plan

Parks Canada 2009-2010 Evaluation Plan

Date Approved
April 16, 2009

Office Of Internal Audit And Evaluation

Table of Contents

Executive Summary

1. Introduction

2. Parks Canada Agency

3. The Evaluation Function

Applicable Policies and Professional Standards
Mission and Services Offered
Governance
Office of Internal Audit and Evaluation (OIAE)

4. Evaluation Universe and Planning Considerations

a) Status of Projects in 2008-2009 Plan
b) Agency's Corporate Risk Profile
c) Agency Renewal
d) Prior Risk Assessments and Current Management Priorities
e) Considerations Arising from TB Submissions, RMAFs and Budget 2009
f) TBS Directed Work
g) Work of External Assurance Providers

5. Planned Projects

Extent and Limitations of Coverage

6. Appendices

1. Assumptions in the Calculation Of Evaluation Capacity
2. Corporate Risk Profile
3. Agency RMAF Evaluation Commitments
4. Results of Risk Ratings of Elements of the PAA Universe (2007 and 2008)

List of Tables

Table 1: Program Activities
Table 2: Planned Evaluation Spending for 2009-2010
Table 3: Past Coverage of Program Activity Elements
Table 4: Five-Year Evaluation Plan
Table 5: Evaluation Project Scheduling for 2009-2010

Executive Summary

The 2008-2009 Parks Canada Evaluation Plan outlines the mandate, organizational structure and resources for Evaluation at Parks Canada, the considerations in developing the Plan and details of individual evaluation projects for the FY 2009-2010, together with the associated resource allocation. The Evaluation Committees of the Agency, chaired by the Chief Executive Officer (CEO) discussed and recommended approval of the plan. The CEO approved the plan on April 16, 2009.

The Office of Internal Audit and Evaluation (OIAE) at Parks Canada adheres to the government's policies and standards for evaluation. Currently, the evaluation function in the Office consists of an executive level director and two staffed evaluator positions (i.e., three positions are currently being staffed). Given a fully staffed function, the Office will have the capacity to conduct approximately two risk-based evaluations (i.e., an average or typical project is 2,200 hours), as well as addressing other non-risk based work (i.e., provision of advice, coordination, follow-up and participation in TBS directed evaluation work).

Planning is based on the Agency's Program Activity Architecture (PAA). Work for the 2009-2010 fiscal year will include completing an evaluation of the through highways management component of the Townsite and Throughway Infrastructure Program and beginning an evaluation of the visitor services component of the Visitor Experience Program. The latter sub-activity is the single largest expenditure in the Agency's PAA, accounting for about one-third of all expenditures in 2007-2008. Additional work in 2009-2010 will focus on developing evaluation frameworks for the establishment of the national parks sub-activity of the Heritage Places Establishment Program and the national parks conservation sub-activity of the Heritage Resources Program. The latter sub-activity is the second largest expenditure in the Agency's PAA accounting for one-fifth of all expenditures in 2007-2008.

The five year evaluation coverage proposed in the plan will cover about 87% of the Agency's direct program expenditures based on the 2007-2008 expenditure data, with over 63% of the expenditures covered prior to the next strategic review anticipated in spring and summer 2011.

Several components of the PAA with lower expenditures are not covered at all during the five-year planning period or are only covered by the development of evaluation frameworks. As noted in previous evaluation plans, it is estimated that an additional four evaluation positions would be required to adequately address the evaluation requirements in the Agency.

1. Introduction

The 2009-2010 Parks Canada Evaluation Plan outlines the mandate, organizational structure and resources for evaluation at Parks Canada, the strategy and process employed in developing the Plan, a project schedule for the five-year period from April 2009 to March 2014, and details of individual evaluation projects for the FY 2009-2010, together with the associated resource allocation.

The plan was designed in conformity with requirements of the recently approved TB Evaluation Policy, and the implications of the Government's Federal Accountability Act, as well as Parks Canada's Evaluation Charter (http://www.pc.gc.ca/docs/pc/rpts/rve-par/47/index_e.asp).

The ultimate goal of evaluation planning is to provide comprehensive evaluation coverage of material program spending over a five-year period in support of the Strategic Review process of the TB Expenditure Management System, consistent with TB Evaluation Policy.

2. Parks Canada Agency

Parks Canada was established as a separate departmental corporation in 1998. The Agency's mandate is to:

“Protect and present nationally significant examples of Canada's natural and cultural heritage, and foster public understanding, appreciation and enjoyment in ways that ensure the ecological and commemorative integrity of these places for present and future generations.”

Parks Canada delivers on its mandate through five major program activities shown in Table 1.

Table 1: Program Activities

Heritage Places Establishment Including the establishment to date of a system of 42 national parks, 3 national marine conservation areas, and the designation of 925 persons, places and events of national historic significance to Canada (e.g., of which 157 national historic sites are administered by the Agency) as well as supporting designation of other heritage places or structures (e.g., federal heritage buildings, heritage rivers, heritage railway, gravesites of prime ministers).
Heritage Resources Conservation Including the maintenance and restoration of ecological integrity in national parks, ensuring the commemorative integrity of national historic sites managed or influenced by Parks Canada; the protection and management of cultural resources under the administration of Parks Canada; and, the ecologically sustainable use of national marine conservation areas. Conservation also includes fulfilling legal responsibilities assigned to Parks Canada by the Species at Risk Act and the Canadian Environmental Assessment Act.
Public Appreciation and Understanding Activities aimed at reaching Canadians at home, at leisure, at school and in their communities through relevant and effective communication and public outreach education initiatives as well as by engaging many stakeholders and partners in the development and implementation of the Agency's future direction to increase Canadians' understanding, appreciation, support and engagement with respect to the natural and historical heritage of Parks Canada administered places.
Visitor Experience Activities aimed at increasing awareness of Parks Canada, supporting visit planning and traveling to and welcoming and orientation upon arrival, provision of recreational and interpretive infrastructure and activities (i.e., campgrounds, infrastructure, hiking trails) activities to ensure visitor safety and the ongoing post-visit relationship.
Town-Site and Throughway Infrastructure Managing, operating and providing municipal services to five townsite communities within Canada's national parks (i.e., provision of safe drinking water, snow removal, garbage pick-up and disposal, sewage treatment, road and street maintenance, and fire services to support residents and visitors). It also involves the operation of provincial and inter-provincial highways and waterways that connect communities and pass through national parks and national historic sites.

Responsibility for the Parks Canada Agency rests with the Minister of the Environment. The Parks Canada Chief Executive Officer (CEO) reports directly to the Minister.

National parks and national historic sites are organized into thirty-two geographically based field-units. About 80% of Parks Canada's work force is based in the field where most of its program expenditures take place. The work of field units is supported by Service Centres located in Halifax, Quebec City, Cornwall/Ottawa and Winnipeg (with small branch offices in Calgary and Vancouver). The Service Centres comprise about 10% of the work force and provide technical and professional services to field units (e.g., science, research, design services). National Office, with less than 10% of the employee base, consists of five directorates (National Parks, National Historic Sites, Strategy and Plans, Human Resources and External Relations and Visitor Experience) who provide legislative, operational policy, planning, program direction, financial management, and human resources functions and services.

3. The Evaluation Function

Applicable Policies and Professional Standards

The evaluation function at Parks Canada adheres to the Evaluation Policies, directives, standards and guidelines of the Government of Canada. In 2007-2008 a new charter for the evaluation function was approved consistent with the then draft Evaluation Policy, directives and standards.

Mission and Services Offered

For evaluation the mission is:

To contribute to the achievement of Parks Canada's mandate by providing the Chief Executive Officer with evidence-based, credible, neutral and timely information on the ongoing relevance, results, and value of policies and programs, alternative ways of achieving expected results, and program design improvements.

Services include:

  • Evaluation and performance frameworks related to programs, activities or initiatives;
  • Formative and summative evaluations of programs, policies and functions;
  • Consulting on RMAFs, and on the development of performance measures, targets and information systems.

Formative or summative evaluations are preceded by the development of evaluation frameworks that describe the elements of a program or sub-activity, its logic and systems for monitoring relevance and performance. These are listed as separate projects in the Agency's evaluation plan (see below).

Follow-up on Management Responses

The evaluation cycle includes a systematic follow-up on the management responses six months after the final approval of the reports by the Evaluation Committee. Managers are requested, by e-mail from the CEO, to complete a template that provides a status report on outstanding recommendations in their area of responsibility. The template is returned directly to the Chief Audit and Evaluation Executive, who reports regularly to the Committee.

Governance

In 2008-2009, Parks Canada implemented a new evaluation committee separate from the previous combined audit and evaluation committee. The CEO chairs the committee composed of seven senior managers in the Agency (i.e., the CAO, and Directors General National Parks, National Historic Sites, External Relations and Visitor Experience, Eastern Canada and Western/Northern Canada and an additional member of Executive Board identified by the CEO). The Evaluation Committee is responsible for reviewing and providing advice or recommendations to the CEO on:

  • Evaluation Function and Products: the Agency's Evaluation Charter, the rolling 5-year Evaluation Plan, the adequacy and neutrality of resources allocated to the evaluation function, the performance of the evaluation function, key elements of an evaluation product lifecycle such as terms of reference, scoping documents, evaluation reports, management responses and action plans including following-up to ensure action plans are implemented
  • Performance Management Framework: and the adequacy of resources allocated to performance measurement activities, and recommend to the CEO changes or improvements to the framework and an adequate level of resources for these activities.

Office of Internal Audit and Evaluation (OIAE)

The organizational chart for the evaluation function is shown below along with planned spending for 2009-2010.

Organizational chart for the evaluation function

Table 2: Planned Evaluation Spending for 2009-2010

  CAEE Office Evaluation Total
FTE 1 5 6
Salary 20,500 380,000 400,500
Benefits     80,100
Accommodations     52,065
Sub Total     532,665
O&M For Staff Support 10,000 50,000 60,000
O&M for Professional Services   50,000 50,000
Sub Total O&M 10,000 100,000 110,000
Grand Total 30,500 480,000 642,665
CAEE Office FTEs includes .5 of the FTE for the Chief Audit and Evaluation Executive and for the Executive Assistant. It only includes the salary, and associated benefit/accommodations for the Executive Assistant position, as the salary for the CAEE is administered centrally for all executive level employees in the Agency. The table does not include carry forward amounts from 2008-2009. Some carry forward is expected but the amount is not known at the time of the plan.

The budget includes $117.7K provided for evaluation (i.e., salary, EBP, accommodations, and O&M) to support the requirement in the Federal Accountability Act to evaluate transfer payment programs on a five-year cycle. Only two of the positions in the evaluation function are currently staffed.

Given a completely staffed function, it is estimated that the OIAE would have the capacity to conduct approximately two to three risk-based evaluations (i.e., an average or typical project is 2,200 hours). The actual number of projects undertaken in any one year may differ from this estimate to the extent that actual project hours differ from average project hours. Details of the calculations involved in the estimate are shown in Appendix 1.

4. Evaluation Universe and Planning Considerations

The key driver of evaluation planning is the requirement in the TB Evaluation Policy to provide comprehensive coverage of direct program spending (i.e., the program activities, sub activities and sub sub-activities in the PAA) over a five-year period in support of the Strategic Review process. For planning purposes therefore, the question is not if an element in the universe will be covered but rather when it will be covered and with what type of evaluation activity. Table 3 shows the evaluation universe (i.e., PAA structure) with associated spending by PA element for 2007-2008 and links to recent evaluations and relevant work by external assurance providers. Past evaluation coverage has largely focused on smaller elements of the universe (i.e., in terms of materiality) and on formative rather than summative evaluations.

Starting in the 2007-2008 plan an evaluation schedule based on summative evaluations of larger elements of the universe was developed. The current plan updates this schedule based on considerations outlined below.

a) Status of Projects in 2008-2009 Plan[1],[2]

  • Evaluation of Issues Related to the National Historic Sites of Canada Cost-Sharing Program: The project was included in the 2007-2008 Evaluation Plan but not shown in 2008-2009 Plan as the report was completed by March 2008. However, the report was not tabled at Evaluation Committee until May 2008 and only approved with management responses in August 2008. It can be found on the Parks Canada web site (http://www.pc.gc.ca/docs/pc/rpts/rve-par/51/index_e.asp).
  • Evaluation of Parks Canada's Asset Management Program: This project, examining the relevance and performance of the Agency's Asset Management Program, was launched in May 2006 and included in the last three evaluation plans. The final report and management responses were recommended for approval but not formally approved when the 2009-2010 Plan was approved.

Table 3: Past Coverage of Program Activity Elements

Program Activities and Sub-Activities 2007-2008 Expenditures Parks Canada Evaluations January 2005 to March 2008 Work of External Assurance Providers
  Heritage Places Establishment 19,807.5    
1 National Park Establishment and Expansion 9,870.9    
2 National Historic Sites Designation (Persons, Places, Events) 5,558.6    
3 National Marine Conservation Area Establishment 2,986.1    
4 Other Heritage Places Designations 1,391.8    
  Heritage Resources Conservation 201,388.9    
5 National Parks Conservation 129,311.8   CESD Chapter 2—Ecological Integrity in Canada's National Parks - September 2005
6 Species at Risk   Formative Evaluation of Federal Species at Risk Programs-July 2006
Contributing to a Environment Canada lead Evaluation of the Habitat Stewardship Component of the Species at Risk Program (2008-2009)
CESD Chapter 5—Ecosystems—Protection of Species at Risk – March 2008
7 National Historic Sites Conservation 54,452.4   AG Chapter 2—The Conservation of Federal Built Heritage Feburary 2007
8 National Marine Conservation Areas Sustainability 2,134.8 Evaluation of Parks Canada's Phase One of Oceans Action Plan -June 2007  
9 Other Heritage Places Conservation 15,490.0    
10 National Historic Sites Cost-Sharing 900.0 Evaluation of Issues Related to the National Historic Sites of Canada Cost-Sharing Program -August 2008  
11 Commercial Heritage Properties Incentive Fund 1,982.1 Formative Evaluation of the Commercial Heritage Properties Incentive Fund (CHPIF) -January 2007  
12 Historic Places Initiative 5,692.6 Formative Evaluation of the Historic Places Initiative -March 2005  
  Public Appreciation and Understanding 33,289.7    
13 Outreach Education and Agency Communications   National Performance and Evaluation Framework for Engaging Canadians: External Communications at Parks Canada -February 2005
Formative Evaluation of Engaging Canadians External Communications Strategy -September 2006
CESD Chapter 2—Ecological Integrity in Canada's National Parks - September 2005
14 Engagement      
  Visitor Experience 286,614.8    
15 Marketing and Promotion 15,444.70    
16 National Parks Interpretation 19,633.50    
17 National Parks Visitor Activities and Services 36,682.70    
18 Public Safety 764.5 Evaluation of Parks Canada's Public Safety Program-February 2005  
19 National Historic Sites Interpretation 130,202.00    
20 National Historic Sites Visitor Activities and Services 9,781.80    
21 National Marine Conservation Areas Interpretation 81,950.60    
22 National Marine Conservation Areas Visitor Activities and Services 1,936.90    
  Town-Site and Throughway Infrastructure 92,939.1    
23 Townsite Management 16,363.4    
24 Through Highways Management 68,870.1    
25 Through Waterways Management 7,705.6    
    634,040.0    
In 2008-2009 the Evaluation of the Parks Canada Asset Management Program was also completed. Assets are a key element in delivery of the Heritage Resources Conservation, Visitor Experience and Town-site and Through Infrastructure Programs and as such this evaluation contributes to coverage of these program activities.
  • Evaluation Framework for the Highways Management Sub-Activity: Consistent with the 2008-2009 Evaluation Plan, a description of the goals, objectives, resources, activities and outputs of the highways program was developed, along with evaluations issues, and methods to prepare for an evaluation in 2009-2010.
  • Evaluation Framework for Visitor Experience Program: Consistent with the 2008-2009 Evaluation Plan, a description of the goals, objectives, resources, activities and outputs of the Visitor Experience Program activity was developed, along with evaluations issues, and methods to prepare for an evaluation in 2009-2010.

b) Agency's Corporate Risk Profile

In 2008-2009 the Agency developed its first corporate risk profile (see Appendix 2 for details). The five major risk areas were:

  • Public and/or Aboriginal support may be insufficient to advance the Agency's mandate and program priorities.
  • The Agency's service offer (i.e., competitive position) may not be attractive or of interest to Canadians.
  • Job market pressures may make it difficult to recruit and retain qualified employees and impair ability to deliver on program.
  • Failure to capture and manage information may hinder the ability to effectively manage all program areas and meet legal requirements.
  • Governance operational imperatives may overtake the corporate agenda, hindering the ability to provide strategic direction and/or make strategic decisions.

Some of these risk areas cut across many of the program activities (i.e., recruiting and retaining qualified staff, information management, keeping a focus on the Agency's strategic directions). Others relate to specific programs particularly the Public Appreciation and Understanding, and Visitor Experience programs.

c) Agency Renewal

In December 2007, the Chief Administrative Officer of the Agency began, with the support of a seconded team, a full time project to focus and coordinate the Agency's efforts directed toward change management and renewal of Agency programs. This renewal responds to a number of external drivers for change as organizational imperatives such as the need to implement a new way of providing law enforcement services. The external drivers include: changing demographics (e.g., an aging more multicultural Canadian population), changing technology, continuing loss of biodiversity and built heritage, changes in patterns of how people spend their leisure time, and increased national and international competition for tourist visits. Visitation at many national parks (i.e., outside of the Mountain Parks and NMCAs) and historic sites has decreased by close to 20% over the past five years. Agency renewal is essential to ensure that Parks Canada continues to be relevant to Canadians and can effectively deliver its mandate into its second century.

The foundation for the Agency Renewal is the development of a case for change and a new Vision Statement. Phase 1 of the Renewal communications was launched in January 2009 and marks the formal start of engaging employees in the renewal process. Phase 2 will include launch of a Renewed Parks Canada brand and associated activities designed to increase Canadians awareness, understanding, product differentiation, and positive images of the products and services offered. Branding is related to the Public Understanding and Appreciation and the Visitor Experience program activities but has wider implications for all program activity areas. These two program activities are also under going a major realignment of the functional structures (i.e., positions) with completion targeted for April 2010. To support this realignment affecting 3000 employees – The Agency is concurrently developing new roles and responsibility statements, accountabilities management tools and training to strengthen capacity in these two functional areas. In addition, the Agency is in the initial stages of renewing the national historic sites program to make the program more responsive to the needs of Canadians.

The Resource Conservation Program is affected by the introduction of a new mechanism for law enforcement in response to Canada Appeals Office (now known as the Occupational Health and Safety Tribunal Canada) Direction on Law Enforcement in Canada's National Parks (May 2007) and the Government of Canada announcement to arm up to 100 law enforcement personnel in Parks Canada (May 2008). The new program will be staffed and operating by April 2009 and already has an approved RMAF with evaluation commitments (see below). At the same time the Agency is undertaking renewal of the resource conservation function (i.e., roles, responsibilities, accountabilities and investments related to all aspect of resource conservation at the national office, service centre and field level.).

A complimentary initiative is the development of a more coherent approach to prevention activities (i.e., preventing incidents before they occur and resolving incidents safely and effectively when they do occur). The approach is now embedded in the new Parks Canada Service Program (service standards and prevention guidelines). Prevention activities cut across the PAA structure of the Agency. This program is being developed concurrently and in coordination with the new law program. Training programs for employees are being finalized for implementation in the Spring 2009 with the target of training a possible 3,500 employees. The evaluation function has had some preliminary discussion with the program lead about the development of a separate RMAF.

In some cases, these change priorities are well advanced (e.g., law enforcement) while in others the change effort is only beginning (e.g., renewal of the NHS program). The OIAE has taken these initiatives into account in planning its priorities.

d) Prior Risk Assessments and Current Management Priorities

The 2007-2008 and 2008-2009 evaluation plans were based in part on having managers rate risks associated with program activities and sub-activities. The methodology used differed across the two planning cycles but led to some broadly consistent results (i.e., two sub-activities, national parks conservation (PA-2) and through highways management (PA-5) were rated as high priorities for evaluation work while two sub-activities, Historic Sites Designation (PA 1) And Townsite Management (PA-5), were consistently rated as low priorities). Appendix 2 reproduces the complete set of ratings from the 2008-2009 Plan.

For 2009-2010 we did not repeat the risk rating process but rather conducted individual interviews with senior management to confirm that the five-year project schedule developed in 2008-2009 remained relevant and to seek input into project planning for the 2013-2014 fiscal year. Interviews were conducted in February and March 2009.

e) Considerations Arising from TB Submissions, RMAFs and Budget 2009

Currently Parks Canada has evaluation commitments resulting from TB submissions and/or RMAFs related to several multi-department programs or initiatives and three of its own programs. The various commitments are listed in Appendix 3 and reflected in the five-year evaluation plan shown below.

Individually the initiatives and funds covered by these evaluation activities represent from less than 1% to about 5% of the Agency's spending in any one year. Therefore, while important to meet commitments to evaluate transfer payment programs as specified in the Federal Accountability Act, or to demonstrate results related to particular initiatives, they do not provide coverage of most of the Agency's core programming activities.

Budget 2009 provided the Agency with $75M for infrastructure projects related to the Visitor Experience Program and an additional $75M for addressing concerns related to built cultural heritage (NHS Conservation) including $8M to new funding for the NHS Cost-Share Program. Both of these amounts are provided over two years. The impacts of these new resources will be considered in the planned evaluations of these programs. Budget 2009 also provided additional funds for completing the twinning of the TransCanada Highway in Banff National Park (i.e., $130M). Previous funding for twinning the highway was obtained through multi-department funding initiatives. The most recent funding provided for a final summative evaluation of the entire twinning project rather than conducting separate evaluations on each portion of the funding (see Appendix 3).

f) TBS Directed Work

The TB Evaluation Policy calls for the creation of a Government of Canada Evaluation Plan to be supported by evaluation activities within departments. The first plan is not yet complete and therefore has not been taken into account for the 2009-2010 fiscal year.

g) Work of External Assurance Providers

Recent relevant studies by the Offices of the Auditor General and the Commissioner of the Environment and Sustainable Development for evaluation planning were noted in Table 3. The Office of the Auditor General also conducts review level assurance work each year on the fairness and reliability of the performance information in the Agency's Annual Performance Report. The OIAE has taken this assurance work into account in planning its priorities.

5. Planned Projects

Based on the factors reviewed, the five-year evaluation plan shown in Table 6 is proposed with development of evaluation frameworks for program activities/sub-activities shown in yellow, internal mandated evaluations shown in green and evaluation commitments arising from RMAFs shown in blue. Table 7 shows more detail on the projects to be completed in 2008-2009.

Extent and Limitations of Coverage

The proposed evaluation coverage the Agency's programs and sub activities represent about 83% of the Agency's direct program expenditures over five years based on the 2007-2008 expenditure data.

The next strategic review of the Agency's program is anticipated in the spring and summer of 2011. Between April 2008 and summer of 2011 evaluations will be completed for the national historic sites cost-sharing program (already done), the visitor services activity of the Visitor Experience Program (i.e., the largest single program in the Agency), national parks conservation and the species at risk sub-components of the Heritage Resources Conservation Program (i.e., the largest single element of the conservation program), and the through highways component of the Townsite and Throughways Infrastructure Program. These programs accounted for about 63% of the Agency's spending in 2007-2008.

By the end of the planning period (i.e., March 31, 2014), evaluations will cover an additional 25% of the 2007-2008 spending. These evaluations include the establishment of national parks and national marine conservation areas in the Heritage Places Establishment Program, national historic sites conservation, the establishment and conservation aspects of the NMCA program, and the national historic sites cost-sharing program, as well as the law enforcement program which relates to both Heritage Resources Conservation and Visitor Experience, the interpretation component of the Visitor Experience Program and some additional work related to twinning projects on the TCH, as well as an evaluation of the general class contribution program.

Some areas of the PAA are not covered or are only covered by the development of evaluation frameworks. These include elements of Heritage Places Establishment Program related to NHS and other heritage place designations, the components of the Heritage Resources Conservation Program related to other heritage places, and some aspects of the marketing and promotion component of the Visitor Experience Program. Finally, there is no evaluation coverage of the townsite and through waterways management aspects of the Townsite and Throughway Management Program. As noted in previous evaluation plans, it is estimated that an additional four evaluation positions would be required to address all the evaluation demands in the Agency.

Table 4: Five-Year Evaluation Plan

Activities (Bold), Sub Activities and
Sub Sub-Activities (indented)
% Of 2007-08
Expend-
itures
2009-2010 2010-2011 2011-2012
(Next Strategic Review)
2012-2013 2013-2014
Heritage Places Establishment 3.1          
National Park Establishment and Expansion 1.6 Framework   Evaluation*    
National Historic Sites Designation (Persons, Places, Events) 0.9          
National Marine Conservation Area Establishment 0.5       Framework   Evaluation
Other Heritage Places Designations 0.2          
Heritage Resources Conservation 31.8          
National Parks Conservation 20.4 Framework   Evaluation   Evaluation of
Law Enfor-
cement
Species at Risk NA Evaluation of the SAR Program    
National Historic Sites Conservation 8.6   Framework   Evaluation    
National Marine Conservations Areas Sustainability 0.3       Framework   Evaluation
Other Heritage Places Conservation 2.4          
National Historic Sites Cost-Sharing 0.1     Evaluation of NHS Cost Sharing Program    
Commercial Heritage Properties Incentive Fund 0.3 Sun Setting Programs        
Historic Places Initiative 0.9
Public Appreciation and Understanding 5.3          
Outreach Education and Agency Communications NA   Framework   Evaluation  
Engagement NA      
Visitor Experience 45.2          
Marketing and Promotion 2.4 Frame-
work
         
National Parks Interpretation 3.1 9       Evaluation  
National Historic Sites Interpretation 5.8        
National Marine Conservation Areas Interpretation 0.1        
National Parks Visitor Activities and Services 20.5 33

Evalu-
ation

       
National Historic Sites Visitor Activities and Services 12.9       Evaluation of
Law Enfor-
cement
National Marine Conservation Areas Visitor Activities and Services 0.3        
Public Safety 1.5          
Town-Site and Throughway Infrastructure 14.7          
Townsite Management 2.6       Framework    
Through Highways Management 10.9 Evaluation       Evaluation of TCH Twinning Project
Through Waterways Management 1.2     Framework    
Other            
General Class Contributions Program     Evaluation      
*Will address the cases relevant to the interdepartmental Evaluation of the Establishing Federal Protected Areas in the NWT
** Will address cases relevant to the Health of the Oceans Initiatives

Table 5: Evaluation Project Scheduling for 2009-2010

Topic Type Description Planned or Actual Dates Resources Required*
2008-2009
Planned
2007-08
Start
Date
Comple-
tion of the fieldwork
Comple-
tion of

Report
Date of Approval by Committee Appro-
ximate hours
O&M
Through Highways Management Evaluation A summative evaluation of the relevance and performance of the sub-activity---through highway Y May 2009 Sep-
tember
Nov-
ember
January 1200 $30K
Visitor Services Evaluation A summative evaluation of relevance and performance of the visitor services component of the Visitor Experience Program Activity. Y Aug
2009
Jan 2010 March May 2010 2000 $120K
National Parks Establishment and Expansion Evaluation Framework A description of the program sub-activity and objectives, evaluation issues and methods for a subsequent evaluation of relevance and performance. Y May
2009
Aug Nov January 1000 $25K
National Parks Conservation Evaluation Framework A description of the program sub-activity and objectives, evaluation issues and methods for a subsequent evaluation of relevance and performance. Y May
2009
Aug Nov January 1000 $25K
* Approximate hours reflect required effort. Projects that extend beyond a fiscal year will require more effort than shown in the table. Staff or consultants may perform the work. O&M required includes costs of professional services and/or costs of staff travel.

6. Appendices

1. Assumptions in the Calculation Of Evaluation Capacity
2. Corporate Risk Profile
3. Agency RMAF Evaluation Commitments
4. Results of Risk Ratings of Elements of the PAA Universe (2007 and 2008)

1. Assumptions in the Calculation Of Evaluation Capacity


  1. Hours Available for Work Per FTE Hours Available
    52 week/year * 5day week *7.5 hour/days 1950.0
    Average days holiday (20 days) -150.0
    Average days sick (5 days) - 37.5
    Average days training (5 days) - 37.5
    Total Work Time Available 1725.0*
    * Equivalent to 230 working days per evaluator

  2. Hours Available for Evaluation Work: On average it is assumed that one hour per working day is allocated to breaks (i.e., 230 hours per evaluator per year), leaving 1,495 hours of actual working time. It is further assumed that approximately 20% of the actual working time per evaluator (i.e., 299 hours per year) is allocated to unit administration, including planning, meetings, supporting internal systems, support to the evaluation committee, human resources issues, follow-up on the status of previous evaluation recommendations, etc. The remaining time is allocated to evaluation work in point C, and to risk based evaluation work.

  3. Types of Evaluation Work: Available work hours are allocated to the following tasks:

    TBS Directed Evaluation Requirements
    • TB Evaluation Policy envisions a Government-Wide Evaluation Plan that could dictate the scope and timing of Agency evaluation work.
    • This category does not include evaluation work required as part of TB submissions or RMAF/RBAF commitments.

    Special Requests
    • Periodic special requests by management outside the approved evaluation plan are typical. Previous examples include the extensive work carried out on quantitative risk assessment associated with warden law enforcement duties.

    Consulting/Advice/Coordination
    • Consulting and providing advice related to evaluation or performance measurement is a normal part of the activities of the OIAE.

  4. O&M Dollars Available for Contracting: Additional capacity can be purchased from contracted professionals based on available O&M budgets. An average per diem of $1,200 per day or $160 per hour is used for this calculation. For 2009-2010, the O&M budget for professional services is $50K.

  5. The resource consumption of the average evaluation project: The average number of hours consumed by an evaluation project is affected by many factors, including its scope and complexity as well as the skills of the personnel performing the work. Based on a review of prior experience with contracted projects and estimates of internal staff time devoted to projects as well as the expectation that future evaluation projects will need to be more complex given the TB policy directions, typical values of 2,200 hours per evaluation project were identified.

2009-2010 Capacity For Evaluation

A) Resources CAEE Office Evaluation Total
FTEs 1 5 6
Hours Available for Work (Less vacation, sick and training days, and 1 hour per working day for breaks leaving 1,495 hours per FTE) 1,495 7,475 8,970
Professional Service Hours Purchased (O&M for professional Services/$160 per hour)   313 313
Total Hours Available 1,495 7,788 9,283
B) Demands      
Administration i.e., (80% of CAEE, 100% of Admin. Assistant and 20% of other staff hours are allocated for activities such as planning, meetings, work on internal systems, support to the evaluation committee, human resources, quality control, follow-up on previous recommendations, etc) -1,345 -1,495 -2,840
TBS Directed Evaluations (none in 2009-2010)     0
Special Requests (.25% of one FTE) -10 -374 -384
Consulting/Advice/Coordination (40% of one FTE) -140 -598 -738
Residual Hours for Risk Based Projects 0 5,321 5,321
C) Project Capacity      
Number of Typical Projects (2,200 hours per project)     2

2. Corporate Risk Profile

Risk Category and
Label
Description Relevant PAs Mitigation OIAE Work
Public Public support Support from local communities, stakeholders, NGOs, and the Canadian public may not exist or be insufficient to advance Parks Canada's programs. 1-5 Risk being mitigated in current business plans. Evaluation Framework For Public Appreciation and Understanding
Inter-governmental collaboration Need for collaboration with other federal departments, provinces, territories, and municipalities may lead to delays and ultimately impede implementation of Parks Canada's programs. 1-5   Audit Framework Collaborative Arrangements and Managing with Others
Aboriginal support Support from Aboriginal peoples may diminish and become insufficient to advance Parks Canada's programs. 1-5 Guide to formalized Aboriginal advisory relationships; Improved linkages with other federal departments for policy.
Socio-
economic
Exploration and development pressures Exploration in oil, natural gas and mining, as well as development pressures, may limit opportunities for NP, NMCA establishment, NHS commemoration, and conservation and presentation of heritage places. 1, 2, 4   Evaluation Framework National Parks Conservation
Economic conditions Economic conditions across the country and internationally may hinder the ability to sustain Parks Canada's operational environment. 4 and 5    
Competitive position Parks Canada's service offer may not be attractive or of interest to Canadians in comparison to other parks and cultural attractions and/or leisure activities. 4 National Marketing and Communication Plan; Branding Identity; National Awareness Campaign; National Historic Sites Renewal.
Visitor experience product development
Evaluation Frameworks for the Public Appreciation and Understanding Program and the Marketing and Promotions and the Visitor satisfaction component
Component of the Visitor Experience Program
Job market pressures Failure to recruit and retain qualified employees may lead to challenges in delivery of all programs and support functions. 1-5 and ISD Agency diversity program; Agency succession approach Audit Framework HR Planning and Policies
Techno-
logical
Adoption of technology Use of information technology in Parks Canada's service offer may not meet employee, visitor and outreach expectations. 2-4    
Environ-
mental
Climate change Climate change may threaten or destroy cultural resources, ecosystem elements and/or assets and affect Parks Canada's service offer. 1, 2 and 4   Evaluation Framework National Parks Conservation
Biodiversity loss Biodiversity loss may threaten or destroy ecosystem elements and affect Parks Canada's service offer. 1,2 and 4  
Disasters Disasters may impair or destroy critical infrastructure and/or assets of national historic significance. 1, 2, 4 and 5   Audit Business Continuity Planning
Parks Canada's Business Operations Information management Failure to capture and manage pertinent data and information may hinder the ability to effectively manage all program areas and meet legal requirements. 1-5 and ISD Records Disposition Authority; Enterprise Information Committee; Information Management and ATIP and Privacy Awareness training, Phase 1. Audit of Information Management
Governance Operational imperatives may overtake the corporate agenda, hindering the ability to provide strategic direction and/or make strategic decisions. 1-5 and ISD Accountability for results; Alignment of planning and reporting Audit Framework Planning and Budgeting Framework
Labour relations Failure to negotiate a collective agreement may hinder the delivery of Parks Canada's programs 1-5 and ISD    
Emergency responsiveness Parks Canada may be perceived as responding inadequately to emergencies. 4-5   Audit Business Continuity Planning
PA-1: Heritage Places Establishment, PA-2: Heritage Resources Conservation, PA-3: Public Appreciation and Understanding, PA-4: Visitor Experience, PA-5: Town-Site and Throughway Infrastructure, ISD: Internal Service Delivery

3. Agency RMAF Evaluation Commitments

  2010-2011 2011-2012 2012-2013 2013-2014
Horizontal Evaluations[3] Evaluation of the Species at Risk Program: Involving Environment Canada, Fisheries and Oceans and the Agency as the lead federal organizations for the Species at Risk Program. Parks Canada received a total of $10.6M between 2007/08 and 2011/12 from this initiative and expects to receive $6.8M per year thereafter, representing about 10% of the overall funding. The RMAF for this initiative commits to a summative evaluation in 2010-2011. Evaluation of the Establishing Federal Protected Areas in the NWT: Involving Indian and Northern Affairs Canada, Environment Canada and Parks Canada as the partners working together to advance the Northwest Territories Protected Areas Strategy. Parks Canada will receive $8.056M between April 2008 and March 2013 (i.e., roughly a third of investment) to assist in establishing a national park in the East Arm of Great Slave Lake, and to help develop and make operational the Sahoyue and Ehdacho National Historic Site of Canada. A summative evaluation of the initiative will be led by DIAND in early 2012. Evaluation of the Health of the Oceans Initiatives: The Agency has or will receive $6.25M between April 2007 and March 2012 to help develop and implement a federal marine areas strategy in concert with Fisheries and Oceans and Environment Canada and to conduct a feasibility study for establishing a national marine conservation area in Lancaster Sound. Both areas relate to the Agency's Heritage Place Establishment Program and would be addressed in an evaluation of the Establishment of Marine Conservation Areas sub-activity proposed for 2012-2013.  
Parks Canada Evaluation of the Parks Canada Agency's General Class Contributions Program: The GCCP is not a program within itself but rather a funding authority that is available to Agency managers to support activities or projects related to National Parks, Marine Conservation Areas, Historic Sites and Canals. Approximately $2.2M in funding is allocated per year. The program was renewed in April 2006 and continues until March 2011. A proposed Results Reporting Capacity Check for 2007-2008 was not completed. A summative evaluation is scheduled for 2010-2011 prior to the renewal of the terms and conditions of the program.   Evaluation of the National Historic Sites of Canada Cost-Sharing Program: The program provides matching contribution funds to eligible national historic sites undertaking conservation and presentation work. It was renewed in 2008-2009 with an annual budget of between $2.3 and 3.3M per year over the next five years. A summative evaluation is scheduled for 2012-2013. Evaluation of the Law Enforcement Program: The program, involving up to 100-armed law enforcement officers responsible for enforcement of national parks, national marine conservation area and national historic site laws and regulations, but not for criminal code enforcement, was funded and developed in 2008-2009 with on the ground activities commencing in 2009-2010. The program will have start-up costs of $8.5M in 2008-2009 and ongoing costs of $2.3M per year thereafter (i.e., less than one percent of the Agency's annual spending). A summative evaluation is planned for in 2013-2014.
Evaluation of the Twinning of the Trans-Canada Highway in Banff National Parks: The recent initiative (i.e., 2004-05 on) involves twining 32 kilometres of the TCH at a total cost of $317M over 10 years. Funding was received through four different TB submissions including portions from the Asia Pacific Gateway and Corridor Initiative Fund and the Gateway and Border Crossing Fund.[4] The TB approval of the last $130M in funding (March 2009) provided an exception from the planned horizontal evaluations of the two funds and instead allowed for a summative evaluation of the whole project in 2013-2014.

4. Results of Risk Ratings of Elements of the PAA Universe (2007 and 2008)

# Program Activities 2006-2007
Expenditures
2007 Assessment
Priority (0-10)
2008 Assessment Priority (-2 to +2)
Average
N Rating
  Heritage Places Establishment 20,488.60      
1 National Park Establishment and Expansion 14,239.10 5.56 11 0.64
2 National Historic Sites Designation (Persons, Places, Events) 4,752.60 2.22 14.5 0.91
3 National Marine Conservation Area Establishment 82.00 6.11* 9.75 0.46
4 Other Heritage Places Designations 1,414.90 5.00* 14 0.43
  Heritage Resources Conservation 193,989.20      
5 National Parks Conservation 126,984.50 9.44 12.75 0.20
6 Species at Risk     10 0.60
7 National Historic Sites Conservation 49,763.40 6.81 17 0.41
8 National Marine Conservations Areas Sustainability 1,466.20 6.11* 7 -0.04
9 Other Heritage Places Conservation 15,775.10 5.00*    
10 National Historic Sites Cost-Sharing     8 -0.09
11 Commercial Heritage Properties Incentive Fund     9 0.53
12 Historic Places Initiative     8.25 0.70
  Public Appreciation and Understanding 39,643.00      
13 Outreach Education and Agency Communications   4.44* 15.25 0.18
14 Engagement   13.25 0.17
  Visitor Experience 253,506.40      
15 Marketing and Promotion 14,316.10 6.67 13 0.35
16 National Parks Interpretation 24,883.80 6.11 11.75 0.87
17 National Parks Visitor Activities and Services 116,884.40 10.5 0.69
18 Public Safety   5.56   0.60
19 National Historic Sites Interpretation 43,901.50 5.56 11.75 0.50
20 National Historic Sites Visitor Activities and Services 51,605.90 10.5 0.50
21 National Marine Conservation Areas Interpretation 942.60 6.11 6 0.25
22 National Marine Conservation Areas Visitor Activities and Services 972.10 5 0.30
  Town-Site and Throughway Infrastructure 96,985.60      
23 Townsite Management 12,715.40 2.22 7 0.93
24 Through Highways Management 75,145.40 7.78 9 -0.06
25 Through Waterways Management 9,124.80 6.11 7 0.50
* Other heritage places establishment and conservation received one rating in 2007 that is shown separately against the two sub-activities. Marine Conservation Area establishment, sustainability and visitor experience received one rating in 2007 shown against each sub-activity.
* Corporate Service costs are proportionally allocated to program activity based on fund centres.

Result of 2008 PAA Assessment
(Average score per respondent across four questions)


[1] The 2008-2009 Plan included reference to an Evaluation Framework for the Parks Canada Learning Strategy had been developed and tabled at the then combined Audit and Evaluation Committee in December 2007 but was returned to the Agency's Human Resource Committee for review and revisions of the management responses prior to re-tabling. In May 2008 it was decided at Evaluation Committee to discontinue the project and focus evaluation resources on the major program based evaluations outlined in the 2008-2009 Plan.

[2] Unplanned work in 2008-2009 included contributing to an Environment Canada led Evaluation of the Habitat Stewardship Component of the Species at Risk Program to be completed in 2008-2009.

[3] Parks Canada has or will receive $1.3M between April 2007 and March 2012 to provide services and support to the International Polar Year Initiative. The Agency is not considered one of the six lead departments with respect to this initiative. The IPY Federal Program Office, housed within DIAND, is to conduct a formative evaluation for 2008-2009. The Agency does not plan any evaluation work related to this initiative.

[4] Five federal departments or Agencies including Parks Canada are immediate stakeholders in the Asia Pacific Gateway and Corridor Initiative. It provides $591M over eight years to invest in transportation infrastructure, initiatives to support competitiveness and develop strategies and foster ongoing collaboration. Parks Canada received $37M (i.e., about 6% of the funds) for twinning of a portion of the Trans-Canada Highway in Banff). A summative evaluation led by Transport Canada is scheduled for 2010-2011. The Gateway and Border Crossing Fund provides $2.01B over seven years between April 2007 and March 2014 for both infrastructure and non-infrastructure projects. Parks Canada will manage $100M from the fund for twinning of the Trans-Canada Highway in Banff (i.e., about 5% of the funds). A summative evaluation is anticipated in 2013-2014.