Parks Canada
Symbol of the Government of Canada

Common menu bar links

Internal Audit and Evaluation Documents

Parks Canada Multi-Year Evaluation Plan 2011-2012 to 2015-2016

Parks Canada Multi-Year Evaluation Plan 2011-2012 to 2015-2016

April 21, 2011

Recommended for Approval by Parks Canada Evaluation Committee: May 11, 2011
Approved by CEO: May 24, 2011

Office of Internal Audit And Evaluation
Parks Canada

I submit to the Treasury Board of Canada Secretariat, the evaluation plan that I approved for the Parks Canada Agency for fiscal years 2011-2012, as required by the Policy on Evaluation.

As per section 6.1.7 and 6.1.8 of the policy, I confirm that this five-year departmental evaluation plan:

  1. aligns with and supports the departmental Management, Resources and Results Structure;
  2. has been designed to help support the requirements of the Expenditure Management System, including strategic reviews; and,
  3. includes all ongoing programs of grants and contributions administered by the department, as required by section 42.1 of the Financial Administration Act.

I will ensure that this plan is updated annually and will provide information about implementation of the Agency’s evaluation plan to the Treasury Board of Canada Secretariat, as required.

 

___________________________
Alan Latourelle
Chief Executive Officer
Parks Canada Agency
_____________
Date

 

Table of Contents

Executive Summary

1. Introduction

2. Parks Canada Agency

3. The Evaluation Function

Applicable Policies and Professional Standards
Mandate and Services Offered
Governance
Office of Internal Audit and Evaluation (OIAE)

4. Evaluation Universe and Planning Considerations

a) Agency Renewal
b) Agency’s 2011-12 Corporate Risk Profile
c) Status of Projects Scheduled in 2010-2011
d) Past Evaluation Coverage and Work of Other Assurance Providers
e) TBS Directed Work
f) Planned Work of External Assurance Providers
g) Considerations Arising from TB Submissions, RMAFs

5. Priority Assessment And Five Year Plan

6. Planned Projects 2011-2012

7. Appendices

1. Assumptions in the Calculation of Evaluation Capacity
2. Agency PAA Structure
3. Priority Assessment Dimensions and Scales
4. Corporate Risk Profile 2011-2012
5. Past Coverage of the Evaluation Universe
6. Agency RMAF Evaluation Commitments
7. Average Expenditures by Program Sub-Activity

Executive Summary

The 2011-2012 Parks Canada Multi-Year Evaluation Plan outlines the mandate, organizational structure and resources for evaluation at Parks Canada, the considerations in developing the Plan and details of individual evaluation projects for the FY 2011-2012, together with the associated resource allocation. The Evaluation Committees of the Agency, chaired by the Chief Executive Officer (CEO) discussed and recommended approval of the plan. The CEO approved the plan on May 24, 2011.

The Office of Internal Audit and Evaluation (OIAE) at Parks Canada adheres to the government’s policies and standards for evaluation. Currently, the evaluation function in the Office consists of an executive level director and five staffed evaluator positions. Given a fully staffed function, the Office will have the capacity to conduct approximately two to three risk-based evaluations (i.e., an average or typical project is 2,200 hours), as well as addressing other non-risk based work (i.e., provision of advice, coordination, follow-up and participation in TBS directed evaluation work).

Planning is based on a slightly modified version of the sub-activities and sub-sub-activities in the Agency’s Program Activity Architecture (PAA). Work for the 2011-2012 fiscal year will include the following projects:

  • Concluding evaluations of the Visitor Service Offer, and Through Waterway Management sub-activities;
  • Continuing an evaluation of National Park Establishment and Expansion sub-activity;
  • Commencing an evaluation of the National Park Resource Conservation sub-activity;
  • Completing an evaluation of the National Historic Sites Cost-Sharing Program; and
  • Participating in various interdepartmental horizontal evaluations in fulfillment of Treasury Board requirements, including concluding the evaluation of the Species at Risk Act.

The five-year evaluation coverage proposed in the plan will cover about 88% of the Agency’s direct program expenditures based on the average expenditures between 2006-2007 and 2008-2009, with approximately 66% of the expenditures covered in the next two years.

1. Introduction

The 2011-2012 Parks Canada Evaluation Plan outlines the mandate, organizational structure and resources for evaluation at Parks Canada, the strategy and process employed in developing the Plan, a project schedule for the five-year period from April 2011 to March 2016, and details of individual evaluation projects for the FY 2011-2012, together with the associated resource allocation.

The plan was designed in conformity with requirements of the TB Evaluation Policy (2009), and the implications of the Government’s Federal Accountability Act, as well as Parks Canada’s Evaluation Charter (http://www.pc.gc.ca/docs/pc/rpts/rve-par/47/index_e.asp). The ultimate goal of evaluation planning is to provide comprehensive evaluation coverage of material program spending over a five-year period to support evidence-based decision making on policy, expenditure management (e.g., Strategic Review) and program improvements and accountability for results.

2. Parks Canada Agency

Parks Canada was established as a separate departmental corporation in 1998. The Agency's mandate is to:

Protect and present nationally significant examples of Canada's natural and cultural heritage, and foster public understanding, appreciation and enjoyment in ways that ensure the ecological and commemorative integrity of these places for present and future generations.

Responsibility for the Parks Canada Agency rests with the Minister of the Environment. The Parks Canada Chief Executive Officer (CEO) reports directly to the Minister.

National parks and national historic sites are organized into thirty-two geographically based field-units. About 80% of Parks Canada’s work force is based in the field where most of its program expenditures take place. The work of field units is supported by Service Centres located in Halifax, Quebec City, Cornwall/Ottawa and Winnipeg (with small branch offices in Calgary and Vancouver). The Service Centres comprise about 10% of the work force and provide technical and professional services to field units (e.g., science, research, design services). National Office, with less than 10% of the employee base, consists of five directorates (National Parks, National Historic Sites, Strategy and Plans, Human Resources and External Relations and Visitor Experience) who provide legislative, operational policy, planning, program direction, financial management, and human resources functions and services.

3. The Evaluation Function

Applicable Policies and Professional Standards

The evaluation function at Parks Canada adheres to the Evaluation Policies, directives, standards and guidelines of the Government of Canada. In 2007-2008 a new charter for the evaluation function was approved consistent with the then draft Evaluation Policy, directives and standards.

Mandate and Services Offered

The mandate of the function is:

To contribute to the achievement of Parks Canada's mandate by providing the CEO with evidence-based, credible, neutral and timely information on the ongoing relevance, results, and value of policies and programs, alternative ways of achieving expected results, and program design improvements.

Services include:

  • Evaluation frameworks related to programs, activities or initiatives (i.e., completed in advance of an evaluation to describe an entity, its logic, inputs, outputs, reach and results and identify evaluation questions, methods and costs);
  • Evaluations of programs, policies and functions (i.e., treating the core issue of relevance and performance); and
  • Special projects and the provision of advice, as required, on performance measures, targets and information systems.

Follow-up on Management Responses

The evaluation cycle includes a systematic follow-up on the management responses, at six months intervals, after the final approval of the reports by the CEO. Managers are requested, by e-mail from the CEO, to complete a template that provides a status report on action plans in response to evaluation recommendations. The template is returned directly to the Chief Audit and Evaluation Executive, and is tabled at the next evaluation committee meeting. The process continues for five-years or until all actions plans are reported to be complete.

Governance

The Agency’s Evaluation Committee is chaired by the CEO and composed of seven senior managers (i.e., Directors General, Chief Financial Officer, and Chief Administrative Officer). The committee approved a revised terms of reference in November 2010.

The Evaluation Committee is responsible for reviewing and providing advice or recommendations to the CEO on:

  • Evaluation Function and Products: including the Agency’s Evaluation Charter, the rolling Five-Year Evaluation Plan, the adequacy and neutrality of resources allocated to the evaluation function, the performance of the function, key elements of an evaluation product lifecycle such as terms of reference, scoping documents, evaluation reports, management responses and action plans including following-up to ensure action plans are implemented.
  • Performance Management Framework: the adequacy of resources allocated to performance measurement in support of evaluation activities, and recommend to the CEO changes or improvements to the framework and an adequate level of resources for these activities.

Office of Internal Audit and Evaluation (OIAE)

The organization structure of the function is shown in the accompanying chart.

The organization structure for the evaluation function.

The table below shows the budget for the function for 2011-2012. Given a completely staffed function, it is estimated that the OIAE would have the capacity to conduct approximately two to three risk-based evaluations (i.e., an average or typical project is 2,200 hours). The actual number of projects undertaken in any one year may differ from this estimate to the extent that actual project hours differ from average project hours. Details of the calculations involved in the estimate are shown in Appendix 1.

  CAEE Office** Evaluation Total
Salaries* 26,000 418,000 444,000
O&M Staff Support 6,000 61,000 67,000
O&M Professional Services and Travel   189,000 189,000
Total     700,000
Does not include benefits and accommodations costs (i.e., about 33% of the salary budget)
CAEE Office includes .5 of the FTE each for the Chief Audit and Evaluation Executive and for the Executive Assistant. It only includes the salary for the Executive Assistant position, as the salary for the CAEE is administered centrally for all executive level employees in the Agency.

4. Evaluation Universe and Planning Considerations

The key driver of evaluation planning is the requirement in the TB Evaluation Policy to provide comprehensive coverage of direct program spending over a five-year period. The sub-activities and sub-sub activities in the Agency’s program activity architecture serve as the evaluation universe for planning coverage of direct spending (i.e., rather than program activities as a whole with some adjustments to amalgamated sub-activities where it makes sense and to add a few programs[1] that are not part of the PAA structure but for which evaluation commitments already exist). Prioritizing elements of the evaluation universe serves to identify the timing, scope and scale of the evaluations to be conducted (i.e., high priority entities are evaluated earlier in the cycle and with more depth and rigour than lower priority entities).

For planning purposes each element in the universe is rated on eight dimension (i.e., with a three point scale for each) adapted from the TBS Guide to Evaluation Planning. The dimensions are shown below. More details are shown in Appendix 3.

  1. Existing Commitments to Conduct an Evaluation
  2. Materiality (i.e., the amount of spending on a sub-activity)
  3. Links to Corporate Risk Profile
  4. Completeness of Performance Framework
  5. Intended Direct Reach
  6. Degree of Direct Control Over Program Outcomes
  7. Importance of Health and Safety Considerations in Program Delivery
  8. Public Interest and Sensitivity

As part of the ratings process the following kinds of information are considered.

a) Agency Renewal

In December 2007, the Agency began a project directed toward renewal of its programs in response to a number of external drivers for change (e.g., changing demographics, changing technology, changing leisure patterns, increased urbanization and increased national and international competition for tourist visits). In January 2009, the Agency officially rolled out its case for change and a new Vision Statement.

Since then the Agency has renewed the Parks Canada brand to communicate the vision outside the Agency and increase the public’s appreciation of the Agency and its programs (PA 3). It has also undertaken an extensive realignment of the External Relations and Visitor Experience program (PAs 3 and 4) creating separate visitor experience and external relations organizational structures within field units and additional specialized capacity particularly in urban outreach, internet web content, and stakeholder and partner relations.

National Historic Sites program renewal (PA 2) is focusing on developing a strategy that will make the Agency’s national historic sites more known to, appreciated by and responsive to the needs of, and connected with Canadians.

As a result of the Government of Canada May 2008 announcement of improvements to law enforcement capabilities in Canada’s National Parks and authorization for PC to arm up to 100 law enforcement officers, the Agency began renewal of the general resource conservation function (i.e., accountability framework,, roles, responsibilities, accountabilities organizational models, work descriptions related to all aspect of natural resource conservation at the national office, service centre and field level.) (PA 2).

The Agency has also developed a more coherent approach to prevention activities (i.e., preventing incidents before they occur and resolve incidents when they do occur safely and effectively). The approach is now embedded in the new Parks Canada Service Program (service standards and prevention guidelines). Prevention activities cut across the PAA structure of the Agency. This program was developed concurrently and in coordination with the new law program and has involved training over 100 trainers who subsequently trained over 3,500 front line employees prior to the 2009-2010 operating season.

In some cases, these changing priorities are well advanced (e.g., vision, brand, law enforcement, external relations and visitor experience program renewal) while in others the change is only starting to take shape (e.g., renewal of the NHS program).

b) Agency’s 2011-12 Corporate Risk Profile

The Agency updated its corporate risk profile for 2011-12 to 2015-16 (see Appendix 4 for details). Eleven risks areas are identified with four categorized as key risks warranting additional mitigation. These are:

  • Competitive Position: Parks Canada’s profile, service and experience offer may be less attractive or of less interest to Canadians in comparison to other parks and cultural attractions and/or leisure activities.
  • Environmental Forces: The Agency’s ability to maintain or improve overall EI in national parks and meet legal requirements related to species at risk may be hindered by environmental forces, such as climate change, biodiversity loss, and exotic/invasive species.
  • Asset Management: Aging infrastructure and inadequate level of recapitalization and maintenance could result in failure of assets and/or significant impairment of built cultural resources, which could compromise public safety, hinder Parks Canada's ability to deliver on its mandate and damage the Agency's reputation.
  • Information Management: Failure to identify, capture, manage, share and report pertinent data and information may hinder the ability to effectively manage all program areas and meet legal requirements.

Many of the risk areas cut across program activities (i.e., asset and information management) while others, such as competitive position and environmental forces relate to specific programs such as Visitor Experience and Heritage Resources Conservation.

c) Status of Projects Scheduled in 2010-2011

Evaluation products not completed at the end of fiscal year are carried over into the next year. The following shows the status, as of March 31, of the projects scheduled in 2010-2011.

Project Status Dates
Frameworks
National Park Establishment and Expansion Approved November 2010
National Parks Conservation Approved November 2010
National Historic Sites Conservation Postponed 2013-2014
Evaluations
Through Highways Management Approved December 2010
General Class Contribution Program Approved December 2010
Species at Risk* In progress September 2011 (delayed)
Visitor Service Offer In progress May 2011 (delayed)
Through Waterways Management In progress June 2011
National Park Establishment and Expansion In progress December 2011 (delayed)
*Species at Risk is a horizontal evaluation of the programs and activities in support of the Species at Risk Act over the period 2005-2010. It is being lead by Environment Canada and conducted jointly with the DFO and the Agency.

d) Past Evaluation Coverage and Work of Other Assurance Providers

Past evaluation work within the Agency and relevant work of the Office of the Auditor General and the Commissionaire of the Environment and Sustainable Development against elements of the evaluation universe is shown in Appendix 5. The table includes the two evaluations completed in 2010-11: the Evaluation of Through Highway Management and the Evaluation of the General Class Contribution Program.

e) TBS Directed Work

The TB Evaluation Policy calls for the creation of a Government of Canada Evaluation Plan to be supported by evaluation activities within departments. It is not known at the time of the planning whether TB would require specific evaluation work.

f) Planned Work of External Assurance Providers

The Office of the Auditor General, conducts review-level assurance work each year on the fairness and reliability of the performance information in the Agency’s Annual Performance Report. The OIAE has taken this assurance work into account in planning its priorities.

g) Considerations Arising from TB Submissions, RMAFs

Currently Parks Canada has evaluation commitments resulting from TB submissions and/or RMAFs related to several multi-department programs or initiatives and three of its own programs. The various commitments are listed in Appendix 3. Individually the initiatives and funds covered by these evaluation activities represent from less than 1% to about 5% of the Agency’s spending in any one year. Therefore, while important to meet commitments to evaluate transfer payment programs as specified in the Federal Accountability Act, or to demonstrate results related to particular initiatives, they do not provide coverage of most of the Agency’s core programming activities.

5. Priority Assessment And Five Year Plan

The elements of the universe are shown below ordered by highest to lowest priority, along with planned frameworks and evaluation studies, and the percentage of direct program expenditures covered by year, for the next five year period.

Given the fact that the fundamentals of the Agency’s programs (i.e., materiality, direct reach, control over outcomes, health and safety implications) do not change significantly from year to year, it is not surprising that the overall priority ratings of the elements of the universe are similar in 2010-2011 and 2011-2012. Dimensions where some change is evident include a commitment to conduct an evaluation (i.e., due dates are nearer increasing the priority of some items) and links to the corporate risk profile (i.e., the extent to which the Agency’s risk profile has changed).

In scheduling evaluations, first priority is given to areas for which there are existing commitments to evaluate the activity. Evaluations of higher priority areas are then scheduled in light of the evaluation unit’s annual capacity for project work (see Appendix 1) and reviewed with the evaluation committee prior to finalizing the plan. In the five year planning period, evaluations will cover 88% of direct program expenditures including all the high and medium priority elements.

2011-2012 Priority Ratings and Five-Year Plan

Universe Elements Priority Ratings
(0 to 32)
2011-12 2012-13 2013-14 2014-15 2015-16 Past Coverage
2010/
11
2011/
12
Through Highways Management 16 20     Evaluation (TCH)     2010-2011
Visitor Activities and Services 16 16 Evaluation          
Through Waterways Management 12 16 Evaluation          
Species at Risk 14 14 Evaluation          
Outreach Education And External Communi-
cations
14 14   Framework Evaluation      
Stakeholder And Partner Engagement 14 14        
Visitor Safety 14 14   Evaluation        
National Park Establishment and Expansion 10 12 Evaluation        
National Parks Conservation 12 12 Evaluation        
Law Enforcement 12 12     Evaluation      
National Marine Conservations Areas Sustainability 12 12     Framework Evaluation    
Market Research and Promotion* 12 12 Evaluation          
Interpretation 12 12   Evaluation        
Townsite Management 8 12         Evaluation  
General Class Contributions Program 8 12         Evaluation 2010-2011
National Marine Conservation Area Establishment 10 10     Framework Evaluation    
National Historic Sites Conservation 6 8     Framework Evaluation    
Other Heritage Places Conservation 8 8         Evaluation  
National Historic Sites Cost-Sharing 6 8 Evaluation          
National Historic Sites Designation (Persons, Places, Events) 6 6         Evaluation  
Other Heritage Places Designations 6 6         Evaluation  
Direct Program Spending Coverage
(as percentage of average spending[2])
33% 33% 6% 9% 7%  
*Aspects of Market Research and promotion are covered in the evaluation of visitor activities and services

6. Planned Projects 2011-2012

The scheduling and resource requirements of the 2011-2012 projects are shown below.

Evaluation Project Scheduling for 2011-2012
Topic Type Description Planned or Actual Dates Resources Required
Planned
2010-11
Start
date
Comple-
tion
of the field-
work
Comple-
tion of
report
Date of approval by
Commit-
tee
App-
rox hours
O&M
Total 5700 189K
Carried Over From 2010-2011
National Park Establish-
ment and Expansion
Evaluation An evaluation of the relevance and performance of this sub-activity. Y January 2011 April 2012 June 2012 September 2012 2000 38K
Species at Risk Horizontal Evaluation Interdepart-
mental evaluation of the programs and activities in support of the Species at Risk Act.
Y June 2010 March 2011 June 2011 September 2011 200 0K
Visitor Service Offer Evaluation A summative evaluation of relevance and performance of the visitor services component of the Visitor Experience Program Activity. Y August
2009
November 2010 May 2011 September 2011 200 0K
Through Waterway Management Evaluation An evaluation of the relevance and performance of this sub-activity. A brief scoping document will precede the evaluation, describing the sub-activity and the evaluation’s issues and scope. Y June 2010 March 2011 May 2011 September 2011 500 12.5K
New in 2011-12
National Parks Conservation Evaluation An evaluation of the relevance and performance of the sub-activity. Y March 2011 June 2012 Septem-
ber 2012
December 2012 1000 117K
National Historic Sites Cost-Sharing Program Evaluation A summative evaluation of the relevance and performance of the program. Y April 2011 October 2011 January 2012 March 2012 1200 20K
Contributions to Interdepartmental Evaluations for 2011-12*
Health of the Oceans Inter-
depart-
mental Evaluation
Parks will participate in a joint, DFO-lead, evaluation of the initiative. N April 2011 August 2011 Septem-
ber 2011
December 2011 100 0K
Advancing Conservation Interests in the NWT Inter-
depart-
mental Evaluation
Parks received funding for feasibility study of East Arm of Great Slave Lake and development and operation of Sahoyue and Ehdacho NHS. N March 2011 September 2011 October 2011 December 2011 400 1.5K
Federal Contami-
nated Sites
Inter-
depart-
mental Evaluation
A horizontal evaluation, involving the sixteen organizations involved in the three components of the Federal Contaminated Sites Action Plan. N June 2011 February 2012 June 2012 August 2012 100 0K
*Parks Canada participates in these joint evaluations, generally contributing to joint evaluation reports. Reporting from these will be tabled at Evaluation Committee for information or approval as appropriate.
Note: Approximate hours reflect required effort. Projects that extend beyond a fiscal year will require more effort than shown in the table. Staff or consultants may perform the work. O&M required includes costs of professional services and/or costs of staff travel.

7. Appendices

  1. Assumptions in the Calculation of Evaluation Capacity
  2. Agency PAA Structure
  3. Priority Assessment Dimensions and Scales
  4. Corporate Risk Profile 2011-2012
  5. Past Coverage of the Evaluation Universe
  6. Agency RMAF Evaluation Commitments
  7. Average Expenditures by Program Sub-Activity

1. Assumptions in the Calculation of Evaluation Capacity

a)
The table below shows our assumptions in calculating the total work time available per FTE.

Time available for work per FTE Hours
52 week/year * 5day week *7.5 hours 1950
Average time for holidays -150
Average sick leave -37.5
Average time for training -37.5
Total Work Time Available 1725*
*Equivalent to 230 days (7.5 hours)

b)
Hours Available for Evaluation Work: On average it is assumed that one hour per working day is allocated to breaks (i.e., 230 hours per evaluator per year), leaving 1,495 hours of actual working time. In 2010-11 the evaluation unit implemented a time tracking system to record hours spent by evaluators on evaluation project work. Evaluation project work includes the development of evaluation frameworks, evaluation work required as part of TB submissions or RMAF/RBAF commitments, horizontal interdepartmental evaluations, and risk-based evaluation projects. Based on this data, evaluators averaged 1182 hours evaluation project work per evaluator over the course of a fiscal year.

Distribution of available work hours Hours
Breaks (hour per day) 230
Evaluation Project work 1182
Administration, special requests, consultation/advice/coordination 313

 

The remainder of the available work hours, 313 hours per evaluator, are assumed to be for administration, special requests by management outside the approved evaluation plan, and consultation and provision of advice related to evaluation or performance measurement.[3] Hours contributed by the CAEE toward evaluation project work, as well as those of the CAEE’s assistant, are not tracked and therefore estimated in the analysis for section 2.

  1. O&M Dollars Available for Contracting: Additional capacity can be purchased from contracted professionals based on available O&M budgets. An average per diem of $1,200 per day or $160 per hour is used for this calculation. For 2011-2012, the O&M budget for professional services is $100K.

  2. The resource consumption of the average evaluation project: The average number of hours consumed by an evaluation project is affected by many factors, including its scope and complexity as well as the skills of the personnel performing the work. Based on a review of prior experience with contracted projects and recent time tracking of projects as well as the expectation that future evaluation projects will need to be more complex given the TB policy directions, typical values of 2,200 hours per evaluation project were identified.
2011-2012 Capacity
A) Resources CAEE Office Evaluation Total
FTEs 1 4.8* 5.8
Hours Available for Work 1,495 7,176 8,671
Professional Service Hours Purchased (O&M for professional Services/$160 per hour/$100K budget)   625 625
Total Hours Available 1,495 7,801 9,296
B) Demands
Administration, special requests, consultation/advice/coordination, etc. i.e., (accounts for 80% of CAEE plus -10 hours special requests and -140 hours consulting, advice, and coordination. We factor 313 hours for evaluator FTE based on past time tracking data.) -1,495 -1,502 -2,997
Residual Hours for Risk Based Projects 0 6,299 6,299
C) Project Capacity
Number of Typical Projects (2,200 hours per project)     2.8
*one employee is 0.8 an FTE because of leave with income averaging and parental leave.

2. Agency PAA Structure

Parks Canada Agency Strategic Outcome and Program Activity Architecture 2009/2010

Agency Program Activity Architecture Structure

Click to enlarge

3. Priority Assessment Dimensions and Scales

  4 2 0
1. TB commitments Required in the next 12 to 18 months Required but not in the next 18 months None required
2. Materiality Greater than 10% (approximately 60+ million) 5% to 10% (approximately 31 to 60 million) 5% (approximately 30 million)
3. Links to Corporate Risk Profile Links primarily to high priority corporate risks Links to primarily lower priority corporate risks No links to corporate risks
Activities linked to the four highest corporate risks: competitive position, environmental forces, Delivery/Management Of Infrastructure Projects and Information Management are rated four (see section 2 above). Activities related to other risks in section 2 are rated a moderate and activities not related to the risk profile are rated one.
4. Completeness Of Performance Framework None or few elements of the framework in place Partially complete Complete
A complete framework consists of defined measurable goals and objectives, baseline measures of performance, quantifiable targets with clear time frames for accomplishment of goals and systems to measure and report on progress and goal attainment, and evidence of monitoring and reporting.
5. Extensiveness of Program Reach Extensive reach to communities, stakeholders, NGOs, Aboriginal peoples, and the public. Moderate and/or regional-level reach to communities, stakeholders, NGOs, Aboriginal peoples, and the public. Limited and/or localized reach to communities, stakeholders, NGOs, Aboriginal peoples, and the public.
High intended direct reach is typified by activities related to building awareness and understanding the Agency and its mandate and promotion and marketing Parks Canada sites as well as the visitor experience program which are intended to reach millions of Canadians and international visitors. Low reach is typified by sub-programs in the Other Heritage Places Establishment Sub-Activity such as Grave Sites of Prime Ministers which is effectively targeted at a few families of former prime ministers whose grave sites are not yet formally commemorated. When the target reach of a program are organizations, or provinces, as in park establishment for example, we count reach as the number of groups targeted and not the size of the constituencies represented by these groups. Most program activities have ultimate beneficiaries i.e., Canadians as a whole, who are not counted as the program or sub-activity reach.
6. Degree of Direct Control Over Outcomes Low Direct Control Moderate Direct Control High Direct Control
Low control over outcomes is exemplified by the national park and national marine conservation areas establishment and expansion sub-activities, which require extensive consultation and negotiations over many years with dozens of different stakeholders, who differ in their capacities and interests, and have the capability to block a particular establishment process. More control is available over a contribution program where the Agency, with TB agreement, has set the terms and conditions for receiving funding and evaluates and recommends who will be funded. An intermediate example might be conservation in national parks and NMCAs where the Agency may have a relatively high degree of control over what occurs within the boundaries of the park but is also interested in influencing regional land use practices than impact on the park’s ecological integrity.
7. Importance of Health and Safety Considerations in Program Delivery High Level of Consideration of health and safety issues in delivery of a sub-activity. Moderate Level of Consideration of health and safety issues in delivery of a sub-activity. Low Level of Consideration of health and safety issues in delivery of a sub-activity.
Many activities involving visitors require consideration of health and safety issues as a fundamental part of the program delivery. Examples include the potential for human wildlife conflicts in national parks, possibilities of contamination when providing potable water, the potential of accidents on highways managed by the Agency, and the potential for accident or injury when conducting law enforcement or search and rescue activities. We do not assess the nature or quality of management measures to mitigate health and safety issues involved in sub-activity delivery only whether and the extent to which these considerations have been inherent in delivery of the activity.
8. Public Interest and Sensitivity High Moderate Low
Activities which have received recent public or political attention are rated higher (i.e., the lead up to the decision to arm park wardens and the new law enforcement program had extensive media coverage but this has largely abated since the new program began operating). Introduction of new legislation such as the Heritage Lighthouse Act, creates temporary political interest in a particular activity (the Act would protect heritage lighthouses in Canada and is considered part of the Other Heritage Places Designation sub-activity). Some consideration is also given to potential for public or political interest. Many of the health and safety concerns reviewed above have high potential interest should they occur (e.g., the failure of a dam or a potable water system resulting in a significant number of injuries or deaths).

4. Corporate Risk Profile 2011-2012

Risk Category and Label Description Risk Owner OIAE Work
Public
Aboriginal Support Support from Aboriginal Peoples may diminish and become insufficient to advance Parks Canada’s programs. Director, Aboriginal Affairs Secretariat  
Inter-governmental Collaboration Resource capacity in other federal departments, provinces, territories, and municipalities may be insufficient to fully collaborate on Parks Canada’s program priorities. DG, National Parks
DG, National Historic Sites
 
Partnering Instruments Existing partnering instruments may limit Parks Canada’s ability to fully leverage partnering opportunities, resulting in its inability to extend its reach and to grow the base of support for Parks Canada’s administered places. DG, External Relations and Visitor Experience  
Public Support Support from local communities, stakeholders, NGOs, and the Canadian public may not exist or be insufficient to advance Parks Canada’s programs. DG, External Relations and Visitor Experience  
Socio-Economic
Competitive Position Parks Canada’s profile, service and experience offer may be less attractive or of less interest to Canadians in comparison to other parks and cultural attractions and/or leisure activities. DG, External Relations and Visitor Experience Evaluation of Visitor Service Offer - March 2011
Development Pressures Development pressures may limit opportunities for NP/NMCA establishment, NHS commemoration, maintenance of EI and CI, and development of connection to place. DG, National Parks
DG, National Historic Site
 
Environmental Forces
Disasters Disasters may impair or destroy critical infrastructure and/or assets of national historic significance, or lead to serious injury or loss of life. Chief Administrative Officer Audit of Business Continuity
Environmental Forces The Agency’s ability to maintain or improve overall EI in national parks and meet legal requirements related to species at risk may be hindered by environmental forces, such as climate change, biodiversity loss, and exotic/invasive species. DG, National Parks  
Parks Canada’s Business Operations
Asset Management Aging infrastructure and inadequate level of recapitalization and maintenance could result in failure of assets and/or significant impairment of built cultural resources, which could compromise public safety, hinder Parks Canada's ability to deliver on its mandate and damage the Agency's reputation. Chief Administrative Officer Evaluation of the Parks Canada Asset Management Program – July 2009
Information Management Failure to identify, capture, manage, share and report pertinent data and information may hinder the ability to effectively manage all program areas and meet legal requirements. Chief Administrative Officer Audit of Information Management
Recruitment and Retention Failure to recruit and retain competent employees may lead to challenges in delivery of all programs and support functions. Chief Human Resources Officer  
Source: Parks Canada Agency Corporate Risk Profile 2011-12

5. Past Coverage of the Evaluation Universe

Program Activities and Sub-Activities Parks Canada Evaluations
January 2005 to March 2011
Work of External Assurance Providers
Heritage Places Establishment
National Park Establishment and Expansion    
National Historic Sites Designation (Persons, Places, Events)    
National Marine Conservation Area Establishment    
Other Heritage Places Designations    
Heritage Resources Conservation
National Parks Conservation   CESD Chapter 2—Ecological Integrity in Canada's National Parks - September 2005
Species at Risk Formative Evaluation of Federal Species at Risk Programs-July 2006
Contributing to a Environment Canada lead
Evaluation of the Habitat Stewardship Component of the Species at Risk Program (2008-2009)
CESD Chapter 5—Ecosystems—Protection of Species at Risk – March 2008
National Historic Sites Conservation   AG Chapter 2—The Conservation of Federal Built Heritage February 2007
National Marine Conservation Areas Sustainability Evaluation of Parks Canada's Phase One of Oceans Action Plan -June 2007  
Other Heritage Places Conservation    
National Historic Sites Cost-Sharing Evaluation of Issues Related to the National Historic Sites of Canada Cost-Sharing Program -August 2008  
Commercial Heritage Properties Incentive Fund Formative Evaluation of the Commercial Heritage Properties Incentive Fund (CHPIF) -January 2007  
Historic Places Initiative Formative Evaluation of the Historic Places Initiative -March 2005  
Public Appreciation and Understanding
Public Outreach and External Communication National Performance and Evaluation Framework for Engaging Canadians: External Communications at Parks Canada -February 2005
Formative Evaluation of Engaging Canadians External Communications Strategy -September 2006
CESD Chapter 2—Ecological Integrity in Canada's National Parks - September 2005
Stakeholder and Partner Engagement    
Visitor Experience
Marketing and Promotion    
National Parks Interpretation    
National Parks Visitor Activities and Services    
Public Safety Evaluation of Parks Canada’s Public Safety Program-February 2005  
National Historic Sites Interpretation    
National Historic Sites Visitor Activities and Services    
National Marine Conservation Areas Interpretation    
National Marine Conservation Areas Visitor Activities and Services    
Town-Site and Throughway Infrastructure
Townsite Management   CESD Chapter 1 – Safety of Drinking Water – March 2009

AG Chapter 4 – Safety of Drinking Water: Federal Responsibilities - 2004
Through Highways Management Evaluation of Through Highway Management – November 2010  
Through Waterways Management    
Evaluations contributing to coverage of multiple program activities Evaluation of the Parks Canada Asset Management Program – July 2009

Evaluation of Parks Canada’s General Class Contribution Program – November 2010
 

6. Agency RMAF Evaluation Commitments

  Horizontal Evaluations Parks Canada
2011-2012 Evaluation of the Species at Risk Program: Environment Canada, Fisheries and Oceans and the Agency are the lead federal organizations for the Species at Risk Program. Parks Canada received a total of $10.6M between 2007/08 and 2011/12 from this initiative and expects to receive $6.8M per year thereafter, representing about 10% of the overall available funding. The RMAF for this initiative commits to a summative evaluation in 2010-2011.

Evaluation of the Establishing Federal Protected Areas in the NWT: Indian and Northern Affairs Canada, Environment Canada and Parks Canada are partners working together to advance the Northwest Territories Protected Areas Strategy. Parks Canada will receive $8.05M between April 2008 and March 2013 (i.e., roughly a third of investment) to assist in establishing a national park in the East Arm of Great Slave Lake, and to help develop and make operational of the Sahoyue and Ehdacho National Historic Site of Canada. A summative evaluation of the initiative, lead by DIAND, is required for early 2012.

Evaluation of the Health of the Oceans Initiatives: The Agency has or will receive $6.25M between April 2007 and March 2012 to help develop and implement a federal marine areas strategy in concert with Fisheries and Oceans and Environment Canada and to conduct a feasibility study for establishing a national marine conservation area in Lancaster Sound.
Evaluation of the National Historic Sites of Canada Cost-Sharing Program: The program provides matching contribution funds to eligible national historic sites undertaking conservation and presentation work. It was renewed in 2008-2009 with an annual budget of between $2.3 and 3.3M per year over the next five years. The budget has been supplemented in 2009 through 2011 with EAP funding. A summative evaluation is required to be completed by 2012-2013.
2012-2013    
2013-2014 Evaluation of Federal Contaminated Sites Action Plan: The three program components identified for this evaluation include the FCSAP Secretariat, expert support departments and all custodians who have received FCSAP funding. An interdepartmental Program Evaluation Working Group will be formed to facilitate and guide the evaluation. Planning for the evaluation is underway with the evaluation starting in June 2011; a final report is planned for August 2012. Evaluation of the Law Enforcement Program: The program, involving up to 100-armed law enforcement officers responsible for enforcement of national parks, national marine conservation area and national historic site laws and regulations, but not for criminal code enforcement, was funded and developed in 2008-2009 with on the ground activities commencing in 2009-2010. The program will have start-up costs of $8.5M in 2008-2009 and ongoing costs of $2.3M per year thereafter (i.e., less than one percent of the Agency’s annual spending). A summative evaluation is planned for in 2013-2014.

Evaluation of the Twinning of the Trans-Canada Highway in Banff National Parks: The recent initiative (i.e., 2004-05 on) involves twining 32 kilometres of the TCH at a total cost of $317M over 10 years. Funding was received through four different TB submissions including portions from the Asia Pacific Gateway and Corridor Initiative Fund and the Gateway and Boarder Crossing Fund.[4] The TB approval of the last $130M in funding (March 2009) provided an exception from the planned horizontal evaluations of the two funds and instead allowed for a summative evaluation of the whole project in 2013-2014.
Parks Canada has or will receive $1.3M between April 2007 and March 2012 to provide services and support to the International Polar Year Initiative. The Agency is not considered one of the six lead departments with respect to this initiative. No evaluation work related to this initiative is planned.

7. Average Expenditures by Program Sub-Activity

($ thousands) 2006/07 2007/08 2008/09 Average
Expenditure
Average
Percentage
Heritage Places Establishment
National Park Establishment and Expansion 14,239 9,871 24,178 16,096 2.5%
National Historic Sites Designation (Persons, Places, Events) 4,753 5,559 4,154 4,822 0.8%
National Marine Conservation Area Establishment 82 2,986 3,680 2,249 0.4%
Other Heritage Places Designations 1,415 1,392 1,581 1,463 0.2%
Heritage Resources Conservation
National Parks Conservation
Includes species at Risk and law enforcement
126,985 129,312 125,850 127,382 19.9%
National Historic Sites Conservation 49,763 54,452 59,786 54,667 8.5%
National Marine Conservations Areas Sustainability 1,466 2,135 2,193 1,931 0.3%
Other Heritage Places Conservation 15,775 14,590 28,859 19,741 3.1%
National Historic Sites Cost-Sharing   900 134 345 0.1%
Public Appreciation and Understanding
Outreach Education and Agency Communications and Stakeholder and Partner Engagement 39,643 33,290 40,126 37,686 5.9%
Visitor Experience
Market Research and Promotion 14,316 15,445 19,952 16,571 2.6%
National Parks Interpretation 24,884 19,634 18,046 20,854 9.7%
National Historic Sites Interpretation 43,902 36,683 40,796 40,460
National Marine Conservation Areas Interpretation 943 765 1,470 1,059
National Parks Visitor Activities and Services 116,884 120,470 129,506 122,287 29.4%
National Historic Sites Visitor Activities and Services 51,606 81,951 60,864 64,807
National Marine Conservation Areas Visitor Activities and Services 972 1,937 1,247 1,385
Visitor Safety   9,732 11,224 6,985 1.1%
Town-Site and Throughway Infrastructure
Townsite Management 12,715 16,363 16,604 15,227 2.4%
Through Highways Management 75,145 68,870 87,618 77,211 12.0%
Through Waterways Management 9,125 7,706 5,927 7,586 1.2%
Other
General Class Contributions Program          
Total 604,613 634,043 683,795 640,814  

[1] These are the law enforcement program and the General Class Contribution Program.

[2] Expenditures by sub-activity are not directly coded into the financial system and must be estimated each year by Finance Branch. The methodology for this is complex and under review. Estimates are not available for 2009-10 and 2010-11. Coverage is therefore based on average spending from 2006-07 through 2008-09 (see Appendix 7 for details)

[3] Administration includes staff hours allocated for activities such as planning, meetings, work on internal systems, support to the evaluation committee, human resources, quality control, follow-up on previous recommendations, etc.

[4] Parks Canada is a stakeholder in the Asia Pacific Gateway and Corridor Initiative which provides $591M over eight years to several departments to invest in transportation infrastructure and other projects. Parks Canada received $37M or about 6% of the funds for twinning of a portion of the TCH. A summative evaluation led by Transport Canada was scheduled for 2010-2011. The Gateway and Boarder Crossing Fund provides $2.01B over seven years between April 2007 and March 2014 for both infrastructure and non-infrastructure projects. Parks Canada received $100M from the fund for twinning of the TCH or about 5% of the funds. Parks Canada will conduct a summative evaluation of the investment in the TCH as a whole in 2013-2014.