Parks Canada
Symbol of the Government of Canada

Common menu bar links

Internal Audit and Evaluation Documents

Parks Canada Multi-Year Evaluation Plan 2014-2015 to 2018-2019

Collage of photographs of locations in Parks Canada parks 

May, 2014
Office of Internal Audit and Evaluation Parks Canada

Recommended for Approval by Parks Canada Executive Management Committee: April 22, 2014
Date approved by CEO: June 10, 2014

Table of Contents

Deputy Head Confirmation

I approve the departmental evaluation plan (DEP) of Parks Canada for the fiscal years 2014-2015 to 2018-2019, which I submit to the Treasury Board of Canada Secretariat as required by the Policy on Evaluation.

As per Sections 6.1.8 of the policy, I confirm that the following evaluation coverage requirements are met and reflected in this five-year DEP:

  • all ongoing direct program spending is evaluated every five years;
  • all ongoing programs of grants and contributions are evaluated every five years, as required by section 42.1 of the Financial Administration Act;
  • the administrative aspect of major statutory spending is evaluated every five years;
  • programs that are set to terminate automatically over a specified period of time, if requested by the Secretary of the Treasury Board following consultation with the affected deputy head;
  • specific evaluations, if requested by the Secretary of the Treasury Board following consultation with the affected deputy head.

As per section 6.1.7, I confirm that this five year DEP:

  • aligns with and supports the departmental Management, Resources and Results Structure; and
  • supports the requirements of the Expenditure Management System, including spending reviews.

I will ensure that this plan is updated annually, and I will provide information about its implementation to the Treasury Board of Canada Secretariat, as required.



Original Signed by

___________________________
Alan Latourelle
Chief Executive Officer
Parks Canada Agency



June 10, 2014

______________
Date

Executive Summary

The Parks Canada 2014-2015 Multi-Year Evaluation Plan outlines the mandate, organizational structure and resources for evaluation in the Agency, the considerations employed in developing the Plan and details of individual evaluation projects for FY 2014-2015, together with the associated resource allocation.

The Office of Internal Audit and Evaluation (OIAE) adheres to the government's policy, directive and standards for evaluation. The evaluation function consists of a Chief Evaluation Executive (CEE) and four evaluator positions.

The evaluation universe (i.e., all the individual "evaluable programs") consists of 19 activities of sub-programs based on the Agency's Program Alignment Architecture (PAA) Evaluable entities are described and prioritized based on eight ratings scales (e.g., materiality, completeness of performance framework, reach of entity, degree of control over outcomes). Under policy, it is expected that each of the entities will be evaluated every five years, with evaluation priority ratings serving to help schedule the timing and the scope and scale of the evaluations.

For this planning cycle, the universe was restructured consistent with the Agency's new PAA. Evaluation priority ratings were adjusted based on consulted with senior management. For 2014-2015, the function will complete three evaluations carried over from 2013-2014, provide on-going support to interdepartmental evaluations and launch two new evaluations.

Introduction

The 2014-15 Parks Canada Evaluation Plan, consistent with the TB Evaluation Policy, outlines the mandate, organizational structure and resources for evaluation at Parks Canada, the strategy and process employed in developing the Plan, a project schedule for the five-year period from April 2014 to March 2019, and details of individual evaluation activities for the FY 2014-2015, together with the associated resource allocation.

Parks Canada Agency

Parks Canada was established as a separate departmental corporation in 1998. The Agency's mandate is to:

"Protect and present nationally significant examples of Canada's natural and cultural heritage, and foster public understanding, appreciation and enjoyment in ways that ensure the ecological and commemorative integrity of these places for present and future generations."

Responsibility for the Parks Canada Agency rests with the Minister of the Environment. The Parks Canada Chief Executive Officer (CEO) reports directly to the Minister.

Evaluation Function

Applicable Policies and Professional Standards

The evaluation function at Parks Canada adheres to the TB Policy on Evaluation, and associated directives, standards and guidelines of the Government of Canada. In 2013-2014, the charter for the evaluation function was updated.

Mandate and Services Offered

The mandate of the function is:

To contribute to the achievement of Parks Canada's mandate by providing the CEO with evidence-based, credible, neutral and timely information on the ongoing relevance, results, and value of policies and programs, alternative ways of achieving expected results, and program design improvements.

Services include:

  • Evaluation plans completed in advance of an evaluation to briefly describe an entity, its logic, inputs, outputs, reach and results and identify evaluation questions, methods and costs);
  • Evaluations of programs, policies and functions (i.e., treating the core issue of relevance and performance); and
  • Consulting projects and advice, as required, on performance measures, targets and information systems.

Follow-up on Management Responses

The evaluation cycle includes a systematic follow-up on the management responses, at six months intervals, after the final approval of the reports by the CEO. Responses are tabled at the next available evaluation committee meeting. The processes continue for five-years or until all planned actions are complete.

Governance

Evaluation Committee is now the Executive Management Committee in the Agency which is chaired by the CEO. Terms of reference for the committee were updated in 2013-2014.

The Evaluation Committee is responsible for reviewing and providing advice or recommendations to the CEO on:

  • Evaluation Function and Products: including the Agency's Evaluation Charter; the rolling Five-Year Evaluation Plan; the adequacy and neutrality of resources allocated to the evaluation function; the performance of the function; and key elements of an evaluation product lifecycle, such as terms of reference, scoping documents, evaluation reports, and management responses and action plans including following-up to ensure action plans are implemented.
  • Performance Management Framework: the adequacy of resources allocated to performance measurement in support of evaluation activities, and recommend to the CEO changes or improvements to the framework and an adequate level of resources for these activities.

Organizational Structure and Resources

Organizational Structure and Resources

The organizational chart for the function is shown at the right. The function currently consists of four funded positions. The effective staff complement for 2014-2015 is estimated to be 2.9 FTEs due to evaluators being away for parts of the year on parental leave.

Budget for evaluation in the Agency includes:

  • Part of the salary and O&M costs for the office of CAEE; typically about $34K per year.
  • Costs of the evaluation function (i.e., the salary and expenditures for the four evaluator positions).

The available budget for the evaluation function along with actual expenditures in 2013-2014 and forecasted expenditures in 2014-2015 are shown in the table below.

  Available
Budget
Expenditures Forecasted Expenditures
as % of
Available Budget
2013-2014 2014-2015
Actual Forecast
  524,730 244,685 350,000 66%
Salaries 385,430 214,355 240,000 62%
Project Costs 139,300 1,668 70,000 94%
Non Project O&M 28,662 40,000

Evaluation Planning Methodology and Considerations

Evaluation planning starts with the identification of all evaluable entities (i.e., the universe of programs or activities that may be subject to evaluation). The universe is based on the Agency's Program Alignment Architecture (PAA) which was restructured in 2013-2014. As a result the evaluation universe for the 2014-2015 Plan consists of 19 entities reflecting sub-programs in the PAA with some adjustments and modifications to amalgamate sub-programs where it makes sense and to add a few entities that are not part of the PAA structure.[1]

Each entity is described, documented and prioritized to inform the sequencing of evaluation activities over a five year period. Priority ratings for evaluation are based on ratings of the entity on eight dimensions (i.e., with a three point scale for each) adapted from the TBS Guide to Evaluation Planning (see Appendices C, D, E, and F for more details). Scheduling of entities for evaluation also takes account of external commitments to conduct evaluations[2], past or planned work of other assurance providers, senior management priorities, and evaluation capacity.

Under the TB Policy on Evaluation the Agency is required to evaluate 100% of its direct program spending over a five year policy starting with the April 2013 to March 2017 cycle. An additional requirement in the FAA is to evaluate all grants and contributions (G&C) programs every five year staring from December 2006.

For this planning cycle, descriptive information for most evaluation entities was updated but final formal priority ratings were not completed for the entities in time to inform the plan. Priorities were assessed through a series of discussions and meetings with members of Executive Management Committee and in some cases their management teams during February and March.

Planned Projects For Next Five Years

The tables below shows project schedule for the next five years, followed by details of the timing and resource requirements for the 2014-2015 fiscal year.

Coverage: All entities in the universe are planned for evaluation over the following five years consistent with the requirement noted above. The Agency's two grant and contribution programs were evaluated in 2011-2012 (i.e., the General Class Contribution Program) and 2012-2013 (i.e., the National Historic Sites of Canada Cost-Sharing Program) respectively. Both are scheduled for re-evaluation in the five year cycle starting April 2013.

Five Year Evaluation Plan
 Sub-Programs 2014-15 2015-16 2016-17 2017-18 2018-19 Last Approved Evaluation
Heritage Place Establishment
National Park Establishment and Expansion           June 2014
National Marine Conservation Area Establishment     Evaluation      
National Historic Site Designations Evaluation            
Other Heritage Places Designations Evaluation            
Heritage Places Conservation
National Parks           June 2014
National Urban Park       Evaluation    
National Marine Conservation Areas     Evaluation      
National Historic Sites   Evaluation        
Other Heritage Places Evaluation            
Law Enforcement Evaluation          
Heritage Places Promotion and Public Support
Heritage Places Promotion     Evaluation        
Partnering and Participation       Evaluation      
Visitor Experience
(NP, NUP, NMCA, NHS, Canals)         Evaluation March 2012
Visitor Safety     Evaluation        
Heritage Canals, Highways and Townsites Management
Townsites Management   Evaluation          
Highways Management Evaluation
(TCH Twinning)
    Evaluation     Jan 2011
Heritage Canals Management       Evaluation   March 2012
Grants and Contributions Programs
General Class Contribution Program     Evaluation     Jan 2011
National Historic Sites Cost-Sharing Program       Evaluation   Dec 2012

Projects for 2014-2015

Proposed timing and costs of the projects are outlined below. Estimated resource requirements are for 2014-2015 only.

Topic Requirement for Evaluation Planned or Actual Dates Resources Required
In Previous Plan Start date Comple-
tion of the fieldwork
Comple-
tion of report
Date of Approval Approx hours O&M
(000)
Total 3,985 70
Carried Over From 2013-2014
Law Enforcement TB Submission Y April 2014 Dec 2014 Feb 2015 March 2015 1,200 20
National Historic Site Designations Policy on Evaluation Y September 2013 March 2014 May 2014 September 2014 750 10
Other Heritage Place Conservation and Designations Policy on Evaluation Y September 2013 Sept 2014 Nov 2014 Dec 2014 750 10
New in 2014-15
Twinning TCH Project TB Submission Y September 2014 January 2015 Feb 2015 March 2015 500 10
Townsites Management Policy on Evaluation Y September 2014 March 2015 May 2015 September 2015 750 20
Contributions to Interdepartmental Evaluations for 2014-15
Climate Change Adaptability Evaluation led by EC that will include nine departments funded for climate change adaptation. PCA is expected to have a small role in the evaluation. Y June 2014 TBD March 2016 TBD 35  

Appendix A. Approval Schedule

Identifier Link to PAA #(s) Title of the Evaluation Planned Deputy Head Approval Date
(M/Y)
 
Estimated G&C Value ($)7
Estimated Value (Including Gs&Cs)
($)
 Sub Total  552,662,971
 Internal Services 59,800,163
Grand Total 612,463,134
2014-2015
FY1.1 P 1 NP Establishment and Expansion June 2014   11,635,581
FY1.2 P 2 National Parks Conservation June 2014   94,302,957
FY1.3 P 2 Law Enforcement March 2015   ?
FY1.4 P 1 National Historic Site Designations October 2014   2,991,194
FY1.5 P 1
and 2
Other Heritage Place Conservation and Designations December 2014   13,717,936
FY1.6 P 5 Twinning TCH Project March 2015    
2015-2016
FY 2.1 P 5 Town sites Management Sept 2015   7,247,704
FY2.2 P 2 National Historic Sites Conservation March 2015   52,036,141
2016-2017
FY3.1 P 1 and P 2 National Marine Conservation Area Establishment and Sustainability March 2017   3,164,631
FY3.2 P 3 Heritage Places Promotion September 2016   28,402,733
FY3.3 P 4 Visitor Safety September 2016   ?
FY3.4   GCCP March 2017 ? ?
2017-2018
FY4.1 P 2 National Urban Park Conservation March 2018   3,598,748
FY4.2 P 5 Heritage Canals Management March 2018   14,004,786
FY4.3 P 2 National Historic Sites Cost-Sharing Program March 2018 ? ?
FY4.4 P 3 Partnering and Participation September 2017   11,054,507
FY4.5 P 5 Highway Management September 2017   72,208,042
2018-2019
FY5.1 P 4 Visitor Experience March 2019   238,298,011

Estimated value from 2014-2015 Main Estimates

Appendix B. Priority Assessment Dimensions and Scales

Dimension Score
4 2 0
TB Commitments Required in the next 12 to 18 months Required but not in the next 18 months None required
Materiality >10% (approximately 60+ million) 5% to 10% (approximately 31 to 60 million) <5% (approximately 30 million)
Links to Corporate Risk Profile Links primarily to high priority corporate risks Links to primarily lower priority corporate risks No links to corporate risks
Activities linked to the 2013-14 key corporate risks: competitive position, natural disasters, environmental forces, asset are rated four. Activities related to other corporate risks are rated a moderate and activities not related to the risk profile are rated one.
Completeness of Performance Framework None or few elements of the framework in place Partially complete Complete
A complete framework consists of defined measurable goals and objectives, baseline measures of performance, quantifiable targets with clear time frames for accomplishment of goals and systems to measure and report on progress and goal attainment, and evidence of monitoring and reporting.
Extensiveness of Program Reach Extensive reach to communities, stakeholders, NGOs, Aboriginal peoples, and the public. Moderate and/or regional-level reach to communities, stakeholders, NGOs, Aboriginal peoples, and the public. Limited and/or localized reach to communities, stakeholders, NGOs, Aboriginal peoples, and the public.
High intended direct reach is typified by activities related to building awareness and understanding the Agency and its mandate and promotion and marketing Parks Canada sites as well as the visitor experience program which are intended to reach millions of Canadians and international visitors. Low reach is typified by sub-programs in the Other Heritage Places Designation sub-program such as Grave Sites of Prime Ministers which is effectively targeted at a few families of former prime ministers whose grave sites are not yet formally commemorated. When the target reach of a program are organizations, or provinces, as in park establishment for example, we count reach as the number of groups targeted and not the size of the constituencies represented by these groups. Most program activities have ultimate beneficiaries i.e., Canadians as a whole, who are not counted as the program or sub-program reach.
Degree of Direct Control Over Outcomes Low Direct Control Moderate Direct Control High Direct Control
Low control over outcomes is exemplified by the national park and national marine conservation areas establishment and expansion sub-programs, which require extensive consultation and negotiations over many years with dozens of different stakeholders, who differ in their capacities and interests, and have the capability to block a particular establishment process. More control is available over a contribution program where the Agency, with TB agreement, has set the terms and conditions for receiving funding and evaluates and recommends who will be funded. An intermediate example might be conservation in national parks and NMCAs where the Agency may have a relatively high degree of control over what occurs within the boundaries of the park but is also interested in influencing regional land use practices that impact on the park's ecological integrity.
Public's Importance of Health and Safety Considerations in Program Delivery High Level of Consideration of health and safety issues in delivery of a sub-program. Moderate Level of Consideration of health and safety issues in delivery of a sub-program. Low Level of Consideration of health and safety issues in delivery of a sub-program.
Many activities involving visitors require consideration of health and safety issues as a fundamental part of the program delivery. Examples include the potential for human wildlife conflicts in national parks, possibilities of contamination when providing potable water, the potential of accidents on highways managed by the Agency, and the potential for accident or injury when conducting law enforcement or search and rescue activities. We do not assess the nature or quality of management measures to mitigate health and safety issues involved in sub-program delivery only whether and the extent to which these considerations have been inherent in delivery of the sub-program.
Public Interest and Sensitivity High Moderate Low
Activities which have received recent public or political attention are rated higher (i.e., the lead up to the decision to arm park wardens and the new law enforcement program had extensive media coverage but this has largely abated since the new program began operating). Introduction of new legislation such as the Heritage Lighthouse Act, creates temporary political interest in a particular activity (the Act would protect heritage lighthouses in Canada and is considered part of the Other Heritage Places Designation sub-program). Some consideration is also given to potential for public or political interest. Many of the health and safety concerns reviewed above have high potential interest should they occur (e.g., the failure of a dam or a potable water system resulting in a significant number of injuries or deaths).

Appendix C. Corporate Risk Profile 2014-2015

Risk Category and Label Risk Statement Risk Owner
Public
Aboriginal Engagement A decrease in Aboriginal engagement with Parks Canada may impact the Agency's ability to deliver on and advance its programs. Director, Aboriginal Affairs Secretariat
Partnering Parks Canada may not be able to effectively collaborate with potential partners due to internal capacity (such as deficiencies in financial authorities) or external factors. This could limit our ability to leverage opportunities, extend our reach, grow our base of support, and advance our programs. VP, External Relations and Visitor Experience
Public Awareness and Support Local communities, stakeholders, NGOs, and the Canadian public may not be sufficiently aware or supportive of Parks Canada, compromising the Agency's ability to fulfill its mandate. VP, External Relations and Visitor Experience
Socio-economic
Competitive Position Parks Canada may fail to attract visitors if it does not maintain a strong competitive position within the tourism industry and respond to the changing needs and expectations of visitors. VP, External Relations and Visitor Experience
External Development Pressures Development pressures may limit opportunities for establishment of new national parks and national marine conservation areas, affect the ecological integrity of national parks and the ecologically sustainable use of national marine conservation areas, as well as impact commemorative integrity at Parks Canada's national historic sites in urban areas. VP, Protected Areas Establishment and Conservation; VP, Heritage Conservation and Commemoration
Environmental
Disasters Natural and human-originated disasters may impair or destroy critical infrastructure and lead to significant unforeseen expenses, serious injury, loss of life and the permanent loss of assets of national significance. Chief Administrative Officer; VP, Operations, Eastern Canada; VP, Operations, Western and Northern Canada
Environmental Forces Environmental forces such as habitat changes, exotic/invasive species may limit the Agency's ability to make improvements in ecological integrity in national parks and meet legal requirements related to Species at Risk. VP, Protected Areas Establishment and Conservation
Parks Canada's Business Operations
Asset Condition Assets are continuing to deteriorate with the result that more than 45% of the Agency's built assets are in poor to very poor condition. Chief Administrative Officer
Information Management Failure to identify, capture, manage, share and report pertinent data, plus maintain security of information and knowledge, may hinder the ability to effectively manage all program areas and meet legal requirements. Chief Administrative Officer
Source: Parks Canada Agency Corporate Risk Profile 2014-15

Appendix D. Past Coverage of the Evaluation Universe (April 2009 to March 2014)

Program and Sub-Programs Parks Canada Evaluations and Interdepartmental Evaluations Work of External Assurance Providers
Heritage Places Establishment
National Park Establishment and Expansion
  • Evaluation of Parks Canada's National Parks Establishment and Expansion 2014
 
National Marine Conservation Area Establishment  
National Historic Sites Designation    
Other Heritage Places Designations    
Heritage Resources Conservation
National Parks Conservation
  • Evaluation of Parks Canada's National Parks Conservation 2014
  • Interdepartmental Evaluation of the Programs and Activities in Support of the Species at Risk Act August 2012
National Marine Conservation Areas Sustainability
  • Interdepartmental Evaluation of the Health of the Oceans (HOTO) Initiative 2012
National Historic Sites Conservation    
Other Heritage Places Conservation    
Law Enforcement    
Public Appreciation and Understanding
Heritage Places Promotion    
Partnering and Participation    
Visitor Experience
Visitor Experience (NP/NUP/NMCA/NHS/Canals)  
Visitor Safety    
Town-Site and Throughway Infrastructure
Townsite Management    
Through Highways Management  
Through Waterways Management  
Other
Grants and Contributions Programs  
Other Functions  
* indicates an interdepartmental evaluation

Appendix E. Agency RMAF Evaluation Commitments 2014-2015

Horizontal Evaluations Parks Canada
Evaluation of Climate Change Adaptation: This evaluation, to be led by EC, will include nine departments that have received funding for climate change adaptation. Parks Canada is expected to have a small role in the evaluation. Planning for this work will begin in 2014-2015 with a target to finish the work in 2015-2016. Evaluation of the Law Enforcement Program: The program, involving up to 100 armed law enforcement officers responsible for enforcement of laws and regulations in the Agency's protected heritage places (excluding criminal code enforcement) was funded and developed in 2008-09 with on the ground activities commencing in 2009-10. The program will have start-up costs of $8.5M in 2008-09 and ongoing costs of $2.3M per year thereafter (i.e., less than one percent of the Agency's annual spending). A summative evaluation is planned in 2013-2014.
Evaluation of the Twinning of the Trans-Canada Highway in Banff National Park: The recent initiative (i.e., 2004-05 on) involves twining 32 kilometres of the TCH at a total cost of $317M over 10 years.

[1] These are the Law Enforcement Program and the Agency's grants and contributions programs.

[2] There are commitments in TB submissions to conduct evaluations as a condition of receiving funding.

Top of page