Parks Canada
Symbol of the Government of Canada

Common menu bar links

Internal Audit and Evaluation Documents

Evaluation of Parks Canada's National Historic Sites Cost-Sharing Program

5 November 2012

Office of Internal Audit and Evaluation

Report submitted to the Parks Canada Evaluation Committee: 22 November 2012
Approved by the Agency CEO: 07 December 2012

Her Majesty the Queen in Right of Canada, represented by the Chief Executive Officer of Parks Canada, 2012

Catalogue No.: R62-433/2013E-PDF
ISBN: 978-1-100-21860-1

Table of Contents

Executive Summary

1 Introduction

2 Program Description

3 Evaluation Design

4 Evaluation Findings

5 Conclusions and Recommendations

Appendix A. Strategic Outcome and Program Activity Architecture

Appendix B. Terms and Conditions of the NHSCCSP

Appendix C. Evaluation Questions, Expectations, Indicators and Data Sources

Appendix D. Key Documents Reviewed

Executive Summary

The National Historic Sites Cost-Sharing Program (NHSCSP) is one of two contribution programs at Parks Canada. Yearly contribution payments have averaged approximately $5.0M between FY2009-2010 and FY2011-2012. The program represents less than 1% of the Agency's direct yearly program spending.

The evaluation responds to a requirement in the Treasury Board (TB) Policy on Evaluation[1] and section 42.1 of the Financial Administration Act, to evaluate all grants and contributions programs over a five-year period. The evaluation is also intended to support the renewal of the current Terms and Conditions (T&Cs) of the program which expire at the end of March 2013. If the program is not adequately managed, there are risks that funds will be wasted on projects that are not aligned with and supportive of the Agency's mandate and overall government priorities and that results will not be achieved, or not achieved efficiently and economically.

Evaluation Issues

Consistent with the requirement of the TB Policy on Evaluation and associated directives, the evaluation addressed:

  1. Relevance: Does the NHSCSP serve an important need or function and is it aligned with Agency and government roles and priorities?
  2. Performance: Have NHSCSP funded projects been implemented and produced the outputs and outcomes intended? Is the process timely and effective and are administrative costs of the program efficient relative to outputs and outcomes produced?
  3. Design and Delivery: Are roles, responsibilities, program structures and processes clear and appropriate?

Methodology

The methodology chosen for the evaluation relied on triangulating data from multiple sources. In order to gain a full understanding of the Program we relied on key informant interviews, document review and analysis from the program database. Key informant interviews were structured around a questionnaire that allowed a semi structured discussion to take place. The document review concentrated on the state of built heritage in Canada, financial support of heritage and policy/legislative requirements. A specific review of the cost-sharing program's database of applications was also completed.

Findings

The NHSCSP is operating as designed with few areas for improvement. It is clear that funds are being distributed to eligible sites and that controls are in place to ensure that the agreed to conservation, preparatory assistance and presentation work is completed. The only areas of weakness in the program are monitoring eligible owner satisfaction with the program and whether service standards are being met. However, even with these minor deficiencies there is general confidence that projects completed through the NHSCSP funding are having a positive effect on the state of heritage in Canada.

Recommendations

Recommendation 1

The recipient survey should be redesigned to strengthen analysis for individual questions and it should be administered such that anonymity is protected. To be more comprehensive, this could also include a survey of applicants not recommended for funding. Additional questions could also be included to address perceived needs of applicants and barriers to application.

Management Response

Agree: The program management agrees that the use of an anonymous end-of-project questionnaire to obtain feedback from applicants could result in useful information on the needs of national historic sites' owners.

While maintaining project satisfaction questions with recipients to help measure program results, program management will develop additional questions that will target program issues relevant to all applicants. Measures to protect anonymity will be examined in order to ensure that the chosen method complies with all applicable federal regulations, is transparent and results in clear and usable responses. The new questionnaire will be used as soon as the final stage of the next intake of projects is reached. The responses to the questionnaire will be compiled and used to analyse the program's direction at the end of each intake.

Recommendation 2

The service standards for the program should be tracked in such a way that the program can determine whether these are being met. This data should be used to review and adjust the program as required in order to meet or revise service standards.

Management Response

Agree: The current service standards are, in large part, based on existing standards in use across the Federal Government for grants and contributions programs. The Program's current tracking of service standards focuses mainly on ensuring that:

  1. Decisions are clearly communicated to applicants - within 6 months - and if this is not possible, that applicants are informed of any delays encountered within that same timeframe period;
  2. Contribution Agreements are signed within a 90 day timeframe; and
  3. Only eligible applicants are retained for further assessment of their proposal; non-eligible applicants are informed of their ineligibility.

Program management will therefore ensure that relevant service standards are tracked, and that the tracked information is compiled and assessed on an intake-by-intake basis. This new tracking model will start with the official announcement and posting of the next intake of projects on PCA's Webpage. In addition, the Program's 1-800 line will be maintained to ensure accessibility to Program staff during regular operating hours.

1 Introduction

The Parks Canada Agency's (PCA) mandate is to:

Protect and present nationally significant examples of Canada's natural and cultural heritage, and foster public understanding, appreciation and enjoyment in ways that ensure the ecological and commemorative integrity of these places for present and future generations.

The Agency is responsible for three major heritage systems:

  • 44 National Parks of Canada
  • 167 National Historic Sites of Canada administered by the Agency
  • 4 National Marine Conservation Areas of Canada

Parks Canada carries out its mandate through five program activities (PA), 20 sub-activities and 3 sub-sub-activities. The major program activities are heritage places establishment, heritage resources conservation, public appreciation and understanding, visitor experience and townsite and throughway infrastructure (see Appendix A for the Program Activity Architecture - PAA).

The focus of this evaluation is the Agency's National Historic Sites Cost-Sharing Program (NHSCSP), one of two contribution programs in the Agency. The Agency's evaluation plan identified the NHSCSP as a low priority for evaluation work based on factors such as its low materiality (the program accounts for less than 1% of the Agency's annual direct program spending), its strong performance framework, and low corporate risk. However, an evaluation of the NHSCSP was identified as a commitment in the Agency's 2011-2012 Evaluation Plan[2] in keeping with the evaluation policies of both TB and PCA and with section 42.1 of the Financial Administration Act, to evaluate all grants and contributions programs over a five-year period. The evaluation is intended to support the renewal of the current Terms and Conditions (T&Cs) of the program, which expire at the end of March 2013.

2 Program Description

National historic sites are places designated by the Minister responsible for Parks Canada on the recommendation of the Historic Sites and Monuments Board of Canada (HSMBC) as places of national historic significance to Canada. To date, 964 historic places have been designated as national historic sites. These sites, located in all provinces and territories, can be found in almost any setting - from urban and rural locales to wilderness environments. They may include but are not limited to sacred spaces, battlefields, archaeological sites, and buildings. Many are still used today for work, worship, commerce, industry, habitation and leisure. They can range in size from a single structure to a historic district encompassing hundreds of resources. Parks Canada is responsible for the administration of 167 of these sites.

The National Historic Sites Cost-Sharing Program assists recipients in conducting activities aimed at ensuring the commemorative integrity of non-federally owned or administered national historic sites. Specifically, the Program provides financial contributions to eligible recipients that share the costs of works necessary to ensure the physical health of a national historic site and to ensure Canadians understand the importance of the site and its role in the history of Canada.

2.1 Outcomes/Goals

Parks Canada's Performance Management Framework was updated in September 2012. The most recent corporate hierarchy of outcomes and targeted performance expectations related to the National Historic Sites Cost-Sharing Program is shown in Table 1.

Table 1. Corporate Performance Expectations and Targets related to NHSCSP
Level Expected Result Performance Expectation
Agency Strategic Outcome Canadians have a strong sense of connection, through meaningful experiences, to their national parks, national historic sites and national marine conservation areas and these protected places are enjoyed in ways that leave them unimpaired for present and future generations. (None specific to NHSCSP)
Other Heritage Places Conservation

(Sub-Activity)
Parks Canada programs support the conservation of cultural resources at historic places administered by others. 100% of Parks Canada advice promotes the conservation of significant cultural resources at historic places administered by others and is targeted to areas most in need.
National Historic Sites Cost-Sharing (Sub-sub Activity) Funded cost-sharing projects in the conservation of national historic sites not owned or administered by the government. 100% of cost-sharing agreements contribute to the conservation of cultural resources of national significance (level 1) within national historic sites owned or administered by others.
Source: Adapted from PCA Performance Management Framework, 2012-13

The NHSCSP is intended to assist recipients in conducting activities and delivering projects that will support the Parks Canada Agency in fulfilling its mandate to protect and present nationally significant examples of Canada's cultural heritage and foster public understanding, appreciation and enjoyment in ways that ensure their commemorative integrity for present and future generations. A national historic site possesses commemorative integrity when it is healthy and whole, and when the site's heritage values are protected, communicated and respected.

Specific expected results and outcomes to be achieved by the program as outlined in its T&Cs include the following:

Table 2. Results and Outcomes of NHSCSP
Immediate Results & Outcomes Intermediate Results & Outcomes Strategic Results & Outcomes
  • Non-federally owned national historic sites receiving funding are conserved, thus reducing the threat to their commemorative integrity;
  • Approved conservation projects are carried out according to the Standards and Guidelines for the Conservation of Historic Places in Canada;
  • The reasons for a site's designation as a national historic site are effectively communicated to the public in both official languages.
  • The federal government provides leadership in creating a culture of conservation;
  • Conservation of national historic sites to national standards through the mandatory use of the Standards and Guidelines for the Conservation of Historic Places in Canada;
  • Reduced threat to commemorative integrity at non-federally owned national historic sites.
  • Canadians feel a greater sense of connection to each other, their history, identity and historic places;
  • More historic places are conserved and given a function in the life of communities (United Nations Educational, Scientific and Cultural Organisation, UNESCO);
  • National historic sites provide improved access to significant, diverse cultural heritage.

The program also has service standards as required under the 2008 TB Policy on Transfer Payments. The specific standards pertain to:

Availability:
hours of operation, service in both official languages.

Promptness:
retuning calls, acknowledging e-mails, providing basic information within two business days, and responding to letters or faxes within five business days.

Acknowledgement:
of applications received within 15 calendar days.

Eligibility:
advising applicants in writing within 30 business days of the eligibility of the application.

Decision:
issuing written notice of a funding decision within six months of the application deadline or informing applicants of any additional required processing time.

Contribution Agreement:
conclusion within 90 days of the written notification.

Payment:
issuing of payments within 28 days of fulfillment of requirements.

 

The program's performance target is to meet these standards for 90% of files.

2.2 Activities and Outputs

There are three key activity areas associated with the NHSCSP. These activities are:

  • Application - Establishment and communication of application criteria and deadlines;
  • Approval - Review of applications to ensure they meet program requirements and that the work being proposed is for conservation of threatened resources that relate directly to the commemorative integrity of the site; and
  • Monitoring - Ensuring that contribution agreement requirements are followed.

More detail on each of these key activities is provided below.

2.2.1 Application

Application requirements, including a description of eligible recipients and project types, are outlined in Parks Canada's National Historic Sites Cost-Sharing Program Guidelines, available on the Agency's website. Projects can be eligible in one of two categories:

Table 3. NHSCSP Project Types
Category Title Description
I Preparatory Assistance Projects Professional and technical services needed to prepare documents required by a national historic site to aid in its planning for conservation and presentation work, as well as site management. These may include such documents as a conservation plan, a management plan, a commemorative integrity statement or a building/site condition assessment, and may require associated architectural and engineering services, technical drawings or photography, historical research, archaeological research/investigation, environmental assessment, fire safety plan, maintenance and inspection plans and manuals, and interpretive plans.

Eligible costs for Category I will be reimbursed up to the lesser of $25,000 or 50% of total eligible costs.
II a) Conservation Projects Materials and labour directly related to the conservation of a national historic site. The conservation work must lead directly to the protection of cultural resources or character-defining elements, in particular the reduction of threat and/or level of impairment. This work may involve preservation measures to protect, shelter, reinforce and stabilize cultural resources, or more invasive conservation measures involving limited restoration of deteriorated parts, or replacement of missing or deteriorated parts, including fittings, machinery, landscape elements, or other components that Parks Canada agrees are integral to the commemorative integrity of the national historic site.

Work may also include fire detection and suppression systems where Parks Canada agrees that these are the most appropriate means of safeguarding the national historic site or its cultural resources from catastrophic fire loss.

Eligible costs will be reimbursed up to the lesser of $425,000 or 50%.
b) Conservation Projects with a presentation component Same as above, with the addition of a component for the implementation of a presentation project to communicate to the public the reasons for the place's designation as a national historic site.

An eligible presentation component will be reimbursed up to the lesser of $25,000 or 50% of total eligible costs.

A Funding Application Form, Project Budget Form, and Project Timeline Form[3] have also been developed by the Agency and are required from applicants to ensure a complete application. Applicants are also encouraged to contact the Program Office prior to preparing an application.

2.2.2 Approval

Parks Canada assesses applications based on eligibility, provided that applications meet the application deadline and that all required information is present. The approval process can take up to 6 months. If the project meets all the assessment criteria, applicants will be contacted with the decision on whether they will be receiving financial assistance. The CEO of Parks Canada provides official written confirmation to funding recipients, while the Program Director informs unsuccessful applicants.

2.2.3 Monitoring

Managers are expected to monitor projects following a risk-based plan developed during the negotiation phase of, and included in, each agreement. They are to maintain files with copies of reports, evidence of results achieved, and details of project expenses and payments.

Overall monitoring of conservation work is by default done through the certification process. Certification is an important monitoring mechanism for the program. In order for recipients to receive the full amount specified in the project's cost-sharing agreement, a Parks Canada Certification Officer (or designate) must certify that work completed meets required standards established at the beginning of the project.

Program Officers ensure that recipients fulfill the full scope of the terms and conditions of the funding agreement, including the financial expenditures and recipient reporting aspects.

2.3 Expenditures

In 2008, the Evaluation of Issues Related to the National Historic Sites of Canada Cost-Sharing Program outlined the program's expenditure history. The NHSCSP was initially launched in 1987. Between 1987 and 2000, the Program entered into contribution agreements with 53 different NHSs for approximately $26.1 M. Between April 2000 and March 2006, no new agreements were signed as funds from the NHSCSP budget were already committed to existing contribution agreements, were used for other related heritage conservation projects, or were reallocated to support other priority conservation projects within the Agency. During this period, the Program funded 15 projects under cost-sharing agreements signed prior to 2000. In June 2007, the terms and conditions of the Program were extended until October 2008 with only one project funded.

By Treasury Board approval, the most recent round of funding for the NHSCSP extends from May 2008 to March 2013. The scope of this evaluation is all funding from April 2009 to March 2012. The projects that received funding during FY2007-08 and FY2008-09 were approved in the previous version of the NHSCSP. Since they did not have to meet the guidelines of the current program in place during the period being evaluated, these fiscal years were not included in our analysis.

Table 4 shows the administrative and contribution amounts of the Program from FY 2009-2010 to FY 2011-2012. In FY2009-2010, the resources available for projects grew significantly as a result of Canada's Economic Action Plan (CEAP). The CEAP resulted in an additional $8 M over two years distributed towards work done at national historic sites through the NHSCSP.[4] In FY2009-2010, all funding for the NHSCSP was from CEAP. This was a short-term infusion of funds; the most recent round of funding was limited to $1.0M for projects taking place in FY2012-2013, indicating a return to a lower level of funding for 2012-2013.

Table 4. Administration, Certification and Contribution Expenditures
Program Administration and Certification
  2009-10 % 2010-11 % 2011-12 % Total %
Salaries $323,990 10% $447,638 5% $402,520 9% $1,174,148 7%
Goods & Services $81,434 3% $33,033 0% $20,365 0% $134,832 1%
Certification[5] $143,974 4% $237,996 2% 56,479 1% $438,449 3%
Total Admin. $405,424 12% $480,671 5% $422,885 9% $1,308,980 8%
Total Overhead $549,398 19% $718,667 8% $479,364 11% $1,747,429 11%
Contribution $2,704,913 83% $8,828,309 92% $3,619,420 88% $15,152,642 89%
Total Program $2,854,913   $8,978,309   $3,769,420   $15,602,642  
Source: PCA Finance (via Program)

For the purposes of program management, this renewed funding fell into three intakes commonly referred to as Intake 1 (2009-2010), Intake 2 (2010-2011) and Intake 3 (2011-2012). The amount of funding approved by the CEO of PCA in these intakes (including EAP money) totals $15,152,642. During this time, 102 contribution agreements were signed with 91 historic sites. As of March 2012, this included 18 projects funded under Category I, 76 under Category II(a), and 10 under Category II(b). The value of these contribution agreements ranged from $678 to $425,000, with an average value of $147,783.

2.4 Reach

The Program is intended for national historic sites designated by the Minister responsible for Parks Canada. Applicants must be an owner or eligible lessee of a national historic site (or a part thereof) or of a contributing property within a national historic district. In addition, eligible applicants must be one of the following:

  • A not-for-profit organization;
  • A provincial, territorial, regional or municipal government;
  • A not-for-profit organization acting on the authority and behalf of an eligible owner or lessee;
  • A not-for-profit Aboriginal organization with a formalized stewardship responsibility directly related to all or part of a national historic site; or
  • A not-for-profit Aboriginal organization acting on the behalf of an eligible not-for-profit Aboriginal organization.

Parks Canada's Heritage Conservation and Commemoration Directorate (formerly National Historic Sites Directorate) maintains a database of owners of national historic sites from which it is possible to estimate the overall number of eligible sites and the number by each type of owner. However, it should be noted that eligible funding recipients and site owners are not always equivalent. For example, there are 167 NHS administered by Parks Canada but two of these were awarded funding because a portion of the designated place belonged to a non-federal owner. The non-federal partner at these sites is eligible for funding under the program, even if Parks Canada is not.

The 2008 Evaluation of Issues Related to the National Historic Sites of Canada Cost-Sharing Program estimated the eligible population of national historic sites to be 356 when the program was launched in 1987 and 605 in 2008 (i.e., a 70% increase). At the time, a group of 44 sites were also identified where eligibility was uncertain, including 21 sites where the boundaries of the site are not well defined and/or owners are not known and 23 historical districts or complexes with multiple owners (i.e., potential hundreds or thousands of individual owners according to program personnel).

In fact, individual owners of these sites may be eligible for funding as long as the proposed work is consistent with program objectives and the owner falls in an eligible category. This is assessed on a case-by-case basis as funding applications are received. In some cases, clarifications may be requested from the HSMBC's Status of Designation Committee (SDC) before a recommendation for funding is made. Six owners of contributing properties within historic districts or complexes were funded during the evaluation period.

In addition, the 'Other federal' category identified in 2008 included HSMBC plaques owned by Parks and other sites related to Parks Canada but not identified as administered by the Agency. Since then, the HSMBC's SDC has completed a review of the status of many of these sites. If they are still considered a NHS, they are now listed under the appropriate category (i.e., as owned by PCA or other federal government departments/agencies).

Table 5 below summarizes our 2008 data and our current understanding of eligible owners by type of owner or site.

Table 5. National Historic Site Ownership and Eligibility for NHSCSP
Type of Owner Totals by type
2008
Totals by type
2012
Sites not eligible for cost-sharing
Parks Canada Agency 157 165
Other federal government departments/agencies 53 65
Other federal 26 0
Under review - 2
Private ownership by individuals or corporations 40 84
Total ineligible population 276 316
Sites may be eligible for cost-sharing
No owner type identified / Designated place is questionable 21 21
Historic districts or complexes 23 24
Total population that may be eligible 44 45
Sites eligible for cost-sharing
Aboriginal group/band 49 75
Educational institution 14 20
Historical society 68 67
Not-for-profit organizations 75 34
Municipal government 174 167
Provincial/territorial government 124 153
Other governmental 11 2
Religious groups 90 85
Total Eligible Population 605 603
Total NHSC 925 964
Source: HCC Directorate Database

2.5 NHSCSP Logic Model

The logic model showing the relationship between inputs, activities/outputs, reach, and intermediate and long-term outcomes is shown in Table 6. The model is a visual summary of the program description.

Table 6. NHSCSP Logic Model
Inputs Activities Outputs Reach Immediate
Results
Intermediate
Results
Long Term
Results
  • Financial ($15.6 M over 3 years)
  • Agency staff
  • Develop strategies, plans, guidelines, tools and templates
  • Receive and evaluate application for funding, record decisions and notify applicants of results
  • Monitor program and project implemen-
    tation, disburse funds, receive and analyze reports of progress, maintain statistics
  • Outreach and engage-
    ment strategy
  • Program guidelines
  • Application form and model agreement
  • Program monitoring templates
  • Records of applications received, decisions made and notification
  • Contribution agreements signed
  • Funds disbursed, program and project reports and statistics
  • Owners and lessees of federally recognized national historic sites not owned or adminis-
    tered by PCA
  • Canadians interested in national historic sites
  • Eligible owners are aware of the program, make inquiries and submit applications
  • Eligible owners are satisfied with the program, the information/
    tools/
    support and application process and believe it adds credibility and legitimacy to their project
  • Recipients are able to undertake work they would otherwise have been unable to afford in whole or in part and/or the timing of a project is advanced
  • Recipients respect the terms and conditions of their agreements
  • Federal leadership is ensured in the protection of cultural resources not owned or adminis-
    tered by PCA
  • Nationally significant examples of Canada's cultural heritage, not owned or adminis-
    tered by the federal government, are maintained and improved.
  • Canadians feel a greater sense of connection to each other, their history, identity and historic places
  • More historic places are conserved and given a function in the life of communi-
    ties

3 Evaluation Design

The evaluation examined the relevance, performance (e.g., effectiveness, efficiency and economy), and design and delivery of the program consistent with the requirements of TB Evaluation Policy (2009). The scope included all NHSCSP contribution agreements involving payments between April 2009 and March 2012. Parks Canada Agency evaluation staff conducted the evaluation between September 2011 and July 2012, using multiple methods of data collection to address the evaluation questions.

3.1 Approach and Questions

The evaluation is based on a review of documents, key informant interviews and program data. The specific methodologies and questions were originally identified in the Integrated Results-based Management and Accountability Framework (RMAF) / Risk-Based Audit Framework (RBAF) for the NHSCSP prepared in 2008 at the time of last renewal of the program's T&Cs. Some questions and methods have been updated to conform to current TB Evaluation Policy requirements. In keeping with evaluation policy, the evaluation approach was calibrated to the low materiality and the low level of program risk identified in evaluation planning. Insofar as the NHSCSP has documented T&Cs, including high-level expected outcomes, the evaluation was goals-based and designed to assess the extent to which the program is meeting its objectives.

The evaluation addressed 8 key questions and 17 associated expectations. The key questions are shown in Table 7. A more detailed matrix of evaluation questions, expectations, indicators and relevant data sources is shown in Appendix D.

Table 7. Evaluation Questions
Relevance
1. Is there a demonstrated need and demand for the NHSCSP?
2. Does the NHSCSP duplicate programs offered by other levels of government or by other granting agencies?
3. Is the NHSCSP aligned to Government of Canada priorities and Parks Canada's mandate and objectives?
Performance
4. To what extent has the program been successful in meeting its expected results?
5. Did NHSCSP projects meet the program funding requirements?
6. Is the administration of the NHSCSP cost-effective and efficient?
Design and Delivery
7. Are the roles, responsibilities and governance structure of the program clear and appropriate?
8. Are the Terms and Conditions of the NHSCSP and clauses in contribution agreements appropriate?

3.1.1 Methodology

Data from multiple lines of evidence was collected for the evaluation. These included document review, key informant interviews and file data review.

Document Review: The documents reviewed included TB policies, directives, legislation and the approved T&Cs for the program. There was also a review of a recent internal audit (2011) and the last evaluation (2008) of the program. In addition to Agency documents, a review was done of T&Cs of similar contribution programs in other departments, evaluations of contribution programs and state of heritage reports federally, provincially and for non-government organizations. Academic and industry literature were also taken into account when developing our understanding of the state of national historic sites. The document review allowed a general understanding of contribution programs and the state of heritage in Canada to be developed and of how the Cost-Sharing Program is intended to function.

Key Informant Interviews: We conducted semi-structured interviews with six program staff and two staff from an NGO to gain an overall understanding of the program, approval process, strengths and issues. Key informants were chosen based on their involvement with the program or expertise in the field of historic site conservation. The interviews followed but were not limited to a questionnaire sent to subjects prior to the interview date.

File data review: The file review focussed on the data available on the contribution agreements rather than completeness of the files since an internal audit had been completed in 2011. A total of 312 projects are contained in the database which was still under construction at the beginning of the evaluation.

3.1.2 Limitations

The major limitation of the evaluation is the limited information available from the eligible program population about either the state of the places they manage or their awareness and views of the program. The decision not to solicit the views of the eligible population, as was done in 2008, was in keeping with the risk-based scoping of the project. Some information was available from secondary source data either in the form of surveys of recipients by program staff or in the form of third party surveys of those involved in heritage conservation in Canada. These surveys are also subject to specific limitations which are reviewed in the relevant sections of this report.

A second limitation involved evaluator access to the primary data on applicants and recipients. The program uses database software to manage all recipient files. The software is an improvement on the previous paper-based process but does not allow data to be exported to other programs. In consequence, the evaluation staff had to rely on the program to provide much of the information on costs and contribution agreements used for the evaluation.

Recipient audits were identified in the RMAF/RBAF for the program as one way of assessing effectiveness of funding. However, no recipient audits have been done and so this information is not available. To address this limitation, the evaluation relied on the results of Parks Canada's National Audit of Parks Canada's National Historic Sites Cost-Sharing Program (2011).

4 Evaluation Findings

4.1 Relevance

Question 1 Indicators
Is there a demonstrated need and demand for the NHSCSP?
  • Evidence from previous reports, surveys and studies indicate external funding is required to maintain sites
  • Reported consequences if the program were not available
  • Interest in the program from owners

Expectation(s):

  • The non-federally owned NHSs continue to require financial support to carry out conservation work.
  • The heritage community and other levels of government are aware of the threats and interested in their conservation.
  • Canadians support efforts to reduce threats to non-federally-owned national historic sites.
  • There is a continued demand by the heritage community for funding from the program.

The World Heritage Convention recognizes that, internationally, cultural heritage is threatened with destruction and that protection of this heritage at the national level often remains incomplete because of the scale of the resources which it requires.

Empirical evidence of the state of cultural heritage in Canada, including national historic sites, is generally lacking. In 2003, the Office of the Auditor General (OAG)'s report on the Protection of Cultural Heritage in the Federal Government[6] found that Parliament received incomplete information on cultural heritage. In general, most federal departments and agencies reported some data on the condition and possible risks to cultural heritage but not on the extent of conservation problems or detailed information on conservation results. In particular, the OAG found that there is no information on the condition of the national historic sites administered by third parties.

The most recent evidence concerning national historic sites not owned or administered by Parks Canada comes from a survey completed for the 2008 evaluation, which found that owners of national historic sites generally believed their sites were in fair or good condition (46% and 45%, respectively). However, further probing revealed that, of owners who thought their site was in good condition, 31% reported at least some compromised resource. Collectively, 69% of owners in this survey recognized that there are some compromised resources.

The key informants for the evaluation believe that built heritage in Canada is under threat. They draw parallels to the Agency's own challenges in maintaining its sites and note that even with the necessary resources, conserving heritage is an on-going effort as environmental conditions require regular work. They believe that it is unrealistic to expect that non-federally owned or administered national historic sites, with generally fewer resources than the Agency, would be doing better. Some key informants also believe that if the funding program was removed, sites would be negatively impacted as maintenance would be deferred resulting in more costly repairs.

In 2007, the OAG's Federal Built Heritage[7] report found that the state of federal built heritage sites is declining and conservation is not a priority. Financial resources for conservation were not growing commensurate with the number of new sites being designated. A 2012 survey by the Heritage Canada Foundation of its members also found that 64% of respondents believed access to funds and incentives for heritage rehabilitation was a significant or very significant issue. However, the specific reasons for their concerns are not clear.[8]

Question 2 Indicators
Does the NHSCSP duplicate programs offered by other levels of government or by other granting agencies?
  • Extent and nature of other funding sources
  • Existence of other appropriate funding instruments within the Agency for projects funded under the NHSCSP

Expectation(s):

  • The NHSCSP as designed does not substantively duplicate other G&C programs.
  • The NHSCSP is not used in place of other more appropriate funding instruments available within the Agency.

In order to better understand the nature of conservation funding in Canada, we explored the nature of other funding sources both within and external to Parks Canada. This was done through key informant interviews and document review.

Within the Parks Canada Agency, there is only one other contribution program - the General Class Contributions Program (GCCP). This program differs from the NHSCSP as the GCCP does not limit funding to conservation and presentation work, nor to work based on the use of the Standards and Guidelines for the Conservation of Historic Places.

Within the federal government, we also examined funding requirements of the Canadian Arts and Heritage Sustainability Program, Museum Assistance Program and the Cultural Spaces Fund (see Table 8). While all these funding programs could be useful to a National Historic Site, they also do not replicate the NHSCSP since they do not focus on conservation work or require use of the Standards and Guidelines for the Conservation of Historic Places in Canada.

Table 8. Federal funding programs relevant to NHSCSP
Fund Objective
Canadian Arts and Heritage Sustainability Program Over the long-term, projects supported through the Canada Cultural Investment Fund are intended to contribute to the organizational, administrative and financial health of arts and heritage organizations.
Museum Assistance Program The Museums Assistance Program (MAP) supports heritage institutions and workers in the preservation and presentation of heritage collections. MAP provides financial assistance to Canadian museums and related institutions for activities that:
  • facilitate Canadians' access to our heritage;
  • foster the preservation of Canada's cultural heritage, including the preservation of representative collections of Aboriginal cultural heritage; and
  • foster professional knowledge, skills and practices related to key museum functions.
Cultural Spaces Fund The CCSF seeks to improve physical conditions for artistic creativity and innovation. It is also designed to increase access for Canadians to performing arts, visual arts, media arts, and to museum collections and heritage displays. The Fund supports the improvement, renovation and construction of arts and heritage facilities, and the acquisition of specialized equipment as well as conducting feasibility studies.

This situation has not changed since the 2008 evaluation which examined many potential alternate funding sources. It was noted that funding sources fluctuate over time and that while the universe of potential funding sources appeared large, most national historic sites would only be eligible for a small sub-group of funding sources and most funding sources provide only limited amounts of per project funding. The NHSCSP is the only contribution program exclusively targeted to NHSs.

Key informants reported that there is no similar funding source to the NHSCSP in Canada, given the level of funding the Program could provide and the credibility any project had if it secured funding. There is the general belief among key informants that provinces have reduced or eliminated their funding envelopes for conservation work. Public sector funding, specifically federal funds, is thus perceived to be required as seed money for most projects.

Question 3 Indicators
Is the NHSCSP aligned to government priorities and Parks Canada's mandate and objectives?
  • Logical links between NHSCSP goals and overall government priorities
  • Perceived alignment with Agency mandate and objectives
  • Presence of legal authority

Expectation(s):

  • The goals, objectives, terms and conditions of the NHSCSP are logically related to the Agency's objectives, mandate and to overall government priorities.
  • A sound legislative and/or policy basis exists for the federal government and Parks Canada's funding of these sites.

Alignment with Government of Canada Priorities

The Government of Canada gives priority to the conservation of Canada's history and heritage, including national historic sites. For example, through its ratification of the UNESCO World Heritage Convention, Canada has formally recognized its duty to ensure the identification, protection, conservation, presentation and transmission to future generations of its cultural and natural heritage.[9] This is also reflected in the Whole of Government Framework, specifically, the federal government's priority for 'A Vibrant Canadian Culture and Heritage' under which program activities aim to support Canadian culture and enhance knowledge of Canada's history and heritage, such as military history and national heritage sites.[10] Through its focus on conservation and presentation, the National Historic Sites Cost-Sharing Program is consistent with these priorities.

Legal Authority

The Terms and Conditions for the Program also outline the relevant federal legislation that exists to provide a legal basis for the program. This includes the Parks Canada Agency Act, which allows the Agency to use funds to:

  • develop or maintain a national historic site, and to make any related contribution or other payment (s. 21.(3)(c)); and
  • Implement a decision by the Minister to recommend the establishment of a national historic site or to commemorate a historic place under section 3 of the Historic Sites and Monuments Act, and to make any related contribution or other payment (s. 21.(3)(d)).

This legal authority also includes the Historic Sites and Monuments Act (s. 3(b)), which states that the Minister may make agreements with any persons for marking or commemorating historic places pursuant to this Act and for the care and preservation of any places so marked or commemorated.

Alignment with Parks Canada's Mandate and Objectives

One of the responsibilities found within the mandate of Parks Canada Agency is to, protect and present nationally significant examples of Canada's natural and cultural heritage.[11] Given this mandate, the Agency's Performance Management Framework (2012-13) and Parks Canada Agency Corporate Plan (2012-2013) includes a priority for Parks Canada programs that support the conservation of cultural resources at historic places administered by others and funded cost-sharing projects in the conservation of national historic sites not owned by the government.

Key informants reported that in their view supporting national historic sites not owned by the federal government, either through advice or financial aid helps to create a culture of conservation to aid in ensuring commemorative integrity. Without the opportunity for financial support, informants noted there would be little incentive to become listed as a national historic site.

Key informants also believe that the Agency benefits from people learning about and experiencing heritage sites, which may encourage them to visit other locations. Since the Agency does not have sites in many urban areas, this may be the first contact Canadians have with Parks Canada. This links the program to the Agency's overall strategic outcome, that Canadians have a strong sense of connection to their national historic sites.

Overall Finding: Relevance

We concluded that the program remains relevant. Empirical evidence of the state of cultural resources in general in Canada or of national historic sites in particular is largely lacking. There is some evidence that access to funds and incentives for heritage rehabilitation is a significant issue for many involved in heritage conservation and that sources of funding are limited. We found no other sources of funding that duplicate the NHSCSP.

The program is clearly aligned with both international commitments and the federal whole of government framework and with the Agency's mandate and objectives. While there is no requirement for Parks Canada to provide financial assistance to the family of national historic sites not owned or administered by the Agency, there is a sound basis in both legislation and policy to provide financial assistance to these sites.

4.2 Effectiveness

Question 4 Indicators
To what extent has the program been successful in meeting its expected results?
  • Design of T&Cs consistent with Agency performance expectations for sub-sub-activity
  • Manager and recipient feedback on the program
  • Projects are completed according to the T&Cs
  • Manager and recipients reports of success

Expectations:

  • 100% of Parks Canada's advice promotes the conservation of significant cultural resources at historic places administered by others and is targeted to areas most in need.
  • 100% of cost-sharing agreements contribute to the conservation of cultural resources of national significance (level 1) within national historic sites administered by others.
  • Recipients comply with the terms and conditions of their agreements.
  • Projects are monitored and performance information is collected consistent with expectations.
  • Eligible owners are aware of the program, make inquiries, and submit applications

As outlined in the introduction, there are three key activity areas associated with the delivery of the program - application (and communication), approval, and monitoring. Findings on the effectiveness of each of these activities are presented below.

Program Framework

We expected that program staff would have developed guidelines, tools and templates to support the application process based on the terms and conditions for the Program. In addition, we expected that the terms and conditions of the program would be consistent with the performance expectations of the sub-sub activity in PA2, i.e., that cost-sharing agreements contribute to the conservation of cultural resources within non-federally owned or administered sites.

We found that the T&Cs of the program are designed to ensure that only non-federally owned or administered national historic sites are supported and that the contribution of the project to the protection and communication of the commemorative integrity of the site is clearly articulated in the application. Further, the T&Cs ensure that Parks Canada's advice promotes the conservation of these sites. Specifically, the Standards and Guidelines for the Conservation of Historic Places in Canada must be followed and proposed interventions must undergo certification by PCA for a project to receive funding.

Program Communication

To achieve program objectives, its existence must be effectively communicated to owners of national historic sites. We expected some process to make eligible recipients aware of the program.

Quote from program staff:
If you are into heritage conservation, you would know about us.

The entire family of national historic site owners have not been surveyed recently to determine their level of awareness of the program. Key informants indicated that, other than on-line information, little official communication takes place with historic sites and most of this was through staff at Service Centres and Field Units.[12] Contact with regional staff was deemed extremely valuable as it put the Agency in direct contact with applicants. Program staff we interviewed were confident that applicants would discover the program as it is one of the last remaining funding programs for conservation work.

Another indicator of awareness of the program is the extent of web traffic accessing the program guidelines and application documents. Table 9 shows that there was considerable traffic to the web site over the past fiscal year. However, traffic to the program website does not measure awareness of owners or lessees of eligible sites.

Table 9. Web Traffic as an Indicator of NHSCSP Awareness
PCA web page or downloadable document (April 2011-March 2012) Number of unique page views
English French
Parks Canada's National Historic Sites Cost-Sharing Program (home page) 3602 625
Program Guidelines (PDF) 441 102
Funding Application (.doc) 75 12
Funding Application (PDF) 141 36
Project Timeline (excel) 49 8
Project Timeline (PDF) 104 38
Project budget (excel) 60 23
Project budget (PDF) 99 48

Application Process

We expected that the application process and application criteria be clear (e.g., submission deadlines clearly communicated) and responsive to changing priorities. The program has had a single deadline for applications for each intake, i.e., April 2009 (Intake 1), October 2010 (Intake 2) and November 2011 (Intake 3). Notification of the deadline is published with program guidelines and application materials, available on the web page for the NHSCSP.

Table 10 summarizes the number of applications received by the program for the period under evaluation, and the number that were assessed as eligible or ineligible as well as the number of funded applications.

Table 10. Applications, eligibility and funded projects by intake year
Intake year #applications #eligible #ineligible #funded
2009-2010 127 99 28 40
2010-2011 76 68 8 41
2011-2012 70 60 10 21
Total 273 230 46 102
Source: Program database

Demand has exceeded resources in each round of funding, with no more than 60% of eligible recipients receiving funding in any given year. Table 11 shows that only about 30% of total funding requested was approved by the CEO. For various reasons, even projects that do receive funding may not receive the full amount requested. As a result, the program must have sound criteria to assess the relative merits of each eligible project.

Table 11. Difference between funding requested and received from NHSCSP, by intake year ('000s)
  Intake 1 Intake 2 Intake 3
Amount requested $29,464 $14,894 $12,816
Amount approved by CEO[13] $8,632 $5,785 $2,471
Difference between funding requested and approved $20,831 $9,110 $10,345

How Funding is Awarded

Decision-making criteria for awarding funds have evolved as a result of lessons learned in previous funding rounds and changing Agency priorities. Criteria and the weighting of each are shown in Table 12 for the last three years.

Table 12. Evolution of Funding Application Evaluation Process
Funding Year(s) and Evaluation Criteria
2009-10 2010-12 2012-13
The criteria defined in the 2008 RMAF/RBAF
  1. Program delivery capacity of the applicant.
  2. Objectives and intended results (adherence to CRM Policy and Standards and Guidelines).
  3. Management capacity of the applicant (including financial risk).
  4. Proposed budget (demonstrated need for funding).
  5. Project design (level of threat/mitigation of threat).
  1. Adherence to CRM Policy and Standards and Guidelines.*
  2. Level of threat/mitigation of threat.*
  3. Demonstrated need for funding.*
  4. Financial risk.*
  5. Completion of project in funding year.*
*each criteria scored out of 5.
  1. Level of threat to commemorative integrity and proposed mitigation strategy/measures.*
  2. Adherence to CRM Policy and Standards and Guidelines.**
  3. Demonstrated need for program funds, including evidence that the project could not proceed effectively without assistance.**
  4. Level of financial risk associated with the project.**
  5. Demonstrated capacity to carry out and complete the proposed activities by March 31, 2013.**
* criteria scored out of 10.

** criteria scored out of 5.

In the first funding year, proposal assessments were completed by program staff and when required by Cultural Resource Management (CRM) experts. Level of threat was included among criteria.

During the second funding round, projects were assessed first by program staff and then by an evaluation committee composed of Agency staff with demonstrated CRM expertise. Once eligibility was confirmed (property ownership and not-for-profit status), individual proposals were assessed using numerical scoring against five revised criteria, again including level of threat.

The evaluation process was again revised for 2012-13 to increase cohesiveness by including, prior to the evaluation committee meeting, a working group meeting composed of all program staff tasked with reviewing individual assessments.[14] The assessment forms were also merged into a single, more streamlined document to facilitate discussion and assessment by the evaluation committee. The evaluation committee maintained a numerical scoring approach, however given direction from senior management to focus on level of threat to the commemorative integrity of sites, the weight for this criterion was doubled from 5 to 10.

Table 13 shows the proportion of eligible sites, applicants and projects funded between 2009 and 2012, with a break-down by specific eligible owner type. From this, it is clear that certain types of owners are more likely to be funded relative to their frequency in the population. For example, owners who are categorized as educational institutions, not-for-profit organizations, municipal governments or religious groups are funded more often relative to their portion of the eligible population than are aboriginal groups/bands or provincial/territorial owners. In many cases, data also shows that these categories also submit applications with greater frequency. The data also shows that of eligible applicants religious groups, municipal governments and educational institutions are more likely to be funded while Aboriginal groups/bands, historical societies and provincial governments are more likely to not have funding requests approved.

Table 13. Eligible Owners, Eligible Applicants, Eligible Applicants not funded and funded
Group % eligible sites % eligible applicants % eligible applicants not funded % funded projects
Aboriginal group/band 12% 5% 8% 2%
Educational Institution 3% 4% 2% 5%
Historical Society 11% 15% 17% 14%
Not-for-profit organization 6% 7% 7% 8%
Municipal government 28% 27% 25% 30%
Provincial/territorial government 25% 16% 18% 14%
Other governmental 0% 0% 0% 0%
Religious groups 14% 21% 18% 22%
Source: Program database

The significance of this is not clear, and could reflect

  1. Applicants' capacity to complete the application process and challenges in raising the necessary matching funds (i.e., the rigour and documentation involved in the application process may be difficult for small volunteer organizations, especially those of a seasonal nature);
  2. Differing awareness of program or need (i.e., either state of place or requirements for external funding) levels in segments of the population; and/or
  3. Some bias on the part of the application process.

Monitoring and Program Satisfaction

We examined whether projects were being completed according to their terms and conditions, and whether the success of these projects is being monitored. The success of funding provided by the program is demonstrated by the closure rate of projects. The program currently has completed 101 projects out of the 102 started. The timeline of the only remaining project not completed extends beyond the scope of the evaluation, with an end date of 2014.

To ensure that the work done on the site complies with the T&Cs of the application, a holdback of at least 25% of the value of the contribution is applied (in line with established financial rules). Holdbacks are released upon demonstrated fulfillment of the contribution agreement, including receipt by the Program Office of final recipient reports, and completion of a final certification report by a Certification Service Provider. The latter certifies that the work was completed in compliance with established conservation practices and satisfies any established conditions.

Program staff indicated that there are no instances where final certification was not performed when a project was pre-certified (i.e., certification took place to establish the status of the project before work was completed). However, for the first intake, Category I projects were not pre-certified and thus no final certification was required.

The NHSCSP's Integrated RMAF/RBAF (2007) also outlines requirements for recipient auditing as part of the program. These audits should be used to ensure funds were used for intended purposes, that recipients complied with the negotiated terms and conditions, and that project results are accurately and fairly reported. Program management is responsible for identifying the need to conduct a recipient audit. Both our review of documents and discussions with program staff indicate that no recipient audit has yet been completed.

However, key informants believe that monitoring activities are sufficient. Parks Canada's National Audit of Parks Canada's National Historic Sites Cost-Sharing Program (2011) also reviewed the payment requirements and submission of reports by recipients. The internal audit found substantial compliance with the authorization process for payments. In addition, it found that no deficiencies have been identified with respect to certification reports and final reports, both key documents to be obtained prior to payment.

Another measure of program success is feedback from recipients. The NHSCSP uses a recipient survey to determine effectiveness of the funding and client satisfaction with the program. This survey is administered to all recipients during the final reporting stage, approximately one month before a project ends. A simple tool, the survey consists of three yes/no questions and one Likert-scale question. In total, 91 funding recipients (90%) responded to the survey.

Respondents to the survey were unanimous that the NHSCSP allowed them to mitigate or reduce threats to heritage resources or create plans to do so, and that the funding received made it possible to undertake work they would otherwise not have been able to afford, in whole or in part. Results for overall satisfaction with the program are shown in Figure 1. In total, 92% of respondents rated their satisfaction with tools, support and the application processes as a three or a four.

Figure 1: Recipient Satisfaction with NHSCSP

Are you satisfied with the Cost-Sharing Program, the information, tools and support provided and the application process Please rate satisfaction on a scale of 1 to 4 with 4 being the most satisfied.

Recipient Satisfaction with NHSCSP

However, there are some limitations noted with the survey data. First, the survey did not provide for anonymity. As respondents are recipients of funding, there may be a built-in bias towards support for the program. Second, the satisfaction question addresses multiple parts of the funding process. While this identifies overall satisfaction with the program, it does not attribute results to specific activities. For example, key informants had anecdotal evidence that applicants may be less satisfied with some components of the program (e.g., the application process was deemed too rigorous in some situations). Lastly, the survey is only directed to funding recipients and so does not provide information on possible program improvements that could be suggested by owners of eligible sites who did not receive funds or owners that do not apply.

Overall Finding: Effectiveness

The expected results of the NHSCSP are being achieved. The terms and conditions of the program as expressed through the application process are consistent with performance expectations for the program. The approval process continues to improve to better target results. Further, there are no deficiencies noted in recipients' adherence to these terms and conditions.

Recipient satisfaction with the Program is also high. However, little data is available regarding satisfaction for those applicants that are unsuccessful or don't apply. There is some suggestion that the application process could be improved to better respond to the needs of some potential recipients.

4.3 Cost-Effectiveness and Efficiency

The evaluation of cost-effectiveness and economy focused on adherence to the program's funding requirements and the cost-effectiveness and efficiency of program administration. There is no information available nor does the program require information on the cost-effectiveness (i.e., efficiency and economy) of individual projects.

Question 5 Indicators
Did NHSCSP projects meet the program funding requirements?
  • Extent of Agency funding compared to total project value (matching funds)

Expectations:

  • NHSCSP funding requirements were met.

As per the Program's terms and conditions, at least 50% of the costs of approved projects must be shared by the applicant. A review of funded projects for 2009-2010 and 2010-2011 (n=80) found that all projects secured at least 50% of their funding from other sources consistent with the program T&Cs. In total, 31% of all projects were funded at the 50% level by the program. Other projects ranged from 51% to 97% of their funding from other sources. The average amount of funding contributed by the Agency is 45%.

Other sources of funding and the total dollar amounts by source are shown in Table 14 for years 2009-10 and 2010-11.

Table 14. Source of total funds to approved cost-sharing projects ($'000s)
Intake PCA Final Contri-
bution
Final Contributions from non-PCA sources
Self financed Fed. Prov. - Terr. Municipal Grant Private Donation In-kind Total non-
PCA
1 8,226 3,217 n/a 7,029 443 n/a 826 144 11,660
2 5,157 4,358 n/a 2,245 13,671 10 569 70 20,923
Total 13,383 7,575 - 9,274 14,114 10 1,395 214 32,583
Source: Program database

Table 14 demonstrates that on average for every dollar that Parks Canada has contributed to these projects, and additional $2.4 dollars were contributed by other funding sources. Other levels of government, either provincial/territorial or municipal, constitute the other major sources of funding.

It should be noted that while the program is contributing only about 45% of the total funding for these projects we cannot conclude that the program was instrumental in leveraging these funds from other sources. This follows logically from the fact that the T&Cs of the program require the applicant to have other sources of funding in place prior to entering into an agreement.

Question 6 Indicators
Is the administration of the NHSCSP cost-effective and efficient?
  • Evidence that the application, approval and project management components of the program are timely and efficiently administered (including risk-based).
  • Ratio of administrative costs to total funds dispersed.
  • Extent management has used available flexibilities to encourage efficient or economical operation.

Expectations:

  • Administrative processes timely and appropriate.
  • Administrative costs of the NHSCSP are reasonable compared to value added of agreements.

The administrative costs of the NHSCSP were shown in Table 4. These include salaries, goods & services and certification.[15] These represent less than 10% of total program costs for the last three fiscal years. In general, for every dollar in administrative costs, $10 have been provided in contribution funding.

Key informants' opinions differed as to the appropriateness of these administrative costs. Senior managers were more likely to believe costs could be reduced. Some key informants were concerned that the cost of travel for certification is too high. However, we did not find evidence to substantiate this concern. While the specific amount allocated to travel could not be confirmed given data limitations of the Agency's financial system, program staff estimate that travel accounts for approximately 25% of the total for certification. Overall, certification costs (including professional fees and travel) account for only about 3% of the program's yearly expenditures (see Table 4). This could not likely be reduced due to the remoteness of many of the sites.

Key informant interviews did not identify significant concerns regarding the timeliness of the administrative process. However, as a related indicator, we also examined whether the program was meeting the service standards it has developed to guide its timely delivery. We found that there is only a mechanism in place to track one aspect of the service standards, i.e., the number of days to complete a contribution agreement. For Intake 1, this service standard had not yet been developed. For this intake, the average number of days to complete an agreement was 184 days. There was some improvement with the introduction of the service standard for Intake 2. The new 90-day deadline was met for 12 agreements (28%). The average number of days was 92.5. For Intake 3, 12 (57%) agreements were done within 90 days. The average number of days was 108.

Management has a variety of flexibilities and constraints in operating the NHSCSP that contribute to or inhibit the economic and efficient operation of the program. These include:

  • The budget for the NHSCSP, which is adjusted depending on available funds. The 2008 evaluation considers whether there is an adequate level of funding to deliver Program objectives and provides a number of possible budget scenarios to consider, but leaves it to the Program to decide optimal funding levels.
  • The provision of multi-year funding. At present, funding agreements stipulate that funded projects must be initiated and completed within the given funding period; multi-year agreements are not being pursued. In addition to being a general risk reduction strategy, this decision reflects the requirements of the time-limited funding provided by the Economic Action Plan. This took effect during Intake 1 and remains in place but does not mean multi-year agreements are not allowed or will not take place in the future (i.e., multi-year projects are not prohibited by program T&Cs).
  • The design of the application process. Currently, all applicants must follow the same application process regardless of the size of the organization or the amount of funding being requested. Key informants suggested that, if projects under a specified dollar threshold were assessed against different criteria, a certain amount of time and money could be saved on both the part of the applicants and the Agency. This has never been done and may require a change to the program's T&Cs.
  • The program has discretion on how it chooses to prioritize eligible applications. As demand for funding consistently exceeds available resources, decisions must be made as to how to assess the relative merit of applications. The most recent criteria give more weight to projects that demonstrate a greater level of threat to commemorative integrity. These criteria could be further adjusted in future to reflect available resources and program priorities. For example, the evaluation criteria and scoring system used during the approval process could be adjusted to target specific groups, such as those that program staff suggested may be under-represented by funding.

Overall Finding: Efficiency and Economy

We found that the funding requirements of the NHSCSP were met, i.e., funding recipients were able to match at least 50% of the proposed costs and total federal government assistance for the projects did not exceed 50% of eligible costs.

The evaluation did not identify any standard that could be used to determine the appropriate ratio of administrative costs. However, in our opinion, the administrative costs are reasonable given overall program expenditures. There is some indication that the program's service standards are not always met, and that some elements of the standard are not being tracked.

4.4 Design and Delivery

Question 7 Indicators
Are roles, responsibilities and the governance structure with respect to overall program management clear and appropriate?
  • Reported clarity of roles
  • Consistency between prescribed roles and practices
  • Extent processes function as intended.

Expectations:

  • Roles and responsibilities are clear and understood, and carried out effectively.

Key informant interviews and the recent internal audit of the program indicate that roles and responsibilities are clearly understood.

The responsibility for administering the NHSCSP resides with the Vice President (VP), Heritage Conservation and Commemoration Directorate. Project agreements are reviewed and signed by the VP prior to being submitted to the CEO as a last challenge function.

Specific responsibility for managing, monitoring and controlling resources used and results achieved rests with the Director, Heritage Conservation Branch of the Directorate. Program officers are primarily responsible for managing agreements with recipients while managers coordinate monitoring and reporting to senior management.

Roles and responsibilities are defined in part in the job descriptions for staff regularly involved in the process. Annual performance evaluations of staff allow specific related tasks to be assigned to certain individuals. Assigning these specific tasks means that expectations can be clearly defined and an evaluation platform is provided for supervisors.

As shown in Table 15, there are relatively few FTE involved in the administration of the program. This contributes to an administrative and governance structure that is clear and simple to understand.

Table 15. Break-down of FTE to NHSCSP
  Prior to 2009 2009-2010 2010-2011 2011-2012 Post Budget 2012 (2012-2013)
Program FTE 1 PM04 1 Program Manager
1 PM02
2 PM04
3 PM05
1 Program Manager
1 PM02
2 PM04
2 PM05
1 Financial Officer
1 Program Manager
1 PM02
2 PM04
2 PM05
1 PM04
1 PM05
Total FTE 1 7 7 6 2

To date, the structure of roles and responsibilities appears to be appropriate given the T&Cs of the program. While the structure of the program was recently revised given Budget 2012, it is too soon to comment on the appropriateness or effectiveness of these revisions.

Question 8 Indicators
Are the NHSCSP Terms and Conditions and the clauses in contribution agreements appropriate?
  • Reports that the Terms and Conditions are sufficient and appropriate for the Agency's needs.
  • Contribution agreement clauses are seen as reasonable, clear and comprehensive.

Expectations:

  • The T&Cs are seen as appropriate to the intended purposes of the NHSCSP and Agency needs and the agreement clauses are understood and appropriate.

Agency key informants indicated that the T&Cs of the Program are appropriate. They are flexible enough to allow a variety of projects to be funded. While the program's focus on built heritage was cited by some as a concern, this focus is logical given that the funding program was developed for conservation. While cultural landscapes can receive funding, the concept of their rehabilitation is not common.

While noting that the Program is generally well designed, program staff provided additional suggestions to help improve its design and delivery. These mostly related to capacity building among the owners of national historic sites. For example, program staff suggested specific initiatives to encourage Aboriginal owners to apply. This may be in the form of more targeted communications, change to the 50% rule and/or a streamlined application process.

Overall Finding: Design and Delivery

The terms and conditions of the NHSCSP are appropriate to achieve current expected results. Areas for adjustment have been suggested by program staff given management flexibilities but whether or not these should be pursued would depend on the priorities for the program going forward.

5 Conclusions and Recommendations

The NHSCSP is one of two contribution programs in the Agency. Yearly contribution payments under the program have averaged approximately $5.0M since FY2009-10. The program represents less than 1% of the Agency's direct yearly program spending. If the program is not adequately managed, there are risks that funded projects will not be aligned with and supportive of the Agency's mandate, projects will not be implemented as intended, and/or that results will not be achieved efficiently and economically.

The evaluation focused on the continuing relevance of the Program, the effectiveness, efficiency and economy of its operations and questions related to program design and delivery. Our broad conclusions are:

  • The NHSCSP continues to be a relevant mechanism for delivery of the Agency's mandate and priorities and is aligned with overall government objectives. Funded projects over the period 2009-10 to 2011-12 can be linked to the Agency's program activity for heritage resources conservation and specifically to the sub-sub-activity of national historic sites cost-sharing. The Program is seen to be useful and offer advantages over other funding mechanisms by both Agency staff and recipients. There is little sense that it duplicates other programs. There are no other contribution programs that target the same eligible recipients and types of agreements.
  • All the contribution agreements contain provisions for final project reporting. NHSCSP funded projects are largely implemented as intended and produce, or are likely to produce, the intended outputs defined in contribution agreements. Funding recipients are generally satisfied with the program.

The issues or challenges identified include:

  • Data regarding program satisfaction is limited to funding recipients. No data is regularly collected to develop a full understanding of the needs of eligible recipients as a whole or the state of heritage at national historic sites not administered by the Agency. It is therefore difficult to evaluate whether there are barriers to application or if the program is targeting areas most in need.
  • There is some indication that the program's service standards are not always met, and that some elements of the standard are not being tracked.

Our recommendation follows from these issues.

Recommendation 1

The recipient survey should be redesigned to strengthen analysis for individual questions and it should be administered such that anonymity is protected. To be more comprehensive, this could also include a survey of applicants not recommended for funding. Additional questions could also be included to address perceived needs of applicants and barriers to application.

Management Response

Agree: The program management agrees that the use of an anonymous end-of-project questionnaire to obtain feedback from applicants could result in useful information on the needs of national historic sites' owners.

While maintaining project satisfaction questions with recipients to help measure program results, program management will develop additional questions that will target program issues relevant to all applicants. Measures to protect anonymity will be examined in order to ensure that the chosen method complies with all applicable federal regulations, is transparent and results in clear and usable responses. The new questionnaire will be used as soon as the final stage of the next intake of projects is reached. The responses to the questionnaire will be compiled and used to analyse the program's direction at the end of each intake.

Recommendation 2

The service standards for the program should be tracked in such a way that the program can determine whether these are being met. This data should be used to review and adjust the program as required in order to meet or revise service standards.

Management Response

Agree: The current service standards are, in large part, based on existing standards in use across the Federal Government for grants and contributions programs. The Program's current tracking of service standards focuses mainly on ensuring that:

  1. Decisions are clearly communicated to applicants - within 6 months - and if this is not possible, that applicants are informed of any delays encountered within that same timeframe period;
  2. Contribution Agreements are signed within a 90 day timeframe; and
  3. Only eligible applicants are retained for further assessment of their proposal; non-eligible applicants are informed of their ineligibility.

Program management will therefore ensure that relevant service standards are tracked, and that the tracked information is compiled and assessed on an intake-by-intake basis. This new tracking model will start with the official announcement and posting of the next intake of projects on PCA's Webpage. In addition, the Program's 1-800 line will be maintained to ensure accessibility to Program staff during regular operating hours.

Appendix A. Strategic Outcome and Program Activity Architecture

Parks Canada Strategic Outcome and Program Activity Architecture 2012-13

Updated on November 7, 2011

View larger image

[long description]

Appendix B. Terms and Conditions of the NHSCCSP

Authority

Transfer payments will be made under the authority of the Parks Canada Agency Act, paragraphs 21.(3)(c) and 21.(3)(d), and the Historic Sites and Monuments Act, paragraph 3(b).

Description of the Program

The National Historic Sites of Canada Cost-Sharing Program (the Program) assists recipients in conducting activities aimed at ensuring the commemorative integrity of non-federally owned or administered national historic sites and, thereby, to support Parks Canada Agency in fulfilling its mandate to protect and present nationally significant examples of Canada's natural and cultural heritage, and foster public understanding, appreciation and enjoyment in ways that ensure the ecological and commemorative integrity of these places for present and future generations.

Specifically, the Program provides financial contributions to eligible recipients that share the costs of works necessary to ensure the physical health of a national historic site and to ensure Canadians understand the importance of the site and its role in the history of Canada.

Objectives and Results

The expected immediate results and outcomes of the funding are:

  • Non-federally owned national historic sites receiving funding are conserved, thus reducing the threat to their commemorative integrity;
  • Approved conservation projects are carried out according to the Standards and Guidelines for the Conservation of Historic Places in Canada;
  • The reasons for a site's designation as a national historic site are effectively communicated to the public in both official languages.

The expected intermediate results and outcomes of the funding are:

  • The federal government provides leadership in creating a culture of conservation;
  • Conservation of national historic sites to national standards through the mandatory use of the Standards and Guidelines for the Conservation of Historic Places in Canada;
  • Reduced threat to commemorative integrity at non-federally owned national historic sites;

The expected strategic (end) outcomes of the Program are:

  • Canadians feel a greater sense of connection to each other, their history, identity and historic places
  • More historic places are conserved and given a function in the life of communities (United Nations Educational, Scientific and Cultural Organisation, UNESCO)
  • National historic sites provide improved access to significant, diverse cultural heritage

Eligible Recipients

  • Owners or lessees (as defined below) of national historic sites, including its associated lands, or parts of a national historic site or national historic site district which are:
    • provincial, territorial, regional or municipal governments;
    • not-for-profit organizations;
  • Not-for-profit organizations acting on the authority and behalf of an eligible owner or lessee may also be considered for funding.
  • Aboriginal organizations with a formalized stewardship responsibility directly related to all or part of a national historic site managed by a related Aboriginal organization.

For the purposes of this section, a lessor must have at least 10 years remaining on the lease as of the date of the application to the program to be eligible.

Stacking Limits

All potential recipients will be required to disclose all sources of funding for a proposed project before the start and at the end of the project. Funding is provided at the minimum level necessary to achieve project objectives. Parks Canada examines all project proposal budgets prior to approval and at various points in the project life period to ensure that only funds necessary for the success of the project are made available.

Decisions will be made on the basis of value-added to the Program's objectives and mandate and the incremental value of the funding by the Agency.

The Agency will require the recipient to disclose any other proposed financial assistance being received for the specific project in accordance with the Treasury Board Policy on Transfer Payments. The stacking limit of Total Government Assistance (federal, provincial, territorial and municipal for the same eligible expenditures) for the project will not exceed 100% of eligible costs.

Recovery of Excess Funds

Parks Canada will ensure that its contributions respect the cost-share provisions of 50% taking into account all federal government funding for the project.

The Agency requests information on other financial sources prior to making a funding decision, and it requests an accounting in the final project report on the actual sources and amounts of funds received.

Where it is evident that an organization has received more funding than required to meet the original project objectives, the Agency may consider the following actions:

  1. Require the organization to repay some portion of the funds received;
  2. Withhold amounts from any final payment owing;
  3. Accept an expansion of the project to encompass a wider range of eligible needs and thus costs.

Eligible Costs

The Program will reimburse up to 50% of eligible costs identified in the cost-sharing contribution agreement.

Eligible Project Categories

From time to time, based on available program resources or as instructed by the Minister, Parks Canada may limit funding to one or the other of the project categories, or a subset of one or both of the project categories, listed below. Parks Canada will ensure information on funding priorities is available to prospective applicants through appropriate mechanisms.

Appendix C. Evaluation Questions, Expectations, Indicators and Data Sources

Issues Evaluation Questions What Should be Observed Indicators Data source
Relevance
Issue 1: Continued Need for Program 1. Is there a demonstrated need and demand for the NHSCSP?
  • The state of non-federally owned NHSs should continue to require improvement.
  • The heritage community and other levels of government are aware of the threats and interested in their conservation.
  • Canadians should support efforts to reducing threats to non-federally-owned national historic sites.
  • There should be a continued demand by the heritage community for funding from the program.
  • Evidence from previous reports, surveys, and studies.
  • Reported consequences if the program were not available.
  • Profile of program applicants (e.g. number and description) and other evidence of interest in the program.
  • Document review
  • Key informant interviews
2. Does the NHSCSP duplicate program(s) offered by other levels of government or by other granting agencies?
  • The NHSCSP as designed does not substantively duplicate other G&C programs.
  • The NHSCSP is not used in place of other more appropriate funding instruments available within the Agency.
  • Extent and nature of other funding sources.
  • Existence of other appropriate funding instruments within the Agency for projects funded under the NHSCSP.
  • Key informant interviews
  • Review of project approval files
  • Document review
Issue #2: Alignment with Government Priorities and Agency strategic objectives 3. Is the NHSCSP aligned to government strategic priorities and Parks Canada's mandate and objectives?
  • The goals, objectives, terms and conditions of the NHSCSP are logically related to the Agency's mandate and to overall government priorities.
  • Agreements support recipient types as specified in the Terms and Conditions.
  • Logical links between NHSCSP goals and Agency mandate and overall government priorities.
  • Perceived alignment with Agency objectives.
  • Profile of recipients and agreement purposes.
  • Document review
  • Review of project approval files
  • Key informant interviews
Issue #3: Alignment with Federal Roles and responsibilities 4. Is there a sound basis for the federal government and Parks Canada provision of funding to non-federally-owned national historic sites?
  • A sound legislative and/or policy basis for the federal government and Parks Canada's funding of these sites.
  • Presence of legal authority for the program.
  • Perceived rationale for the program.
  • Document review
  • Key informant interviews
Performance
Issue #4: Achievement of Expected Outcomes 5. To what extent has the program been successful in meeting its expected results, as identified in the Agency's PAA?
  • Sub activity Parks Canada programs support the conservation of cultural resources at historic place administered by others
  • Sub sub-activity Funded cost-share projects in the conservation of national historic sites not owned by the government
  • 100% of Parks Canada's advice promotes the conservation of significant cultural resources at historic places administered by others and is target5ed to areas most in need.
  • 100% of cost-sharing agreements contribute to the conservation of cultural resources of national significance (level 1) within national historic sites administered by others.
  • Recipient feedback on support they received.
  • Projects are completed according to the Ts & Cs
  • Document review
  • File review
6. To what extent has the program achieved other articulated outcomes?[16]
  • Compliance with terms and conditions.
  • Nationally significant examples of Canada's cultural heritage, not administered by the federal government, are maintained and improved (i.e. threats to or deterioration of heritage resources are mitigated or reduced)
  • Heritage community is aware of the program.
  • Heritage community is satisfied with the program.
  • Funded organizations have an increased capacity to undertake projects.
  • Recipients comply with the terms and conditions of their agreements.
  • There is evidence of funded projects maintaining and/or improving threats to or deterioration of heritage resources.
  • Eligible owners are aware of the program, make inquiries, and submit applications
  • Eligible owners are satisfied with the program, believing it adds credibility and legitimacy to their project.
  • Recipients can undertake work they would have otherwise not been able to and / or timing is advanced.
  • Analysis of project reports and findings from past studies.
  • Level and trends of inquiries and submissions of applications.
  • Manager and recipient's reports of success.
  • File review
  • Document review
  • Key informant interviews
Issue #5: Demonstration of Efficiency and Economy 7. Are NHSCCSP projects cost-effective?
  • NHSCCSP projects provide value for the dollars spent.
  • Project managers and recipient reports of cost-effectiveness
  • Extent of Agency funding compared to total project value
  • Key informant interviews
  • Document Review
  • Expenditure Data
8. Is the administration of the NHSCCSP cost-effective and efficient?
  • Administrative processes are seen to be timely and appropriate.
  • Administrative costs of the NHSCCSP are reasonable compared to value added of agreements.
  • Evidence and perception that the application, approval and project management components of the program are timely and efficiently administered (including risk-based)
  • Ratio of administrative costs to total funds dispersed
  • Document review / expenditure data
  • Key informant interviews
Design and Delivery 
  9. Are roles and responsibilities and the governance structure with respect to overall program management clear and appropriate?
  • Roles and responsibilities are clear and understood, and carried out effectively.
  • Reported clarity of roles
  • Consistency between prescribed roles and practices
  • Extent processes function as intended
  • Document review
  • Key informant interviews
  10. Are the NHSCSP's Terms and Conditions and the clauses in contribution agreements appropriate?
  • The T&Cs are seen as appropriate to the intended purposes of the NHSCSP and Agency needs and the agreement clauses are understood and appropriate.
  • Reports that the Terms and Conditions are sufficient and appropriate for the Agency's needs
  • Recipients meet the T&Cs of their agreement
  • Contribution agreement clauses are seen as reasonable, clear and comprehensive
  • Key informant interviews
  11. Are mechanisms in place for monitoring project progress and are interim and/or final results documented?
  • Projects are monitored and performance information is collected consistent with expectations.
  • Reports of monitoring activities
  • Existence and quality of performance information
  • Correspondence between expected and actual types of performance information
  • Key informant interviews
  • Results data for a sample of agreements
  • Case studies

Appendix D. Key Documents Reviewed

Legislation

Financial Administration Act

The Historic Sites and Monuments Act (1985; current to 2007-06-02; last update 2007-07-06),

The Parks Canada Agency Act (December 3, 1998)

Parks Canada Documents

Canadian Perceptions of Parks Canada 2005. Release I - National Findings. (June 2006)

Internal Audit and Evaluation Office (2010). Evaluation of Parks Canada's General Class Contributions Program.

General Information about Contributions. Intranet source (February 2010).

Integrated Results-based Management and Accountability Framework (RMAF) and Risk-based Audit Framework (RBAF), National Historic Sites of Canada Cost-Sharing Program (June 2007).

Internal Audit and Evaluation Office, (2008). Evaluation of Issues Related to the National Historic Sites of Canada Cost-Sharing Program.

Internal Audit and Evaluation Office. (2011) National Audit of Parks Canada's National Historic Sites Cost-Sharing Program.

Internal Audit and Evaluation Documents: Parks Canada multi-year Evaluation Plan 2011-2012 to 2015-2016.

National Historic Sites of Canada Cost-Sharing Program, Program Guide (draft) (March 2001).

National Historic Sites of Canada Cost-Sharing Program Guidelines (2012-2013).

National Historic Sites of Canada Cost-Sharing Program, Terms and Conditions, Annex A (2007).

Parks Canada Agency Corporate Plan (2006/7 - 2010-11 Report on Plans and Priorities), (2006).

Register of Government of Canada Heritage Buildings 1982-2002 (March 2003).

Report on Plans and Priorities 2006 - 2007

Standards and Guidelines for the Conservation of Historic Places in Canada, 2nd edition (2010)

The Canadian Register of Historic Place: Documentation Standards Handbook. Historic Places Program Branch National Historic Sites Directorate, Parks Canada (August 2006).

Treasury Board Submission, Extension of the National Historic Sites of Canada Cost-Sharing Program Authority, T.B. 833752 (June 2007).

Treasury Board Submission: Renewal of the National Historic Sites of Canada Cost-Sharing Program Authority, draft (June 2007).

Wright, J. (1984). Architecture of the Picturesque in Canada. Parks Canada Agency.

Other References

British Columbia (2011). Heritage-Funding Opportunities. Ministry of Forests, Lands and Natural Resource Operations.

Browne, K. (April 10, 2012). When it comes to historical preservation, the Brits have us beat. National Post.

Canada's Economic Action Plan (2009-2010). National Historic Sites of Canada Cost Sharing Program.

Canadian Heritage (2011). Building Communities Through Arts and Heritage. Her Majesty the Queen in Right of Canada.

Chianello, J. (February 25, 2012). Another piece of history is gone. Ottawa Citizen.

City of Dawson (2012). Heritage Management Plan.

Community Foundations of Canada - all for community. Benjamin Moore Community Restoration Program.

Environics Research Group limited (May 2005). 2004 Survey of National Historic Sites

Flaherty, J.M. (2012) Jobs Growth and Long Term Prosperity: Economic Action Plan 2012. Her Majesty the Queen in Right of Canada.

Government of Canada (2009). Canada Cultural Spaces Fund. Canadian Heritage.

Heritage Canada Foundation (2012). HCF's 2012 Top Ten Most Endangered Places List.

Heritage Canada Foundation (2012).The State of Heritage in Canada Survey.

International Council on Monuments and Sites (2011). Charter for the preservation of Quebec's Heritage (Deschambault Declaration)-1982.

Jack Jedwab (April to October 2006). Survey Results: What Would You Like to Know about Canada: Canadian Interest Investigated

Jack Jedwab (July 2007) Historic Pride, National Identities and Diversity: Situating Canada on the World Map. (Conducted in participation in the International Social Survey Program) Association for Canadian Studies.

LeBlanc, D. (Feb 17, 2011). Toronto's heritage buildings face 'demolition by neglect'. Globe and Mail.

Manweiler, B. (January 2007). The State of Heritage Conservation in Canada: Provincial and Territorial Levels. Prepared for Heritage Branch - Ministry of Tourism, Sport and the Arts (Province of British Columbia).

Nephin Consulting Partners (October 1997). A Review of the National Cost-Sharing Program for Historic Sites, External Evaluation

Office of the Auditor General of Canada (2003). Report of the Auditor General of Canada to the House of Commons: Chapter 6 Protection of Cultural Heritage in the Federal Government. Minister of Public Works and Government Services, Canada.

Office of the Auditor General of Canada (2007). A Status Report of the Auditor General of Canada to the House of Commons: Chapter 2 The Conservation of Federal Built Heritage. Minister of Public Works and Government Services Canada.

Office of the Auditor General of Canada (February 2007) Status Report, The Conservation of Federal Built Heritage (Chapter 2)

Office of the Auditor General of Canada (November 2003). Protection of Cultural Heritage in the Federal Government (Chapter 6)

Public Works and Government Services (2011). Final Report 2008-608 Evaluation of Heritage Conservation Services. Office of Audit and Evaluation.

Sharpe, C.A. (2003). The Devil's In The Details: Benign neglect and the erosion of heritage in St. John's, Newfoundland. Newfoundland and Labrador Studies, Vol 19, No 2.

Shipley, R., & Reyburn, K. (2002). The Loss of Heritage Properties in Ontario.

Solutions Research Group (March 2006). Diversity in Canada: A Parks Canada Proprietary Report.

Statistics Canada (2008). Construction Price Indexes, (Apartment Building) CANSIM 327-0040.

Statistics Canada (2008). Construction Price Indexes, (Non-Residential Building) CANSIM 327-0039.

Statistics Canada (2008). Consumer Price Index, Historical Summary (1988 to 2007). CANSIM 326-0021.

Treasury Board of Canada Secretariat (2011). Descriptors for Government of Canada Outcome Areas.

Treasury Board of Canada Secretariat (2011). Directory of Federal Real Property.

Treasury Board of Canada Secretariat (2009). Policy on Evaluation.

Treasury Board of Canada Secretariat (June 2000), Policy on Transfer Payments

Treasury Board of Canada Secretariat, (2007), The Government of Canada Report on Plans and Priorities

UNESCO (1972). Convention Concerning the Protection of the World Cultural and Natural Heritage.

Zlomislic, D. (April 20, 202). Thousands of buildings owned by Ottawa are crumbling and need repair, database shows.


[1] http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?section=text&id=15024

[2] http://www.pc.gc.ca/docs/pc/rpts/rve-par/ep-2010-2011_e.asp

[3] http://www.pc.gc.ca/progs/lhn-nhs/ppf-csp/index.aspx

[4] For details of the CEAP, see: http://actionplan.gc.ca/initiatives/eng/index.asp?mode=7&initiativeID=128

[5] Certification is a service provided to the program and is thus calculated separately from program administration.

[6] http://www.oag-bvg.gc.ca/internet/English/parl_oag_200311_06_e_12929.html

[7] http://www.oag-bvg.gc.ca/internet/English/parl_oag_200702_02_e_17468.html

[8] The survey was distributed to approximately 3,000 members and had a completion rate of approximately 10%. The survey instrument was developed around five questions with multiple choice responses not limited to a single response. Members are not necessarily owners of heritage sites.

[9] http://whc.unesco.org/en/convention/

[10] http://www.tbs-sct.gc.ca/ppg-cpr/descript-eng.aspx

[11] http://www.pc.gc.ca/eng/agen/chart/chartr.aspx

[12] An exception to this practice took place in March 2009, when all known owners were sent a letter notifying them of the renewal of the program. It is not know if the letter was received by an appropriate person at sites as contact information is not necessarily kept up to date. In the case of historic districts no contact information is available for all individual owners within the designated site.

[13] The amount approved by the CEO does not match the contribution amount in Table 4 in each fiscal year. This is due to the contribution amount being the actual dollars released in a fiscal year. Although the program is currently focused on single-year projects, financial disbursements may carry over into the next fiscal year. Some multi-year projects also remain from past intakes.

[14] Funding in FY2012-13 is outside the scope of the evaluation. This data is included to better demonstrate the continued progress being made towards program improvement.

[15] Certain costs are tracked by program management (e.g., travel) and are not available in the Agency's financial system. We are not able to verify the accuracy of these costs.

[16] These are based on residual outcomes from the program's 2008 Results-based Management Framework.