Embed Size (px)
Citation preview
Evaluating the Aboriginal Programs and Initiatives at the
Department of Canadian Heritage
Canadian Evaluation Society Conference 2003Vancouver, British Columbia
Purpose of Presentation
Provide context to the evaluations within the Aboriginal Affairs Branch at PCH
Present two methodological approaches Aboriginal Languages Initiative – project
site visits Aboriginal Women’s Program – Family
Violence Initiative – learning circle
Discuss evaluation challenges and lessons learned
Discuss next steps
Aboriginal Affairs Branch - PCH
13 programs and initiatives Focussed mainly off-reserve Pan-Aboriginal in nature
Developed over some thirty years Multifaceted approach to complex
Aboriginal issues Facilitate participation in Canadian society Strengthen Aboriginal cultural
distinctiveness
Types of Program Support
Sustaining – established Aboriginal organizations Representative, women’s, friendship centres,
broadcasting National, provincial/territorial, regional
Project
Partnering Accords with territorial governments –
mutual goals Aboriginal organizations
Program Delivery – 3 approaches
Canadian Heritage administration Aboriginal Affairs Branch Regional offices
Aboriginal organizations Six western cities – youth advisory
councils
Comprehensive Review
Retrospective study Environmental scan Consultations – Aboriginal
organizations/groups Evaluation of 13 programs and initiatives
Evaluation Challenges
Complex approach to program delivery
Ensuring a collaborative approach (involvement of delivery partners)
Ensuring cultural sensitivity
Availability of baseline data and performance indicators
Evaluation of the Aboriginal Languages Initiative
Evaluation Methods Document and File Review Secondary Research and Statistical Review Key Informant Interviews Case studies of delivery organizations (AFN,
ITK and MNC) Project Site Visits – 10
Working group and steering committee comprised of representatives from PCH (Aboriginal Affairs Branch and Corporate Review Branch) AFN, ITK, MNC)
ALI Project Site Visits
2 Inuktitut sites, 2 Michif sites, 6 First Nations languages sites
Sites were selected to include all regions and geographic locations (near urban, rural, and isolated) and from each type of “project cluster”
Opportunities and Benefits Opportunities
Conduct face-to fact interviews with those involved directly and indirectly with the project
Conduct focus groups/dialogue circles with project beneficiaries and other community people
Benefits Observe projects and activities directly Place project in context of the community within which
it operates Obtain information on program impact directly from
beneficiaries - how they define success - Obtain information in a culturally sensitive manner Communities see themselves as part of evaluation
process
Evaluation of the AWP Family Violence Initiative
Evaluation Methods
Document& File
Review
Key Informant Interviews
LearningCircle
What is a Learning Circle?
Gathering of people from different organizations
Opportunity for open sharing and exchange Inclusion of range of projects and
perspectives
Learning Circle Process
Sharing knowledge
What have we learned?
Issues and challenges
Stories and journeys
Common threads
Future direction
Learning Circle Opportunities and Benefits
Opportunities to:
Share experiences and insights
Learn
Explore ideas
Reduce isolation
Benefits:
Connection at the community
level
Meaningful results
Lessons learned
Inspiration & momentum
Future program directions
Disadvantage of methodological approaches
Costly and time consuming
Results from site visits or learning circles are not statistically representative of the results of the program as a whole – findings need to be balance with other lines of evidence
One time assessment rather then continuous measurement of results
Importance of including Aboriginal communities in evaluation work – legitimizes results
Qualitative data should be balanced with quantitative information
Evaluations can be designed that meet both government and Aboriginal community information needs
Importance of information sharing
Lessons Learned
Analysis of information Development of position paper Consultation Development of a new consolidated
policy
Aboriginal Affairs BranchNext Steps…
Evaluation reports available electronically
Department of Canadian Heritage Website
www.pch.gc.ca
Presenters Audrey Greyeyes,
Aboriginal Affairs Branch, PCH(819) 994-2121 [email protected]
Dianne Lepa, Corporate Review Branch, PCH (819) 994-7485, [email protected]
Wanda Jamieson, JamiesonHartGraves Consulting (613) 730-2595 [email protected]
