NWT Literacy Strategy Evaluation and Renewal

NWT Literacy Strategy Evaluation and Renewal

Overview

International Adult Literacy and Skills Survey (IALSS)

NWT Literacy Strategy Summative Evaluation

NWT Literacy Strategy Renewal


IALSS measures adult skill proficiencies on a 5-point scale (with 5 as the highest level).

Level 3 is considered the international standard for functional literacy.

Prior to IALSS, the best data on adult literacy available to the NWT was self-reported grade levels from censuses.

Data from the 2003 IALSS provides direct measurements of NWT adult skill levels.


NWT adult literacy levels were close to the Canadian average in the all skill domains.

IALSS confirmed a wide discrepancy between NWT Aboriginal and non-Aboriginal adult literacy levels in all skill domains and for all age groups.

NWT Adult Skill Proficiencies

Prose Literacy

Document Literacy Numeracy

Problem Solving

Level 1 19.3 20.2 25.0 35.7Level 2 25.8 25.9 28.0 36.3Level 3 35.1 33.8 31.7 24.1Level 4/5 19.8 20.1 15.3 3.9

Level 3 Document Literacy Task

Describe the relationship between

fireworks sales and injuries

Prose Literacy

2003 IALSS NWT Prose Proficiency, 16+

0%10%20%30%40%50%60%70%80%90%

100%

Aboriginal Non-Aboriginal

Level 4/5

Level 3

Level 2

Level 1

Document Literacy

2003 IALSS NWT Document Literacy, 16+

0%10%20%30%40%50%60%70%80%90%

100%


Level 4/5

Level 3

Level 2

Level 1

Numeracy

2003 IALSS NWT Numeracy Proficiency, 16+

0%10%20%30%40%50%60%70%80%90%

100%


Level 4/5

Level 3

Level 2

Level 1

Problem Solving

NWT Problem Solving Proficiency, 16+

0%10%20%30%40%50%60%70%80%90%

100%


Level 4/5

Level 3

Level 2

Level 1

IALSS and the Literacy Strategy

IALSS data has been cross-referenced with goals, objectives and actions from the 2001 NWT Literacy Strategy.

Recommendations arising from IALSS research are informing the renewal of the NWT Literacy Strategy.

Example 1: NWT Adult Participation in Training

2003 IALSS Percentage of NWT Population, 16-65,

Receiving Adult Education and Training in 12 Months Preceding IALSS Testing

0

10

20

30

40

50

60

70

80

90

Prose Literacy

Document Literacy

Numeracy Problem Solving

Level 1

Level 2

Level 3

Level 4/5

Example 1: NWT Adult Participation in Training

IALSS data on NWT adult participation in reveals that low literacy adults, who need training the most are accessing it the least.

Recommendations for new Literacy Strategy: – Targeted recruitment of low literacy NWT

adults into training; – Promotion of a lifelong learning perspective.

Example 2: Workplace Literacy IALSS reveals that a significant percentage of NWT Level

1 and 2 adults are in the workplace, so effective workplace literacy programs are crucial to increasing adult skill levels.

Level 1 Level 2 Level 3 Level 4/5Prose 12.4 24.5 39.0 24.1Document 13.3 24.4 37.8 24.5Numeracy 17.7 28.0 35.8 18.6Problem Solving 28.4 38.6 28.1 4.9

2003 IALSS Percent Distribution of Employed NWT Adults,

16 - 65, by Skill Level

Example 2: Workplace Literacy

IALSS draws attention to the issue of skills losses:

“Skills are like muscles that need to be exercised in order to be developed and maintained.”

Of particular concern is the low workplace reading, writing and numeracy engagement rates of NWT Level 1 and 2 adults.


-0.8-0.6-0.4-0.2

00.20.40.6

ReadingIntensity

Writing Intensity NumeracyIntensity

2003 IALSS NWT Reading, Writing and Numeracy Engagement

at Work by Prose Literacy Level

Level 1

Level 2

Level 3

Level 4/5


Continuing to deliver workplace literacy programs that only address skills deficiencies is akin to pouring water into a leaky bucket.

Recommendation for new Literacy Strategy:– Educate employers about skills losses and

encourage them to provide employees with opportunities to use and retain their skills in the workplace.

NWT Literacy Strategy Summative Evaluation

Background Purpose Methodology Successes Challenges Recommendations

Background

Motion – July 7, 2000. January 2001, the GNWT approved the

Northwest Territories Literacy Strategy. Allocation of an additional $2.4 million

annually for literacy initiatives in the NWT.

Towards Literacy: A Strategy Framework 2001-2005 was developed in consultation with the social envelope departments and other GNWT departments and agencies.

Background

Accountability framework was developed in 2002/2003 fiscal year.

Results-based Management and Accountability Framework (RMAF).

RMAF is the de facto standard for evaluation and accountability in Canada.

RMAF used for Literacy Strategy interim and summative evaluations.

Background

Annual reports. Interim evaluation of the Literacy

Strategy (2004)– covered the first 2 fiscal years (2001/2002

and 2002/03) .

Summative evaluation (2007)– covers first 5 fiscal years (2001/02 to

2005/06).

Purpose Assess progress after 5 years of

NWT Literacy Strategy programming.

Gather feedback from stakeholders.

Inform renewal of the Literacy Strategy.

Methodology 4 goals of Towards Literacy: A Strategy

Framework – 2001-2005:1. to increase the number of people in the NWT who are literate2. to increase awareness of the importance of literacy in the official languages of the NWT3. to ensure government departments work together to increase literacy levels in the NWT 4. to develop literacy partnerships across society.

8 target groups – 5 under the Goal #1.

Methodology

Document and data review – program stats, budgets, formative

evaluation results, annual reports, etc.

Consultations in 10 NWT communities:– Aklavik, Inuvik, Norman Wells, Deline,

Fort Simpson, Fort Providence, Fort Smith, Hay River, Hay River Reserve, Yellowknife.

Methodology

Interviews and focus groups with program managers and stakeholders– NGOs, ECE regional offices, ECE HQ, Aurora

College, Aboriginal Language Communities.– Focus on relevance, success, cost-effectiveness.

Surveys of stakeholders – Focus on impact of Literacy Strategy activities

on objectives, sub-goals, goals.

Methodology

Report analyzed and synthesized the information collected to determine relevance, success, and cost-effectiveness.

Examined 4 Literacy Strategy goals, including a roll-up of the 5 sub-goals under Goal #1.

Literacy Strategy overall.

Successes

Increased awareness of literacy throughout the NWT .

$2.4M was a significant investment in literacy (literacy resources, research, capacity).

Increased partnerships and networking. Evaluation and reporting are positive

steps to Literacy Strategy improvement.

Successes

Some individual programs were very successful:– NWTLC – Northern Parenting & Literacy

Program– NWT IALSS data.

Literacy more than just reading and writing– Technology and Internet.

Challenges

Literacy Strategy was a good idea on paper – but complex and difficult to implement– interlinking strategies not well understood– unfunded areas were problematic– literacy needs are long term – no quick fixes– lack of baseline data.

Data collection and reporting were difficult at all levels.

Challenges

Stakeholders indicated there were not enough resources to achieve all goals and objectives– spread thin, rather than focussed on 1 or 2

areas. Staff turnover. No multi-year funding at the project level. Communication and coordination of the

Literacy Strategy were not consistent.

Recommendations

Renew the Literacy Strategy based on results of IALSS and NWT Literacy Strategy Summative Evaluation.

Need a dedicated Literacy Strategy Coordinator.

Take a “targeted” approach.

Recommendations

Design goals and objectives realistically. Adopt a multi-year funding approach. Ensure proper data collection. Design and implement Literacy Strategy

communications plan. Ensure Literacy Strategy partners play an

active role in design and delivery of renewed Literacy Strategy.


The new NWT Literacy Strategy will be informed by IALSS data and research from the Literacy Strategy summative evaluation.

The Literacy Strategy renewal process provides an opportunity to build on literacy initiatives of the past six years and make improvements to literacy programming in the NWT.

Literacy Strategy Renewal

Acting upon recommendations from the summative evaluation, ECE established a multi-stakeholder Literacy Strategy renewal working group.

Representation from ECE, Health and Social Services, Aurora College, NWT Literacy Council, NGO literacy service providers, Aboriginal organizations and industry.


Technical subgroup drafting goals, objectives and actions.

Draft objectives and actions will be presented to larger NWT Literacy Strategy Renewal Working Group on December 13 – 14.


Literacy Strategy Renewal Working Group recommends a 10-year literacy strategy.

Evaluation framework for new Literacy Strategy will be developed concurrent with the Literacy Strategy.

Target date for completion of draft NWT Literacy Strategy is February 2008.

Thank You!

Documents

NWT Literacy Strategy Evaluation and Renewal