National Developmental National Developmental evaluationevaluation

Kauko Hämälainen Kauko Hämälainen Heikki K. LyytinenHeikki K. Lyytinen

The Finnish Education Evaluation CouncilThe Finnish Education Evaluation Council

The Miracle of PISA –Conference 10. -12-Sept. In HelsinkiThe Miracle of PISA –Conference 10. -12-Sept. In Helsinki

ContentContent 1.Where did we come from and where are we going to 1.Where did we come from and where are we going to

in the national evaluation?in the national evaluation? 2. Within what kind of administrative and 2. Within what kind of administrative and

organizational structure we act?organizational structure we act? 3. What kind of demands/requirements we make for the 3. What kind of demands/requirements we make for the

national evaluation?national evaluation? 4. What are the values and principles we set on the 4. What are the values and principles we set on the

evaluation?evaluation? 5. How do we carry out the developmental evaluations 5. How do we carry out the developmental evaluations

in practice ?in practice ? 6. What have been our evaluation themes?6. What have been our evaluation themes? 7. Examples of evaluation practices (evaluation 7. Examples of evaluation practices (evaluation

projects in practice)projects in practice)

Change of Educational Steering System in FinlandChange of Educational Steering System in Finland

Situation in 1970s and 1980sSituation in 1970s and 1980s SituationSituation in 1990s/2000 in 1990s/2000

Centralised control and Centralised control and decision-makingdecision-making

Devolution of power Devolution of power

centralisedcentralised curricula curricula long-termlong-term plans plans budgeting basedbudgeting based on expenditures on expenditures controlling follow upcontrolling follow up external evaluation: external evaluation: inspections inspections

conducted by state authoritiesconducted by state authorities etc. etc.

self-governanceself-governance school-basedschool-based curricula curricula distinctivedistinctive educational profile of educational profile of

educational institutionseducational institutions self-directionself-direction and - and -regulationregulation learning organisationlearning organisation as a model of as a model of

functioningfunctioning self-evaluationself-evaluation and own control and own control performance-based performance-based funding etc.funding etc. new evaluationnew evaluation organisations organisations National evaluation planNational evaluation plan

Evaluation instead of inspectionEvaluation instead of inspection

no separate school inspectorate and no separate school inspectorate and inspection visits to schoolsinspection visits to schools

the activitis of education providers are the activitis of education providers are quided by objectives laid down in legislation quided by objectives laid down in legislation and the national core curricula and the national core curricula the system relies on the proficiency of the system relies on the proficiency of teachers in their efforts to meet the objectivesteachers in their efforts to meet the objectives

strong focus on self-evaluation and external strong focus on self-evaluation and external evaluationevaluation

The Forms of Steering and Improving the Quality of The Forms of Steering and Improving the Quality of Education in the Finnish School System :Education in the Finnish School System :

Focus in steering by information, monitoring and Focus in steering by information, monitoring and evaluation and steering by professional expertiseevaluation and steering by professional expertise

Steering by information, monitoring and evaluationSteering by information, monitoring and evaluation

Steering by Steering by professional professional expertise:expertise: freeing freeing the creative the creative resources of the resources of the schoolschool

Aim:Aim: improved improved quality of quality of educationeducation

Market-driven Market-driven steering:steering: competition competition between schoolsbetween schools

Norm- and resources-based steering

Challenges for renewing evaluation in Challenges for renewing evaluation in educational serviceseducational services

• responsibility to organise educational services also includes evaluating them

•directing educational services requires goals and evaluating how well they have been met (so-called profit responsibility)

• evaluation seen as a comprehensive process (cf. e.g. quality assurance)

• evaluation as a tool for learning and development( developmental evaluation)

• evaluation adds to the active role of actors(empowerment)

• knowledge of the current state of affairs essential when setting goals for development

• controlled change when steering towards future goals (controlled change based on facts/evidence based cahnge management)

TOWARDS NEW EVALUATION CULTURE

1 Explorative/evaluative work community – EVIDENCE BASED

• continuing evaluation – transparency• making use of research and evaluation knowledge

2 Reflective work community - towards culture of EMPOWERMENT

• independence, motivation, atmosphere, appreciation, security

3 Network-oriented operation cultures

4 Making use of parallelisms• peer evaluation, benchmarking …

5 From data management to knowledge management – collective awareness

6 Demands for involvement, multiple perspectives, and consensus – authentic dialog

7 Managing the whole of evaluation

Towards a new evaluation systemTowards a new evaluation system

Internal administrative

evaluation system

New evaluation awareness2-3 years

Transition period2-3 years

Change

Continuous development (new evaluation culture)

Change of evaluation culture

Towards developmental evaluation

External evaluation system of

expert networks

Administrative dependence (”internal evaluation”)

Expert network –based independence (external/internal)

Benefitting from the network both in Benefitting from the network both in evaluation and in project planningevaluation and in project planning

1. Planning group: 1. Planning group: expertise of the field and interest groupsexpertise of the field and interest groups draws up a plan and schedule for the project and its draws up a plan and schedule for the project and its

different stages different stages proposes assessors to be appointed in the evaluation proposes assessors to be appointed in the evaluation

group to the Council group to the Council 2. Evaluation group: 2. Evaluation group: independent experts who work in assessment and research independent experts who work in assessment and research evaluate and draw conclusions with further recommendationsevaluate and draw conclusions with further recommendations write a report and are responsible for itwrite a report and are responsible for it

Evaluation projects are organised in two phases to maximiseEvaluation projects are organised in two phases to maximise best expertise best expertise commitment of the field, partners and interest groups commitment of the field, partners and interest groups independence and independence and impactimpact

National production of evaluation information to National production of evaluation information to serve policy-makingserve policy-making

Parliament

The Ministry of EducationThe Government

The EducationEvaluation Secreteriat

The Evaluation Secreteriat

The Finnish EducationEvaluation Council

14 members

The Finnish HigherEducation Evaluation

Secreteriat12 members

National evaluation and information production network

- Regional Authorities- Provincial State Offices- Provincial Councils- Employment and EconomicDevelopment Centres

UniversitiesPolytechnics

StatisticsFinland

Research Institutesin Univeristies

National Board ofEducation

EducationProviders

Private Evaluation OrganisationsSchools

The other evaluationorganisations(national andinternational)

EducationEvaluation council in EducationEvaluation council in conjunction with the Ministry of Education conjunction with the Ministry of Education since April 2003:since April 2003:

planning, co-ordinating, managing andplanning, co-ordinating, managing and

developing the evaluation of basic educationdeveloping the evaluation of basic education

and upper secondary education and trainingand upper secondary education and training

The polytechnics and univeristies are The polytechnics and univeristies are responsible for the evaluation of their own responsible for the evaluation of their own operations and outcomes (support from operations and outcomes (support from the Higher Education Evaluation Council) the Higher Education Evaluation Council)

The Finnish Higher Education Evaluation The Finnish Higher Education Evaluation

Council Council (FINHEEC)(FINHEEC) (FINHEEC)(FINHEEC) is an independent expert body assisting is an independent expert body assisting

universities, polytechnics and the Ministry of Education universities, polytechnics and the Ministry of Education in matters relating to evaluation. in matters relating to evaluation.

The evaluations conducted by FINHEEC can be divided The evaluations conducted by FINHEEC can be divided into three rough categories: into three rough categories: – Evaluations of higher education institutions: Evaluations of higher education institutions:

Institutional evaluations (universities and Institutional evaluations (universities and polytechnics), audit of quality work, and polytechnics), audit of quality work, and evaluations for the accreditation of the evaluations for the accreditation of the polytechnics polytechnics

– Programme and thematic evaluations Programme and thematic evaluations – Accreditation of professional courses offered by Accreditation of professional courses offered by

higher education institutionshigher education institutions

NATIONAL EVALUATION SYSTEM NATIONAL EVALUATION SYSTEM

Indicators-quantitative-qualitative

Learning outcomes

General knowledge/School subjects

Professional competence

General qualifications-learning to learn

Situation and system-evaluations

Thematicevaluations

Methods and models

International cooperationPISACopenhagenprocess

Evaluation research

Self-assessmentQuality controlExternal and internalauditsPaperless enviromentIndicators

AIMS OF EVALUATIONAIMS OF EVALUATION IN THE FINNISHIN THE FINNISH EDUCATION SYSTEMEDUCATION SYSTEM

The aims of evaluation are to gather and analyse information which serves educational decision-making at the national policy and local levels and contributes to the development of education in general.

In addition, the aim is to support students’ learning and the work of educators, as well as institutional development.

Educational evaluation: Focus on Educational evaluation: Focus on many levelsmany levels

Individuals (pupils, students, teachers)Individuals (pupils, students, teachers) Classrooms/coursesClassrooms/courses Curriculum/programmesCurriculum/programmes Organizations (schools, universities)Organizations (schools, universities) Fields (all schools in a municipality, all Fields (all schools in a municipality, all

programmes in a discipline)programmes in a discipline) The national level (national quality The national level (national quality

development and quality assurance systems)development and quality assurance systems) The international level (PISA,IEA, EQUIS in The international level (PISA,IEA, EQUIS in

the business school area)the business school area)

Regional evaluationRegional evaluation

The Provincial Government Act (22/1997) obliges the provincial The Provincial Government Act (22/1997) obliges the provincial governments to evaluate the accessibility of basic services governments to evaluate the accessibility of basic services within their borderswithin their borders

○ ○ The Ministry of the Interior and other The Ministry of the Interior and other minitstries minitstries define shared evaluation targets for a number of define shared evaluation targets for a number of

years (i.e. the term of the government)years (i.e. the term of the government)

○ ○ evaluations should be carried out co-operatively, evaluations should be carried out co-operatively, consistently, and drawing primary on existing statistics consistently, and drawing primary on existing statistics and and department specific databases department specific databases

DISTRIBUTION OF WORK AND CO-OPERATION IN DISTRIBUTION OF WORK AND CO-OPERATION IN EVALUATIONEVALUATION

Ministry of Education and the Education Evaluation Council are Ministry of Education and the Education Evaluation Council are responsible for external evaluation and its development. Education responsible for external evaluation and its development. Education

providers are primarily responsible for local evaluation and its providers are primarily responsible for local evaluation and its development. development.

The National Board of Education The National Board of Education participates in evaluating syllabuses and learning participates in evaluating syllabuses and learning

achievements within degree regulationsachievements within degree regulationsas a part of educational evaluation expert network as a part of educational evaluation expert network

Links between external evaluation, local evaluation and evaluation of Links between external evaluation, local evaluation and evaluation of basic services should be taken into account when drawing up basic services should be taken into account when drawing up

evaluation programmes and plans. evaluation programmes and plans. Educational institutions and providers who have participated in the Educational institutions and providers who have participated in the evaluation must have access to the evaluation reports on their own evaluation must have access to the evaluation reports on their own

activities. activities.

EVALUATION OF LEARNING EVALUATION OF LEARNING ACHIEVEMENTSACHIEVEMENTS

The National Board of Education The National Board of Education participates in evaluating syllabuses and learning participates in evaluating syllabuses and learning

achievements within degree regulationsachievements within degree regulationsas a part of educational evaluation expert networkas a part of educational evaluation expert network

Evaluation of education in The Evaluation of education in The Finnish National Board of EducationFinnish National Board of Education

quantitative and qualitative indicators will be definedquantitative and qualitative indicators will be defined information in the form of indicators will be producedinformation in the form of indicators will be produced learning achievements will be comparedlearning achievements will be compared learning achievements will be evaluated in different learning achievements will be evaluated in different

school types and levels as follows (sample-based school types and levels as follows (sample-based approach): approach):

* basic education: mathematics and * basic education: mathematics and mother tongue, regularlymother tongue, regularly* basic education: other subjects and * basic education: other subjects and themes (for ex. sustainable development) themes (for ex. sustainable development) * methods of evaluation will be developed* methods of evaluation will be developed

REQUIREMENTS FOR EVALUATION SET BY THE REQUIREMENTS FOR EVALUATION SET BY THE PARLIAMANENT EVALUATION POLICY)PARLIAMANENT EVALUATION POLICY)

strengthening the role of evaluation in developing strengthening the role of evaluation in developing education, adds the neutrality of national evaluation of education, adds the neutrality of national evaluation of education and promotesco-operation and communication education and promotesco-operation and communication between actors in evaluation and interest groupsbetween actors in evaluation and interest groups promotes equalitypromotes equality main emphasis should be in supporting the development main emphasis should be in supporting the development of teaching and learning at school and municipality level of teaching and learning at school and municipality level education providers should be given priorityeducation providers should be given priority evaluation must concern the whole learning environmentevaluation must concern the whole learning environment

Organising evaluationOrganising evaluation

evaluation adequately independent of the rest of the evaluation adequately independent of the rest of the education administrationeducation administration

independent and impartial evaluation actors independent and impartial evaluation actors increasing the impact of evaluation especially from the increasing the impact of evaluation especially from the

viewpoint of educational institutions viewpoint of educational institutions a system must be established to monitor the impact of a system must be established to monitor the impact of

evaluationevaluation

VALUE BASIS OF EVALUATIONVALUE BASIS OF EVALUATION

Equality means promoting educational equality by

means of evaluation.

•Fairness means creating an ethically sustainable foundation for evaluation and refraining from comparison which may be harmful for the target of evaluation.

•Truthfulness means high ethical responsibility and critical examination, as well as optimisation of the reliability of evaluation.

Principles for educational evaluationPrinciples for educational evaluation

Evaluation is based on education policy Evaluation is based on education policy decisions (development aspect)decisions (development aspect)

The legitimacy is founded on educational The legitimacy is founded on educational legislationlegislation

Education evaluation should be transparentEducation evaluation should be transparent The essential outcomes need to be publishedThe essential outcomes need to be published Methods used are to be recorded and justified, Methods used are to be recorded and justified,

as reliable, valid and comparable as possibleas reliable, valid and comparable as possible

In this regard, evaluation must provideIn this regard, evaluation must provide

central and local administration with validcentral and local administration with valid

data on how succesfully educationaldata on how succesfully educational

objectives are being achieved. Thus,objectives are being achieved. Thus,

evaluation should foster self-control andevaluation should foster self-control and

self regulation at the level of central andself regulation at the level of central and

local administration and promotelocal administration and promote

accountability to the various partiesaccountability to the various parties

involved. involved.

From the perspective of educational policy,From the perspective of educational policy,

it is important to ensure that educationit is important to ensure that education

providers deliver the educational servicesproviders deliver the educational services

that they are intended to produce and thatthat they are intended to produce and that

they do this in accordance with the goalsthey do this in accordance with the goals

set for these services and on the basis ofset for these services and on the basis of

professional skills of adequately highprofessional skills of adequately high

standard. standard.

Evaluation process and decision Evaluation process and decision makingmaking

Decision on evaluation

Appointing the planning group

PLANNING GROUP

Planning the evaluation

Evaluation group proposal Developing data gathering and feedback systemsDrawing up instructions for self- assessment Evaluation visits and panels

EEC

Accepting the planAppointing the evaluation groupAccepting methods and measures and self-assessment models

INSTITUTIONS Self-assessment, other material

EVALUATION GROUP

Evaluation visitsInterpreting results

Writing a report

Publishing a report

EEC

1.

2.3.

4.feedback

Quality assurance

THE IMPLEMENTATION OF EVALUATIONTHE IMPLEMENTATION OF EVALUATION (PLANNING AND EVALUATION NETWORK(PLANNING AND EVALUATION NETWORK) )

Educationproviders

National Board of

Education

Education evaluationsecretariat

University of Jyväskylä

School Y (7-9)

School X (1-6)

Collaborative seminars

Education evaluation

council

University of Oulu

Lapp district court sessions

Åbo Akademi

University

Administrative Board of Western Finland University of

Joensuu

University of Turku

University ofLapland

University of Helsinki

University of Jyväskylä

Educationevaluation

council

Implemented evaluation projectsImplemented evaluation projects

Basic educationBasic education1. The implementation and quality of basic security in 1. The implementation and quality of basic security in educationeducation2. The implementation, availability and quality of 2. The implementation, availability and quality of special needs education, remedial teaching and special needs education, remedial teaching and student welfare services in basic educationstudent welfare services in basic education

Upper secondary educationUpper secondary educationThe need for special needs education in upper The need for special needs education in upper secondary school educationsecondary school education

Liberal Adult Education: Institutional Structure, Liberal Adult Education: Institutional Structure, Service Ability and Societal EffectsService Ability and Societal Effects

Training Opportunities and Provider Network of Training Opportunities and Provider Network of Vocational Adult Education on the Secondary Vocational Adult Education on the Secondary LevelLevel

Both projects long lasting: three years in two phases (2004-2005, and Both projects long lasting: three years in two phases (2004-2005, and 2006-2007)2006-2007)

Based on available data: statistics, research reports, documents, the Based on available data: statistics, research reports, documents, the variety of material produced in various development projects etc; no variety of material produced in various development projects etc; no primary data gatheredprimary data gathered

Process: (1) multiperspective planning group comprised of education Process: (1) multiperspective planning group comprised of education providers and relevant interest groups, (2) independent, scientific providers and relevant interest groups, (2) independent, scientific research oriented evaluation groupresearch oriented evaluation group

Adult education

Basic educationBasic education Teaching methods in the comprehensive school from January 1Teaching methods in the comprehensive school from January 1stst

2007 to December 312007 to December 31stst 2007 2007

Upper secondary educationUpper secondary education Collaboration in upper secondary school education and vocational Collaboration in upper secondary school education and vocational

education from October 1education from October 1stst 2005 to October 10 2005 to October 10thth 2006 2006

Vocational educationVocational education Workplace learning from Sebtember 2004 to October 2006Workplace learning from Sebtember 2004 to October 2006 The regional effectiveness of vocational education from January 1The regional effectiveness of vocational education from January 1stst

2006 to December 312006 to December 31stst 2007 2007 Key competencies in vocational education from January 1Key competencies in vocational education from January 1stst 2007 to 2007 to

December 31December 31stst 2007 2007

Adult educationAdult education The competence-based examination systemThe competence-based examination system

Different evaluation Different evaluation frameworksframeworks

EFQM EXCELLENCE MODEL

Leadership& Management

10 %

Processes 14 %

Key Performance

Results 15 %

Personnel (HR)9 %

PersonnelResults

9 %

Policy and Strategy

8 %

Partnership and Resources 9 %

Customer Results

20 %

Society Results

6 %

Action 50 % Results 50 %

Innovation and learning

A Common Quality Assurance FrameworkA Common Quality Assurance Framework

Model (Common Core Criteria)

Methodology (Self assement/Peer Review)

Monitoring(External)

Measurement(Indicators)

GainsMutual Trust Trancparence

Credit Transfer

Quality assurance methods and process, indicators, standards and norms, approach of selfevaluation

QUALITY ASSURANCE MODEL QUALITY ASSURANCE MODEL EUROPEAN COMISSION (LISBON STRATEGY)EUROPEAN COMISSION (LISBON STRATEGY)

PLANNING i.e. strategies, goals, objectives, resources,

anticipation and evaluation mechanism

IMPLEMENTATIONi.e. action plan, resources, partnerships, leadership, process management,

training of trainers, didactical materials

MONITORING AND EVALUATION

i.e. self evaluation, external inspection, internal quality control

Results of teaching, training and learning staff-oriented results,

key performance results, sosietal results

REVIEW feed back and prosedures

for changecontinue improvement

development

METHODOLOGY: quality management,

self evaluation, stakerholders’s role

management of information external evaluation

Examples on evalaution Examples on evalaution projectsprojects

EVALUATION MODEL FOR LEARNING ON THE JOB LEARNINGEVALUATION MODEL FOR LEARNING ON THE JOB LEARNING

1. Quantitative indices as part of self-assessment

2 Self-assessment by education providers-multisectoral evaluation-consensus

4 Description of best practices- Portfolio

Initial assessment by an evaluator team

In-depth analysis of self-assessments, and related visits to educational institutes by the evaluator team

Field-specific evaluation seminars

Provider feedbackFinal evaluation and evaluation report

Bac

kgo

un

d d

ata

web-based evaluation environment

5. Stakeholder questionnaireeducation committees

5. Authority interviews

FROM GOALSFROM GOALSTO INTERACTIONTO INTERACTION

The evaluation The evaluation of basic education of basic education

pedagogy pedagogy

What was evaluated?What was evaluated?

1.1. Key principles of pedagogyKey principles of pedagogy

2.2. Teachers’ goal awareness Teachers’ goal awareness

3.3. Range of teaching methodsRange of teaching methods

4.4. Attendance to the pupils’ unique characteristicsAttendance to the pupils’ unique characteristics

5.5. Equality of learning processes Equality of learning processes

6.6. Functionality of the learning environments’ factors Functionality of the learning environments’ factors

7.7. Teaching, studying and ???Teaching, studying and ???

18.04.2340

How was information produced?How was information produced?

1)1) Survey of teachers (N = 2351)Survey of teachers (N = 2351)

2)2) Survey of rectors (N = 410)Survey of rectors (N = 410)

3)3) Visits to schools ( N = 12)Visits to schools ( N = 12)

4)4) Hearings ( N = 12)Hearings ( N = 12)

5)5) Data and research results from Data and research results from previous statistics, studies and previous statistics, studies and evaluationsevaluations

18.04.2341

TEACHER’S IMPORTANTTEACHER’S IMPORTANTPEDAGOGIC PRINCIPLESPEDAGOGIC PRINCIPLES

– FairnessFairness– Student oriented approachStudent oriented approach– Goal awarenessGoal awareness– Importance of classroom pieceImportance of classroom piece– Stressing action and variationStressing action and variation

– The “objectivity” of pedagogic thinkingThe “objectivity” of pedagogic thinking

– Socio-constructivism has its “opponents”Socio-constructivism has its “opponents”

18.04.2342

TEACHERS’ GOAL AWARENESSTEACHERS’ GOAL AWARENESS

The status of curriculum is significant.The status of curriculum is significant. Goal awareness highlights many aim fields of Goal awareness highlights many aim fields of

education and upbringing.education and upbringing. Text book still has quite a significant role.Text book still has quite a significant role.

Critical approaches to curriculumCritical approaches to curriculum Wishes for time division (lesson plan)Wishes for time division (lesson plan) Goal awareness given – contemplation on aims thinGoal awareness given – contemplation on aims thin

18.04.2343

TAKING STUDENTSTAKING STUDENTSINTO ACCOUNTINTO ACCOUNT

Students appreciated as individualsStudents appreciated as individuals Equity central / significantEquity central / significant Students’ differences and needs importantStudents’ differences and needs important Student oriented approach has increasedStudent oriented approach has increased Making use of students’ activityMaking use of students’ activity

No new remedies for bullying. No new remedies for bullying. Few means of managing disruptive students.Few means of managing disruptive students. Special needs students cannot always be supported.Special needs students cannot always be supported. Possibilities of differentiation limited in practice.Possibilities of differentiation limited in practice.

18.04.2344

RANGE OF RANGE OF TEACHING METHODSTEACHING METHODS

The range and variety of teaching methods is The range and variety of teaching methods is highlighted.highlighted.

Externally most often teacher oriented teaching – Externally most often teacher oriented teaching – with student oriented elements.with student oriented elements.

Instrumental use of teaching materials is becoming Instrumental use of teaching materials is becoming more common.more common.

Information technologies have enriched working Information technologies have enriched working methods and materials.methods and materials.

Examining and experimental methods of working not Examining and experimental methods of working not usual.usual.

Minor changes in teaching.Minor changes in teaching.18.04.2345

PREREQUISITES FOR ACTIONPREREQUISITES FOR ACTION A positive atmosphere for development and shared perception A positive atmosphere for development and shared perception

of teaching positive (rectors’ view)of teaching positive (rectors’ view) The pedagogic know-how of teachers good in this country.The pedagogic know-how of teachers good in this country. Technical equipment and options for teaching material are Technical equipment and options for teaching material are

usually of good quality. usually of good quality. Great variation in the preconditions of action between Great variation in the preconditions of action between

schools. schools. Large and heterogeneous teaching groups is a problem.Large and heterogeneous teaching groups is a problem. Too much variation in the resources for complementary Too much variation in the resources for complementary

education. education. Haste and the feeling of haste in school work have increasedHaste and the feeling of haste in school work have increased Messages of problems with air-conditioning and schools with Messages of problems with air-conditioning and schools with

mold and mildew problems!mold and mildew problems!

18.04.2346

THE IMMEDIATE CHALLENGES OF PEDAGOGY?THE IMMEDIATE CHALLENGES OF PEDAGOGY?

1)1) Large + heterogeneous groups Large + heterogeneous groups (taking into account the (taking into account the broad questions of differentiation)broad questions of differentiation)

2)2) The frequency of class-room disturbance The frequency of class-room disturbance – remedies?– remedies?

3)3) The needs of developing self-evaluation – how? The needs of developing self-evaluation – how?

4)4) The curriculum includes too much content, too The curriculum includes too much content, too demanding goals, and too high criteria for good learning demanding goals, and too high criteria for good learning outcomes, the instructions for evaluation call for outcomes, the instructions for evaluation call for clarifications. clarifications.

5)5) Teachers’ supplementary education has to be taken Teachers’ supplementary education has to be taken more seriouslymore seriously: more time and funding, more equity, : more time and funding, more equity, more long-term planning, teacher oriented approach.more long-term planning, teacher oriented approach.

18.04.2347

LONG-TERM NEEDS IN THE LONG-TERM NEEDS IN THE DEVELOPMENT OF PEDAGOGYDEVELOPMENT OF PEDAGOGY

1.1. Supporting pedagogic leadership at schoolsSupporting pedagogic leadership at schools2.2. General education in the future – launching the discussionGeneral education in the future – launching the discussion

1.1. What are the contents?What are the contents?2.2. How do we learn? How do we learn? 3.3. What do we evaluate?What do we evaluate?

3.3. Enhancing goal awareness and pedagogic thinkingEnhancing goal awareness and pedagogic thinking1.1. levelslevels2.2. steps of advancementsteps of advancement3.3. goalsgoals

4.4. Increasing the amount of the studying, evaluating and developing Increasing the amount of the studying, evaluating and developing approach in the teachers’ work … in the work communities approach in the teachers’ work … in the work communities

5.5. Enhancing the understanding and encountering of the students’ Enhancing the understanding and encountering of the students’ differences, and the pedagogy of developing. differences, and the pedagogy of developing.

6.6. Enhancing learning as collective construction of knowledge Enhancing learning as collective construction of knowledge 7.7. Enhancing the appropriation of media, information technologies, and Enhancing the appropriation of media, information technologies, and

information flows as part of e.g. developing the learning to learn skillsinformation flows as part of e.g. developing the learning to learn skills8.8. Increasing the amount of interaction with the surrounding community Increasing the amount of interaction with the surrounding community

/society (e.g. children’s and young people’s lifestyles and cultures) /society (e.g. children’s and young people’s lifestyles and cultures) 9.9. Developing the evaluation skills (of the schools / teachers /students)Developing the evaluation skills (of the schools / teachers /students)

18.04.2348

DISSEMINATION OF EVALUATION DISSEMINATION OF EVALUATION INFORMATIONINFORMATION

1)1) Publication / announcementsPublication / announcements2)2) Report, feedback for organizers?Report, feedback for organizers?3)3) National, regional, local eventsNational, regional, local events4)4) Articles etc. Articles etc. 6)6) Developing the overall evaluation process from the Developing the overall evaluation process from the

point of view of making use of information point of view of making use of information (collaboration, networking, participation, client (collaboration, networking, participation, client oriented approach)oriented approach)

18.04.2349

The process in shortThe process in short

The evaluation groupThe evaluation group brings all information together brings all information together describes the state of affairsdescribes the state of affairs defines the criteriadefines the criteria evaluates the state of affairs evaluates the state of affairs points out the needs of developmentpoints out the needs of development makes some general recommendationsmakes some general recommendations takes part in the distribution of the resultstakes part in the distribution of the results