ORIENTE and Indicators ORIENTE and Indicators of Change for of Change for

Innovation in Education Innovation in Education

Friedrich Scheuermann

Institute for Future Studiesweb:www.futurestudies.org

mail: scheuermann@futurestudies.org

And input from

Dr. Mario BarajasUniversity of Barcelona

Contents

The ORIENTE network

Example: DELPHI project

Reflections

○●○○○○○○○○○○○○○○○○○○○○○

ORIENTE

O bservatory of

R esearch on

I nnovation in

E ducation:

N ew

T echnology

E valuation

European network of researchers specialised in monitoring and evaluation ongoing research tackling educational, institutional, organisational and symbolic aspects of new learning environments mediated by ICT.

○○●○○○○○○○○○○○○○○○○○○○○

Steering Committee

Dr. Katherina KikisFoundation for Research and Technology-Hellas (FORTH)Institute of Applied and Computational Mathematics Evaluation - Education and Training DivisionHeraklion, Crete (Greece)

Dr. Mario BarajasUniversity of Barcelona, SpainDepartment of Didactics and Educational Organisation

Dr. Sofoklis SotiriouEllinogermaniki Agogi, GreeceDepartment of Research and Development

Friedrich ScheuermannAssociation for Flexible Learning (Austria)Institute for Future Studies

○○○●○○○○○○○○○○○○○○○○○○○

Basis of Work

• Individual base• Evaluation activities on a national and European level• Own research

• Institutional base• EU projects

• Network base

• Evaluation Research• EU projects• Publications, conferences

○○○○●○○○○○○○○○○○○○○○○○○

Common projects: Institutional base

Implementation of Virtual Environments in Training and Education. A Thematic Network (Ivette). Programme Targeted Socio-Economic Research. 1998-2000

Implementation of Virtual Environments in Training and Education-Workshop (Ivette-W). Programme Improving the Human Potential and the Socio-Economic Knowledge Base, Accompanying Measures. 2000-2001

Monitoring and Evaluation of Research in Learning Innovations (Merlin). Programme Improving the Human Potential and the Socio-Economic Knowledge Base, Accompanying Measures. 2000-2002.

Monitoring and Evaluation of Research in Learning Innovations (DELPHI). Elearning Action Plan. 2002-2004. ○○○○○●○○○○○○○○○○○○○○○○○

Methods of Investigation

The Network uses different approaches to evaluation, from cluster evaluation, case studies, using quantitative and qualitative methodologies, monitoring progress and appraising outcomes through both external assessment or participatory designs.

○○○○○○●○○○○○○○○○○○○○○○○

Example

DELPHIDELPHI European Observatory of e-Learning European Observatory of e-Learning practicespractices

e-Learning , 2nd. Calle-Learning , 2nd. Call

http://www.ub.es/euelearning/delphttp://www.ub.es/euelearning/delphi/hi/

○○○○○○○●○○○○○○○○○○○○○○○

Aims of DELPHI

The analysis focuses on the outcomes from EU The analysis focuses on the outcomes from EU projects of the 5th framework, programmes and projects of the 5th framework, programmes and actions IST, MINERVA and IHP. The overall objective actions IST, MINERVA and IHP. The overall objective of DELPHI is to capitalise on the results of a of DELPHI is to capitalise on the results of a selection of projects in their different contexts, selection of projects in their different contexts, looking for approaches to innovation in projects looking for approaches to innovation in projects where ICT play a key role in teaching and e-learning where ICT play a key role in teaching and e-learning processesprocesses

A key goal of DELPHI is the implementation of an A key goal of DELPHI is the implementation of an Internet-based Internet-based ObservatoryObservatory on learning innovation, to on learning innovation, to present and disseminate research work of EU funded present and disseminate research work of EU funded projects, and toprojects, and to Establish a Establish a LaboratoryLaboratory as an active networking as an active networking promoter of researches and as a disseminator of promoter of researches and as a disseminator of learning innovations.learning innovations.

○○○○○○○○●○○○○○○○○○○○○○○

Projects selected

18 MINERVA projects18 MINERVA projectsOpenTess, AgroWeb, Emile, Arte, OpenTess, AgroWeb, Emile, Arte, Sett, Euro.Geo, Benvic, Client, Sett, Euro.Geo, Benvic, Client, Suessis, Aen, NetCampus, KidNet, Suessis, Aen, NetCampus, KidNet, I-Sea-U, ODL, Eurex, Vitalagell, I-Sea-U, ODL, Eurex, Vitalagell, Vimims, GaleciaVimims, Galecia

8 IST projects8 IST projectsCuber, Trial Solutions, Weblab, Cuber, Trial Solutions, Weblab, Time2Learn, Mobilearn, K2, ItcoleTime2Learn, Mobilearn, K2, Itcole

4 IHP cluster projects4 IHP cluster projects

○○○○○○○○○●○○○○○○○○○○○○○

Foci of Analysis

The foci of analysis for the DELPHI The foci of analysis for the DELPHI project cases were on:project cases were on:pedagogic innovation pedagogic innovation institutional/organizational innovation institutional/organizational innovation socio-cultural and socio-economic socio-cultural and socio-economic innovationinnovation

Qualitative IndicatorsQualitative Indicators (QI) point to (QI) point to relevant changes of ICT-based learning relevant changes of ICT-based learning that are present in the projects that are present in the projects reviewedreviewed

○○○○○○○○○○●○○○○○○○○○○○○

Qualitative Indicators of Change

The study of emergent practices, The study of emergent practices, given their not very well defined given their not very well defined characteristics, require characteristics, require QIQI of change. of change.

QIQI shape the innovations by shape the innovations by investing them with characteristics investing them with characteristics that could be found in related that could be found in related innovations that occur in similar innovations that occur in similar contextscontexts

○○○○○○○○○○○●○○○○○○○○○○○

Pedagogical Dimension: Indicators

Teacher and student rolesTeacher and student rolesApproaches and scenarios to e-Approaches and scenarios to e-

Learning Learning Teaching and learning techniques, Teaching and learning techniques, methods, and devicesmethods, and devicesTeacher/students interactionTeacher/students interactionTeacher workloadTeacher workloadTeacher collaborationTeacher collaborationAssessmentAssessment

○○○○○○○○○○○○●○○○○○○○○○○

Institutional/organisational Dimensions: Key Indicators

Institutional changes resulted from the use of ICT into existing structuresThe role of staff trainingRaising the capacity of the teachers to usebasic ICTs applicationsMain actors, adopters and resisters to the adoption ofthe innovation as identified in the projectOrganisational conditions that are (un)supportive toinnovation FlexibilityAccessibility

○○○○○○○○○○○○○●○○○○○○○○○

Other socio-Economic and cultural aspects of the innovations

E-learning standardsGlobalisationOther socio-cultural factors influencing learning processesFunding & commercialisationImplications for LLL

○○○○○○○○○○○○○○●○○○○○○○○

Final Questions

What was considered innovative?What was considered innovative? What was the specific roles of ICT in the What was the specific roles of ICT in the

innovations?innovations? Were the innovations studied Were the innovations studied

sustainable/scalable/transferable?sustainable/scalable/transferable?

○○○○○○○○○○○○○○○●○○○○○○○

Some reflections in the light of the conference topics

An invitation for a common discussion…An invitation for a common discussion…

○○○○○○○○○○○○○○○○○●○○○○○

Reflections

“…“…knowledge is produced by children themselves.”knowledge is produced by children themselves.”

Can we take it for granted that children learn and Can we take it for granted that children learn and learn effectively (in the context of formal settings and learn effectively (in the context of formal settings and pre-defined goals), even when the rightpre-defined goals), even when the right

-technological context (infrastructure) technological context (infrastructure) -““Content” (material)Content” (material)-EnvironmentEnvironment

is there?is there?

○○○○○○○○○○○○○○○○○○●○○○○○

Claim

Adequate pedagogical concepts are needed in Adequate pedagogical concepts are needed in order to support learningorder to support learning, adapted to the , adapted to the

-Context of the learnerContext of the learner-Context of the school environment and teachersContext of the school environment and teachers-Socio-cultural contextSocio-cultural context-Subject matterSubject matter-Available technological infrastructure, which is not Available technological infrastructure, which is not

limited limited to ICT componentsto ICT components

The ICT integration encourages better pedagogical The ICT integration encourages better pedagogical methods to be applied, however the rich variety of tools methods to be applied, however the rich variety of tools and possibilities make the pedagocial work even more and possibilities make the pedagocial work even more complex.complex.

○○○○○○○○○○○○○○○○○○○●○○○○

Some observations and Concerns

Despite given potentials we can sometimes observe Despite given potentials we can sometimes observe increasing pedagogical poverty instead of richness increasing pedagogical poverty instead of richness supported by ICT.supported by ICT.

The development and application of effective The development and application of effective pedagogical concepts require intensive reflection. Do we pedagogical concepts require intensive reflection. Do we have / get the time for it?have / get the time for it?

Are we (esp. teachers, learners) prepared for new Are we (esp. teachers, learners) prepared for new student-centred approaches for active learning?student-centred approaches for active learning?

○○○○○○○○○○○○○○○○○○○●○○○

Teacher Training

The new roles of teachers and students require new The new roles of teachers and students require new concepts for teaching and learning.concepts for teaching and learning.

Teacher training needs to better support teachers with Teacher training needs to better support teachers with their needs to learn about adequate pedagogical their needs to learn about adequate pedagogical methodologies in their given context (ideas, scenarios, methodologies in their given context (ideas, scenarios, support, evaluation). support, evaluation).

○○○○○○○○○○○○○○○○○○○○●○○

Conclusion

1)1)Projects (technologies, applications) presented Projects (technologies, applications) presented provide a good potential for pedagogical improvements. provide a good potential for pedagogical improvements. More emphasis is needed in the future to the process of More emphasis is needed in the future to the process of teaching and learning and support need for adequate teaching and learning and support need for adequate implementation.implementation.

2)2)Better connection of different contexts of formal, Better connection of different contexts of formal, non-formal and informal learning.non-formal and informal learning.

3)3)Flexible and individualised concepts are needed for Flexible and individualised concepts are needed for the future.the future.

4)4)Educational policy and financial investments must Educational policy and financial investments must reflect that what we consider to be effective in reflect that what we consider to be effective in education.education.

○○○○○○○○○○○○○○○○○○○○○●○

Future Plans

•Open network for European Research and Evaluation activities

• Become a member ! Free subscription...• Non-profit association

•Web-Site with manyfold information and communication facilities:

http://www.oriente.info

○○○○○○○○○○○○○○○○○○○○○○●

Some DELPHI conclusions so far…

●○○○○○○○○○○

a.a.Effectiveness of the methodological Effectiveness of the methodological approachesapproaches to the ICT-based teaching/learning is to the ICT-based teaching/learning is dependent on the degree to which the activities dependent on the degree to which the activities implemented with ICT are consistent with the regular implemented with ICT are consistent with the regular school practices and/or school missionschool practices and/or school mission

a.a.The application of ICT in teaching/learning The application of ICT in teaching/learning does not follow one particular methodological does not follow one particular methodological approachapproach. HOWEVER, indications suggest that ICT . HOWEVER, indications suggest that ICT facilitates a progressive shift from traditional paradigms facilitates a progressive shift from traditional paradigms to more collaborative and participative oriented onesto more collaborative and participative oriented ones

b.b.Motivation appears to be increasedMotivation appears to be increased as a result as a result of the personal control of the learning process through of the personal control of the learning process through the integration of ICTthe integration of ICT

Some DELPHI conclusions so far…

●○○○○○○○○○○

g. the culture of collaborationg. the culture of collaboration. . In online learning In online learning this entails this entails close links being built between all actors close links being built between all actors including those involved from a distance. Attached to including those involved from a distance. Attached to this is the issue of language and cultural sensitivity.this is the issue of language and cultural sensitivity.

h.h. flexibility in the curriculumflexibility in the curriculum: : Curricular Curricular structures are aware of the notion of “alternative structures are aware of the notion of “alternative teaching and learning approaches”, different time teaching and learning approaches”, different time schedules, interdisciplinarity, and different assessment schedules, interdisciplinarity, and different assessment approaches.approaches.

i.i. the institutional approach to achieve its the institutional approach to achieve its mission: mission: institutions develop a positive view towards institutions develop a positive view towards provisions of educational services for online learning and provisions of educational services for online learning and for the added value of ICT-based learning.for the added value of ICT-based learning.

DELPHI conclusions

●○○○○○○○○○○

d.d.Changes emerge as teachers come to Changes emerge as teachers come to understand of the roles they need to play in ICT-understand of the roles they need to play in ICT-based learning settingsbased learning settings. In other words, it is the . In other words, it is the interplay between the innovation and the perceived role interplay between the innovation and the perceived role that shapes the new support provisions required in ICT-that shapes the new support provisions required in ICT-based learningbased learning

e.e.The role of the teachers is shiftingThe role of the teachers is shifting from an from an agent for knowledge transmission and designer and agent for knowledge transmission and designer and leader of student activities to an agent for supporting leader of student activities to an agent for supporting learnerslearners

f.f.In terms of the learners, they take a more In terms of the learners, they take a more active and self-reliant roleactive and self-reliant role. Students can also play . Students can also play roles traditionally executed by the teacherroles traditionally executed by the teacher