Christina PrestonInternational Doctorate in Education

Faculty of Culture and PedagogyDepartment of Learning, Curriculum and Culture

How do

multimodal concept maps

provide insights into teachers’ priorities in a digital

technologies CPD context?

Draft, 40,000 Ed.D. Professional study Chapter One

Research question and literature survey Chapter Two

The research design and toolsChapter Three

The control group: information transmission international computer-based trainingChapter Four

Cohort one: mentored constructive action-based research between England and South AfricaChapter Five

Cohort Two: social interactional practice-based research in England (that includes members of my CoP)Chapter Six

Conclusions that compare the results from the three cohorts (and includes the observations of the co-researchers who have published in this area).

Questions for your consideration

Your advice in this seminar ….

Background to the study

Appeal of the maps

Research questions

Data collection

Quantitative analysis by number

Qualitative analysis by word

Qualitative analysis by sign

Unexpected data

The role of the researcher

Recommendations to the researcher

Further research

Where to stop?

The background to the study

Despite the complexity of pedagogical practice, and indeed as a result of its interdependence

with the regulatory frameworks of the national community, there is very strong evidence that

innovation in pedagogy can be introduced rapidly if they are tied to changes in what is assessed.

B. Somekh (2007) Pedagogy and Learning with ICT Routledge Oxford p.42

My professional practice-based research Preston, C. (2004). Learning to use ICT in Classrooms: teachers' and trainers'

perspectives : an evaluation of the English NOF ICT teacher training programme

1999-2003. London, MirandaNet and the Teacher Training Agency

www.mirandanet.ac.uk/tta.

Preston, C. and J. Cuthell (2007 ). The Perspectives of Professional Educators' on

ICT CPD: Past, Present, Future. N. Pachler. London, NAACE, MirandaNet

Fellowship, WLE, Institute of Education, University of London.

Davis, N., C. Preston, et al. (2008 in press). ICT teacher training impacts multiple

ecologies: evidence from a national initiative. British Journal of Education

Technology (BJET).

Davis, N. E. (2008 in press). How may teacher learning be promoted for educational

renewal with IT? . International handbook of information technology in education. J.

Voogt and G. Knezek. Amsterdam, Kluwer Press.

Davis, N. E., C. Preston, et al. (2008 in press). Theoretical and evaluation

frameworks to inform technology-related professional development for teachers,

tested with evidence from a national study of ICT professional development for

teachers. British Educational Research Journal.

Impact2 (2002) BectaMavers, M., B. Somekh, et al. (2002). "Interpreting the externalised images of pupils' conceptions of ICT: methods for the analysis of concept maps." Computers and Education 38: pp 187-207.

Impact2 (2002) BectaMavers, M., B. Somekh, et al. (2002). "Interpreting the externalised images of pupils' conceptions of ICT: methods for the analysis of concept maps." Computers and Education 38: pp 187-207.

The appeal of MCM

‘Fascinating cultural artifacts’

Mavers 2002

Amy ECDL ICT skills course: Amy ECDL ICT skills course:Figure Two Before Figure Two AfterFirst map Amy CBT

Amy ECDL ICT skills course: Amy ECDL ICT skills course:Figure Two Before Figure Two After

Second map Amy CBT

First map : Tom CBT

Bill ECDL ICT skills course : Figure Three AfterSecond map : Tom CBT

The research questions

The research question

Can semiotic theory increase insights into teachers’ priorities as illustrated in MCM produced in the context of ICT CPD?

The three sub-questions are:

In which ways do MCM scoring techniques provide insights into teachers’ priorities about digital technologies?

In which ways does the labelling of nodes in MCM provide insights into teachers’ priorities about digital technologies?

In which ways does the analysis of sign-making in MCM provide insights into teachers’ priorities about digital technologies?

The themes emerging from the grounded research study, the

attendant literature and my practice

The varied ways in which teachers’ MCM design reflects the cognitive theory that underpins their CPD programme

How the interpretation of learning quality illustrated by the MCM is affected by the multiple roles of the researcher and the co-researchers

• 52 pairs of multimodal concept maps (MCM) (from international collection of 250)

• all map-makers were self-declared ‘advisors’ (teaching other teachers)

• all pairs collected at the beginning and end of a one year course in digital technologies

• same main question reworked from Impact2

How do computers impact on your personal and professional life?

• same procedures reworked from Impact2

20 minutes, pen and paper

Data collection 2003 -2006

Three cohorts Control group (CBT) x 18 ‘advisory’ teachers taking an international computer based training course (CBT) based on information transmission (selected from 100)

First cohort (MAR) x15 ‘advisory’ teachers working in one province in South Africa taking a MirandaNet mentored action research (MAR) CPD programme based on constructive learning (selected from 30)

Second cohort (SCoP)19 ‘advisory’ in England at the IOE taking a MirandaNet designed a scholarly community of practice CPD programme based on social interaction (selected from 25) Some were also members and co-researchers in the CoP, MirandaNet).

Data collection 2003 -2006

Tools for semiotic analysis of the MCM

Number : quantitative scores• connectivity scoring method from Impact2 showing cognitive activity

• new activity score presenting accumulation of concepts

Word: qualitative content analysis• Score sheet developed Impact2 headings that concentrated on basic computer concepts, concrete and abstract

• Transcript analysis that

- establishes learning processes adapted from Daly and Pachler (2007)

- establishes the map makers in SCoP as co-researchers

Word: qualitative content analysisSemiotic framework developed from the work of Jewitt, Kress, Mavers, Somekh, Van Leeuwen (with words included as one element of the analysis)

Theme one: the cognitive theory underpinning the three CPD programmes (Pachler 2005)

Information transmission: learning concepts to reproduce for a test

Constructive learning: mentored action based research

Social interaction: sharing practice based research in a CoP that continues beyond the end of a course with the intention of changing practice and policy

Theme two : the perspective of the researcher (Adler and Adler 1987)

Peripheral member researcher

Active member researcher

Complete member researcher / CoP co-researchers

Quantitative analysisby number

Quantitative Analysis Marton and Booth, (1997), Pearson and Somekh, (2000) & Marton, (1994)

Node - discrete item or set of items

Links - lines from one node to another

Connectivity - number of links

number of nodes

Impact2Method One

Quantitative analysisMarton 1994,Marton and Booth1997, Pearson and Somekh 2000, Mavers, Somekh, Rosterick 2002 (4,000 maps)

88 = 1:1 ratio

SCoP group: Table of connectivity and activity scores

FIRST MAP SECOND MAP FIRST MAP SECOND MAP FIRST MAP SECOND MAP FIRST MAP SECOND MAP FIRST MAP SECOND MAP FIRST MAP SECOND MAP FIRST MAP SECOND MAP1 ID001 Tim 20 36 13 12 1.54 3.00 0 5 0 1 2 2 2 82 ID002 Bernice 31 56 24 42 1.29 1.33 7 11 3 0 2 4 12 153 ID003 Dennis 19 18 20 22 0.95 0.82 3 8 2 1 0 2 5 114 ID004 David 27 52 38 67 0.71 0.78 5 9 0 0 3 2 8 115 ID005 Larry 17 7 22 20 0.77 0.35 6 4 1 0 0 1 7 56 ID006 Kevin 6 18 14 16 0.43 1.13 3 10 3 0 2 5 8 157 ID007 Pippa 57 36 44 40 1.30 0.90 9 16 0 0 3 3 12 198 ID008 Alex 18 9 23 15 0.78 0.60 5 9 0 1 2 3 7 139 ID009 Mark 64 39 46 39 1.39 1.00 10 15 0 0 3 5 13 2010 ID010 Richard 26 51 26 46 1.00 1.11 3 6 3 2 2 1 8 911 ID011 Mel 5 12 6 10 0.83 1.20 3 4 2 1 2 1 7 612 ID012 Paul 28 22 26 21 1.08 1.05 6 12 2 4 1 5 9 2113 ID013 Martin 24 24 24 32 1.00 0.75 10 4 2 4 5 3 17 1114 ID014 Neil 69 29 59 26 1.17 1.12 12 8 2 0 5 4 19 1215 ID015 Malcolm 37 34 39 24 0.95 1.42 7 14 1 4 3 4 11 2216 ID016 Kate 11 29 12 27 0.92 1.07 7 14 2 2 3 2 12 18

AVERAGE 28.69 29.50 27.25 28.69 1.01 1.10 6.00 9.31 1.44 1.25 2.38 2.94 9.81 13.50

ACTIVITY SCOREID NAME

LINKS NODES SCORE SPHERE OF THINKING SPHERE OF THINKING ZONES OF USE

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

SCORE FIRST MAP

SCORE SECOND MAP

Figure 5.2.b. Chart of ScoP connectivity scores

0

5

10

15

20

25

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Series1

Series2

Figure 5.2.c. Chart of ScoP activity scores

ECDL ACTIVITY SCORE CHART

0200400600800

100012001400160018002000220024002600280030003200340036003800400042004400460048005000520054005600580060006200

0001 0002 0010 0012 0015 0023 0082 0085 0095 0096 0098

STUDENT ID

SCORE

ECDL ACTIVITY SCORE1ST MAP

ECDL ACTIVITY SCORE2ND MAP

TAR ACTIVITY SCORE CHART

0

200

400

600

800

1000

1200

1400

1600

1800

2000

2200

2400

2600

2800

3000

3200

3400

3600

3800

4000

4200

4400

4600

4800

5000

5200

5400

5600

5800

6000

6200

001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016STUDENT ID

SCORE

TAR ACITIVITY SCORE1ST MAP

TAR ACTIVITY SCORE2ND MAP

Significant differencesbetween the cohorts in cognitive activity

2 cohorts of x20 teachers who had

good IT skills

Draft charts

Neil’s first map

Second map Neil

…Talking about mess… no drawings, there are absolutely no drawings on this because I tend to concentrate on maps for developing linguistic skills. I’ve been doing mapping for a very long time and teaching the children how to map so it

becomes almost second nature,…

Neil: long term member of MirandaNet and expert practice based research mapper

From Bloom’s perspective, my first map is overcomplicated, difficult to read and inadequate in categorising the concepts to prove higher order thinking. In the second map, therefore, I have consciously used the affordances of digital mapping to refine and categorise my ideas to illustrate higher order thinking.

Tentative conclusions about MCMInteresting results for groups if you are working as a peripheral researcher (PMR) but only tells you about qualitity of connections not quality

But inconsistent when individual scores are considered….

Professional peripheral researchers may still want to adapt scoring techniques to follow patterns of behaviour in one parameter….

But for classroom practice based researchers the numbers too small, techniques too time-consuming…

Although discussions about the scores between the learners might be profitable….they will require triangulation with other methods

If relying on scores it seems necessary to teach mapping first, like Novak, and explain how the formulas are being used. Impact2 side-stepped this issue by interviewing the map makers…

Qualitative analysis

Word

Impact 2Method Two

Qualitative analysis

Signs and words

Spheres of Thinking

Uses of computers Abstract thoughts about computers

Zones of Use

Score sheet establishing frequency of concepts across a group

START FINISH START FINISH START FINISH START FINISH START FINISH START FINISH START FINISHCOMPUTER SYSTEMS COMPUTER SYSTEMS PUBLISHING/ PUBLISHING/

& NETWORKS & NETWORKS CORRESPONDENCE CORRESPONDENCE

Computer x 4 Computer x 6 Search the Net x 1 0

Networks of communicating Professional x 2

Networks of communicating Professional x 1

Preparation of lessons x 1 0 0 PC games x 1

Music x 2 Music x 2

Stimulating new thoughts x 1 0

0 Smart board x 2 Websites x 3 Websites x 1 Communication x 5Communication x 3 Printing of lessons x 1 0 0 Playstation x 1 0 Personal Music x 1

0 Programmes - maths x 1 Internet x 6 Internet x 8 E-mail x 10 E-mail x 10Scanning of lessons x 1 0 Sport x 1 Sport x 2 0 Entertainment x 1 0

Stimulating innovative ideas x 1

0 ICT suite x 2 0 Broadband x 3 Texting x 1 Texting x 3 Reports x 1 0 Online gambling x 1 0 0 Records x 1 Improving standards x 1 0Multimedia x 2 multimedia x 1 The Web x 3 The Web x 4 Forum x 4 Forum x 3 0 Greeting cards x 2 0 Betting clubs x 1 0 TV x 1 Maths reader x 1 Maths/reader x 1

0 Laptop x 3 Surf Net x 1 0 0 Satellite x 1 0 History project x 2 0 Hobbies x1 0 Radio x 1 GTC course x 1 GTC course x 1

0 IWB x 1 Intranet x 1 0 0 Messenger x 2 0Lesson curriculum x 1 0 Box games x 1 0 Sound x 1 CPD x 2 CPD x 3

0 Desktop x 1 Downloads x 1 0 Discussion x 4 Discussion x 4 0School newspaper x 1 Games x 1 Games x 1 0 Self development x 1

PC x 1 PC x 1 Ideas x 1 Ideas x 2 Chat x 2 Chat x 1 0 Scripts x 1 Gaming x 1 0 0Professional development x 1

0Interactive white board x 2 Info x 3 Info x 5 0 Net meeting x 2 0 Notes on meeting x 1 GNVQ x 1 0

0 Network x 4 Information x 2 Information x 2 0 Social interaction x 2 0 Journal x 1 Applied GCSE x 1 0Hardware x 1 0 Research x 3 Research x 4 0 ejournal x 3 E-learning x 2 E-learning x 3Software x 3 0 Resources x 3 Resources x 2 Chatrooms x1 0 Word processing x 1 Distance learning x 1 0

0 Whiteboard x 1 Searches x 3 0 Faxes x 1 0 Publishing x 1 0 0 Digital course x 10 Wireless network TTA Project x 1 0 0 Facilitating learning x 20 Wireless x 1 PowerPoint x 1 0 0 T + L x 1

0 Scanner x 1 0PowerPoint presentation x 1 0 SATs x 1

0 GTCE x 1 Records x 1 0 0 M.Ed course online x 1Storage systems x1 0

Communication assessment x 1 0 0 Learning Group x 1

0 CD-Roms x 1 Tasks x 1 0Photoalbums x 1 0

0 Paperwork x 1Excel modelling x 1 0Word x 1 0

W/sheet documents x 1 0

0Resources text visual x1

Research Project x 1 00 Creating Resources

12 29 31 32 30 32 13 12 4 8 2 8 9 17

LEARNING LEARNING

Bars 1 and 2 Bars 3 and 4 Bars 5 and 6 Bars 7 and 8 Bars 9 and 10 Bars 11 and 12 Bars 13 and 14

INFORMATION GAMES SOUND SOUNDINFORMATION COMMUNICATION COMMUNICATION GAMES

0

5

10

15

20

25

30

35

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Series1

The SCoP group content analysis

Greatest gain Greatest gain

Results: the same analysis of the signs?Daly and Pachler

2007CBT MCM MAR MCM SCoP MCM

& transcripts &collaborative map

cognition abstract and concrete concept decrease

‘learning’ becomes ‘work’

significant increase in abstract and concrete concepts

significant increase in abstract, concrete and scholarly concepts

autobiography despondency and stress about tests

decrease in humour

great hopes of the potential of computers in learning

concentration on the processes and emotions associated with learning

knowledge construction

(constructivism)

Limited - majority of scores drop

Knowledge construction by individuals and the group

meta-learning none (An increase in the use of verbs of doing)

Evidence at the start as well as at the end. Some significant higher order thinking.

Community

(social interaction)

20% comment on communicating socially ; none professionally

Hopes of local, national and global connections envisaged

Professional community links a strong feature of the second maps

WordsTentative conclusions

A researchers find what they are looking for!

CPD programme content

Concrete and abstract concepts related to what is known. Very useful for needs analysis. Surprising what was known before the course started. Also indicated different nature of cohorts that were superficially ‘advisors’.

The cognitive theory underpinning the CPD programme

Daly and Pachler (2007) - social interaction with teachers (MTeach)

Manifest and latent concepts about learning process are valued

in words for the tutor.

cognition

autobiography

knowledge construction (constructivism)

meta-learning

community (social interaction)

Signs(with words as one semiotic unit)

The framework criteriaInformation Transmission

Concepts: graphic and words

Modalities

Materiality of Meaning

Dimensionality

Constructive learning process (autobiography)

Compositional elements and their interrelations

Narratives

Affectual and ludic qualities

Dynamics

Social interaction process

Representations and interaction between the map maker, audience/viewer (and co-researcher)

SignsTentative conclusions

Fascinating opportunity for researchers, tutors, practice-based researchers, and co-researchers

• to view a snapshot of a learner’s priorities at one point in time • to deduce how those priorities might merge with those of the tutor• to increase skills in organising and communicating concepts• to benefit those who are digitally literate• to offer an innovative means of assessment of and for learning• to act as a scaffold in conversations with and between learners about learning progress over a period of time• to act as a scaffold for promoting social interaction

Suggestions for researchers:• decide to teach mapping first, or not• develop data collection procedures about graphics, digital maps, collaborative maps and remotely web maps• avoid scoring……

Adjusting the semiotic framework?

Information Transmission

Concepts: graphics and words

Modalities

Materiality of Meaning

Dimensionality

Constructivism (autobiography)

Compositional elements and their interrelations

Narratives

Affectual and ludic qualities

Dynamics

Social interaction (add Daly and Pachler elements)Representations and interaction between the map maker, audience/viewer (and co-researcher)

Dynamics?

Compositional elements and their interrelations?

N.B. Only one collaborative map by accident and one digital map

Adding to the framework for semiotic analysis in terms of constructive learning and social interaction?

?Daly and Pachler

2007CBT MCM MAR MCM SCoP MCM

& transcripts &collaborative map

cognition abstract and concrete concept decrease

‘learning’ becomes ‘work’

significant increase in abstract and concrete concepts

significant increase in abstract, concrete and scholarly concepts

autobiography despondency and stress about tests

decrease in humour

great hopes of the potential of computers in learning

concentration on the processes and emotions associated with learning

knowledge construction

(constructivism)

Limited - majority of scores drop

Knowledge construction by individuals and the group

meta-learning none (An increase in the use of verbs of doing)

Evidence at the start as well as at the end. Some significant higher order thinking.

Community

(social interaction)

20% comment on communicating socially ; none professionally

Hopes of local, national and global connections envisaged

Professional community links a strong feature of the second maps

Adding to the framework for semiotic analysis?

Constructive learning and social interaction

Somekh and Pearson (2003-2006) - PERLS - with young learners

A theory of transformatory learning mediated by context

Learning creatively

Learning as an active citizen (not quite community)

Engaging powerfully with new ideas

Reflecting on own learning

Social interaction and learning

The unexpected…..

The unexpected…..

The pain of learning….

Malcolm : anxiety and concern

Malcolm : the ‘fun’ narrative%urse:Figure Nine Before

Malcolm: Right well one of the immediate things I see of mine is that last

time there were, I think, concerns and question marks as to what’s going to

happen in the future, seem to figure more … the diagram is much bigger

its got more things on it,…I seem to have developed more ideas on what e-

learning might be. They’re questions still but they’re not necessarily

concerns. My one concern within the evaluation, is in the forum so I’ve got

that on my new diagram here, but words like fun and how I’m going to use

it at school and my increasing use of the internet shows that using all this

stuff in various aspects in my lif e has increased. It doesn’t mean that I

know anymore about what I’m doing but it seems to be that I’ve got some

idea of how it might fit in with these other aspects of education.

A. So the concerns that you had here haven’t mapped across into this

second map. Have they dissipated, or you’ve come to terms with that?

The unexpected…..

The power of collaborative theory development

Malcolm and other co-researchers: a scholarly contribution to e-learning theoryen After

The unexpected…..

A practitioner and an academic journal volume about MCM

Who are the researchers? Who are the learners?

CBT : Peripheral member researcher only

Evaluated a course designed, taught and tested by three different groups

MAR: Active member researcher only

Involved for the duration of the CPD as a member of a MN team: designer, tutor of tutors, mentor of action research, evaluator and researcher

SCoP : Complete member researcher

During CPD as a member of a MN team: designer, tutor of tutors, mentor of action research, marker, evaluator and researcher.

But….

Involved course members as map makers and co-researchers who provided evidence from a valuable perspective

One group develop a MCM to explain e-learning theory based on their practice based evidence

Beyond duration Involved in MirandaNet Fellowship MCM working parties and

publications that have influenced me

Further research work

Further development of

analysis criteria to explore evidence of learning processes

the uses of the MCM for assessment for learning for individuals and groups

the value of collaborative and digital mapping

the potential for internal and external assessment

I’d like to do my thesis as a digital concept map