- 0 -

RESEARCH PLAN DUDOC UU4

Use of ICT for acquiring, practicing and assessing relevant mathematical skills

Christian Bokhove

March 31st 2008

Table of content

1. Fact sheet ........................................................................................................................................................................ 1

2. Abstract........................................................................................................................................................................... 1

3. Problem statement .......................................................................................................................................................... 1

4. Conceptual framework ................................................................................................................................................... 2

5. Methodology................................................................................................................................................................... 5

6. Preparatory research ....................................................................................................................................................... 7

7. Timeline and products .................................................................................................................................................... 8

8. Changes of plan .............................................................................................................................................................. 9

9. References ...................................................................................................................................................................... 9

Appendix A: Overview of intervention ............................................................................................................................ 11

Appendix B: Schematic overview of research.................................................................................................................. 12

Appendix C: Matrix of sub-questions versus cycles and data types................................................................................. 13

- 1 -

1. Fact sheet

Project Title: Use of ICT for acquiring, practicing and assessing relevant mathematical skills.

PhD student: Christian Bokhove, St. Michaël College, Leeghwaterweg 7, 1509 BS, Zaandam

Supervisor(s): Promotor: Prof. Dr. Jan van Maanen. Co-promotor: Dr. Paul Drijvers

University: Freudenthal Institute for Science and Mathematics Education, Faculty of Science,

Universiteit Utrecht, PO Box 9432, 3506 GK Utrecht

2. Abstract

In recent years the mathematical skill level of students graduating from secondary education is not on a par with the

expected skill level of higher education, which leads to complaints from higher education. This research aims to

investigate in what way the use of ICT in upper secondary education can enhance acquiring, practicing and assessing

mathematical skills relevant for tertiary education. Three key concepts are involved: assessment, ICT tool use and

algebraic skills. Together with existing design principles they provide the basis for a prototype of a tool for the

acquisition of global insights in algebraic expressions. This tool – structure, interface and content – is used and revised

in three cycles of design research. The results are extrapolated to knowledge on acquiring, practicing and assessing

relevant mathematical skills, as well as an empirically grounded theory for algebraic skills and best practices for

mathematical tool use inside and outside the classroom.

3. Problem statement

For several years now the skill level of students leaving secondary education in the Netherlands has been discussed.

Lecturers in higher education –for example– often complain of an apparent lack of algebraic skills. This problem seems

to have grown larger during the last few years and is not restricted to the Netherlands. In 1995 already, the London

Mathematics Society reported:

(i) a serious lack of essential technical facility—the ability to undertake numerical and

algebraic calculation with fluency and accuracy;

(ii) a marked decline in analytical powers when faced with simple problems requiring

more than one step;

(iii) a changed perception of what mathematics is — in particular of the essential place

within it of precision and proof. (London Mathematical Society, 1995)

In the years following, many Dutch institutions for higher education established entry exams and bridging courses

to assess and improve students’ mathematical skills (Heck & Van Gastel, 2006). In 2006 a Dutch project called NKBW

was started to address and scrutinize this gap in mathematical skills between secondary and higher education. The final

report of the NKBW project (NKBW, 2007) reaffirmed the fact that there is a problem with mathematical skills,

algebraic skills in particular, but concluded that more research on the nature of the problem has to be undertaken. To

tackle the issues of limited algebraic skills, this research focuses on two relevant issues in mathematics education at

Dutch upper secondary level. On the one hand signals from higher and secondary education that students lack algebraic

skills compared with other mathematical subjects (Tempelaar, 2007; Tempelaar & Caspers, 2008; Vos, 2007), on the

other hand the use of ICT in mathematics education. To stress the importance of both issues they are also addressed in

the vision document of cTWO1 commission. Amongst others, the report stresses the importance of numbers, formulas,

functions, and change (viewpoint 4). ICT should be "used to learn" and not "learned to use". Other important issues in

the report are:

• A specific case is made for the transition of students from secondary education towards higher

education. It is stressed that this transition needs more attention.

• The importance of assessment of algebraic skills is stressed.

• Viewpoint 16 mentions the pen-and-paper aspect of mathematics. In the end transfer should not only

take place mentally, but also notation wise.

We conclude that there is ample reason to address the issue at stake. As algebraic skills will have a more prominent

position in the 2013 mathematics curricula, the research is closely related to the reform of science and mathematics

education in the Netherlands. There also is overlap with national and international mathematics and ICT projects like

Sage, Stack and Activemath.

1 commission on the future of mathematics education, see www.ctwo.nl

- 2 -

4. Conceptual framework

Research question

This research focuses on the following central research question:

In what way can the use of ICT support acquiring, practicing and assessing relevant mathematical skills?

As a first step, this question is briefly analyzed word-by-word:

• In what way The premise of the research is that ICT can be used to support learning, testing and assessing

mathematical skills. The question is how this should take place.

• Use of ICT

The second premise is that the use of ICT should have an important role in the math curriculum, as it

has the potential for developing skills any time, any place.

• Acquiring, practicing and assessing Not only final results, grades and scores are important, but also the ways in which mathematical

concepts are learned and tested diagnostically.

• Relevant mathematical skills When students leave secondary education they are expected to have learned certain skills. Here we

focus on algebraic skills, with particular attention given to skills in relation to conceptual

understanding. In the literature, this is addressed by the notion of symbol sense.

The hypothesis is that the use of ICT tools, if carefully integrated, can increase algebraic skill performance in general

and symbol sense in particular.

The main research question is elaborated into the following sub-questions:

a) What are characteristics and criteria for an appropriate tool for assessment of algebraic skills?

b) What role can feedback play when using an ICT tool for acquiring algebraic skills?

c) How does instrumental genesis take place when learning algebraic skills, both basic skills and symbol sense?

d) How can transfer of algebraic skills take place from the tool towards pen-and-paper?

e) How can formative and summative assessment be successfully combined in one didactical scenario for an ICT

tool for acquiring, practicing and assessing algebraic skills?

Three key topics emerge from these research questions: ICT tool use, assessment and algebraic skills.

ICT Tool use

Problem statement

It is important to study the way in which tools2 can be used to facilitate learning. How are tools used and what

characteristics do they have to have?

Theoretical perspective

Tool use is an integrated part of human behaviour. Vygotsky (1978) sees a tool as a mediator, a "new intermediary

element between the object and the psychic operation, directed at it". Verillon and Rabardel (1995) distinguish artifact

and instrument. The artefact is just the tool. The instrument is a psychological notion: the relationship between a

person and the artefact. Only when this relationship is established one can call it a "user agent". The mental processes

that come with this are called schemes. In short: instrument = artefact + instrumentation scheme. Trouche (2003)

distinguishes instrumentation (how the tool shapes the tool-use) and instrumentalisation (the way the user shapes a

tool). Instrumental genesis is the process of an artefact becoming an instrument. In this process both conceptual and

technical knowledge play a role ("use to learn" and "learn to use"). To overcome the contrast between pen-and-paper

and ICT based learning, an ICT environment has to correspond with traditional techniques (Kieran & Drijvers, 2006).

The instrumental approach provides a good framework for looking at the relation between tool use and learning

from an individual perspective. Yackel and Cobb (1996) argued that coordinating both perspectives is expected to

explain a lot on the advent an use of computer tools. The role of ICT use in mathematics education remains unclear. Do

we “learn to use” or “use to learn”? The math curriculum reform committee cTWO is hesitant in using ICT and would

like to see more research on the subject (cTWO, 2007). Research suggests that “learn to use” and “use to learn” are

intertwined (Lagrange, 2000). Integration in the classroom is essential, and to understand this we need to observe

instrumentation.

According to several studies (Artigue, 2002; Guin, Ruthven, & Trouche, 2005), instrumental genesis in the case of

2 With tool we mean ICT tool

- 3 -

computer algebra systems is a time-consuming and lengthy process. When focusing on particular aspects of

instrumental genesis, for example instrumentation, instrumentalization and technique (Guin & Trouche, 1999), it

becomes clear how students can use tools more effectively and what obstacles hinder conceptual and technical

understanding. Trouche sees three functions: a pragmatic one (it allows an agent to do something), heuristic (it allows

the agent to anticipate and plan actions) and epistemic (it allows the agent to understand something). Instrumental

orchestration concerns the external steering of students’ instrumental genesis (Guin & Trouche, 1999).

Artigue (2002) notices problems with the ‘technical-conceptual cut’ of using ICT tools. Two theoretical approaches

are combined to overcome this problem. First, Verillon and Rabardel’s work on instrumentation (1995), the so-called

ergonomic approach and second Chevallard’s (1999) anthropological approach. The ergonomic approach focuses on

instrumentation though mental schemes. In the anthropological approach the balance between task, technique and

theory in acquiring skills, is crucial. Technology and theory can be defined as knowledge per se. Task and technique

are know-how relevant to a particular theory and technology.

ICT tool use in this research

We claim that tools can facilitate in the learning of algebraic skills. Therefore we want to study how instrumental

genesis takes place when using a prototypical assessment tool. In other words, how does a student adopt use of an ICT

assessment tool in relation to the development of algebraic concepts and skills?

Assessment

Problem statement How can we efficiently make use of formative assessment for acquiring, practicing and assessing algebra skills?

Theoretical perspective

Black and Wiliam (2004) distinguish three functions for assessment:

− supporting learning (formative)

− certifying the achievements or potential of individuals (summative)

− evaluating the quality of educational programs or institutions (evaluative)

Summative assessment is also characterised as assessment of learning and is contrasted with formative assessment,

which is assessment for learning. Black and Wiliam (1998) make a case for more room for formative assessment. They

state that "improving formative assessment raises standards". Actively involving the student, implementing formative

assessment as an essential part of the curriculum and motivating students through self-assessment are key benefits of

formative assessment. Means to do this are providing feedback, self-assessment, reflection and interaction.

Black and Wiliam (1998) define assessment as being ‘formative’ only when the feedback from learning activities is

actually used to modify teaching to meet the learner's needs.

Feedback can be direct (Bokhove, Koolstra, Heck, & Boon, 2006), but also implicit by providing the possibility of

using multiple representations of a given task (Van Streun, 2000). It also is an essential ingredient of eleven conditions

under which assessment might support student learning and improve chances of success (Gibbs & Simpson, 2004), as

well as the seven principles of good feedback practice (Nicol & MacFarlane-Dick, 2006). There clearly is a tension

between summative and formative assessment. As Black & Wiliam (2004) put it:

Teachers seemed to be trapped between their new commitment to formative assessment and the different, often

contradictory demands of the external test system (p. 45).

A reciprocal relationship between formative and summative assessment is sorely needed (Broadfoot & Black, 2004) In

this sense one could argue that summative and formative assessment are potentially complementary (Biggs, 1998) and

should be integrated more (Shavelson, Black, Wiliam, & Coffey, 2002).

Several frameworks for assessment exist, for example PISA’s assessment pyramid (De Lange, 1999), and the

TIMMS framework. They have in common that they provide a framework for finding a balance between the level of

thinking called for, mathematical content, and degree of difficulty. In the NKBW project (2007) a classification for

skill level of a question was used, involving the letters A, B and C. This classification was used earlier in the

Webspijkeren project (Kaper, Heck, & Tempelaar, 2005), and drew on work by Pointon and Sangwin (2003). These

three levels correspond with the levels in PISA’s assessment pyramid:

1. Reproduction, definitions, computations. (lower level)

2. Connections and integration for problem solving. (middle level)

3. Mathematization, mathematical thinking, generalization, and insight. (higher level)

Assessment in this research

We see ‘acquiring, practicing and assessing skills’ as forms of assessment, ranging from formative towards summative.

Through practice skills are acquired (formative assessment), after which it is concluded with summative assessment.

Feedback then, is essential for our intervention. We will use the assessment pyramid and the criteria for feedback and

assessment –along with other practical criteria– for choosing and designing a tool for assessment.

- 4 -

Algebraic skills

Problem statement

There are clear indicators that algebraic skills of students do not meet the standards compared with other mathematical

subjects (Vos, 2007). We want to make sure that students really understand algebraic concepts, so just testing basic

skills is insufficient. What defines real algebraic understanding?

Theoretical perspective The lack of algebraic skills in higher education deserves more research attention. Although some researchers have tried

to initiate discussions in this field (Drijvers, 2006; Tempelaar, 2007; Vos, 2007) there is no clear view on what skills

are lacking and in what way algebraic skills are dwindling. Also, it is unclear what skills are meant: is it “real

understanding”, the concepts, or the procedural skills? Algebraic skills have a basic skill component and a symbol

sense component (Arcavi, 1994). Both basic skills and symbol sense should be addressed in education, as they are

closely related: understanding of concepts makes basic skills understandable, and basic skills can reinforce conceptual

understanding. Freudenthal (1991) promoted “training with insightful learning”. Zorn (2002) defines symbol sense as

“a very general ability to extract mathematical meaning and structure from symbols, to encode meaning efficiently in

symbols, and to manipulate symbols effectively to discover new mathematical meaning and structure."

Taken from (Drijvers, 2007)

Symbol sense is an intuitive feel for when to call on symbols in the process of solving a problem, and conversely, when

to abandon a symbolic treatment for better tools. Arcavi (1994) describes several ‘behaviours’ of symbol sense.

The behaviour of flexible manipulation skills requires a certain ‘Gestalt’ skill: one has to recognize certain features of

algebraic expressions. Wenger (1987) and Gravemeijer (1990) also studied this aspect of these global characteristics of

expressions. Here, ‘Gestalt’ played a role: does he or she recognize similar parts of an equation? Not recognizing

patterns often leads to ‘circularity’: rewriting an expression to finally end up where you began. Drijvers (2006) sees an

important role for both basic skills and symbol sense in the acquisition of algebraic skills.

Algebraic skills in this research Our intervention will address both basic skills and symbol sense. The focus is on symbol sense, and in particular the

flexible manipulation skills. This requires a Gestalt quality of recognizing meaningful parts within an expression or

equation. At the same time, basic skills should also be addressed.

Relations between key elements

The following graph represents the main elements of the conceptual framework and the relations between them.

Acquiring, practicing and assessing relevant mathematical skills with ICT can be scrutinized in three relations:

− Tool use and assessment.

ICT tools can be used efficiently for formative and summative assessment, providing functional feedback.

− Tool use and algebraic skills.

Through instrumentation and instrumental genesis an ICT tool can be used for acquiring algebraic skills and

vice versa. Because of the fact that technology, task and theory go hand in hand, ones understanding of algebra

also shapes ICT tool use.

− Algebraic skills and assessment.

Formative assessment enables us to study the qualitative aspect of basic skills and ‘real understanding’. This

focus on formative aspects is closely related to symbol sense. Assessment frameworks for mathematics form

the background for the relation between algebraic skills and assessment.

We aim to integrate the assessment of algebraic skills into one prototypical design, used for giving us insight into these

- 5 -

three aspects and their relations. For this we conducted the preparatory research that is described in section 6. The table

in appendix A, based on the curriculum spider web (Van den Akker, Gravemeijer, McKenney, & Nieveen, 2006), sums

up the characteristics of our intervention. Appendix B gives a schematic overview of our research.

Expected results

The research results include:

• Design and evaluation of a prototypical digital module for acquiring, practicing and assessing relevant

mathematical skills;

• A local instruction theory for using an ICT tool for acquiring, practicing and assessing algebraic skills;

• Knowledge on the acquisitions of a ‘Gestalt’ view on algebraic expressions;

• Extrapolation of the findings to a more general level;

5. Methodology

Research strategy

As we aim to design an intervention in several iterations, the research is based on the principles of design research

(Van den Akker et al., 2006). Research will take place in one preparatory cycle and three subsequent cycles. Within

each macro level research cycle we distinguish three phases, the preliminary design phase, the experiment phase and

the phase of retrospective analysis (Gravemeijer, 1994). This corresponds globally with preparing for an experiment,

conducting the experiment and processing the data.

In every macro-cycle we aim to use mini-cycles by modifying the prototype according to the given feedback, and to

keep in mind the three key concepts of our research. Preparatory research (see 6.) leads to an initial design. The design

of this intervention is based on criteria that follow from the conceptual framework. For their positioning in the total

framework of this research I refer to section 7 on the planning.

The development throughout the cycles can be characterized by a shift in focus: from more qualitative formative

towards quantitative summative. This involves upscaling from a small target audience towards a larger target audience.

This pragmatic approach requires our methodology to be mixed: at first more qualitative where we use grounded theory

(Glaser & Strauss, 1967) for our analysis, and more quantitative later on, using a more quasi-experimental approach

with pre- and post-tests.

We will describe the cycles globally the instruments used, the sample, the target population (schools, teachers,

students) and analysis. Appendix C gives an overview of the used methods.

Cycle 0: preparation phase (7 months)

Before starting the first cycle the preparatory cycle is concluded. For a more detailed description of this phase and its

results we refer to 6. Sub question a. is primarily addressed in this cycle.

First cycle: feedback (7 months)

Goal of this cycle

Sub questions b. and c. are primarily addressed in this cycle In general, the goal of this cycle is to design a prototype of an effective module for acquiring, practicing and assessing

the algebraic skills and Gestalt view on algebraic expressions.

Instruments and method Expert reviews and one-to-ones with selected students and experts are used to decide on:

− the overall design of the tool

− feedback for the student

− item design on symbol sense

Four one-to-ones and two expert reviews are conducted each in a time span of three weeks, for 2 hours. These three

weeks will lead to modification of the initial prototype and a report of the review(s) stating goals, results, and “lessons

learned” plus implications for the prototype. Focus will be on the feedback mechanism. Research methods will include:

observations, audio and screencam recording, interviews. To aid this we will use a computer with large LCD screen

and a microphone. We will ask students to make the tasks in the prototype and talk aloud when clicking, but also while

thinking. We will use screencam software to record the action on the screen.

Sample and target population Experts will be from the NKBW and cTWO network in national expert meetings, as well as four to eight students of

- 6 -

different abilities from St. Michael College, Zaandam.

Retrospective analysis A qualitative analysis of the observations, screencam recordings and interviews will take place. For this we will use a

method in line with the constant comparative method (Glaser & Strauss, 1967; Strauss & Corbin, 1998), using Atlas.TI

software.

Second cycle: algebraic skills (16 months)

Goal of this cycle Sub questions b., c., d. and e. are addressed in this cycle.

Instruments and method

In this cycle we aim at refining the prototypical module, and particularly the feedback embedded in it, through mini-

cycles of design and field test. We want to make sure that transfer takes place. Therefore, a scenario with both

computer tools and paper-and-pencil tests will be used:

Scenario setup:

1. Paper entry test

2. Discussing the results

3. Tool available, usage at least two hours per week

(any time, any place, shift from more feedback to less feedback to no feedback)

4. Final digital exam after three weeks

5. Paper exam one week later

This scenario setup will also be used in the third and last cycle.

Research methods will include: observations, audio and screencam recording, interviews and questionnaires.

Screencam software will be used to record the action on the screen. All data will be stored centrally.

Sample

The sample consists of fifty 1st year students at university and fifty 6th form students at secondary school, from

different backgrounds (science, medicine, business). We will conduct the tests in several groups.

Target population: Two schools participating in the Universum Program and in cTWO curriculum experiments, and relevant groups from

higher education.

Retrospective analysis As the first cycle was of a qualitative nature, this cycle will be more quantitative. This involves the use of quantitative

data analysis. From the quantitative data a sample will be used for a more qualitative approach, leading to a “mixed”

methodology. We will use the constant comparative method (Glaser & Strauss, 1967; Strauss & Corbin, 1998) and

Atlas.TI software for the qualitative analysis, quantitative methods for the scores and numerical data.

Third cycle: upscaling and assessment (8 months)

Goal of this cycle Sub questions d. and e. are primarily addressed in this cycle (upscaling).

Instruments and method Same instruments as previous cycle. Only minor changes made to the module, larger target audience and upscaling.

Sample

The sample consists of one hundred 1st year students at university and hundred 6th form/grade students at secondary

school, from different backgrounds (science, medicine, business). We will conduct the tests in several groups.

Target population:

Two schools participating in the Universum Program and in cTWO curriculum experiments, and relevant groups from

higher education.

- 7 -

Retrospective analysis

Apart from collecting data on qualitative performance, insights on mixing formative and summative assessment are

expected. The scale of this experiment enables us to analyze both qualitative and quantitative aspects. We will use the

same methods of analysis as the preceding cycle.

Reliability and validity

Internal reliability We will ensure internal reliability by using internal consistency measures when constructing our observation criteria,

interview questions and questionnaire.

External reliability Our list of criteria for choosing a tool will be peer reviewed by experts, thus providing inter-rater reliability. Using a

mixed method approach for every sub question improves the reliability of the results.

Internal validity

We use data of good quality, based on a sound operationalization, leading to a good content and criterion-related

validity. Data types will be underpinned by theoretical reflections.

External validity

To improve generalization we will conduct our experiments in different groups with different backgrounds. The data is

analyzed through the constant comparative method (Glaser & Strauss, 1967; Strauss & Corbin, 1998).

6. Preparatory research

Choosing content that makes Symbol Sense

Suitable symbol sense type questions were selected carefully, to be authored in the tool that would be selected in the

inventory of tools. Sources were found in literature:

- Questions from existing exams (NKBW project (NKBW, 2007) , exit exam Dutch Association of

Mathematics Teachers (Rozenhart, 2007), Mathmatch University of Amsterdam (Heck & Van Gastel,

2006).

- Examples from Arcavi (1994).

- Categorization of skill level. We used the approach of the Webspijkeren (Kaper et al., 2005) project, that

was also used in the NKBW project, and has its roots in work from Pointon and Sangwin (2003).

Emphasis is on flexible manipulation skills, in particular the Gestalt view (Arcavi, 1994).

Tool for mathematics assessment

This part consists of an inventory of existing tools that are used for ICT based assessment with feedback. For this I

used criteria based on existing research on tools, feedback and assessment, but also own criteria that were sensible in

the light of the research question.

Sources/results:

- Criteria for tools for assessment. These will be discussed with experts to improve the validity.

- Principles of assessment and good feedback. (Nicol & MacFarlane-Dick, 2006).

- Design criteria for educational applets (Underwood et al., 2005).

- Existing reviews and listings of ICT tools for mathematics, especially for assessment.

This research was concluded with a report and a choice of a tool.

Both tool and content are used for making our preliminary design in the first cycle. The prototype will be authored at

http://www.fi.uu.nl/dwo/voho.3 The results of this preparatory cycle will be the basis for an article for the International

Journal of Computers for Mathematical Learning.

3 Login vohodemo password omedohov

- 8 -

7. Timeline and products

Timeline

Start End Description Cycle 0: preparatory research 7 months

1-9-2007 30-9-2007 Literature research and clarifying research goal

1-10-2007 1-11-2007 Writing down the theoretical and conceptual framework.

1-10-2007 1-2-2008 Finishing research plan

1-10-2007 31-3-2008 Choosing content, constructing and authoring digital tests in

several tools. Choosing final tool, based on clear criteria. Cycle 1: feedback 7 months

1-4-2008 30-4-2008 Designing first prototype

1-5-2008 14-5-2008 Formulating detailed questions for 1st cycle

15-5-2008 31-7-2008 1st cycle: expert review and one-to-ones

1-8-2008 31-10-2008 Writing report and article on results of the 1st cycle

1-9-2008 31-10-2008 Annotating digital exam

Cycle 2: algebraic skills 16 months

1-11-2008 31-12-2008 Formulating detailed questions for 2nd cycle

1-1-2009 30-9-2009 2nd

cycle: focus on algebraic skills, classroom experiments

and field tests

1-10-2009 31-12-2009 Writing report and article on results of the 2nd

cycle

1-1-2010 28-2-2010 Formulating results and hypothesis

Cycle 3: upscaling 8 months

1-3-2010 30-6-2010 3rd cycle: upscaling, classroom experiments and field tests

1-7-2010 31-10-2010 Writing report and article on results of the 3rd cycle

1-11-2010 31-8-2011 Writing dissertation 10 months

48 months

Publication plan

Period Publications and presentations

Summer 2008 Poster presentation ORD2008 (submitted)

Poster presentation EARLI/ENAC Berlin (submitted)

Submit article on criteria for tools for mathematical assessment, to be submitted to

International Journal of Computers for Mathematical Learning

http://www.springer.com/education/mathematics+education/journal/10758

Summer 2008 Presentation at ISDDE conference (http://www.fi.uu.nl/isdde/)

Autumn 2008 Report on the expert review and one-to-ones (1st cycle: feedback)

Article on feedback, to be submitted to a journal (on formative assessment) and to be

presented at a conference.

Summer 2009 Conference presentation on 1st cycle and/or feedback.

- 9 -

Autumn 2009 Report on the 2nd cycle: algebraic skills

Article on assessing symbol sense, to be submitted to journal ‘For the learning of

mathematics’ (http://flm.educ.ualberta.ca/) and to be presented at a conference.

Summer 2010 Conference presentation on 2nd cycle and/or assessing symbol sense.

Autumn 2010 Report on the 3rd cycle: upscaling

General article on whole research, to be submitted to a general journal and to be

presented at a conference.

Summer 2011 Article and general report on finished research. Dissertation.

Products

Period Product

Summer 2008 1st version digital exit/entry exam/symbol sense test

October 2008 After 1st cycle: annotated test.

8. Changes of plan

No major changes are made in this research plan, compared to the initial proposal drawn up by the supervisors.

9. References

Arcavi, A. (1994). Symbol Sense: Informal Sense-Making in Formal Mathematics. For the Learning of Mathematics, 14(3), 24-35.

Artigue, M. (2002). Learning mathematics in a CAS environment: The genesis of a reflection about instrumentation and the

dialectics between technical and conceptual work. International Journal of Computers for Mathematical Learning, 7, 245-

274.

Biggs, J. (1998). Assessment and classroom learning: a role for summative assessment? Assessment in education, 5(1), 103-110.

Black, P., & Wiliam, D. (1998). Inside the Black Box: Raising Standards Through Classroom Assessment. Phi Delta Kappan,

80(2), 139-144.

Black, P., & Wiliam, D. (2004). The formative purpose: assessment must first promote learning. In M. Wilson (Ed.), Towards

Coherence Between Classroom Assessment and Accountability - 103rd Yearbook of the National Society for the Study of

Education (pp. 20-50). Chicago: University of Chicago Press.

Bokhove, C., Koolstra, G., Heck, A., & Boon, P. (2006). Using SCORM to Monitor Student Performance: Experiences from

Secondary School Practice. Math CAA series Retrieved March 28, 2008, from http://mathstore.ac.uk/articles/maths-caa-

series/apr2006/index.shtml

Broadfoot, P., & Black, P. (2004). Redefining assessment? The first 10 years of Assessment in Education. Assessment in

Education: Principles, Policy & Practice, 11(1), 7-27.

Chevallard, Y. (1999). L’analyse des pratiques enseignantes en théorie anthropologique du didactique. Recherches en Didactique

des Mathématiques, 19, 221-266.

cTWO. (2007). Rijk aan betekenis, visie op vernieuwend wiskundeonderwijs. Utrecht: Utrecht University.

De Lange, J. (1999). Framework for Classroom Assessment in Mathematics. Utrecht: Freudenthal Institute.

Drijvers, P. (2006). Context, abstractie en vaardigheid in schoolalgebra. Nieuw Archief voor Wiskunde, 5/7, 198-203.

Drijvers, P. (2007). School algebra in the Netherlands: the ‘back-to-the-basics battle' (in dutch) [Presentation held at the occasion

of the international symposium "Wiskundeonderwijs in Vlaanderen en Europa: een stand van zaken en perspectieven voor

de toekomst", Leuven, 23 august, 2007]. Leuven.

Freudenthal, H. (1991). Revisiting Mathematics Education: China Lectures. Norwell: Kluwer Academic Publishers.

Gibbs, G., & Simpson, C. (2004). Conditions under which assessment supports students’ learning. Learning and teaching in higher

education, 1, 3-31.

Glaser, B. G., & Strauss, A. L. (1967). The Discovery of Grounded Theory: Strategies for Qualitative Research. New York: Aldine

Publishing Company.

Gravemeijer, K. (1990). Globaal kijken, een kenmerk van algebraïsche deskundigheid. Nieuwe Wiskrant, Tijdschrift voor

Nederlands Wiskundeonderwijs, 10, 29-33.

Gravemeijer, K. (1994). Developing Realistic Mathematics Education. Utrecht: Cdbeta.

Guin, D., Ruthven, K., & Trouche, L. (Eds.). (2005). The didactical challenge of symbolic calculators: Turning a computational

device into a mathematical instrument. New York: Springer.

Guin, D., & Trouche, L. (1999). The complex process of converting tools into mathematical instruments: The case of calculators.

International Journal of Computers for Mathematical Learning, 3, 195-227.

Heck, A., & Van Gastel, L. (2006). Mathematics on the Threshold. International Journal of Mathematical Education in Science &

- 10 -

Technology, 37(8), 925-945.

Kaper, W., Heck, A., & Tempelaar, D. (2005). Beschrijving van kennis in twee dimensies geïllustreerd aan twee cursussen aan de

UM en UvA. Retrieved March 28, 2008, from http://www.web-spijkeren.nl/

Kieran, C., & Drijvers, P. (2006). The Co-Emergence of Machine Techniques, Paper-and-Pencil Techniques, and Theoretical

Reflection: A Study of CAS Use in Secondary School Algebra. International Journal of Computers for Mathematical

Learning, 11(2), 205-263, 259.

Lagrange, J.-B. (2000). L'integration d'Instruments Informatiques dans l'Enseignement: Une Approche par les Techniques (The

Integration of Informational Instruments in Instruction: A Technical Approach). Educational Studies in Mathematics,

43(1), 1-30.

London Mathematical Society. (1995). Tackling the mathematics problem. Retrieved March 28, 2008, from

http://www.lms.ac.uk/policy/tackling/report.html

Nicol, D. J., & MacFarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of

good feedback practice. Studies in Higher Education, 31(2), 199-218.

NKBW. (2007). Eindrapport Nationale Kennisbank Basisvaardigheden Wiskunde. Amsterdam: ZuidamUithof Drukkerijen.

Pointon, A., & Sangwin, C. J. (2003). An analysis of undergraduate core material in the light of hand held computer algebra

systems. International Journal of Mathematical Education in Science and Technology, 34(5), 671-686.

Rozenhart, H. (2007). Instaptoetsen! Uitstaptoetsen? Euclides, 83(2).

Shavelson, R. J., Black, P. J., Wiliam, D., & Coffey, J. E. (2002). On Aligning Formative and Summative Assessment. Paper

presented at the National Research Council’s Assessment In Support of Instruction and Learning: Bridging the Gap

Between Large-Scale and Classroom Assessment Workshop.

Strauss, A., & Corbin, J. (1998). Basics of Qualitative Research. Techniques and Procedures for Developing Grounded Theory (2

ed.). Newbury Park: Sage.

Tempelaar, D. (2007). Onderwijzen of bijspijkeren? Nieuw Archief voor Wiskunde, 8, 55-59.

Tempelaar, D., & Caspers, W. (2008). Instaptoetsen wiskunde in een internationaal perspectief. Euclides, 83(5), 250-253.

Trouche, L. (2003). Managing the Complexity of Human/Machine Interactions in Computerized Learning Environments: Guiding

Students’ Command Process through Instrumental Orchestrations. International Journal of Computers for Mathematical

Learning, 9(3), 281-307.

Underwood, J. S., Hoadley, C., Lee, H. S., Hollebrands, K., DiGiano, C., & Renninger, K. A. (2005). IDEA: Identifying Design

Principles in Educational Applets. Educational Technology Research and Development, 53(2), 99-112, 114.

Van den Akker, J., Gravemeijer, K., McKenney, S., & Nieveen, N. (Eds.). (2006). Educational design research. London and New

York: Routledge.

Van Streun, A. (2000). Representations in applying functions. International Journal of Mathematical Education in Science and

Technology, 31(5), 703-725.

Verillon, P., & Rabardel, P. (1995). Cognition and artifacts: A contribution to the study of though[t] in relation to instrumented

activity. European Journal of Psychology in Education, 9, 77-101.

Vos, P. (2007). Algebra-prestaties van tweedeklassers. Euclides, 82(4), 129-132.

Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University

Press.

Wenger, R. H. (1987). Cognitive science and algebra learning. In A. Schoenfeld (Ed.), Cognitive science and mathematics

education (pp. 217-251). Hillsdale: Lawrence Erlbaum Associates.

Yackel, E., & Cobb, P. (1996). Sociomathematical Norms, Argumentation, and Autonomy in Mathematics. Journal for Research

in Mathematics Education, 27(4), 458-477.

Zorn, P. (2002). Algebra, computer algebra and mathematical thinking. Paper presented at the 2nd International Conference on the

Teaching of Mathematics.

- 11 -

Appendix A: Overview of intervention

Rationale Why are they learning?

The algebraic skill level of students going from secondary to

higher education can be improved, possibly with the use of

ICT.

Aims & Objectves Toward which goals are they learning?

Improving algebraic skills, on the complete range from basic

skills to symbol sense.

Content What are they learning?

Algebraic skills, both basic skills and symbol sense.

Learning activities How are they learning?

By making use of an ICT tool that supports acquiring,

practicing and assessing relevant algebraic skills.

Teacher role How is the teacher facilitating learning?

By monitoring use and performance of the tool and

providing feedback on mistakes. Providing instruction on

usage of the tool. Motivating the students.

Materials & Resources With what are they learning?

With digital tasks within an ICT tool

Grouping With whom are they learning?

Students are learning individually

Location Where are they learning?

In a computer environment with internet access (any place)

Time When are they learning?

Whenever students want (any time), including

Assessment How far has learning progressed?

We hope to see a substantial increase in flexible

manipulation skills when making an ICT and paper based

exam on algebraic skills.

- 12 -

Appendix B: Schematic overview of research

- 13 -

Appendix C: Matrix of sub-questions versus cycles and data types

Cycles and Data

Types

Sub-

Questions

Cy

cle

Lit

erat

ure

rev

iew

Ob

serv

atio

n

Au

dio

and

scr

een

cam

reco

rdin

gs

Inte

rvie

ws

Qu

esti

on

nai

res

An

aly

sis

of

qu

anti

tati

ve

dat

a

a) What are characteristics and criteria for an

appropriate tool for assessment of algebraic

skills?

Prep. ����

b) What role can feedback play when using an

ICT tool for acquiring algebraic skills? Prep. ����

1 ���� ���� ����

2 ���� ���� ���� ���� ����

3 ����

c) How does instrumental genesis take place

when learning algebraic skills, both basic skills

and symbol sense?

Prep. ����

1 ���� ���� ����

2 ���� ���� ���� ���� ����

3 ����

d) How can transfer of algebraic skills take place

from the tool towards pen-and-paper?

Prep. ����

1

2 ���� ���� ���� ���� ����

3 ���� ���� ���� ���� ����

e) How can formative and summative assessment

be successfully combined in one didactical

scenario for an ICT tool for acquiring,

practicing and assessing algebraic skills?

Prep. ����

1

2 ���� ���� ���� ���� ����

3 ���� ���� ���� ���� ����