25
Konstantinos Mathiopoulos Fotini Paraskeva 1 Department of Technology Education and Digital Systems University of Piraeus Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education Konstantinos Mathiopoulos Fotini Paraskeva University of Piraeus, Department of Technology Education and Digital Systems

Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

  • Upload
    zody

  • View
    30

  • Download
    0

Embed Size (px)

DESCRIPTION

Department of Technology Education and Digital Systems University of Piraeus. Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education. Konstantinos Mathiopoulos Fotini Paraskeva - PowerPoint PPT Presentation

Citation preview

Page 1: Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

Konstantinos Mathiopoulos Fotini Paraskeva

1

Department of Technology Education and Digital SystemsUniversity of Piraeus

Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

Konstantinos MathiopoulosFotini Paraskeva

University of Piraeus, Department of Technology Education and Digital Systems

Page 2: Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

Konstantinos Mathiopoulos Fotini Paraskeva

2

Department of Technology Education and Digital SystemsUniversity of Piraeus

Contents at a glance

• Theoretical background of Authentic Assessment• Definitions of Authentic Assessment• Traditional Assessment VS Authentic Assessment• Advantages and disadvantages of Authentic Assessment• Authentic Assessment’s expectations • Research gap• Goal of the paper• Study on Authentic Assessment in Computer Supported

Collaborative Learning Environments in Higher Education• Conclusions• Future research

Page 3: Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

Konstantinos Mathiopoulos Fotini Paraskeva

3

Department of Technology Education and Digital SystemsUniversity of Piraeus

Page 4: Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

Konstantinos Mathiopoulos Fotini Paraskeva

4

Department of Technology Education and Digital SystemsUniversity of Piraeus

Definitions of Authentic Assessment:

"A form of assessment in which students are asked to perform real-world tasks that demonstrate meaningful application of essential knowledge and skills "-- Jon Mueller 2008

"...Engaging and worthy problems or questions of importance, in which students must use knowledge to fashion performances effectively and creatively. The tasks are either replicas of or analogous to the kinds of problems faced by adult citizens and consumers or professionals in the field." -- Grant Wiggins -- (Wiggins 1993, p229).

"Performance assessments call upon the examinee to demonstrate specific skills and competencies, that is, to apply the skills and knowledge they have mastered." -- Richard J. Stiggins -- (Stiggins, 1987, p. 34).

Page 5: Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

Konstantinos Mathiopoulos Fotini Paraskeva

5

Department of Technology Education and Digital SystemsUniversity of Piraeus

Traditional Assessment VS Authentic Assessment  

Multiple choice or other tests that are easily marked, often revealing only whether students can recognise, recall or “plug in” what was learned out of context.

Linking learning and working by creating a correspondence between what is assessed in school and what students need to do in the workplace.

Page 6: Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

Konstantinos Mathiopoulos Fotini Paraskeva

6

Department of Technology Education and Digital SystemsUniversity of Piraeus

Mandernach, B. J. (2003). Incorporating Authentic Assessment. Retrieved 01-05-09, from Park University

Page 7: Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

Konstantinos Mathiopoulos Fotini Paraskeva

7

Department of Technology Education and Digital SystemsUniversity of Piraeus

Authentic Assessments in general are expected to:

• stimulate students to learn more deeply (Birenbaum, 1996; Dochy & McDowell, 1997; McDowell, 1995; Frederiksen, 1984)

•stimulate students to develop professionally relevant skills and thinking processes used by professionals (Gielen et al; Savery & Duffy, 1995)

•motivate students to learn by showing the immediate relevance of that which is learnt for professional practice (Herrington & Herrington, 1998; Lizzio & Wilson, 2004a; Martens, Gulikers, & Bastiaens, 2004; McDowell)

Page 8: Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

Konstantinos Mathiopoulos Fotini Paraskeva

8

Department of Technology Education and Digital SystemsUniversity of Piraeus

Research gap:

New ways of assessment are required so as to replace standardized tests and quizzes.

It is important to examine how authentic assessment could be integrated in a web based environment in higher education.

The impact of authentic assessment on student learning in a computer supported collaborative learning environment is an open hot issue in higher education.

Page 9: Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

Konstantinos Mathiopoulos Fotini Paraskeva

9

Department of Technology Education and Digital SystemsUniversity of Piraeus

This paper is a report on findings of a study focused on examining perceptions of assessment authenticity and the influence on student learning in a computer supported collaborative learning environment at higher education level. A growing body of literature and research on new modes of assessment stresses that the effects of assessments on student learning should always be examined in the light of the whole learning environment along with student perception of the learning environment (Biggs, 1996; Birenbaum, 1996; Segers, Dierick, & Dochy, 2001; Struyven, 2005).

Goal of the paper

Page 10: Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

Konstantinos Mathiopoulos Fotini Paraskeva

10

Department of Technology Education and Digital SystemsUniversity of Piraeus

The method: Research Questions

• Does authentic assessment change the way that students think about assessment in general?

• Does implementation of authentic assessment improve study approach(deep/surface)?

• Does authentic assessment promote reflective thinking?

• Does authentic assessment increase motivation of learners in a computer supported collaborative learning environment?

Page 11: Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

Konstantinos Mathiopoulos Fotini Paraskeva

11

Department of Technology Education and Digital SystemsUniversity of Piraeus

The Method: The Participants

Participants were thirty-six post graduate students who attend a two-year Master's Degree (M.Sc.) in a department of "Information and Digital Systems" (120 ECTS) in the area of e-Learning. Students had various educational and professional backgrounds and age range from 22 to 55 years. Data was collected during the course of Computer Mediated Communication and Collaboration.

Page 12: Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

Konstantinos Mathiopoulos Fotini Paraskeva

12

Department of Technology Education and Digital SystemsUniversity of Piraeus

The Method: The Instruments

The moodle course management system was used (http://eptlab.unipi.gr), in order for students to be able to communicate through chat and forums, access documents and presentations, participate in workshops (project submission, self-assessment and peer assessment) and answer to questionnaires.

Page 13: Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

Konstantinos Mathiopoulos Fotini Paraskeva

13

Department of Technology Education and Digital SystemsUniversity of Piraeus

A 41- item questionnaire for perception of authenticity based on 5-dimensionsal framework for authenticity (Gulikers, Bastiaens, & Kirschner, 2004; 2006) and on perceptions of the purpose of formative computer based assessment through quiz items (Sandra T. Miller, 2009)•The 20-item deep and surface study approach questionnaire based on Revised-Study Process Questionnaire - 2 Factors (R-SPQ-2F; Biggs, Kember, & Leung, 2002).

The 16-item reflective thinking questionnaire based on the Level of Reflective Thinking Questionnaire (Kember D. et al, 2000).

The 81-item motives, cognitive and meta-cognitive strategies questionnaire based on the Motivated Strategies for Learning Questionnaire (Pintrich, R. R., & DeGroot, E. V. (1990).

The Method: The Instruments

Page 14: Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

Konstantinos Mathiopoulos Fotini Paraskeva

14

Department of Technology Education and Digital SystemsUniversity of Piraeus

The Statistical Results of the Research Pearson r Correlation

Deep Study

Approach

Surface

Study

Approach

Overall

Authenticity Task

Physical

Context

Form/

Result Criteria

Criteria

Transparency

Perception of

Purpose

Overall Authenticity ,390* -,176

Task

(Low Cronbach a)

,410* -,244 ,834**

Physical Context ,297 -,175 ,633** ,680**

Form /Result ,364* -,255 ,745** ,798** ,727**

Criteria ,297 -,324 ,768** ,777** ,751** ,833**

Criteria Transparency ,348* -,157 ,579** ,544** ,656** ,672** ,660**

Perception of

Purpose

,401* -,339* ,737** ,722** ,709** ,766** ,680** ,570**

Value of AA ,395* -,524** ,699** ,675** ,761** ,796** ,725** ,580** ,795**

Note: * Correlation is significant at the 0.05 level (2-tailed).

** Correlation is significant at the 0.01 level (2-tailed).

Sample size: N=36

 

Table1: Pearson r correlation analysis of dimensions of authentic assessment, perceptions of authentic assessment and study approach.

Page 15: Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

Konstantinos Mathiopoulos Fotini Paraskeva

15

Department of Technology Education and Digital SystemsUniversity of Piraeus

The Statistical Results of the Research

Pearson r correlation analysis indicates that authentic assessment indicators have positive linear relationship with deep study approach.

Statistically significant is the correlation between deep study approach and overall authenticity r(36) = .390 p< .05, Form /Result r(36) = .364 p< .05, Criteria Transparency r(36) = .348 p< .05, Perception Of Purpose r(36)= .401 p< .05 and Value of Authentic Assessment r(36)= .395 p< .05.

On the contrary, authentic assessment indicators have negative linear correlation with surface study approach.

Statistically significant factors are Perception Of Purpose r(36)= - .339 p< .05 and Value of Authentic Assessment r(36)= - .524 p< .01.

Page 16: Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

Konstantinos Mathiopoulos Fotini Paraskeva

16

Department of Technology Education and Digital SystemsUniversity of Piraeus

The Statistical Results of the Research

There is positive linear correlation between Perception of Purpose of Authentic Assessment and the dimensions of authentic assessment. Overall authenticity r(36)=.737 p< .01, Physical Context r(36)=.709 p< .01, Form/Result r(36)=.766 p< .01, Criteria r(36)=.680 p< .01, Criteria Transparency r(36)=. 570 p< .01 apart from task factor due to low internal consistency Cronbach a=.436.

There is positive linear correlation between Value of Authentic Assessment and the dimensions of authentic assessment. Overall authenticity r(36)=.699 p< .01, Physical Context r(36)=.761 p<.01, Form/Result r(36)=.796 p< .01, Criteria r(36)=.725 p< .01, Criteria Transparency r(36)=.580 p< .01. Perception of Purpose of Authentic Assessment and Value of Authentic Assessment are positive connected by r (36) = .795 p< .01.

Page 17: Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

Konstantinos Mathiopoulos Fotini Paraskeva

17

Department of Technology Education and Digital SystemsUniversity of Piraeus

The Statistical Results of the Research Pearson r Correlation

Overall Authenticity

Physical

Context

Form

/Result Criteria

Criteria

Transparency

Perception of

Purpose

Value of

AA

Reflection ,188 ,327 ,255 ,196 ,349* ,489** ,472**

Critical Reflection ,280 ,363* ,366* ,330* ,395* ,480** ,333*

Note: * Correlation is significant at the 0.05 level (2-tailed).

** Correlation is significant at the 0.01 level (2-tailed).

Sample size: N=36

There is a positive connection between authentic assessment and reflection. Reflection is related with Criteria Transparency by r(36) = .349 p< . 05, with Perception of Purpose by r(36)= .489 p< .01 and with Value of Authentic Assessment by r(36)= .472 p< .01. Reflective thinking is positively correlated with Physical Context r(36) = .363 p< .05, Form /Result r(36) = .366 p< .05, Criteria r(36) = .330 p< .05, Criteria Transparency r(36) = .395 p< .05, Perception of Purpose

r(36)= .480 p< .01 and Value of Authentic Assessment r(36)= .333 p< .05.

Table2: Pearson r correlation analysis of dimensions of authentic assessment, perceptions of authentic assessment and reflection.

Page 18: Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

Konstantinos Mathiopoulos Fotini Paraskeva

18

Department of Technology Education and Digital SystemsUniversity of Piraeus

The Statistical Results of the Research Pearson r Correlation

Value of AA Perception of Purpose

intrinsic goal orientation ,313 ,362*

extrinsic goal orientation ,145 ,092

task value ,451** ,410*

self-efficacy for learning and performance ,345* ,266

Note: * Correlation is significant at the 0.05 level (2-tailed).

** Correlation is significant at the 0.01 level (2-tailed).

Sample size: N=36

 

Authentic assessment increases learners’ motivation. Goal Orientation is positively correlated with Perception of Purpose of authentic assessment r(36) = .362 p< . 05. Task value is positively correlated with Perception of Purpose of authentic assessment r(36) = .410 p< . 05 and Value of Authentic Assessment r (36) = .451 p<. 01. Self-efficacy for learning and performance is positively correlated with Value of Authentic Assessment r (36) = .345 p<. 05.

Table3: Pearson r correlation analysis of perceptions of authentic assessment and motivation.

Page 19: Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

Konstantinos Mathiopoulos Fotini Paraskeva

19

Conclusions

Department of Technology Education and Digital SystemsUniversity of Piraeus

• Authentic assessment in computer supported collaborative learning environments is highly preferable and benefits not only the assessment procedure, but also the learning experience for learners.

• According to perceptions of authentic assessment, for successful application, all dimensions (Overall Authenticity, Task, Physical Context, Form/ Result, Criteria, Criteria Transparency) should be taken into consideration.

• Students perceived authentic assessment to be more valuable and efficient, when the purposes (instruction planning, self directed learning, collaboration, identification of strengths and weaknesses, self-assessment, peer assessment) were clear and understandable.

Page 20: Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

Konstantinos Mathiopoulos Fotini Paraskeva

20

Conclusions

Department of Technology Education and Digital SystemsUniversity of Piraeus

• The value of authentic assessment (topic understanding, guidance of student learning, application of mastered skills and knowledge, fair assessment) was enhanced by a suitably designed e-learning environment, which implement all aspects of authentic assessment.

• Moodle could be used in this way, since activities could be created or modified so as to become authentic.

• This study revealed that an increased perception of assessment authenticity in a web based collaborative learning environment resulted in more deep learning.

Page 21: Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

Konstantinos Mathiopoulos Fotini Paraskeva

21

Conclusions

Department of Technology Education and Digital SystemsUniversity of Piraeus

• Intrinsic motivation of learners was also augmented when goals of authentic assessment were considered more transparent. On the contrary, there was no sign of connection between extrinsic motivation and perceptions of authentic assessment. Authenticity did not seem to be imposed by the teacher or the curriculum and it became part of the learning procedure.

• High perception of the value and the purpose of authentic assessment is also reflected in students’ increased appraisal of the task they perform and self-efficacy for learning and performance.

• Authentic assessment may be part of the learning process and it may be better used as assessment for learning and not as assessment of learning.

Page 22: Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

Konstantinos Mathiopoulos Fotini Paraskeva

22

Future Research

Department of Technology Education and Digital SystemsUniversity of Piraeus

Future research should:

• focus on how curriculums may be changed so as to take advantage of the benefits of authentic assessment

• orientate to the production or transformation of applications, in order to support authentic learning environments.

• Examine ways of changing instructional design in Computer Supported Collaborative Learning Environments in Higher Education so as authentic assessment to be part of the ongoing learning process and not an extra assessment procedure.

Page 23: Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

Konstantinos Mathiopoulos Fotini Paraskeva

23

References

Department of Technology Education and Digital SystemsUniversity of Piraeus

• Herrington A. & Herrington J. (2006). Authentic learning environments in higher education. United States of America, Information Science Publishing

 • Herrington, J., & Oliver, R. (2000). An instructional design framework for authentic learning environments. United States

of America. Educational Technology Research and Development • Gulikers, J. T. M., Bastiaens, Th. J., & Kirschner, P. A. (2004). A five-dimensional framework for authentic assessment.

Maastricht, Educational Technology Research & Development. • Gulikers, J. T. M., Bastiaens, Th. J., & Kirschner, P. A. (2006). Authentic assessment, student and teacher perceptions:

the practical value of the five-dimensional framework. Maastricht, Journal of Vocational Education and Training,  • Gulikers, J. T. M., Bastiaens, Th. J., Kirschner, P. A., & Kester, L. (2006). Authenticity is in the Eye of the Beholder Beliefs and perceptions of authentic assessment and the influence on student learning. Maastricht, Studies in

Educational Evaluation. • Pintrich P., Smith D., Garcia T., McKeachie W. (1991). A Manual for the Use of the Motivated Strategies for Learning

Questionnaire. Technical Report 91-B-004. The Regents of the University of Michigan. 1991  • Miller, T. (2009). Formative computer –based assessments: the potentials and pitfalls of two formative computer based

assessments used in professional learning programs. Canada, Queen’s University. 

Page 24: Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

Konstantinos Mathiopoulos Fotini Paraskeva

24

References

Department of Technology Education and Digital SystemsUniversity of Piraeus

Biggs, J. B., Kember, D., & Leung, D. Y. P. (2001). The revised two-factor Study Process Questionnaire: R-SPQ-2F. British Journal of Educational Psychology Kember et al. (2000) Development of a questionnaire to measure the level of reflective thinking. Assessment & Evaluation in Higher Education. Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong

Mandernach, B. J. (2003). Incorporating Authentic Assessment. Retrieved 01-05-09, Park University ,USA

 

Page 25: Authentic Assessment in Computer Supported Collaborative Learning Environments in Higher Education

Konstantinos Mathiopoulos Fotini Paraskeva

25

Thank you

Department of Technology Education and Digital SystemsUniversity of Piraeus