Embed Size (px)
DESCRIPTION
Citation preview
A theoretical framework for e-assessment
in higher education
Pereira, A.1
Oliveira, I.1
Tinoca, L.2
1 LEaD, Universidade Aberta2 Institute of Education, University of Lisbon
ContextPedagogical model for online education
Bologna Process (European Commission, 2008), challenged higher education to promote learning environments that are centered in the development of competences
New assessment culture supported by edumetric criteria – assessment for learning
New learning landscapesThe e-learning explosion and the emergence of new
digitally supported learning environments
Garrisson & Anderson (2003), McConnell (2006), Pereira et al.
(2009), and Anderson (2011)
• Collaborative
• Student centered
A pedagogical model for online education (Pereira et al. 2007)
Based on four cornerstones: student-centered learningflexibilityinteractiondigital inclusion
“these principles guide the organization of instruction, planning, the design and management of activities for the students, the type of materials to develop and the nature of the assessment of competences”
New learning landscapesLanguage and communication
Four main types of metacompetences (Pereira et al., 2009):
Problem solving
Group work
Metacognitive
Fluency in ICT use
An edumetric approach to assessment
A new assessment culture arose from the growing criticism of traditional testing methods relating to the unrealistic nature of the tests, the loss of faith in them as valid measures of learning, and an over-reliance on tests as the ultimate goal of the instruction process. (McDowell, 1995)
Edumetrics criteria are recognized as more valid and fair for competence based assessment, given their emphasis in flexibility and authenticity, as well as their integration into the learning process valuing the formative function of assessment. (Dierick and Dochy, 2001)
“assessment of learning”, as assessment focused on measurement and scaling, from “assessment for learning” as assessment meant for the students, through feedback, to understand their own learning processes and the goals that they intend to achieve. (Elwood and Klenowski, 2002)
The challenge of e-assessment“Confusion of tongues”
“e-assessment occurs when there is an automated marking/response to student input on-screen in a test, informing on the process of answering a question and providing feedback to learners and their teachers through well-crafted advice and reports”. (Beevers, 2010)
“e-assessment is sometimes used to refer solely to on-screen assessment but, in its broadest sense, can refer to all technology-enabled assessment activities”. (JISC, 2010)
e-assessment – our definition
e-assessment refers to all technology-enabled assessment
activities where the design and student activities
(complete, present, submit) must be mediated by
technologies.
It is regarded as optional the format in which the instructor
presents the assignment, as well as the way feedback is
provided
Steps of an e-assessment strategy
• design• presentation
Instructor
• complete• present• submit
Student • classification• feedback
Instructor
must be mediated by technologies
e-assessment benefits• Greater variety and authenticity in assessment designs
• Improved learner engagement
• Choice in the timing and location of assessments
• Capture of wider skills and attributes
• Efficient submission, marking, moderation and data storage
• Consistent, accurate results
• Increased opportunities for learners to act on feedback
• Innovative approaches based around the use of creative media and online peer and self-assessment
Method (Jabareen, 2009)
Literature review
Discussions among experts from three different universities, representing a broad array of scientific areas (literature, natural sciences, social sciences, and education) and with large experience in both face-to-face and online education
Confrontation of the literature reviewconstruct validity (Messick 1994, 1995)quality criteria for competence assessment programs
(Baartman et al., 2007)
Reconstruction of the concepts into a new conceptual framework
Reevaluation of the framework
Conceptual framework for e-assessment
four dimensions identified as especially relevant for
online contexts
authenticity
transparency
practicability
consistency e-assessment
E-assessment
authenticity
similarity
complexity
adequacy
significance
consistency
instruction-assessment alignment
multiple indicators
relevant criteria
competences-assessment alignment
transparency
democratization
engagement
visibility
impact
practicability
cost
efficiency
sustainability
Authenticity
Similarity – competeces needed in real/professioanl life
Complexity – cognitive chalenge
Adequacy – adequate performing conditions
Significance – value for students, instructors and employers
Consistency This dimension emerges as an answer to the traditional demands for validity and reliability, associated with psychometric indicators.
Instruction-assessment alignment
Relevant criteria
Competences-assessment alignment
Multiple indicators – assessment methods, contexts and assessors
TransparencyDemocratization – availability and participation
Engagement – participation in the definition of the learning goals
Visibility – present/share processes and/or products
Impact – effect on the learning process and on course design
PracticabilityCost – time, digital resources, training, …
Efficiency
Sustainability – implement and sustain the proposed assessment design, taking into account the learner profiles and the contextual constraints, both for the organizations and for the assessors
DiscussionThese dimensions are articulated, representing
several degrees of reciprocal interdependence.
The criteria, more than just illustrating
the different features of each dimension,
allow for the operational description of
each criterion stage of implementation, and so
contribute to the evaluation of the achieved
assessment strategy quality level.
Discussion
What can be the contributions of e-assessment
for the assessment culture?
How can e-assessment be used for internal
improvement and external accreditation?
From a research standpoint should all dimensions be
regarded as equal?
Thank You!
http://www.slideshare.net/luistinoca
[email protected]@[email protected]
@ssess – project financed through FCT (PTDC/CPE-CED/104373/2008)
new criteria particularly relevant for e-assessment
adequacy to online contexts
the distinction between instruction-assessment alignment and competences-assessment alignment
democratization
engagement
visibility
sustainability
