Embed Size (px)
DESCRIPTION
E-Assessment: Evaluation, vision and plan. www.jisc.ac.uk John Winkley, e-assessment JISC E-learning Programme. Agenda. A vision for e-assessment Findings from our evaluation Recent e-assessment activity in the e-learning programme A shift of emphasis. - PowerPoint PPT Presentation
Citation preview
Joint Information Systems Committee
E-Assessment: Evaluation, vision and planE-Assessment: Evaluation, vision and plan
www.jisc.ac.ukwww.jisc.ac.uk
John Winkley, e-assessmentJohn Winkley, e-assessmentJISC E-learning ProgrammeJISC E-learning Programme
Joint Information Systems Committee
A vision for e-assessment
Findings from our evaluation
Recent e-assessment activity in the e-learning programme
A shift of emphasis
Agenda
20 April 2010 | slide 2
Joint Information Systems Committee 22/04/23 | slide 3
JISC – The structure of the e-learning programme
Joint Information Systems Committee
A draft vision for e-assessment (by 2015) E-assessment is used appropriately and with confidence … in a wide variety of HE curricula.
E-assessment … adds to traditional assessment processes with efficiency improvements, as well as improvements in assessment quality...
E-assessment is used at all stages of the learning programme…
Information from different types of assessment is interoperable...
The day-to-day use of e-assessment is encouraging change in the nature of assessment... E-assessment is embedded in institutional policy…
A wide range of technologies are used to support e-assessment in diverse forms…
Developments in JISC’s infrastructure projects… provide a development platform which allows new technical and pedagogical assessment opportunities to be researched, developed, piloted and exploited efficiently.
A vibrant world-class community of research underpins technical and pedagogical developments in e-assessment….
20 April 2010 | slide 4
Joint Information Systems Committee
e-learning programme – activity types Investigates - Institutional-level demonstrators/pilots
Describes - Research, studies, mapping and modelling
Analyses - Review and Evaluate
Creates - Commission Development
Informs - Consult and Inform (Knowledge Exchange )
Guides - Communication and Dissemination
Project and Programme Support, Programme Management
Recent move toward larger pedagogy-focused programmes with “cross-cutting themes”, eg OER, Curriculum Design and Delivery.
20 April 2010 | slide 5
Joint Information Systems Committee
eAssessment - Baselining
Programme Management
Underpinning Research
Technology
Content
Embedding and Rollout
20 April 2010 | slide 6
Joint Information Systems Committee
Programme ManagementVISION
Established vibrant, open and mixed community of technical developers, educators and assessment specialists working to form, support and develop JISC’s vision for e-assessment.
Vigorous stakeholder engagement
TODAY
Vibrant but small and specialised.
Too many technologists, not enough educationalists.
Recent efforts to get the two sides talking appear to be working.
Not enough vendors. Slight reluctance from academics.
Hard to maintain but important. Complex education sector, many agencies at the intersection of technology, pedagogy and exams.
20 April 2010 | slide 7
Joint Information Systems Committee
Underpinning ResearchVISION
Regular measurement of progress with embedding.
A clear understanding of institutional barriers to take-up and the best methods to fix.
Clear understanding of international best practice.
TODAY
Anecdotal evidence:
Much adoption, but limited evidence of widescale use of e-assessment in HE. Different elsewhere.
Relies on early adopters and enthusiasts. Poor scaling.
large gap behind the ‘bleeding edge’ and the day-to-day life of departments.
Policy and decision makers are critical.
Ok for Int’l HE, little wider view.
20 April 2010 | slide 8
Joint Information Systems Committee
eAssessment - Underpinning Research (2)VISION
Technical research underpinned by clear pedagogy.
Alignment of best practice between e-assessment and assessment.
Successful tracking, influencing and adoption of interoperability standards.
TODAY
Patchy. Importance very clear.
E-assessment generally not in HE policy and strategy.
Leading role in IMS. QTIv2 thoroughly explored. Killer business case not found yet.
20 April 2010 | slide 9
Joint Information Systems Committee
eAssessment - TechnologyVISION
Successful group of OSS toolkit and demonstrator projects provide inspiration and seedcorn for larger developments.
Technical framework
TODAY
Successful OSS projects: Reference player, item bank, item dev kit, peer review tools. But often poor takeup and sustainability.
Excellent proof of SOA and OSS in practice.
Needs extending for workflow and enterprise applications.
Human processes
“System” integration for information sharing
Struggling to make e-framework work for practical e-assessment applications.
20 April 2010 | slide 10
Joint Information Systems Committee
eAssessment - Technology (2)VISION
E-assessment embedded in formative assessment, supported by range of OSS tools and components
TODAY
Formative e-assessment pedagogy not quite there, but big strides made recently toward a clear framework to support development.
Wide range of tools and processes under development in a wide variety of settings.
20 April 2010 | slide 11
Joint Information Systems Committee
eAssessment - ContentVISION
Content Interoperability is supported, removing concerns about vendor lock-in and allowing sharing of content.
Content management of assessment content is well understood.
TODAY
The QTI1.x and 2.x specifications are reasonably stable. Untried on large scale in UK. Attempts within FE and schools sector have had mixed success.
The two “killer applications” of content interoperability : avoidance of lock-in and content sharing both require a more widespread use of e-assessment than currently seen.
Some doubt about whether HEIs really wish to share content. (Also some examples of success where external QA is important).
It isn’t yet, in most HEIs. The tools are not yet ready to support it.
20 April 2010 | slide 12
Joint Information Systems Committee
eAssessment – Rollout, embeddingVISION
HEIs have a clear understanding of how best to measure and manage quality in e-assessment systems including at policy level.
There is clear evidence of the benefits of e-assessment.
E-Assessment is widely used in HEIsand embedded in institutional and sectoral policy: “crossed the chasm”.
Content management of assessment content is well understood.
TODAY
Significant shortcomings and challenges faced by HEIs. E-assessment commonly run by small groups of “enthusiasts” who spend much of their effort on maintaining a reliable service rather than on managing assessment performance.
Yes, where its use is driven from pedagogy.
E-assessment appears to have a low profile within QAA and HE institutional policies (not fully investigated).
Communities of subject specialists appear to provide the richest community for successful takeup.
20 April 2010 | slide 13
Joint Information Systems Committee
eAssessment – Rollout, embedding (2)VISION
Information from assessment including e-assessment is used effectively within learning programmes. Information from different types of assessment is interoperable, allowing it to be combined seamlessly, to provide a rich and dynamic picture of learning to date and planning for future learning, supporting personalised learning programmes.
TODAY
Technically possible but difficult to organise.
Empirical benefits very clear. Some examples of good practice.
Likely to be a major focus of next phase of JISC e-assessment work.
20 April 2010 | slide 14
Joint Information Systems Committee
Focus of recent activities
Research to support development of a more diverse set of e-assessment approaches and technologies – “technology-enabled assessments”
Investment in formative e-assessment development
Research to investigate quality processes around e-assessment
Investment in e-assessment as a more integrated part of the ‘learning environment’
Community development and engagement
20 April 2010 | slide 15
Joint Information Systems Committee
E-Assessment: Evaluation, vision and planE-Assessment: Evaluation, vision and plan
www.jisc.ac.ukwww.jisc.ac.uk
John Winkley, e-assessmentJohn Winkley, e-assessmentJISC E-learning ProgrammeJISC E-learning Programme
