Identifying the key issues that will persuade staff and

students to engage with ePortfolios: the results of

nine pilot modules

Peter Chalk

London Metropolitan University

‘Telling stories’, Wolverhampton, June 2008

or ‘Stuck in the mud’

• A very mixed story…

Background

• Autumn 2007 London Met ePortfolio pilot

• Nine modules from Foundation Degree to Honours Degree 1-3 to Masters

• Existing PDP framework includes specified tasks and PDP related assessment in core spine modules (www.londonmet.ac.uk/ugstudy/pdp)

Why go electronic?

• Increasing expectation by employers – will improve students’ employability (currently low at London Met)

• Part of London Met’s e-enablement policy (all modules to use VLE)

• Students’ evidence/ artefacts increasingly digital – multimedia/ CV – as are tools

• Improves sharing with multiple viewers

Preparation

• Summer of 2007 tutors introduced to the new ePortfolio system by workshop, online and face to face support

• Encouraged to integrate it into their module preparation, materials, teaching, learning strategy, classroom organisation and assessment (pre-pilot findings, Chalk 2007)

Examples of ePortfolio use

• In addition to those following:– Levels 1 and 2 to upload CV in HE orientation

(HEO) and employability modules– Level 3 project blog shared with supervisor– Level 1 Film Studies HEO module, to store a

log of film criticisms– Level 2 maths employability module with six

PDP related tasks.

ePortfolio evidence

• Evidence of student (and staff) engagement includes – Examples of student work – Templates, tools and other resources developed

by staff – Results of observations and discussions with

staff and students

Foundation Degree Community Sports Coaching students

Evidence of prior achievement

Evaluation of coaching session –evidence of reflection

SMART analysis – use of a PDP tool

Learning object (assignment) – a collection of files

Positive image – role model for non-traditional student – FDSc students enthusiastic compilers of ePortfolios

Tutor’s initial template for Events Management (M level) ePortfolio

Evaluation

• Evaluation of the Autumn pilot was carried out by – Two questionnaires to students, – Two workshops for staff, Nov & Jan– Email discussion, – Observation, and – Analysis of actual student ePortfolios

Selective feedback from 3 staff

• Very few students not in the pilot tutorial group chose to use the ePortfolio.

• Very few students have engaged any more than was absolutely necessary. 30% experienced a significant degree of difficulty in using the system

• The ePortfolio was not assessed in any way so little engagement: one uploaded their CV, two provided some evidence of reflection

Staff feedback 2 (events management)

• 71% of the students in one tutorial group used the ePortfolio for their reflective commentary, and 28% of these had invited guests to view it (but not the module leader),

• In another group only 37.5% used the ePortfolio to record their reflections and none invited in guests,

• Students’ reasons for non-engagement: costly, confusing, unstable internet, too much effort.

Staff feedback 3 (film studies)

• ePortfolio is an improvement over paper-based PDP, customisable by student, told “this is London Met’s Facebook”, but it’s not as user-friendly,

• Tutor can see lots of potential but needs to be integrated into teaching and learning throughout the course and not just on one module,

• Can use multimedia binder to store short films, only place to store seminar logs throughout their course.

Student feedback 1 (events)

• “I don’t think I will find it useful since a lot of my courseworks have to be scanned if I want it in my portfolio and that’s quite time consuming for me”,

• “I am using the reflections tool… it will be useful”,

• “I use the blog, not the CV tool as I use my own version, I don’t like the layout”

Student feedback 2 (Sports FDSc)

• “I find this portfolio system safe and also fun… I can post my work on it and add other accessories, e.g. poems, pictures, videos etc. Using this portfolio has been one of my greatest experiences”

• Non-traditional students compiled ePortfolios – introduced across whole course, assessed and supervised in lab.

Student feedback 3 (maths employability)

• 16 students filled in November questionnaire with 90% stating they had uploaded files and added guests, 70% posted a comment and adapted the presentation, only 40% using the CV tool.

• The most positive ratings were for guest control, showcasing achievement, improving employability, developing self and subject awareness, reflecting on progress.

• The least positive ratings were for developing creativity and learning about ePortfolios.

Results of evaluation questionnaires

• In November generally positive on a Likert scale questionnaire about features

• In January, at the end of the modules, a free text email drew 26 responses, 6 positive and 20 negative

• Evidence from ePortfolios seen by author is patchy (CVs uploaded here, blog written there) except Sports Coaching – see earlier slide – for showcasing/ feedback & session evaluation/ SMART/ standards evidence/ learning object design

Conclusions• London Met has decided not to adopt one single

ePortfolio system at present,• Insufficient evidence of ‘stickability’,• Need user friendly ePortfolio system, integrated

into all aspects of the student’s life, including regular assessment related tasks and enthusiastically supported by academic staff,

• Considering alternatives (new Blackboard, Pebble Pad, ELGG/Moodle, Web 2 Services),

• Researching staff/student opinion…

Future work

• Working with the IV Cohort of the International Coalition for ePortfolio Research, using repertory grid analysis technique of personal construct theory (Kelly 1955, Steed & McDonnell 2003)

• Identifying the key features required for the future ‘stickability’ of ePortfolios in HE

Repertory Grid Analysis

• Conference delegates may wish to consider its possible adoption, its strengths and weaknesses.

• First, identify a category such as ‘storing, presenting and reflecting upon our own evidence of achievement’ (e.g. research papers, reports, other products)

RGA – selecting elements to compare

• What elements in the above discourse are important?

• Examples:– Web site - PC disc – USB - protected web site– Blog - Wiki – Stored emails - Diary– Professional portfolio – CV tool– Brainstorming tool - Action plan tool

RGA (continued) constructs

• Choose any 3 elements (and select two alike and the other different ‘in some way’ – this is the personal construct (& its opposite) – and repeat

• Grade all the elements in a grid for each pair of constructs (including the ‘ideal’)

• Analyse the pattern of graded constructs to identify important features required for ‘ideal’ ePortfolio

References

• Chalk, P (2007) ’The WebLearn Portfolio and HEO pilot – implications for a blended learning approach’, workshop at London Met Teaching & Learning Conference 10/7/07. URL: homepages.north.londonmet.ac.uk/~chalkp/res/pdp-july-conf.ppt.

• Kelly, G A (1955) The psychology of personal constructs, vol 1 and 2, Norton, New York.

References (continued)

• Steed, A., & McDonnell, J. (2003). Experiences with repertory grid analysis for investigating effectiveness of virtual environments. In Proceedings of the 6th International Workshop on Presence. Aalborg, Denmark, 6-8 October 2003, www.cs.ucl.ac.uk/staff/A.Steed/presence2003_fourpage.pdf (accessed 15.2.08)

Acknowledgements

• The author would like to thank the contributions of the module leaders to this pilot project and to this paper: Mehryar Adibpour, Sarah Atchia, David Blundell, Dafna Hardbattle, Maureen Kendal, Justin Lance, Tom Lunt, Rosemary Stott and Heather Wanstall.

• The author is a member of Cohort IV of the International Coalition for Electronic Portfolio Research (http://ncepr.org/).