The Faculty of Engineering and Science The New Project Examination Hans Httel Anette Kolmos Kathrin Otrel-Cass Egon Moesby Jette Holgaard Picture: The Faculty of Engineering and Science Why we are here today Information about the new exam format. Project exams will be group-based. This was the case at Aalborg University until But not everything will be as it was pre- 2007! Remember: while project exams are group- based, the assessment is individual (this has always been the case!) The Faculty of Engineering and Science Implementing the new exam format Group project examination to start in the winter semester of 2012 (exams held in January 2013) for all first-year bachelor programs and selected first-year master programs As from summer semester 2013 all project examinations will be group-based Exemptions will be granted for final projects at Bachelor and Master level. 2 The Faculty of Engineering and Science The strategy at the faculty level The faculty asks each department to identify one or more ambassadors willing to provide support to other (new and existing) staff with questions regarding group examinations In addition there will be faculty organised workshops run on group-based project exams in December and the first week of January, see:es+/es+/ The Faculty of Engineering and Science A comparison Individual project examsGroup-based project exams The group presentation at the start is not part of the examination. Individual assessment is not possible. The group presentation at the start is part of the examination. Individual assessment is possible. About 30 minutes of individual questions based on selected aspects of the project. A joint discussion involving the entire group. All aspects of the project are examined. Supplemented by rounds of individual questions for each student. Feedback from the supervisors and examiners after the project has been assessed and grades are given. Feedback becomes part of the examination itself. The Faculty of Engineering and Science Presentation of project (students) Comments to the presentation (examiners) The examiners agree on the grades Announcement and justification of grades (Some students might prefer this individually) 1.A general, question-based discussion with more methodological questions 2.Specific questions for each student break An example of the structure of the new project exam break The Faculty of Engineering and Science Supervisors and examiners Group of students to be assessed Blackboard An example of the physical setup of the new project exam The Faculty of Engineering and Science Assessment in the new project exam format The assessment is individual; grades may (and often will) vary. The following elements must be taken into consideration: The written report tells the examiners what to expect while the oral presentation presents the project as a whole. The individual assessment for each student is based on the quality of the report and on his/her contributions to the oral presentation contributions during the general discussion of the project answers to the specific questions asked The Faculty of Engineering and Science Some potential pitfalls Good to know as an examiner that: Low achievers may feel under pressure and high achievers may feel compromised. Competition amongst students. Students may feel a lack of recognition of what has been achieved individually. This can be addressed by supervisors during the supervision process by highlighting and discussing these issues such that students are better prepared. The Faculty of Engineering and Science 11 The Faculty of Engineering and Science Learning from the past Picture: markjuleen.com The Faculty of Engineering and Science The preferences Kolmos and Holgaard, 2008 The Faculty of Engineering and Science The reactions at the time of the ban in 2007 There was a strong preference towards the group based exam from: 94% of the internal staff examiners (facilitators) (n=162) 87% of the external examiners (n=131) 69% of the students (n=516) Picture:Kolmos and Holgaard, 2008 The Faculty of Engineering and Science Some quotes from the research project I dread a transition to an individual exam because I believe that this would result in a significant decline in the academic level. Besides I am of the opinion that the team based exam is the type of exam which best relate to the working environment which the students will be in later on. (External examiner) I think it is important that the members of the group are tested in each others co-presence and in a collaborative process. It opens the possibility for partially evaluating their team play and co-operation, ability to stand out in a group, ability to follow up and to have an effect on a discourse. (External examiner) I think it is important that the members of the group are tested in each others co-presence and in a collaborative process. It opens the possibility for partially evaluating their team play and co-operation, ability to stand out in a group, ability to follow up and to have an effect on a discourse. (External examiner) All the students are present during the assessment to be able to learn something from the questions and comments to the other students. But it is crucial that there is an individual part where the students draw e.g. two questions which are then stated (Faculty member). All the students are present during the assessment to be able to learn something from the questions and comments to the other students. But it is crucial that there is an individual part where the students draw e.g. two questions which are then stated (Faculty member). Kolmos and Holgaard, 2008 The Faculty of Engineering and Science Three concerns that were often raised by some. It may be harder for shy students to get attention during the exam. Individual attention provides better individual judgment. Weaker students may try to hide. Picture: The Faculty of Engineering and Science Addressing the concerns Think of the project exam as the last meeting about the project. It is the examiners responsibility to ensure equal opportunities for everyone. The examiners note what has been said and will distinguish between students. Students cannot hide for several hours. The Faculty of Engineering and Science Strong points for group assessment Easier to examine all parts of the project thoroughly. Less temptation to ask textbook questions that ignore the actual project. Identification and recognition of the diversity of students knowledge. Opportunity to demonstrate applied knowledge and competence. Students can build on and benefit from each others contribution. The feedback during the exam makes it easier for students to understand their grades. Picture: loridennis.com The Faculty of Engineering and Science Skills tested - TekNat The ability to:GRP 2006IND 2008 Answer questions quickly Remember concepts and definitions Argue for methodological choices Relate different concepts to each other Show theoretical overview Show analytical skills Discuss the pros and cons of a solution Transfer knowledge to other situations Make an oral presentation Enter into dialogue and collaboration Comment and elaborate on arguments Work in teams More than 7 out of 10 felt that this ability was tested to a large or to some extent More than 9 out of 10 felt that this ability was tested to a large or to some extent More than 5 out of 10 felt that this ability was tested to a large or to some extent Fewer than 5 out of 10 felt that this ability was tested to a large or to some extent The Faculty of Engineering and Science Studies of the transfer from group based examination to individual in 2007/2008 Project: group based assessment surveys to students, academic staff and external examinersotion/Working+Papers/otion/Working+Papers/ Statistical analysis of the markshttp://vbn.aau.dk/da/publications/ideologi% eb0-92e5-11de-90ca-000ea68e967b%29.html External examiners evaluation The Faculty of Engineering and Science 19 The Faculty of Engineering and Science Starting over learning from experience Picture: infinitevitality1.wordpress.com The Faculty of Engineering and Science What students said about group-based exams The students that were in favour of group-based exams also emphasized: that the team setting create a pleasant atmosphere that they have learned a lot during the exam because of the thorough discussions. However, more than 80% either disagreed or partly disagreed that it is easy to hide in a group-based exam (82%), and that you do not have to cover the whole curriculum in a group-based exam (80%). Kolmos and Holgaard, 2008 The Faculty of Engineering and Science Advocates for individual exam International students, with a different cultural and educational background, felt considerably more confident with the individual assessment (32 % compared to 13% for Danish students). 36% of the Danish students, who experienced individual project exams preferred this compared to 10% for those who had never experienced it because: The relatively long term group-based exam is exhausting It can be a struggle to get the examiners attention and some find it even difficult to speak up in a group setting Kolmos and Holgaard, 2008 The Faculty of Engineering and Science Conducting a good exam (for all involved) Prepare the joint discussion really well. Plan the individual round of questions well and make sure you have discussed this with students in advance. Keep track of contributions using a detailed scorecard and extensive notes. Prepare to give feedback on the performance of each student The Faculty of Engineering and Science How can you prepare for the new format? All exams, no matter what the format, are a challenge. Make sure that you and your colleagues agree on your interpretation of the exam format. Share information about the exam format and strategies on how to get through and be prepared early on in the teaching and supervision process. Meet the students some days before the exam to explain/discuss the format again. Agree on what constitutes good behaviour during the exam. Talk to experienced colleagues. The Faculty of Engineering and Science Lets start peer-learning right now! Form groups that consist of Those of you that have chaired project exams in a group setting Those of you that have attended such exams as external examiners or students Those of you that have a completely new perspective on things The Faculty of Engineering and Science Lets start peer-learning right now! Discuss the following: How can you best use the project exam to provide feedback to the students about the quality of their project? How can you best keep track of the contributions made during the project exam in order to give individual feedback on the performance of each student? The Faculty of Engineering and Science Last but not least, remember the faculty guidelines. who your department ambassador(s) is/are!! the people you have spotted today that have experience from back in the day. to use any opportunity you get to act as an internal examiner before chairing an exam of your own. to use any possibility to get support at the exam e.g. by having a peer (who is even more experienced than you) to provide feedback on your performance during breaks and after the exam. Thank you for the your attention