Embed Size (px)
Citation preview
BioMed CentralBMC Medical Education
ss
Open AcceResearch articleDetermining the quality of educational climate across multiple undergraduate teaching sites using the DREEM inventoryRajesh Varma*, Ekta Tiyagi and Janesh K GuptaAddress: Academic Department of Obstetrics and Gynaecology, Birmingham Women's Hospital, Birmingham, B15 2TG, UK
Email: Rajesh Varma* - [email protected]; Ekta Tiyagi - [email protected]; Janesh K Gupta - [email protected]
* Corresponding author
AbstractBackground: Our obstetrics and gynaecology undergraduate teaching module allocates 40–50final year medical students to eight teaching hospital sites in the West Midlands region. Based onstudent feedback and concerns relating to the impact of new curriculum changes, we wished toobjectively assess whether the educational environment perceived by students varied at differentteaching hospital centres, and whether the environment was at an acceptable standard.
Methods: A Dundee Ready Education Environment (DREEM) Questionnaire, a measure ofeducational environment, was administered to 206 students immediately following completion ofthe teaching module.
Results: The overall mean DREEM score was 139/200 (70%). There were no differences in theeducation climate between the teaching centres.
Conclusion: Further research on the use of DREEM inventory, with follow up surveys, may beuseful for educators to ensure and maintain high quality educational environments despite studentsbeing placed at different teaching centres.
BackgroundThe undergraduate curriculum at our medical school wasredesigned in 1998/99 to bring it in line with recommen-dations suggested by the General Medical Council (GMC)in Tomorrow's Doctors [1]. Obstetrics and Gynaecology istaught as a final year module. Around 20–30 students, ofa total year group of around 200 students, are allocated toeight teaching hospital sites in the West Midlands region,and remain their for the length of the module (eightweeks). Throughout the placement, all formal lecturestake place at the principal Teaching Hospital (Birming-ham Women's Hospital). A comprehensive course hand-book and web-based multiple choice formativeassessment accompany the module, and detail the teach-
ing, practical and assessment objectives for students andclinicians. We have aimed to ensure there are no signifi-cant differences in the way the curriculum is deliveredbetween centres. All 200 students sit the final exam inObstetrics and Gynaecology straight after completing the8-week course module.
Based on previous student feedback reporting differencesin educational experiences, together with our concernsrelating to the impact of new curriculum changes, wewished to objectively assess whether the educational envi-ronment perceived by students varied at different teachinghospital centres, and whether the environment was at anacceptable standard. In particular, was there any potential
Published: 21 February 2005
BMC Medical Education 2005, 5:8 doi:10.1186/1472-6920-5-8
Received: 26 November 2004Accepted: 21 February 2005
This article is available from: http://www.biomedcentral.com/1472-6920/5/8
© 2005 Varma et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Page 1 of 4(page number not for citation purposes)
BMC Medical Education 2005, 5:8 http://www.biomedcentral.com/1472-6920/5/8
loss of teaching experience when students were placedaway from the principal Teaching Hospital. Thus, the nullhypothesis we wished to test was that there was no differ-ence in the learning environment between centres. Severalquestionnaire-based educational tools are available thatset out to 'quantify' the educational environment [2-4].However, we chose to use the Dundee Ready EducationEnvironment Measure (DREEM) inventory, as more stud-ies had evaluated and validated this method [5,6]. TheDREEM inventory consists of 50 questions, each scoring4, giving a total maximum individual DREEM score of200. The five domains that comprise the DREEM aredepicted in Table 1.
MethodsThe DREEM questionnaire, based on a Likert scale, wasadministered to the full class of 206 final-year Birming-ham University medical students undertaking the exammodule in Obstetrics and Gynaecology in 2000. All ques-tionnaires were distributed and returned the same day ofthe exam, which allowed us to achieve a 100% response
rate. Students were told to only comment on their recent8-weeks experience of Obstetrics and Gynaecology. Statis-tical analysis was performed using Microsoft Excel andArcus Quickstat Biomedical Statistical software, and uti-lised single-sample T test and One-way analysis of vari-ance (ANOVA).
ResultsThe year group comprised 42% male and 58% female. Theoverall mean DREEM score for the study group was 139/200 (95% CL 136.1 to 141.9), or expressed as percentageof the maximal score, 70% (95% CL 68% to 71%). Therewas no statistically significant difference between themean scores for the contributory DREEM domains, whichwere as follows: perception of learning, 34.52/48 (72%);perception of teaching, 32.05/44 (73%); academic self-perception, 19.46/32 (61%); perception of atmosphere,34.07/48 (71%), and for social self perceptions, 18.90/28(68%). The DREEM scores for each hospital, with compar-ison of all contributory elements of the DREEM inventory,are depicted in Table 2 and Figure 1.
Table 1: Domains of DREEM questionnaire
TOPIC Number of questions Maximum DREEM Score
Students' Perception of Learning 12 48Students' Perception of Teachers 11 44Students' Academic Self-Perceptions 8 32Students' Perception of Atmosphere 12 48Students' Social Self-Perceptions 7 28Total 50 200
Table 2: The DREEM domains and overall score for each hospital
HOSPITAL Number of Students
LEARNING Mean Score/48
TEACHERS Mean Score/44
ACADEMIC SELF-PERCEPTION Mean Score/32
ATMOSPHERE Mean Score/48
SOCIAL Mean Score/28
OVERALL DREEM Score/200
DREEM percentage for each hospital
(total of 206) Percentage of maximum score
Percentage of maximum score
Percentage of maximum score
Percentage of maximum score
Percentage of maximum score
Percentage of maximum score
BWH 53 33.77 31.89 19.77 33.40 19.32 138.15 69%Good Hope 20 33.30 30.10 18.15 33.40 18.20 133.15 67%B'ham Heartlands 26 34.15 32.73 18.92 32.77 19.58 138.15 69%Walsall Manor 20 34.10 34.35 19.90 34.30 19.05 141.70 71%City 32 35.13 28.31 19.41 34.97 17.75 135.56 68%Wolverhampton 22 35.77 32.59 20.41 33.64 18.86 141.27 71%Shrewsbury 13 34.77 32.85 18.69 35.00 19.15 140.46 70%Wordsley 20 35.15 33.60 20.40 35.10 19.30 143.55 72%
Mean overall 34.52 72% 32.05 73% 19.46 61% 34.07 71% 18.90 68% 139.00 70%Lower 95% CL 33.83 70% 30.41 69% 18.77 59% 33.33 69% 18.38 66% 136.13 68%Upper 95% CL 35.21 73% 33.69 77% 20.14 63% 34.82 73% 19.42 69% 141.88 71%
CL Confidence Limit
Page 2 of 4(page number not for citation purposes)
BMC Medical Education 2005, 5:8 http://www.biomedcentral.com/1472-6920/5/8
When converting the raw DREEM score to percentages,two-sided P-value single-sample Student's T test showedno statistically significant difference between hospitals byeach DREEM domain, or between each DREEM domainwithin the same hospital. Greatest variation between hos-pitals occurred in the Students' Perception of Atmospheredomain, where there were four hospitals beyond the 95%Confidence Limits; this compared to three hospitalsbeyond 95% Confidence Limits in all other DREEMdomains (Table s2). One-Way analysis of variance(ANOVA) yielded F (variance ratio) = 0.5222, P = 0.8111,which indicated no statistically significant differencesbetween hospitals, DREEM domains, or overall DREEMscores (Table 3).
DiscussionWe have used the Dundee Ready Education EnvironmentMeasure (DREEM) in 'diagnosing' the educational envi-ronment of eight different teaching centres and makingcomparative analysis between these centres. The overallmean DREEM score was 139/200, or expressed as a per-centage, 70% (95% CL 68–71%). The educational learn-ing environment did not vary between centres. The twolowest scoring contributory domains, academic self-per-ception (61%) and social self-perceptions (68%), werenot statistically significantly different from the other threeDREEM domains or overall mean DREEM score.
This study has benefited by using an established educa-tional measure and obtaining a 100% response rate. Nostudents had been previously taught at the principalteaching hospital as this was solely used for Obstetrics andGynaecology teaching. However, some of the students(surveyed to be 16/206, 8%) had previously attended theother seven teaching hospital centres due to prior clinicalteaching attachments. Thus, previous experiences mayhave biased the teaching assessment completed by somestudents. Furthermore, the DREEM questions are of sucha nature that it is likely that the environment of the entirecurriculum was being assessed. However, by performingthe DREEM survey immediately at the end of the obstet-rics and gynaecology module, and emphasising reportingonly the last eight weeks experience, we believe thismaximised the chance that the DREEM measure assessedonly the recent hospital teaching site and minimised anyrecall bias. Other groups [7] have highlighted the poten-tial flaws in using means and parametric statistical tests onordinal data from Likert scales. As there is no firmly estab-lished consensus, we adopted to use the Student's T testand ANOVA calculation to fulfil best statisticalmethodology.
The DREEM domains are unlikely to be independent var-iables, and may be less of an environment test but moreof a measure of the overall motivation and learning atti-tude of the individual. The Course Valuing Inventory(CVI) score is made up of five domains: worthiness oflearning experience, emotional awareness, personal devel-opment, cognitive enhancement and task drive. A recentstudy of first year medical students showed a correlationbetween higher Course Valuing Inventory (CVI) scores,female gender, stronger self-confidence as a learner,greater motivation to learn and higher DREEM scores [8].
There is no accepted agreement on what is an acceptableDREEM inventory score from published literature. Never-theless, our DREEM score of 139/200 was higher thanother reports. A study of final year medical students inTrinidad reported an overall mean DREEM of 109.9/200[5]. A larger scale study, involving students from bothfinal and earlier undergraduate training years, showed aDREEM score of 118/200 in a Nigerian medical school,and 130/200 in a Nepalese medical school [9]. Our higherscore is reassuring, and is perhaps an indicator of betterhospital teaching environment, the positive value of usinga comprehensive course handbook, and the encourage-ment of formative self-assessment as guided by the coursehandbook and web-based package.
Graphical representation of the contribution of each DREEM domain to the overall mean DREEM scoreFigure 1Graphical representation of the contribution of each DREEM domain to the overall mean DREEM score
50% 60% 70% 80% 90% 100%
Percentage DREEM Score
LEARNING
TEACHERS
ACADEMIC
ATMOSPHERE
SOCIAL
OVERALL
Page 3 of 4(page number not for citation purposes)
BMC Medical Education 2005, 5:8 http://www.biomedcentral.com/1472-6920/5/8
Publish with BioMed Central and every scientist can read your work free of charge
"BioMed Central will be the most significant development for disseminating the results of biomedical research in our lifetime."
Sir Paul Nurse, Cancer Research UK
Your research papers will be:
available free of charge to the entire biomedical community
peer reviewed and published immediately upon acceptance
cited in PubMed and archived on PubMed Central
yours — you keep the copyright
Submit your manuscript here:http://www.biomedcentral.com/info/publishing_adv.asp
BioMedcentral
The non-significant differences between the DREEM
domains and between hospitals were significant findings.This was conveyed to our tutors based at the various teach-ing centres as a positive and encouraging result. In practi-cal terms, this meant that regardless of hospital capacity orstudent group size, their education delivery and environ-ment was no different to other centres in the student's cur-riculum. The DREEM inventory may thus be a useful toolfor educators to ensure and maintain high quality educa-tional environments and uniformity in educational deliv-ery despite students being placed at different teachingcentres.
Competing interestsThe author(s) declare they have no competing interests.
Authors' contributionsRV and ET carried out the statisitcal analysis, data interpre-tation, and drafted the manuscript. JKG conceived andcoordinated the study, acquired the results, and maderevisions of the manuscript. All authors read andapproved the final manuscript.
AcknowledgementsRV is funded by the MRC, UK; JKG is employed by Birmingham University; ET works voluntarily as a clinical attachment.
References1. General Medical Council: Tomorrow's Doctors. recommenda-
tions on undergraduate medical education. London, GeneralMedical Council; 1993.
2. James PA, Kreiter CD, Shipengrover J, Crosson J: Identifying theattributes of instructional quality in ambulatory teachingsites: a validation study of the MedEd IQ. Family Medicine 2002,34:268-273.
3. Pololi L, Price J: Validation and use of an instrument to meas-ure the learning environment as perceived by medicalstudents. Teach Learn Med 2000, 12:201-207.
4. Robins LS, Gruppen LD, Alexander GL, Fantone JC, Davis WK: Apredictive model of student satisfaction with the medicalschool learning environment. Academic Medicine 1997,72:134-139.
5. Bassaw B, Roff S, McAleer S, Roopnarinesingh S, De Lisle J, Teeluck-singh S, Gopaul S: Students' perspectives on the educational
environment, Faculty of Medical Sciences, Trinidad. MedicalTeacher 2003, 25:522-526.
6. Till H: Identifying the perceived weaknesses of a new curricu-lum by means of the Dundee Ready Education EnvironmentMeasure (DREEM) Inventory. Medical Teacher 2004, 26:39-45.
7. Jamieson S: Likert scales: how to (ab)use them. Med Educ 2004,38:1217-1218.
8. Sobral DT: Medical students' self-appraisal of first-year learn-ing outcomes: use of the course valuing inventory. Med Teach2004, 26:234-238.
9. Roff S, McAleer S, Ifere OS, Bhattacharya S: A global diagnostictool for measuring educational environment: comparingNigeria and Nepal. Med Teach 2001, 23:378-382.
Pre-publication historyThe pre-publication history for this paper can be accessedhere:
http://www.biomedcentral.com/1472-6920/5/8/prepub
Table 3: ANOVA analysis between different hospitals for the differing DREEM domains
Percentage of maximum score for each DREEM component for each hospital
Birmingham Women's
Good Hope Birmingham Heartlands
Walsall City Wolver-hampton
Shrewsbury Wordsley
Learning 70% 69% 71% 71% 73% 75% 72% 73%Teachers 72% 68% 74% 78% 64% 74% 75% 76%Academic 62% 57% 59% 62% 61% 64% 58% 64%Atmosphere 70% 70% 68% 71% 73% 70% 73% 73%Social 69% 65% 70% 68% 63% 67% 68% 69%
One-way analysis of variance (ANOVA) yielded F (variance ratio) = 0.5222, P = 0.8111.
Page 4 of 4(page number not for citation purposes)
