Advances in Health Sciences Education 1: 119-123, 1997. 119) 1997 Kluwer Academic Publishers. Printed in the Netherlands.

The Development of a 'Standardised Learner' inResearching Teaching Behaviours

JOHN PITTS and COLIN COLESInstitute of Health and Community Studies, Bournemouth University, Royal London House,Christchurch Road, Bournemouth BHI 3LT, U.K.E-mail: colin. coles@bournemouth.ac.uk

Abstract. Standardised patients are increasingly used in medical education and the reliability andvalidity of their use as an assessment method are supported by published research. This paper detailsthe application of this methodology to the educational interaction between a teacher and a learnerusing a standardised learner.

In contrast with standardised patients, a standardised learner must use a level of medical knowl-edge and experience appropriate for the role assumed. Some aspects of reliability and validity havebeen considered and appear supportive of this methodology.

The feasibility and value of this method for researching actual teacher behaviour are discussed.

Key words: teaching skills, standardised learner, analysis of teaching

Introduction

In this paper we describe the development of a 'standardised learner' which wehave used to identify, study and measure teaching approaches used by generalpractitioners (family physicians) in the United Kingdom attending a mandatoryteaching skills course to prepare them for the education of general practice traineesattached to them for the final year of their postgraduate specialist training. Thecourse was intensive, residential, run in tutor-led small groups and lasted five days,and taught the principles of learner-centredness through a process of 'reflection onpractice'.

Earlier work demonstrated a significant shift in attitudes of people attendingthe course towards a more learner-centred approach (Pitts, 1993) that persistedone year later (Pitts, 1994). More recently, in a controlled study, we demonstratedsimilar changes in actual teaching behaviour (Pitts and Coles, 1995), that can alsobe shown to persist one year later (Pitts and Coles, 1996). The methodology thisinvolved included a 'standardised learner', whose development we present here.

The place and potential of using 'simulated patients' has been increasinglyexplored in the context of clinical medicine to teach and assess skills not easilyaddressed by other methods, and has been shown to be both valid and reliable(Norman et al., 1985; Stillman and Swanson, 1987; Van der Vleuten and Swanson,1990). Attempts have also been made to create teaching situations using 'standard-

JOHN PITTS AND COLIN COLES

ised students' in a medical school that allow teachers to reflect on their handlingof brief, often straight-forward clinical problems such as low back pain (Simpsonet al., 1992), and in a multi-station 'objective structured teaching exercise' used asthe basis for a workshop on teaching skills (Lesky and Wilkerson, 1994). However,these accounts are descriptive and do not contain psychometric data or performancecriteria to evaluate the effectiveness of standardised learners in changing teachingbehaviour.

Earlier reports of our work (Pitts and Coles, 1995, 1996) with a standardisedlearner demonstrated the effectiveness of the teaching course by using a before-and-after strategy and a control group. Briefly, before attending the course, eighteenprospective teachers held a simulated teaching situation (tutorial) with the standard-ised learner. This was repeated after they had attended the course. Twelve controlssimilarly held two tutorials with the standardised learner but did not attend thecourse. Tutorials were recorded and a typed transcript made available for analysis.The measures of teaching we used were the demonstration of previously agreedbehaviours, the percentage of open questions used by the teacher and the ratio ofteacher/learner talk to shown who spoke the most.

Development of the 'standardised learner'

For the purpose of this study, our standardised learner was the same person usedthroughout, a young woman who had recently completed her training for familypractice. In each tutorial, she presented to each participant in turn a 'problemclinical case' that she might have experienced as a general practitioner in training,based on an actual scenario.

In the precourse/first tutorial the case presented was a middle-aged profes-sional man attending as an 'emergency' consultation with headache. We trainedthe standardisd learner to express as her major concern the exclusion of seriouspathology, while recognising the nature of the headache was almost certainly stress-related. We discussed with our standardised learner the range of educational topicsthat could potentially be recognised as arising from this scenario: difficulty in clin-ical examination, wariness in taking a psychosocial history, dealing with personalexasperation at such an 'emergency' presentation, managing time, and the use ofan inappropriate investigation.

For the post-course/second tutorial, the same clinical problem was developed.The standardised learner presented concern following a subsequent consultationwith the patient, complicated by the fact that he returned accompanied by his wifewho then proceeded to dominate the consultation, raising further difficulties forthe general practitioner in training. Once again, a range of potential educationaltopics was discussed in training our standardised learner: dealing with third-partyconsultations, further pursuing and confirming the diagnostic possibilities, anddealing with clinical uncertainty.

120

DEVELOPMENT OF A 'STANDARDISED LEARNER'

The standardised learner was trained to commence each tutorial with the phrase"I have just seen this patient; I did not really know what to make of him; I am notreally happy with the way that it went . ".

Preparation of the standardised learner was achieved by means of a pilot studyof one-to-one tutorials with a group of ten general practice teachers. From ananalysis of the tutorial transcripts we discussed with the standardised learner theconsistency of the presentations and how to avoid leading the teacher towards thepotential educational agenda.

Reliability and validity

Standardised patients have been shown to be reliable and valid (Colliver andWilliams, 1993). In view of the overall similarity of that approach with a standard-ised learner, we were encouraged to check these from our data.

RELIABILITY

At one level, our method was reliable because it used the same learner. But howconsistent was she in performing in the same educational setting many times over?

Firstly, analysis of tutorial transcripts, examined by means of a Spearman prank order correlation calculation, showed that the sequence in which the tutorialswere carried out was independent of the total word counts. This showed that earlytutorials were not longer or shorter than later ones. Tutorial length was independentof the learner's familiarity with the role.

Secondly, the significant changes in the measures of teaching behavioursoccurred in the study group but not in the controls. While this suggests thesechanges are most likely to be due to the effects of the course, it also indicates,with no differences between the measures of teaching behaviour of the controlgroup, they were unlikely to be caused by any influence of the standardised learner.There was a high degree of consistency of the standardised learner in her tutorialswith our subjects over the two clinical scenarios and over the six week time spanbetween the two tutorials.

These data support the claim that the learner did not behave differently withincreasing experience and familiarity with the role.

VALIDITY

How valid, though, were the tutorials educationally? Qualitative data from bothteachers and the learner indicate that the cases were similar to problems commonlyexperienced and that the teaching followed a typical pattern for these types ofcases. None of the subjects commented that the clinical problem nor the educationalproblems that followed in the teaching were in any way unreal. Comments of theteachers included "Is this a real case?" or "I'm always seeing cases like this".

121

JOHN PITTS AND COLIN COLES

Discussion

This work has demonstrated that a 'standardised learner' can be developed andthis approach used successfully in the evaluation of a teaching course. Further-more, using quantitative methods we have shown that improvements in teachingbehaviours can be measured.

The advantages of a standardised-learner over actual general practice traineesas students are the same as those for standardised patients. However, in contrastwith standardised patients, who may be 'real' patients or actors trained to play arole, to be plausible a standardised learner must have a requisite and appropriateknowledge and experience. It may also be necessary for this person to 'unlearn'certain knowledge and skills - to 'forget' something already known and to display'knowing' behaviour as (simulated) learning occurs. Detailed attention was givento the training of the standardised learner, and discussion of educational issues.This was found to be a very necessary aspect of her development.

If this methodology were applied to the assessment of teachers, it would beimportant to know how many tutorials would be required for a valid assessment.With standardised patients, studies have shown that 'expert' criteria tend to exceedactual levels of performance (Norman et al., 1985; Kopelow et al., 1992; Tamblyn,1992). In practice, however, not all the aspects of a patient's condition may beaddressed at just one visit (Norman et al., 1985). Similarly, important issues forlearning might be 'noted' in a tutorial but not addressed in just one teaching session.

Preliminary assessments of reliability and validity using our study data aresupportive, though within the area of simulated patient encounters, concerns havebeen expressed about the validity of the observations (Kinnersley and Pill, 1994).Clearly, actions performed under test conditions should be similar to those carriedout with real patients. Kinnersley and Pill (1994) suggest that the ultimate test iswhether doctors can detect simulated patients in a blind trial, and point out that abody of evidence suggests that simulated patients successfully pass this test, and arepresumably treated in the same way as a real patient would have been (Woodwardat al., 1985; Rethans and van Boven, 1987; Rethans et al., 1991a, 1991b). Themain limitation of our work so far is that the tutorials are 'staged' and knowinglyrecorded, and may invite atypical behaviour. A follow-up study, using the samemethodology, after one year, showed sustained changes in behaviour suggestingthat 'under test' they behaved 'normally' (Pitts and Coles, 1996).

Implications for the future

This approach represents a methodology for the analysis of actual teaching behav-iour. As such, it has the potential for fostering improvement in many areas - coursedevelopment and teacher accreditation being some. Standardised learners, like theircounterparts, standardised patients, can contribute significantly to the education ofdoctors and hence improved health care.

122

DEVELOPMENT OF A 'STANDARDISED LEARNER'

References

Colliver, J.A. & Williams, R.G. (i993). Technical Issues: Test Application. Academic Medicine 68:454-460.

Kinnersley, P. & Pill, R. (1994). Potential of Using Simulated Patients to Study the Performance ofGeneral Practitioners. British Journal of General Practice 43: 297-300.

Kopelow, M.L., Schnabl, G.K., Hassard, T.H., Tamblyn, R.M., Klass, D.J., Beazley, G., Hechter, F. &Grott, M. (1992). Assessing Practicing Physicians in Two Settings Using Standardised Patients.Academic Medicine 67: (Supplement) 19-21.

Lesky, L.G. & Wilkerson, L. (1994). Using 'Standardised Students' to Teach a Learner-CentredApproach to Ambulatory Precepting. Academic Medicine 69: 955-957.

Norman, G.R., Neufeld, V.R., Walsh, A., Woodward, C.A. & McConvey, G. (1985). MeasuringPhysicians' Performances by Using Simulated Patients. Journal of Medical Education 60: 925-934.

Pitts, J. (1993). Changes in Attitudes to Teaching and Consulting Following the Wessex General Prac-tice Trainers' Course - the Urchfont Experience. Postgraduate Education for General Practice4: 130-135.

Pitts, J. (1994). Persisting Change in Attitudes to Teaching and Consulting One Year After a GPTrainers Course. Education for General Practice 5: 174-178.

Pitts, J. & Coles, C. (1995). Objective Evaluation of Clinical Teaching: A Controlled Trial. Educationfor General Practice 6: 284-293.

Pitts, J. & Coles, C. (1996). Changes in Teaching Behaviour One Year After a General PracticeTrainers Course: Implications for Course Planning. Education for General Practice 7: 199-207.

Rethans, J.J.E. & van Boven, C.P.A. (1987). Simulated Patients in General Practice: A DifferentLook at the Consultation. British Medical Journal 294: 809-812.

Rethans, J.J., Drop, R., Sturmans, F. & van der Vleuten, C. (1991). A Method for IntroducingStandardized (Simulated) Patients into General Practice Consultations. British Journal of GeneralPractice 41: 94-96.

Rethans, J.J., Sturmans, F., Drop, R. & van der Vleuten, C. (1991). Assessment of the Performance ofGeneral Practitioners by the Use of Standardized (Simulated) Patients. British Journal of GeneralPractice 41: 97-99.

Simpson, D.E., Lawrence, S.L. & Krogull, S.R. (1992). Using Standardised Ambulatory TeachingSituations for Faculty Development. Teaching and Learning in Medicine 4: 58-61.

Stillman, P.L. & Swanson, D.B. (1987). Ensuring the Clinical Competence of Medical GraduatesThrough Standardised Patients. Archives of Internal Medicine 147: 1049-1052.

Tamblyn, R.M., Abrahamowicz, M., Berkson, L., Dale Dauphinee, W., Gayton, D.C., Grad, R.M.,Isaac, L.M., Marrache, M., McLeod, P.J. & Snell, L.S. (1992). First-Visit Bias in the Measurementof Clinical Competence with Wtandardised Patients. Academic Medicine 67: (Supplement) 22-24.

Van der Vleuten, C.P.M., & Swanson, D.B. (1990). Assessment of Clinical Skills with StandardisedPatients: State of the Art. Teaching and Learning in Medicine 2: 58-76.

Woodward, C.A., McCovey, G.A. & Neufeld, V. et al. (1985). Measurement of Physician Performanceby Standardised Patients; Refining Techniques for Undetected Entry into Physicians Offices.Medical Care 23: 1019-1027.

123