Lancashire School of Health and Postgraduate Medicine

MSc Medical Education

Module GP4990

An exploratory study of perceptions of International Medical

Graduates (IMGs) in relation to peer group study and

preparation for the Membership of the Royal College of General

Practitioner (MRCGP) Clinical Skills Assessment (CSA)

Dr David Andrew Harniess

October 2016

2

Contents

Acknowledgements 4

Abstract 5

Chapter 1 Introduction

IMG performance in the MRCGP CSA examination

Brief history of the CSA and its evolution that

affected IMGs performance

Demographic and other risk factors attributed to

failing the CSA exam

What are the possible factors underlying the increased CSA failure rate amongst IMGs?

Peer group learning and preparation for the CSA amongst IMGs: Where does this research fit in?

6 6 7 9 10 13

Chapter 2 Background and Literature Review

Theoretical models for peer assisted learning (PAL)

Conceptual framework for PAL

Purpose of literature review

Scope of the review

Mini – Systematic Review: IMGs and PAL for the acquisition of consultation and communication skills

Broadening the literature search: Peer group learning and the acquisition and competency of communication and consultation skills in the medical education

o Postgraduate studies o Undergraduate studies

Conclusion

Limitations of the literature review

16 17 18 21 23 32 39 41 47 48

Chapter 3 Methodology and Study Design

Personal context of research topic

Choosing qualitative methods

Sampling and recruitment

Method of data analysis

Strengths and weaknesses of the study

Ethical considerations

Reflections on the process of ethical approval

50 50

51

52

57

58

60

62

3

Chapter 4 Data Analysis

Exploratory interviews

Thematic areas seen in code matrix browser

Evidence gained which confirmed findings in the literature relating to IMGs preparing for the CSA

New emergent themes o Group size, use of multiple groups and

frequency of meeting o Forming peer groups

Perceived advantage of having UKG in peer group

Barriers to forming groups with UKGs Pull factors to form with IMG-only groups

o Group dynamics o Planning learning activities and learning

strategies employed o Difference in peer group feedback style in

different peer groups o Peer group study as motivation for learning o Poor self-esteem and self-monitoring o The universal essence: `It’s different from where

I’m from’

Where does peer group learning fit into the context of other learning strategies for the CSA?

65

66

67

68

76

76

77

78

79

79

82

86

90

94

94

96

98

Chapter 5 Discussion

What does this research confirm around IMG factors affecting performance in the CSA?

New emergent themes from the data on peer group learning

Peer assisted learning (PAL) in the context of other learning strategies

Recommendations

Limitations of this study

Further research

99 102 104 111 112 114 115

Appendices Appendix 1: Table summarizing research trials analysed in both medical undergraduate and postgraduate setting

Appendix 2: Semi-structured interview questions

Appendix 3: Information sheet of study and consent form

Appendix 4: Proposed questionnaire of IMG’s perceptions of peer group study for the CSA

116 126 129 135

Bibliography

References

138 140

4

Acknowledgements

I would like to express my personal thanks to the following people for their support

and understanding and without whom this research project would not have been

possible:

My supervisor Dr Anne Milston for her ready support and advice and especially her

willingness to guide me through coding of the data,

my co-supervisor Dr Paul Milne for his counsel and direction at the beginning of this

project,

Dr Steve Agius, Senior Research Fellow in Medical Education at HENW for further

advice and support with the study design,

UCLAN STEMH Ethics committee for their recommendations and advice on the

research methods and ethical considerations,

Professor Aneez Esmail for his helpful insight into overcoming barriers of recruitment

of IMGs into the study,

the IMGs, from the Northwest of England, who volunteered to participate in the

study,

colleagues at the HENW who have expressed interest in and encouragement with

this study: Dr Bob Kirk and Dr Rebecca Baron,

Fiona for your ongoing love; for giving me space, looking after our boys, to write up

this project and help in proof reading my final draft,

Sam and Joel thank you for your patience and understanding of deprived daddy

time!

and finally at times this has been hard and a struggle particularly with delays with the

ethics committee, difficulty recruiting IMGs into the study and motivation to keep

going so I would like to acknowledge divine inspiration that has kept me going.

`Not only so, but we also rejoice in our sufferings, because we know that suffering produces

perseverance; perseverance, character; and character, hope. And hope does not put us to shame,

because God’s love has been poured out into our hearts through the Holy Spirit, who has been given

to us.’ Romans 5:3-5.

5

Abstract

This study attempts to explore how international medical graduates (IMGs) perceive

the use of peer group learning to prepare for the MRCGP CSA examination in the

northwest of England. A systematic review of the medical literature was carried out

evaluating the evidence for peer assisted learning (PAL) and subsequent improved

competency in consultation and communication skills. Data was gathered from

interviews which were then coded using thematic analysis. The report concludes

that IMGs face challenges in adapting from their own culture and educational

background to both the mode of study, and the focus on patient-centred

communication in the CSA exam. It recommends the formation of mixed IMG / UK

graduate groups and a focus amongst IMGs on development of certain PAL skills.

6

Chapter 1: Introduction

`If you have knowledge, let others light their candles at it.’ Margaret Fuller

International Medical Graduate (IMG) Performance in the MRCGP Clinical

Structured Assessment (CSA) Examination

A recent General Medical Council review of the MRCGP CSA examination

highlighted that black and minority ethnic (BME) international medical graduates

(IMGs) are 15 times more likely to fail the CSA than their white UK graduate peers1.

This was confirmed in a retrospective analysis of IMG and BME candidates sitting

the CSA examination2. This difference between IMGs and white UKGs performance

in the CSA has been demonstrated in a systematic review looking at academic

performance in both undergraduate and postgraduate settings and correlates with

other postgraduate medical examinations3,4,5.

Failing the MRCGP CSA exam has significant implications for trainees, given the

personal financial cost of £1780 (as of 2014)6, possible extensions to training

programmes cost the NHS some £40 000 per trainee, and in addition there is the

human resource of extra training7. The CSA is a high-stakes examination for

trainees in that if a candidate fails the examination four times, they are unable to

complete their GP training and practice general practice in the UK8.

7

Brief history of the CSA and its evolution that affected IMGs performance

Prior to 2007, the old MRCGP examination assessment of self-selected video

consultations was deemed inadequate, and the need for a more standardised real-

time assessment was identified. A two year design period resulted in the

introduction of the CSA in October 2007. The CSA uses the principles of an objective

structured clinical examination (OSCE) to test a number of competencies from the

MRCGP curriculum in a simulated surgery format. These include patient-centred

care, problem-solving skills, a comprehensive approach, community orientation and

holistic manner9. Each of the thirteen cases is marked by a different examiner, using

a generic marking schedule which comprises of three domains: data gathering,

clinical management (using evidence based medicine), and interpersonal skills (the

doctor-patient relationship). The CSA has been defined as ‘an assessment of a

doctor’s ability to integrate and apply appropriate clinical, professional,

communication and practical skills in general practice’9.

Shaw argued in a letter to the BMJ that due to the incentivisation to increase training

numbers, between 2007 and 2009 deaneries accepted candidates with lower scores

than previous years (including IMGs with poorer English)10. This factor coupled with

a change in the CSA in 2010 that raised the standard of communication required to

pass had a huge impact on the performance of IMGs in the examination, whereas in

previous years they would have `been good enough to pass’10.

8

Is the CSA fair? Recent controversies around racial bias and the CSA putting

white UK candidates at an advantage

Esmail, in his analysis of academic performance of BME graduates versus white UK

graduates (UKGs), controversially concluded that racial bias couldn’t be excluded in

accounting for the difference in academic performance between them2. His

argument followed that whilst BME IMGs have cultural and linguistic factors to

contend with, UK-born BME graduates don’t and there is still a discrepancy in

performance between UK BME graduates and their white counterparts.

In response to Esmail’s claims, Wakeford undertook a cross-comparison study of

candidates from ethnic minorities who had sat both the Membership of the Royal

Clinical Physicians (MRCP) examination and the MRCGP, and found there was a

mirroring of Esmail’s results in both examinations. In fact, performance in the new

CSA format from 2010 showed stronger correlations with performance in the

Practical Assessment of Clinical Skills (PACES) examination than MRCP

performances when studied in relation to ethnicity11.

A recent detailed collaborative study between the RCGP and King’s College London

criticised the CSA examination’s lack of diversity in case mix. In particular there were

relatively few patients from linguistic minority backgrounds and so the UKG may

have advantaged12. Furthermore, the weighting of interpersonal skills advantaged

UKGs relative to IMGs and these `soft skills’ were felt to be unrepresentative of the

real world challenges of consulting in a diverse patient population.

9

Demographic and other risk factors attributed to failing the CSA exam

There have been a few studies looking at the demographic and attributable factors in

candidates who are at higher risk of failing the CSA examination. These studies

have highlighted that IMGs, BME candidates, male versus female candidates and

older versus younger candidates are all at higher risk of failing the CSA examination

as outlined by the Table 1 below1,3,13.

Risk factor for failing CSA Increased

likelihood of

failing the CSA

Study

Black and minority ethnic

(BME) UK candidate

3.8 times Esmail 2012

BME International medical

graduate

14.5 times Esmail 2012

Older candidate over 35 years

of age (UKG/ BME IMG when compared to

own ethnic group)

4.3 times / 1.8

times

GMC

Independent

Enquiry 2013

Male candidate (UKG/ BME IMG when compared to

own ethnic group)

2 times / 1.4

times

GMC

Independent

Enquiry 2013

Low PLAB Part 2

communication score (<3 compared to >4)

2 times GMC

Independent

Enquiry 2013

Low IELTs score (overall score of 7 versus 9)

2.6 times GMC

Independent

Enquiry 2013

Deanery placement (Mersey

Deanery* 78% failure rate versus

London 46% failure rate)

*excluding East Scotland (80% failure

rate) as only 4 IMGs

1.7 times GMC

Independent

Enquiry 2013

Table 1 Risk factors identified from the literature for likelihood of failure of CSA

10

What are the possible factors underlying the increased CSA failure rate

amongst IMGs?

Language and linguistic factors of the CSA examination and its link to IMG

performance

A 2014 detailed collaborative study between the RCGP and King’s College London

performed an analysis of 40 videoed CSA examinations with a range of IMG, BME

UKGs and white UKGs to identify linguistic and cultural factors in the CSA12. The

researchers used qualitative and quantitative sociolinguistic methods that revealed

that poorly performing candidates found it difficult to give explanations to role playing

patients, had greater misunderstandings in the consultation and experienced more

difficulty repairing these misunderstandings.

In addition to this, the GMC independent enquiry found a correlation between

candidates with poorer scores in the IELTs (particularly in the English understanding

scores) and higher risk of failing the MRCGP CSA. This fits with the King’s College

London study where misunderstandings in the consultation led to irreparable

misunderstandings and misalignment with the simulated patient 9.

Wider cultural factors of communication

Whilst there are no specific studies looking at cultural factors of communication in

relation to the CSA, it is discussed widely in the literature. One UK qualitative study

looking at the experiences of 26 non-UK doctors working in hospital posts in the UK

found that cultural aspects of communication often led to misunderstandings and a

feeling of isolation14. Interviewees identified a range of difficulties in communication

which included misunderstandings about the use of eye contact, tone of voice, facial

expressions and gestures and more subtle issues of cultural expectations of social

11

behaviour, none of which would be picked up in a standard English language test.

This further highlights the complexity of cross-cultural communication and aspects of

rapport building that area a challenge for IMG doctors.

The adjustment to the UK perspective of patient-centred communication

When considering the cultural aspects of the doctor-patient relationship, IMGs come

from a range of countries with differing perceptions of the nature of the doctor-patient

relationship and how doctors ought to communicate with patients. International

studies of IMGs working in Australasia and North America highlight the lack of

understanding of patient-centred models of communication (desired by the RCGP in

the CSA)15,16. A systematic review of the issues of training IMGs highlighted how

IMGs often come from cultures where the doctor operates from a position of

considerable power in the community17. Thus, difficulty arises when the IMG moves

to a culture in which the doctor–patient relationship is more equitable.

One Australian observational study looked at IMGs in role play and noted IMGs

tended to be physician centred, see the doctor-patient interaction as an interview

rather than a conversation, use checklist questions and were highly structured rather

than flexible in allowing the patient to have equal share in contributing18.

12

IMGs coping with simulation in the CSA

The Royal College of London study highlighted that one cultural aspect of the CSA

exam that IMGs struggled with was simulation. According to one MRCGP examiner

in a letter to the BMJ, having to `imagine and suspend belief’ in a simulated

consultation adds to the complexity of the task19. In the `simulation game’, role

players behave differently from patients in that they are more dominant and ask

more questions, according to an analysis of a 100 medical students and simulated

patients20.

Lack of exposure of IMGs to UK general practice

In the GMC independent review, one of the reasons cited to account for this

differential between IMGs and UKGs was preparedness of the candidates, based on

amount of prior UK education experience of general practice1. The vast majority of

IMGs are from the Indian subcontinent, and other regions where the discipline of

general practice is underdeveloped. There is therefore a lack of exposure to high

quality general practice putting IMGs at a disadvantage compared to their UK

colleagues.

Lack of exposure of IMGs to undergraduate communication and consultation

skills teaching

In the UK, consultation skills are now taught and developed at undergraduate level

using video recording of consultations, the use of simulated patients and role play21.

IMGs highlight that one of the factors leading to poorer outcomes in the CSA is often

the lack of communication training in their home countries22. The process of working

with simulated patients and using videoed consultations was sometimes met with

13

resistance from IMGs, and two UK qualitative studies reported IMGs as saying it was

`invasive’ and `unnatural’15,16.

Study skills and reflective practice of IMGs

Given that the MRCGP CSA examination requires an integration of consultation and

communication skills with up to date medical knowledge and practice, candidates

preparing for this examination need to use a variety of learning methods23. In terms

of planning one’s learning, Warwick found that the majority of IMG trainees struggled

with reflective practice, which some IMGs viewed as a separate practice for

`academics’24.

Peer group learning and preparation for the CSA amongst IMGs - where does

this research fit in?

Patterson, in her review of how to help struggling CSA candidates, highlighted the

need for further research into the study skills of CSA candidates25. Little is known of

IMG’s perceptions as to how they plan and carry out their study for the CSA. There is

some anecdotal evidence from GP trainers that IMGs do struggle with reflective

practice but there has been no detailed exploration from an IMG perspective.

Jamieson and Browne described on a West Scotland revision course how many

IMGs would form their own study groups comprised solely of IMGs, with a tendency

to praise doctor-centred and biomedical behaviour in practice consultations, in

contrast to the patient-centred approach required for the CSA26.

14

Small group peer learning is an important aspect of learning and preparing for the

CSA but there are no studies exploring how peer-assisted learning is linked to

performance in the CSA examination. There is a need for greater understanding as

to how these small study groups form, function and perform as regards IMGs

preparing for the CSA. This qualitative study aims to explore, from an IMG

perspective different aspects of peer group learning from a theoretical framework.

15

16

Chapter 2: Background and Literature Review

`When one teaches, two learn.’ Robert Half

The previous chapter outlined the particular research aim and argued for its

relevance and importance. In order to start gathering data on the perceptions of

IMGs on peer group study for the CSA, it is first necessary to explore the relevant

background to the study. Current theoretical frameworks will first be reviewed,

followed by a literature review.

Definition of terms

“Peer assisted learning (PAL) is a generic term for a group of strategies

that involve the active and interactive mediation of learning

through other learners who are not professional teachers” Topping27.

More briefly, it is the development of knowledge and skill through explicit and active

helping and supporting among status equals, with the deliberate intent to help others

with their learning goals. There are a variety of PAL methods described in the

literature including peer tutoring, modelling, monitoring and assessment. Feedback

from peer assessment is usually intended to be formative – enabling the learner to

improve performance.

17

Theoretical Models for Peer Assisted Learning

Several theories from psychology have been applied to explain and predict positive

and negative effects of peer assisted learning. The concept of cognitive congruence

takes the view that learning is an extension of an existing knowledge base.

According to Vygotsky, learning is optimized if the distance between what is already

known and understood and what must still be learned is just enough to stimulate

active inquiry by the student, a distance called the `zone of proximal development’28.

Near peers may sense this zone of proximal development much more easily than

content experts, who may not always understand the cognitive problems students

experience when processing new information29.

Role theory comes from primary education cross-age tutoring (e.g. higher class

children tutor younger ones) where the interpersonal rewards, such as offering

friendship and serving as a role model, motivate the tutor at the same time as

stimulating learning in the younger student (social congruence). The trusting

relationship of a peer might facilitate self disclosure of ignorance and cognitive

errors, enabling subsequent diagnosis and correction4. Thus, a near-peer tutor may

be a better catalyst then a more senior teacher, provided that this near peer has

sufficient content expertise on the topic.

Another more recent theory has been self-determination theory (SDT) which relates

to role theory and seeks to explain why intrinsic motivation may be increased by

having a teaching role. SDT claims that intrinsic motivation is caused by three

features: competence, autonomy and relatedness to significant others30. Teaching

may very well serve to generate these particular feelings of competence, autonomy

and esteem before others, which in turn could increase motivation for further study.

18

Ten Cate summarised these different theoretical perspectives and the postulated

benefits on the student taught versus student tutor31.

Classification of theoretical perspectives on peer teaching

Postulated benefit for the student being taught

Postulated benefit for the teaching student

Cognitive and metacognitive level of learning

Cognitive congruence Goal-oriented information processing and verbal elaboration

Affective and motivational level of learning

Social congruence Role theory and adjoining theories

Table 2 (borrowed from Ten Cate & Durning 2007)

Conceptual Framework

Topping and Ehly developed a conceptual framework for how PAL works from

theoretical underpinnings32. They summarised this in a single chart (see below).

Cognitively PAL involves conflict and challenge (reflecting Piagetian schools of

thought - see Footnote1). The benefit of verbalisation and recitation in peer teaching

has been demonstrated in several studies and participants might never have truly

grasped a concept until they had to explain it to another, embodying and crystallizing

thought into language (Vygotskian concept)33,34,35,36.

1 Piagetian schools of thought emphasize the `constructivist theory of knowing’ which focus on how

humans make meaning in relation to the interaction between their experiences and their ideas.

19

The five sub processes described feed into a larger onward process of extending

each other’s declarative knowledge leading to a joint construction of shared

understanding between peer tutor and the tutee; the `intersubjective’. The

intersubjective might not represent absolute truth but forms a foundation for further

progress. PAL enables and facilitates a greater volume of engaged practice, leading

to consolidation, fluency and automaticity of core skills. As this occurs, both peer-

tutor and tutee give feedback to each other, implicitly or explicitly.

Both peer tutor and tutee generally begin to become more consciously aware of what

is happening to them in their learning interaction, and more able to monitor and

regulate the effectiveness of their own learning strategies in different contexts. This

development into fully conscious explicit and strategic metacognition not only

promotes effective onward learning; it should also increase confidence to achieve

more.

20

Conceptual Framework of Peer Assisted Learning – Topping and Ehly27

21

Literature Review

The purpose of a literature review

Cohen et al, postulate that a literature review should establish a theoretical

framework for research, and describe significant prior research, defining constructs

and concepts, and reporting methodologies used37. As well as setting out the key

issues, a literature review identifies gaps that need to be explored (further reasons

are listed in Table 3).

Why is conducting a literature review important?

Learning from researcher’s mistakes and avoid making the same ones.

It may help one consider the inclusion of variables in your own research that

you otherwise might not thought of.

It may suggest further research questions for you.

It will help with the interpretation of your findings.

It gives you some pegs on which to hang your findings.

Table 3 Bryman’s reasons for conducting a literature review38

As Silverman sets out, there is also a critical element to a literature review and an

assessment of the quality of the research already gone before39. With this in mind

the Critical Appraisal Skills Programme (CASP) tool was used to assess papers

identified and reviewed here40. The CASP tool was originally developed to help

researchers critically appraise research studies in different standardised domains.

22

Scope of the review

The focus of this study is how IMGs utilise peer group study methods in preparation

for the MRCGP CSA examination. Firstly the literature review sought to determine

whether there were there any research papers comparing IMGs and UKGs in their

utilisation of methods in acquiring consultation and communication skills. The

second aspect of the literature review is to determine whether there is evidence for

peer group study in assisting with the acquisition of competency in consultation and

communication skills. Finally it sought to identify an emerging thematic framework

on PAL in this area, in order to apply further exploration in this study.

23

Mini – Systematic Review: IMGs and PAL for the acquisition of consultation

and communication skills. Looking for the needle in the haystack!

Search Strategy

Due to the time limitations of a Masters research project, a mini systematic review

was conducted, with a sole reviewer of the research papers, not in keeping with the

recommended two independent reviewers required for a fuller systematic review

outlined by Prospero41. Systematic reviews are the most appropriate method of

conducting an unbiased, structured review of good quality research to limit bias and

random error that can occur in traditional narrative reviews42. Evans and Benefield

set out 6 principles for undertaking systematic reviews43:

1. A clear research question to be addressed

2. Systematic, comprehensive and exhaustive search for relevant studies

3. Clear criteria for inclusion and exclusion of studies

4. Evaluation of the quality of the methodology in the studies

5. Specification of strategies for reducing bias in selecting and reviewing

studies

6. Transparency in the methodology adopted for reviewing the quality of the

studies

24

The research question was broken down into components to fit the Population

Intervention Comparator Outcome (PICO) format44. The PICO format was

developed by Sacks et al as a way of formulating research questions that can be

broken down into their different components in order to test hypotheses and build

literature reviews on research questions.

Research Question

The research question was distilled into the following: `Reflective analysis of medical

education research on peer group learning amongst international medical graduates

compared to UK graduates for communication and consultation skills’. This question

was then further subdivided into its different components of the PICO question

shown in Table 4:

Population(s)

International medical graduates

Intervention

Peer group learning

Comparator(s)

UK trained graduates

Outcome(s)

Communication and consultation skills

Study Designs to be included Systematic review or synthesis / Randomised Control Trials / Mixed methods/ Qualitative studies

Table 4 PICO research question `IMGs vs UKGs in acquiring consultation and communication skills’ broken down into different components

This literature review is narrowed in that it is limited to the UK context. However

IMGs travel the world to practice family medicine and work and train in other

countries with well-established primary care which assess communication and

consultation skills. With this in mind the comparator of `UK trained graduates’ was

not included in the initial search to allow a broader review of the literature. If a large

25

number of studies were found, this extra parameter of `UK trained graduates’ would

be included in addition in the final search.

There was some internal debate as to what the outcome measure should be in that

the CSA similar to an OSCE in testing both communication and clinical skills. Thus

the search should include studies looking at peer group learning for the acquisition of

clinical skills. However after consultation with CSA candidates, a CSA examiner and

review of the reference material regarding the exam it was felt the primary focus of

the examination was testing communication and patient-centred consultation skills45.

The outcome measure of examination and academic performance could have been

considered. However it was felt that whilst the ultimate aim of peer group learning

amongst all MRCGP candidates is to pass the CSA examination, the actual specific

objective of the peer group learning was to acquire and practice communication and

consultation skills.

Databases to be included

As research based studies in medical education were to be considered, medical and

educational databases will be used in the relevant literature as listed below:

Cochrane (systematic review and synthesis database)

Medline Full text (comprehensive medical journal database)

Cinahl (allied health professional and emphasis on qualitative research)

ERIC (education database that includes medical educational studies)

PsychINFO (psychology and behavioural science database, including medical

educational research)

26

These databases were selected as being health-specific and education databases

containing all the journals of relevant to the research area. The use of more than

one database widens the scope of the literature search increasing

comprehensiveness.

Research Study Strategy

Keywords were put into each of the databases and specific MeSH terms (Medline)

and Subject Titles (Cinahl, ERIC and PsycINFO) were then ascertained to broaden

the search. All the keywords, MeSH terms and Subject Titles were then employed to

interrogate the individual databases rather than running the search on all the

databases simultaneously2. Advanced searches using Boolean operators of `OR’

were used between the different terms to capture as many studies as possible and

`AND’ between the different subgroups to narrow the search to more specific and

relevant studies. The full search is outlined in the PICO format in Table 5 below.

2 Running separate searches on the individual databases means you do not lose your specificity and

sensitivity of your search as gleaned after a learning session with a UCLAN librarian. If you include all the databases simultaneously in your search you may miss relevant articles.

27

PICO domain

Keyword, MeSH term or subject title

Population `International medical graduate*’ OR `Foreign Medical Graduate*’ (MeSH and subject term used in both Cinahl and Medline databases)

AND

Intervention `Peer group learning’ OR `peer group’ OR `peer assessment’ OR `small group’ OR `peer assisted learning’ (MeSH term on Medline) OR `Group processes’ (subject title used in Cinahl) OR `peer teaching’ OR `peer evaluation’ (both subject titles used in ERIC)

AND

Outcome `Communication skill*’ OR `consultation skill*’ OR (no extra MeSH or subject titles deemed appropriate as covered by keywords)

AND (if many studies)

Comparator `UK medical graduate’ OR `UK trained’ OR `UK qualified’

Table 5 to summarise search strategy using PICO method

The search filters used were `research’ to exclude opinion pieces and review

articles. Another filter adjusted was to select only papers written after 1990.

Inclusion / exclusion criteria

In considering inclusion and exclusion criteria, the principle of the hierarchy of

evidence46 (as outlined in Figure 1 below) was used to capture higher level studies

such as systematic, randomised control trials and qualitative studies, but exclude low

level research such as case study and expert review articles.

28

Figure 1 - Hierarchy of Evidence

Inclusion criteria

Exclusion criteria

Systematic or original research

International medical graduates

Involved peer group studying involved

Acquisition of communication and consultation skills

English only articles

Full text article available

Articles prior to 1990

Inter-professional, nursing or allied health learning

Clinical skills acquisition

Expert review articles, case studies or descriptive studies

Table 6 Inclusion and exclusion criteria in search strategy

29

Search results

The initial search results uncovered 3 research articles and no articles when the

comparator of `UK medical graduates’ was added as a further subgroup to the

literature search. Here are the number and types of articles uncovered using the

search terms above.

Database Articles found

Cochrane No relevant reviews (Konowicz et al are due to review virtual patient simulations for health professional education which is a commonly used learning method in peer groups for acquiring and assessing

communication and consultation skills)47

PsychoINFO 1 descriptive Australian study (communication skills project with IMGs using poetry)

Medline with full text No relevant studies

ERIC 1 abstract (no full text article available)

Cinahl 1 descriptive American study (communication skills teaching programme over 1 year for IMGs in Internal Medicine)

Table 7 Search results from databases looking at IMGs and peer group learning for the acquisition of consultation and communication skills

These three international studies were teacher and programme orientated in their

approach rather than peer group centred and led. The ERIC database research

article had no full text available so was excluded from the critical review. The 2

remaining papers were analysed using the critical appraisal skills programme

(CASP) tool as previously outlined and a narrative critique is given4,48.

30

Study

Was there a clear statement of the

aims of the research?

Is a qualitative methodology appropriate?

Was the research design

appropriate to address the aims of the research?

Was the data collected in a way that

addressed the research issue?

Has relationship of researcher

and participant been

considered?

Have ethical issues been taken into

consideration?

Was the data analysis

sufficiently rigorous?

Is there a clear statement of

findings?

Score out of 8

Duncan 2007 N Y N N N N N N 1

Ramaswamy 2014 Y Y Y Y N N Y Y 6

Table 8 CASP tool evaluation of qualitative studies of IMGs using peer group work to acquire communication and consultation skills

31

The Australian study described a novel approach to developing communication skills with cultural

understanding by stimulating small group discussion of selected poetry, and videoeing role plays

of 13 IMGs in rural Australia49. There were no clear objectives in terms of what communication

and consultation skills were acquired or assessed, and no clear research design outlined. There

was some quoting of phrases picked up on the video assessments by the language experts but no

formal qualitative research methods were used. This was a poor descriptive study of a novel

teaching approach to communication skills and cultural understanding for IMGs. Furthermore it

was very specific to the rural Australian context and therefore not easily generalizable to the UK

context.

The American study looked at 23 IMGs on an Internal Medicine residency programme which had

developed a communication skills curriculum looking specifically at communicating with families

and breaking bad news50. This was a higher quality study with clearly defined specific goals and

measurement of outcomes with the use of validated tools. The teaching programme concluded

small group workshops with facilitators using case based learning and role play. Participants in

this study did show an improvement in confidence and competency in communication skills in

specific contexts of breaking bad news and speaking with families, which represents a higher level

consultation skill. However the study was limited as it lacked a non-intervention comparator

group. Time in the speciality and more exposure to live patient encounters at work might have

also led to increased confidence and competence in these specific communication skills. Like the

Australian study, however these were not self-monitoring peer groups and an external facilitator

was guiding the learning process in the groups.

On reading the references of these 2 studies did not reveal any further research studies. To widen

the search I changed the population studied to include `undergraduates’, but no additional relevant

studies were identified.

32

Broadening the literature search: Peer group learning and the acquisition and competency

of communication and consultation skills in the medical education literature. Seeking the

pearl of great price!

In light of the paucity of evidence considering IMGs and peer group learning, with regard to

acquisition of consultation and communication skills, it seems appropriate to broaden the search

strategy to look at peer group learning and communication and consultation skills acquisition and

competence in the medical education literature.

In a bid to increase the breadth of the literature review it was felt studies attaining professional

competency and academic performance should also be included, not just the acquisition of

communication and consultation skills. So the search strategy to be considered in this new review

is `Reflective analysis of the medical education research on peer group learning amongst medical

doctors and students for the acquisition of communication and consultation skills for professional

competency’. Table 8 outlines this question in PICO format.

Population(s) Medical students and postgraduate medical doctors

Intervention Peer group learning

Comparator(s) None

Outcome(s) Communication and consultation skill acquisition, competency and examination performance

Study designs to be included Systematic reviews or synthesis / RCT / Mixed method / Qualitative studies

Databases to be accessed Cochrane, Medline with Full text, Cinahl, ERIC and PsychINFO.

Table 8 PICO method to break down research questions into different components

33

Study methodology

Research based studies in medical education were sought and therefore the same medical based

databases as previously were used. As before, each individual database was interrogated

separately with keywords to be able to determine which specific subgroup of MeSH or Subject

Titles were relevant to that particular database as outlined below:

PICO domain Keyword, MeSH term or subject title

Intervention `Peer group learning’ OR `peer group’ OR `peer assessment’ OR `small group’ OR `peer assisted learning’ (MeSH term on Medline) OR `Group processes’ (subject title used in Cinahl) OR `peer teaching’ OR `peer evaluation’ (both subject titles used in ERIC)

AND

Outcome `Assessment’ OR `examination’ OR `competency assessment’ (CINAHL subject heading) OR `professional competence’ (CINAHL and PsychINFO subject heading) `communication skills evaluation’ (CINAHL subject heading) OR `educational measurement’ (MEDLINE MeSH term) OR `performance based assessment’ (ERIC subject heading)

AND

Population `postgraduate medical education’ OR `doctor*’ OR `postgraduate family medicine’ OR `postgraduate primary care’ OR `postgraduate teaching’ OR `general practitioner' OR `family medicine doctor' OR `primary care physician' OR family education (MeSH term Medline) OR `medical student’

AND (if many studies)

Specific Population `UK postgraduate general practice’ OR ‘UK postgraduate medical education’

Table 9 Keywords, MeSH terms, subject titles used in different databases for adapted research question

34

The only search filter used was `research’ in order to exclude opinion pieces and review articles.

Table 10 highlights the inclusion and exclusion criteria used:

Inclusion criteria Exclusion criteria

Systematic reviews OR original research (randomised trials or mixed methods or qualitative only methods)

Medical student OR doctor

Peer group studying involved

Acquisition of communication and consultation skills

Professional competence or examination performance of communication and consultation skills

English only articles

Full text article available

Articles prior to 1990 excluded

Inter-professional, nursing or allied health learning

Clinical skills acquisition

Expert review articles, case review studies and descriptive studies

Table 10 Inclusion and exclusion criteria for adapted research strategy

The following flow chart shows the initial 70 articles which were found (5 excluded as duplicates).

Further exclusions were made for inter-professional studies, lack of relation to peer group learning,

clinical skill only acquisition, foreign language articles and not original research. This left 13

studies which were English language with full text. A further 2 relevant studies were identified on

studying the references of the systematic reviews.

The studies were summarised and critically appraised using various CASP tools (depending on

the type of study) in tabulated format below. The studies have been synthesized and assessed

separately in terms of postgraduate and undergraduate studies in tabulated format in Appendix 1.

35

Flow chart to show search strategy including the inclusion and exclusion

36

Study

Did review address clearly focused question?

Did the authors look for right type of papers?

Do you think all important, relevant studies included?

Did review authors do enough to assess quality of studies?

If results of review combined, was it reasonable to do so?

What are overall results of review? How precise are the results?

Can results be applied to local popn?

Were all important outcomes considered?

Are benefits worth harms and costs?

Yu 2011 Y Y Y Y

Y (4

categories identified)

Category 1 - Peer-teaching vs tutor-teaching showed comparable learning outcomes for medical students Cateogry 2 - Does peer teaching have supplemental benefits for learners? 2 studies Yes, 2 studies No Category 3 - Peer-tutors do have better learning outcomes and do better academically Category 4 - Qualitative aspects of PAL - more relaxed and cooperative learning but learner concerns about reduced time with faculty.

Mixed method research - qualitative data verified in similar studies, some of the quasi randomised trials didn't all agree in categories 1 and 2

Y Y

Not known. In selected contexts PAL just as good as tutor led teaching. No data on effect of PAL medical student teaching on long term learning outcomes.

Burgess 2014

Y Y

N (too narrow timeline - 2002-2012)

N

NA (results

presented as what and how

questions)

Mixed results regarding accuracy of peer assessment and feedback. Many perceived learning benefits for student tutors but no evidence that there was improvement in tutor's examination performance. There is variation in recruitment processes, and tutor training, with little evidence of related effects on student tutor outcomes.

No summarisation of results or qualitative tool used

Y Y

Further research required to verify perceived benefits for student tutors and assessment of training of student tutors.

Williams 2016 (Scoping review)

Y Y

N (lack of MeSH and key terms

and not all UK

studies included

)

N

NA (studies

measuring different

PAL activities)

Improved academic performance of student tutors as evidenced by 5 studies. Positive effect also demonstrated on student learner's outcome in 10/17 studies showed statistical evidence (particularly in OSCE scores). May not be as much benefit of PAL in learning more complex practice skills. 5 studies showed no improvement in student learning outcomes and 2 studies showed PAL detrimental to learning outcomes.

Precise results of studies recorded

N (incl

nursing as well as

med student studies)

Y

Further research needed to determine the cost effectiveness, employability prospects and generalisability of PAL in healthcare education (only medical and nursing student studies with harder outcome measures of examination performance).

Table 12 CASP tool evaluation of systematic review studies assessing peer group work to acquire communication and consultation skills, professional competence or improve academic performance of consultation or communication skills

37

Study

Did trial address focused issue?

Have they truly randomised participants?

Has there been blinding of researchers and participants?

Were control and intervention groups similar at start?

Apart from intervention were the groups treated similarly?

Were all participants accounted for at finish of trial?

How large was treatment effect?

How precise was the estimate of treatment effect?

Can results be applied to your context?

Were all clinical outcomes considered?

Are benefits worth harms and costs?

Hobma 2006

Y Y

Y (2nd

observation) &

N (1st time not

feasible due to time

constraints)

Y

N (comm skills assessment

scores at start not given to control arm)

Y

Regression analysis showed a significant effect on both the treatment and the pre scores (MAAS-Global consultation score) - effect size (d-value) 0.66 - moderate to large effect. Greatest improvement in pt-centred skills.

Precise - although only 8 videos per participant selected to be analysed (many ignored)

Y Y

Depends on value placed on patient-centred consultation skills. Higher patient satisfaction but financially costly and requires skilled facilitators.

Cave 2007

Y Y N

(not possible) Y Y Y

No difference in OSCE performance of students receiving standard teaching, intervention A (assessment criteria given to students for feedback) or intervention B (mini-OSCEs with SP and feedback from SP, peers and tutor). Self-score significantly correlated with tutors and peer scores but not statistically significant correlation with SP scores.

Precise - OSCE marks given in 3 groups showed no difference in OSCE performance (p=0.5)

N (undergrad

uate study)

Y

No benefit of educational intervention seen in giving of mini-OSCE together with SP, peer and tutor feedback. SP feedback doesn’t correlate with peer or tutor feedback given.

Table 13 CASP tool evaluation of RCT studies assessing peer group work to acquire communication and consultation skills, professional competence or improve academic performance of consultation or communication skills

38

Study Was there a clear statement of the aims of the research?

Is a qualitative methodology appropriate?

Was the research design appropriate to address the aims of the research?

Was the data collected in a way that addressed the research issue?

Has relationship of researcher and participant been considered?

Have ethical issues been taken into consideration?

Was the data analysis sufficiently rigorous?

Is there a clear statement of findings?

Score out of 8

Perera 2010 Y Y Y Y Y Y Y Y 8

Cushing 2011 Y Y Y Y N Y Y Y 7

Chou 2013 Y Y (mixed methods) Y Y Y (blinding) N Y Y 8

Lau 2001 Y Y Y Y N N Y Y 6

Hulsman 2014 Y Y (mixed methods) Y Y N Y Y Y 7

Ahem 2013 Y Y Y Y Y Y Y Y 8

Shield 2011 Y N (descriptive) N N Y Y N Y 4

Nestel 2005 Y Y (mixed methods) Y Y N N Y Y 6

Henley 2014 Y Y (mixed methods) Y Y Y (blinding) N Y Y 7

Table 14 CASP tool evaluation of qualitative studies assessing peer group work to acquire communication and consultation skills, professional competence or improve academic performance of consultation or communication skills

39

Postgraduate Studies

Only three original papers relevant to postgraduate medical studies were found

researching the role of peer group work in acquiring and assessing consultation and

communication skills. The highest quality and most relevant of these was a Dutch

randomised control trial. In this a control group of GPs were randomly allocated to

individual self-directed learning or an intervention group. The latter group were

offered immediate feedback on a videoed consultation and they participated in

facilitated peer group work that was peer group directed51. There was evidence that

peer group work is more effective in improving consultation skills than individual

reflective work alone particularly in terms of patient-centred skills. The more peer

group meetings GPs attended, the greater the effect on their performance in

assessed video consultations (using the MAAS-Global rating scale). It must be

noted, however, these groups were facilitated by experts, and therefore differed from

self-monitoring peer groups that form for the CSA examination.

The second paper described a Scottish pilot study which looked at the development

of a new generic consultation peer assessment tool for GPs and pharmacists

particularly in light of professional appraisals52. The tool showed good inter-rater

reliability and validity for GP consultations, however it failed to show the same

consistency when used to assess pharmacist consultations, where there was greater

variation in the level of performance. The usefulness of this tool is questionable for

CSA candidates however it does raise the useful question of which, if any,

assessment tools do IMGs use in peer group learning? This is a subject for the

qualitative interviews.

40

An Australian qualitative study explored the feasibility, advantages and

disadvantages of vertical peer group learning made up of different levels of learners

in general practices in New South Wales. The agenda behind this study was to

justify and consider new models of learning in overstretched primary care

environments. Whilst its aims were not to study the acquisition of communication

and consultation skills it did shed some light on some aspects of peer and near peer

group learning in the postgraduate medical context as outlined in Table 15 below.

Benefits of vertical peer group learning

Disadvantages of vertical peer group learning

group session is more likely to be planned leading to better learning outcomes

easier and safer to ask questions

group can learn from knowledge and skills of others in the group

discover different approaches to the same problem

extra questions can be asked that an individual didn't think of

group participants challenge each other, encouraging debate and discussion.

learning in peer groups can motivate further individual learning

allow benchmarking against peers that can lead to improved self-confidence

not as helpful for shy learners

not easy to deal with sensitive topic areas

junior learners may need more 1 to 1 teaching

not as tailored for individual learning needs

Table 15 summarising benefits and disadvantages of peer group learning amongst different level of near-peer learners in general practice in an Australian setting. These specific elements highlighted above are relevant to IMGs in peer groups,

particularly in considering the positive aspects of peer group learning such as

motivation for learning, benchmarking against peers and practicing skills in a safe

environment. The highlighted disadvantages may be relevant to IMGs who may

have greater knowledge and skill gaps which would be better addressed by

increased 1 to 1 teaching with their GP trainer.

41

Undergraduate studies

There were a far greater number of undergraduate studies looking at peer assisted

learning (PAL) in the context of communication and consultation skills teaching. PAL

is a recent trend in medical undergraduate training and supplements faculty teaching

particularly in the current context where the increasing number of global medical

students is increasing and medical school resources are stretched. PAL is also

justified as helping to develop professionalism and teaching skills in peer tutors53.

The data from these different studies will now be synthesized into thematic areas as

described in the table below.

Peer assisted learning and undergraduate academic performance in consultation and communication skills

Systematic Reviews Yu et al found that peer-teaching achieved short term learner outcomes that were comparable with those produced by faculty-based teaching but data on long term learning outcomes was lacking54. Burgess demonstrated mixed results in terms of learner outcomes in PAL with 10/17 studies showing a positive effect, 5/17 studies no effect and 2/17 studies a possible detrimental effect (Burgess)32

Randomised trials Cave conducted a UK based randomised control trial where medical students were given the assessment criteria of the OSCE format to help aid peer feedback and assessment in peer group learning. The study showed there was no difference in OSCE performance compared with standard teaching with `mini-OSCE’ format teaching with simulated patients (SPs)55. Nestel another UK study compared near peer teaching of 3rd year medical students facilitating 1st year medical students in patient-centred interviewing skills with medical tutor teaching. This showed no difference in quality of videotaped assessments of student-SP consultations56.

42

Perera compared an intervention group using an objective structured self-assessment and peer-feedback (OSSP) in small group communication skills teaching sessions of 1st year medical students to a matched control group who were not using the OSSP tool57. The mean total score at OSCE for the intervention group was significantly higher than the control group (13.3 vs 12.3, p=0.0001). Conclusion The majority of studies show a positive effect on learning outcomes of peer tutored students but the evidence for improved academic performance is variable. The majority of studies don’t show any difference in academic performance between peer-tutored and medical faculty tutored students. Thus peer assisted learning in certain contexts has no detrimental effect on academic performance of medical students.

Peer Assisted Learning and Peer Tutor Learning Outcomes and Academic Performance

Systematic Reviews Williams concluded that there was a positive effect on medical student tutor performance confirmed in five studies32. Burgess agreed that while there were perceived learning benefits for student tutors participating in PAL activities, there was no substantial evidence of improvement of one’s examination performance. Yu concluded that there was an overall improved academic performance (particularly in OSCE based examinations) and benefits in terms of professionalism33. Conclusion All systematic reviews showed a significant beneficial effect on learning and some effect on academic performance in peer tutors (medical students) involved in PAL32,33,58.

43

Peer Assessment and Feedback on Consultation and Communication Skills Quality of feedback

Systematic Review Burgess reported mixed results in terms of peer assessment, two studies showed peer assessors as being more lenient than academic assessors whilst one study found the opposite32. Burgess showed that peer assessors could not competently determine a global mark in an OSCE practice examination59. Systematic Review One study reviewed by Burgess found that the quality of feedback given by peers was superior to medical tutor feedback32. Mixed methods studies Cushing et al showed that medical students had a greater desire for medical faculty feedback than peer feedback particularly when closer to OSCE examinations60. This qualitative study revealed there was anxiety in the peer group on giving negative and corrective feedback and the presence of peers prompted a mixed emotional response, some finding it easier to relax and others more pressurising. These learners were novices in giving feedback and a long way from postgraduate medical colleagues who have developed more intrinsic feedback skills. Chou was assessing whether longitudinal relationships among peers allowed students to give and receive more effective feedback on communication skills in 3rd year medical students61. Students with prior peer-learning relationships were more likely to deliver corrective feedback on communication skills to their peers, compared to those with no such experience. Conclusion This mixed result of peer feedback is confirmed in wider research on peer assessment in PAL and still remains a difficulty in PAL1. Chou’s study is interesting in looking at how peer groups are formed for practicing for the CSA and whether the longevity or brevity of being part of a learning group with their colleagues has any effect on the quality of the feedback given in peer group learning.

44

Peer Assisted Learning and the Acquisition of Consultation and Communication Skills

Systematic review In terms of communication and consultation skills Williams et al in their systematic review found that PAL positively affected OSCE scores32. Williams however did conclude that more complex consultation and practical skills were better taught by more experienced staff. It remains to be seen, whether this same principle applies of complex consultation skills acquisition and assessment within a postgraduate setting.

Patient-centred interviewing skills

Randomised Trial A quasi randomised trial by Nestel et al compared near peer teaching of 1st year medical students taught by their 3rd year colleagues versus medical faculty tutors in patient-centred interviewing. Interviews videotaped and rated by an independent assessor, showed no difference in the patient-centred interviewing skills between the two groups35. This was verified by simulated patient ratings. However on further analysis the peer tutors contributed to just 1 of 6 sessions in the communication skills programme so it is uncertain whether this one intervention had a defining effect. Conclusion This is an important aspect of assessment of the CSA examination in terms of patient-centred communication and consultation skills. This study does seem to support PAL as being on a par with medical faculty tutoring in acquiring these specific patient-centred skills.

45

Self-Assessment in the Context of Peer Assisted Learning

Randomised Control Trial Cave showed in his RCT that self-scores on mini-OSCEs correlated significantly with peer and tutor scores62. This shows that self- assessment of consultation and communication skills can be accurate and peer assessments in this study did marry with tutor based assessments. This study would also seems to suggest thatf SP assessments of consultation and communication skills in medical students as their scores had poor correlation with self, tutor or peer scores have little value. Mixed method trial Hulsman studied the characteristics of self-evaluation and peer-feedback annotations of video recorded communications skills63. Fourth year Dutch medical students video recorded a consultation with a simulated patient for formative assessment, made some self-evaluations using a web digital programme and invited peers to review the video and offer feedback. This study found self-evaluations were more specific than peer feedback particularly when negative. Students who were more specific in their self-evaluation stimulated their peers to be more specific in their peer feedback on data analysis. Hanley et al, an American study, looked at self-assessment and goal setting (SAGS) skills and the development of interviewing skills in 1st year medical students64. Higher baseline SAGS skills (as rated by a blinded researcher) were associated with an initial decline and then substantial increase in communication scores. This may be attributable to a phenomenon known as `expertise reversal’. Low quality SAGS ability at baseline that remained poor at reassessment at 10 weeks was a risk factor for failing to sustain growth in interviewing skills. This is an interesting area that needs more research to understand the longer-term implications for remediation of students. Perera, a Malaysian study, looking at 1st year medical students, found that those exposed to self- assessment and peer feedback in a small group setting may learn better than students who received only SP and tutor feedback on a communication skills teaching course65. Students gained fresh insights into specific areas such as empathy, addressing patients’ concerns and interview style during objective structured self-assessment and peer-feedback (OSSP). This was reflected in the OSCE scores which were significantly higher in the intervention group (13.3 vs 12.3, p=0.0001) particularly in building rapport with SP, listening and interview style.

46

Conclusion RCT evidence does show a correlation between self-assessment of consultation and communication skills with tutor and peer assessment but not with simulated patients. This is an interesting observation in terms of GP specialist trainees using SP in peer group sessions for assessment and feedback on their consultation and communication skills. The ability of self-assessment and goal setting is an important study skill which translates into the postgraduate setting where doctors are trying to attain professional consultation and communication skills. Other studies have shown a poor ability of medical students and doctors to self- assess their own performance with a tendency of male students to overestimate and female students to underestimate their performance66,67,68,69.

Table 15 Summarisation of thematic areas of PAL in medical undergraduate literature in terms of consultation and communication skills

47

Conclusion

This systematic review of the literature found no studies analysing the perceptions of

IMGs in forming peer groups to learn and acquire communication and consultation

skills. There is also a paucity of papers looking at postgraduate peer group learning

for gaining competency in consultation and communication skills. A few papers

focused on the positive aspects of peer group learning including motivation for

learning, benchmarking against peers and a safe environment for practicing skills.

There are no studies looking at the effectiveness of postgraduate peer led groups,

left to function without outside facilitation, to improve communication skills. There

are knowledge gaps in the medical postgraduate literature in terms of PAL and

academic performance in postgraduate examinations.

The main body of evidence for PAL in terms of improved academic performance is in

the medical undergraduate literature. It would seem that peer tutors benefit more

than peer learners in terms of improved academic performance. However there is

evidence to support that peer tutoring is as effective as faculty tutoring. In particular

patient-centred consultation skills, can be acquired through PAL. The evidence is

mixed in terms of the accuracy and quality of peer assessments, which is also

reflected in the wider literature. The ability of the student to make good self-

assessments and goal setting for future learning does have an impact on future

performance.

48

Limitations of the literature review

One of the limitations of this literature review is that only one researcher has critically

reviewed and appraised the process of selecting and critiquing the research papers.

Another criticism that could be made is the limited databases utilised as some of the

systematic and scoping reviews of peer-assisted learning accessed Web of

Knowledge, Embase and Proquest as additional medical educational databases.

49

50

Chapter 3: Methodology and Study Design

Introduction

The last chapter reviewed the relevant background literature to this study which

informed the study method and design. This chapter gives a personal history of the

selection and application of the research and data analysis methods employed in this

study (using Murcott’s questions see Footnote 3) and the course that the research

subsequently took. On the ethical issues raised and process of gaining ethical

approval reflection is also included.

Personal context of research topic

The stated aim of this study is: `An exploratory study of perceptions of International

Medical Graduates (IMGs) in relation to peer group study and preparation for the

Membership of the Royal College of General Practitioner Clinical Skills Assessment

(CSA).’

My interest in this area originates in my development as a GP trainer for general

practice where I coached GP specialist trainees (GPSTs) for the CSA examination. I

was also involved in GP teaching as a Primary Care Medical Educator (PCME) in

Stockport GPST training scheme, where I developed a course for training GPSTs in

peer assessment and giving structured feedback on videotaped GP consultations.

Having worked in Libya as an undergraduate lecturer in a medical school, I was

drawn to the issue of IMG’s general poor performance in the CSA. A review of the

research demonstrated the gap in the literature around self- study skills of GPSTs

3 Murcott’s questions are 1) How did you go about your research? 2) What overall strategy did you

adopt, and why? 3) What design and techniques did you use? 4) Why these and not others? Murcott A, `The PhD: some informal notes.’ Unpublished School of Health and Social Care, South Bank University, London 1997.

51

preparing for the CSA. Initially I considered studying self-regulated learning, but

then decided to focus on IMG’s perceptions of peer assisted learning (PAL) for the

MRCGP CSA.

Choosing qualitative methods

This research was an exploratory study and concerned with the perceptions of IMGs

in regarding peer group learning. As Britten states `qualitative methods are

particularly appropriate when researching a previously unexplored topic, or one that

is poorly understood or ill defined’70. Perceptions here are defined as “what we

understand through our own observation and thoughts” and this is best measured by

qualitative methods. Qualitative research aims to “study things in their natural

setting, attempting to make sense of, or interpret, phenomena in terms of the

meanings people bring to them”71.

In this study, the qualitative methods of semi-structured interviews and a focus group

were employed. Semi-structured interviews allow a loose structure consisting of

open ended questions used to define the area to be explored72. In comparison, a

questionnaire of closed questions would not allow such a rich picture of an

individual’s perceptions concerning PAL. The aim of the interviews was to get the

broad themes of the different aspects of peer group learning to consider, which could

then form, a questionnaire. This could go out to a greater population of IMGs in the

deanery, the basis of the next phase of a larger study.

52

The template of the semi structured interview was developed from a sample of

grounded theory interview questions from Charmaz73. This broke down different

sections of the interview into initial open-ended questions, intermediate questions

and ending questions which intentionally overlapped to permit going back over

earlier threads in the interview4. Around this I reviewed the thematic areas of the

literature from the previous chapter to add in question constructs around peer group

formation, feedback and assessment, self-assessment and the role of PAL strategies

in preparation for the CSA. The final questionnaire (see Appendix 2) was reviewed

and critiqued by other peer medical educationalists and my supervisors.

The aim of the focus group was to discuss the emergent themes from the interviews.

Focus groups allow the researcher to develop an understanding of why people feel

the way they do. In a normal individual interview the interviewee is often asked

about his or her reasons for holding a particular view, but the focus group allows

people to probe each other’s reasons for holding a certain view74. Furthermore using

two different research methods allows triangulation of the data and validation75.

Another purpose of the focus group was to assist in prioritising the most important

perceived aspects of PAL for the CSA, in order to formulate a questionnaire (another

focus group role74).

Sampling and Recruitment

Initially my plan to have a large qualitative study of at least 12 to 16 interviews with

IMGs was scaled back by my supervisors (Dr Paul Milne and Dr Anne Milston) to 4

to 6 interviews comprise a pilot study. It is difficult to recruit, interview, transcribe and

analyse all this data within the time constraints of a Masters project. However,

53

during the ethical approval stage HEE Research Governance Ethics Committee

questioned the validity of the size of the sample in terms of reaching data saturation.

When considering how many qualitative interviews is enough one review paper

suggested 12 to 14 is enough for a Masters projects and data saturation is often

reached at this number76. It was agreed to aim for 12-16 interviews to appease the

ethics committee recommendation.

From the outset there was considerable debate and controversy between the

researcher and supervisors about exploring successful versus unsuccessful IMG

candidates. One of the things highlighted to me was that qualitative research

doesn’t try to test hypotheses like quantitative research does. A mixed method

approach with a larger sample size would need to be used to test if there is any

statistical significance in the type of PAL activities that were carried out in the two

different groups. Furthermore there was concern around the ethics of interviewing

failed candidates. Patterson et al decided in their qualitative study, when looking at

risk factors for struggling GPSTs, to interview GP trainers rather than GPSTs

directly. This was due to concerns over the ethics of questioning and publishing

information relating to personal sources of difficulty77. It was decided only successful

candidates should be recruited for the study.

54

Once again the HEE Research Governance Ethics Committee questioned me

directly as to why I hadn’t considered interviewing unsuccessful candidates to

compare with successful candidates. During the recruitment process I was

approached by three IMGs who had failed the CSA and were happy to be

interviewed. This highlighted to me the fact that as researchers we may be more

concerned about the sensitivities in speaking about failure than trainees themselves.

To recruit IMGs, I had to try several different approaches. It was difficult to gain

access to IMGs in Northwest England. Firstly I approached the HENW who advised

me to ask the Royal College General Practice (RCGP) whether they would allow

access to their national database of CSA candidates. Initially I approached the

Clinical Innovation and Research Centre (CIRC) for help with this project, and they

signposted me to the RCGP examination board. The chief examiner replied:

`Unfortunately, we don’t have permission to release the personal details of

examination candidates to third parties but HENW will have as much information

about these individuals as the College… so we would suggest that you make your

approach to them’(Footnote 4). On returning back to the HENW requesting access to

their database they declared their `database had incomplete data and the status of

GPSTs is volunteered information that many IMGs do not declare’.

I e-mailed a local expert Professor Aneez Esmail who had experience in doing

research with IMGs. He wasn’t surprised to hear of my problem accessing data, due

to the huge sensitivity in releasing it because of fear of what it may show. His

suggestion was to go via the primary care medical educators (PCMEs) of the

different local GPST training schemes. He also suggested the classic 4 E-mail sent from MRCGP chief examiner Paul Foreman on 30

th December 2016.

55

epidemiological `snowballing’ approach (identify 1 or 2 people in the group and then

use them to find others)78. This seemed the best approach. Convenience sampling

was not an option to me as I did not have any personal contact with GPST IMGs79.

The next phase in recruitment was to contact all the PCMEs in the HENW with

details of the research study by e-mail available on the deanery website80. I received

a mixed reaction from PCMEs regarding the study. Some were very positive and

strongly encouraged their IMGs to take part in the study. However others were more

resistant and felt it singled out IMGs as a `special case’ from other GPSTs, which

could be seen as stigmatising.

Despite several leads and interest, I only succeeded in interviewing four IMGs who

had successfully passed the CSA. I was unable to use the snowballing approach as

previously described, as the index IMGs’ peers had failed the CSA. One of the

major difficulties was by the time the study had been ethically approved by both

ethics committees it was towards the end of the GPST3 placement.

56

Figure 4 Summary flowchart of the research protocol with timescale

57

Method of data analysis

All four interviews were done face to face rather than as telephone interviews. The

advantage of face to face interviews is that richer data can be acquired through non-

verbal communication. In particular with IMGs who may speak English as a second

language, there is value in being able to restate and clarify questions where

puzzlement or unease is noted81. The interviews were recorded using an Olympus

voice recorder and then transcribed verbatim by an online UK transcription service

GoTranscript. All the scripts were checked and re-read by the researcher to ensure

rigor in detecting errors, misspellings and data that misunderstood. This also

assisted in familiarisation with the data which is the first of a five stage process of

analysing qualitative data outlined by Pope et al (see Footnote 5)82.

A thematic analysis of the data was considered to be the best approach in data

analysis in order to devise a questionnaire. This requires a phenomenological

approach that reduce the experiences of persons (IMGs) with a shared phenomenon

(peer assisted learning) to a description of the universal essence (a “grasp of the

very nature of the thing”)83. The literature describes two main types of

phenomenological approaches: hermeneutical or transcendental, empirical

approach84. A hermeneutic phenomenology approach was used. That places

emphasis on describing experiences (a Husserl concept – see Footnote 6)

encouraging the investigator to set aside as far as possible their own experiences in

5 Pope et al 5 stages of data analysis: 1) Familiarisation of data 2) Identifying a thematic framework 3)

Indexing (annotating transcripts to thematic areas with numbers in margins of transcripts n.b. one passage of transcript may be indexed to several themes) 4) Charting (rearranging data to appropriate part of thematic areas) 5) Mapping and interpretation (map the range and nature of the phenomena and find associations between themes to provide explanations for the findings). Pope C, Ziebland S, Mays N. Analysing qualitative data. BMJ 2000. 6 Edmund Husserl (1859-1938) was a German mathematician / philosopher who established the

school of phenomenology. He sought to develop a systematic foundational science based on the so-called phenomenological reduction. For a more accessible text on his work Dan Zahavi. Husserl's Phenomenology. Stanford: Stanford University Press. 2003 is a good read.

58

order to gain fresh perspective of the phenomenon under examination. Moustakas

outlines a practical approach to this method (see Footnote 7).

MaxQDA85, qualitative data analysis software, was used to help code, index and

chart the data. The first interview script was analysed by the researcher and

supervisor, Dr Anne Milston, together and the further three analyses were checked

as a process of quality control (confirming that the assigned meaning to coded data

is agreed)86.

Strengths and weaknesses of the study

In my opinion this study highlighted an important area that hasn’t been considered in

the literature regarding IMGs, their preparation and subsequent performance in the

CSA. The IMGs were not connected to one another, coming from different locality

groups thus allowing the data to be representative of a greater range of experiences

and increasing its generalizability. Furthermore there were quality measures of

respondent validation of the data (although only one consented) and consensus of

two researchers for agreement of the meaning of the data87.

One of the biggest weaknesses in this study has been the failure to recruit sufficient

numbers of IMGs to reach data saturation. Lack of provision to access databases in

the RCGP and HENW, and the response of some PCMEs made it difficult to access

a larger number of IMGs directly. The long delay in the granting of ethical approval

also had an effect on timing of recruitment. By June 2016 GPST3s are towards the

7 Moustakes talks through different stages of analysing the data :1) initial highlighting of data for

significant statements 2) quotes understanding the overall theme 3) next these would be collapsed into meaning units or broader themes and 4) finally there would be a need to describe the overall essence of the phenomena

7.

59

end of their GPST placements, often take annual leave at this time and then move

on to new positions at the beginning of August thus the window of recruitment was

narrowed significantly.

If we had initially broadened out the recruitment criteria of the study to include IMGs

who had failed the CSA then larger numbers could have been recruited, and

differences emerging between the two different groups regarding their experiences

of peer group study could have been studied.

In terms of analysing the data, there was difficulty in interpreting some of the English

words and phrases as the majority of IMGs use English as a second language. This

led to particular difficulty when trying to assign meaning and code the data, and as a

result some of the data had to be ignored. Other difficulties experienced with the

recordings were strong accents and noise interference.

60

Ethical considerations

Although this study did not involve an active intervention on the research participants

(beyond interviews), there are still some important ethical issues to consider. The

following section will discuss how the issues of confidentiality, consent, recruitment

and management of the data were handled.

Confidentiality

All comments made concerning IMGs perceptions and experiences of peer group

study for the CSA were confidential as per the requirements of the Data Protection

Act 1998, and Freedom of Information Act 200088,89. This is important in that it

allows a fuller voicing of views than would be otherwise gained if the participants

believed that comments would be traced to them. The participants should have no

fear that any views expressed could influence their relationships with other doctors,

the RCGP or affect future career prospects.

Interviews were held in private, in a GP surgery room, so that confidentiality could be

assured. The transcripts were anonymised and the recordings destroyed once

transcription was complete. All direct quotes taken from the data were made non-

attributable to any individual. None of the participants were known personally to the

researcher.

All personal details of those taking part in the study were held securely on my

encrypted computer and these details were deleted after completion of the research

project.

61

Consent

Consent to participate in this study was voluntary and participants were explicitly

informed that they could withdraw from the study at any point without giving a

reason. The participants were fully informed by e-mail about the nature and

purposes of the study in advance of the interview (see Appendix 3 for information

sheet and consent form). The consent form was discussed and signed prior to

starting the interviews and hard copies of the collected consent forms stored in a

separate file in a locked drawer in an UCLan office, accessible only to the research

team. Furthermore UCLan STEMH Ethics Committee requested that all the GP

practice managers were informed of the interviews scheduled to take place on NHS

GP premises, and this was adhered to.

Recruitment

As previously stated, recruitment into the study was entirely voluntary and no

coercion was used. All participants in the study were given a £15 Amazon voucher

with thanks for their professional time. Apart from the small gift voucher there were

no direct potential benefits to the participants, but it is hoped the information

gathered will be useful in informing further research and feedback for future IMGs

and medical educators. The researcher had no competing outside interests and the

MSc was entirely self-funded.

62

Data Handling

After data was collected, it was transcribed by GoTranscript who abide by

confidentiality rules and anonymised the data90. Copies of the coded transcript will

be stored separate to the codes linking names to the data, in a file in a locked drawer

in UCLan office. The chief investigator is responsible for the collection, recording and

the quality of the data. All primary data collected will be securely stored for at least 5

years as per the University of Central Lancashire’s Code of Conduct for Research91.

Reflections on the process of ethical approval

I found the process of seeking ethical approval a difficult and frustrating experience.

There was significant delay in the process, having submitted an initial proposal in

mid-January 2016. I finally got ethical approval at the end of May 2016. Initially I

was advised that I would only need to gain approval from UCLan Faculty of Health

Ethics Committee, however as HEE Research Governance Committee requested

approval from UCLan directly, the Faculty of Health decided it should go to a higher

university ethics committee board. This decision came to light in mid-March 2016.

There was a further two month delay in meeting and approving the study due to

illness of UCLan STEMH Ethics Committee members.

UCLan STEMH Ethics Committee thanked me for a `clear and well set out’ study

and requested only minor conditions to be made (see Table 16 below). Due to the

delay in the ethics approval, I sought an extension to the research project which was

granted until October 2016. Despite my personal frustrations regarding the lengthy

process of research proposal adjustment before gaining agreement from both ethics

committees, it did result in a valuable personal educational experience.

63

HENW Ethics Committee Recommendations

UCLan STEMH Ethics Committee Conditions and Recommendations

Applicant Response

1. Correction of Northwest Postgraduate Medical Deanery (NWPMD) to Health Education Northwest (HENW)

1. The Data Protection Checklist has the wrong project title – please check the form and correct this and any other errors.

The data protection checklist now has the correct and amended research project title. Correction of NWPMD made to HENW on research proprosal.

2. Number of 4-6 qualitative interviews insufficient to achieve data saturation therefore recommended to extend to 12-15 interviews

2. Can you please supply the focus group interview guide for our records, prior to conducting the focus group?

A focus group guide has now been included and is electronically attached). Agreed to increase scope of interviews to 12-15 to achieve data saturation.

3. Please ensure that you have permission from managers at the practices concerned to use the various premises for interviews.

A statement has been put on the consent form that the practice manager of the practices need to be informed of the interviews and their permission sought. The researcher will speak directly with the practice manager to gain verbal consent to go ahead with the interviews in the practice.

Table 16 Recommendations and Conditions made from HEE Research Governance Ethics Committee and UCLan STEMH Ethics Committee

64

65

Chapter 4 – Data Analysis

Introduction

This chapter will analyse the qualitative data from semi-structured interviews with

four International Medical Graduates (IMGs). A thematic analysis will be used, with

a view to developing a questionnaire that could then be validated in a focus group.

The first part of the data will be considered in light of thematic areas which have

arisen from Chapter 1, including prior experience of peer group work, cultural

aspects of communication in the UK setting and linguistic issues in relation to the

CSA exam. The second part of the data analysis will consider the different aspects

of peer assisted learning using the theoretical framework from Chapter 2.

66

Exploratory Interviews

The participants were recruited from the Health Education Northwest (HENW)

Deanery and were located in practices in Runcorn, Warrington, North and South

Manchester. The demographics of the participants are shown in the table below.

None were from within the European Economic Area (EEA) and all would be

categorised as BME (non-EEA) IMGs. Pseudonyms were given to each of the

participants for anonymity and to allow the data to be read as a narrative.

Pseudonym Sex Age Country

graduated

Year

Graduated

Sky F 33 India 2005

Obie M 33 Nigeria 2007

Mulan F 33 Pakistan 2008

Lise F 35 India 2004

All participants were recruited through PCMEs who forwarded my research outline,

protocol and consent forms to the potential participants. One IMG requested to see

the transcript to verify the data, and the other three declined.

The data was coded using MaxQDA software analysis and emergent themes were

assigned. `Sky’s’ transcript was jointly coded by the index researcher and

supervisor Dr Anne Milston. The other three transcripts were also verified for

accuracy and consensus agreed with Dr Milston.

67

It is recognised that the interviewer himself is a white UKG and this could potentially

affect the openness of BME interviewees. This was evident in one interview with

Obie when discussing what the CSA is testing (see Footnote 8).

Another issue that arose was that the IMGs were speaking English as a second

language. Some of the English spoken was hard to understand and determine the

true meaning. There is the potential for misinterpretation of the data, and some of the

data couldn’t be analysed.

8Obie: I think it's just purely how good is your English language, and I think that is really, really unfair. DH: Do you think it's testing anything else? Obie: It all gets covered up with communication skills, but I do really think it's communication when-- Fine I get it. I may not be able to… probably having a moan now. DH: No. It's fine. Obie: I may not be able to explain to a white person. I'm sorry, using the word white – I’m not being personal DH: Yes. It's fine. Obie: -or sort of use an everyday type language that they are used to. For me growing up in Nigeria, my everyday English is already formed, and it would literally mean having to learn a second language to speak in that fluency that white person is already used to.

68

Thematic Areas Seen in Code Matrix Browser

Subcodes of the major themes can be seen in `Forming peer groups’ and `Peer

group activity’ in the browser above; as these were the largest major themes that

emerged. The larger boxes represent more codes assigned to that thematic area

and it can be seen that the largest box relates to Obie’s transcript where coded data

has been linked to assessment and feedback in IMG-only peer groups. These

emergent themes helped form the structure of a questionnaire (Appendix 4) that will

be validated with a focus group at a future date.

69

Evidence gained which confirmed findings in the literature relating to IMGs

preparing for the CSA

The exploratory interview data analysis replicated some of the issues already

identified in chapter one of introduction. These are outlined below:

1. Little prior undergraduate experience of consultation and communication

skills teaching

`We used to do a lot of group studying (for undergraduate exams) but that is more of a written exams than more of an interview type, because the practical exams themselves is more of a hands-on, and part of it is, obviously, interviewing but I can't remember this sort of practice, because that approach is kind of different. It's more of a clinical orientated than psychosocial’ Sky

`Initially when I was doing it I think I realized that I am the only one in the groups of probably people who are trained here they have had formal education about consultation skill.’ Mulan

DH: `Had you had any experiences before the CSA in learning communication and consultation skills?’

Obie: `No. [chuckles] No, absolutely. Again it’s all of these different things, because with the way we train in Nigeria it’s absolutely different.’

DH: `In your undergraduate education in India, was there any kind of experience of communication, consultation skills teaching or learning?’

Sky:`Not much. As I mentioned before, even the exit exams and the graduation is more focused on clinical knowledge and skills than communication.’

This data aligns with Remedios’s22 findings that IMGs have little or no prior

consultation and communication skills teaching at undergraduate level. However

most of these IMGs graduated over 10 years ago and Mulan described how her

medical school had since changed their curriculum to include communication and

consultation skills.

70

2. Cultural Factors

a) Different perception of doctors by patients

`Doctor knows everything’ `There's a concept of doctors there (Pakistan) where they're visualized as someone who knows everything, even if you're a fresh medical graduate you're considered to be a doctor then.’ Mulan Doctor seen as `demi-god’ `And also doctors, there is that hierarchical attitude as well. Your patients see you as demi-gods basically. Respect for you as a doctor.’ Obie Fear of doctors `So they come in literally trembling and shaken, I give them words of wisdom on what to do.’ Obie

Doctor-centred behaviours usual in international setting `Tell your patient what to do’ `The idea of having to share management is completely alien to us. We just make that and say to the patient, “Take this medication do that, do this and that.” Obie `I worked there for a year and there's no concept of shared management. As a doctor you only tell your patients what they need to do. You don't really ask or share that, “Why don't we do this. What do you think about that? How do you feel about it?” These are the things I had to learn, these never existed.’ Mulan `And you can tell patients, “This is what I'm giving you have to take it for five days three times a day and then if you get better it's fine, if you don't get better then come back.” And that's about it, it ends there.’ Mulan `Patient’s fault if don’t do as told’ `And if you don’t do as you’re told, or you come back the next time the problem is all your fault.’ Obie Use of medical jargon `Not being too medical in our descriptions (in UK setting) which I must say was very difficult, was completely alien to what we know.’ Obie

71

2b) Different style of doctor-patient communication

Directive interviewing versus conversational approach `That sort of exposed me to what is expected of a doctor in terms of communication, because for us it’s all about answering from scripts, we have parrot questions, we're expected to get a yes or no answer.’ Obie `For us all, communication for doctors is the way it is done here is completely alien to us. Here it's a natural free flowing discussion and chat. But for us it’s about you asking the questions and getting the answers you want from the patients.’ Obie `I used to be very abrupt; there were not lots of open-ended questions.’ Lise

`The main thing what I felt was their (UKGs) consultation was more conversational and flowing. Mine was more like asking the questions, getting the answers type of consultation.’ Lise

Different medical model of care: clinical not holistic in international setting `We have clinical, we have psychosocial, as a person, as a whole (in UK). But in undergraduation (undergraduate training), that sort of approach is quite different. What we have been through is more a clinical knowledge testing.’ Sky

The difference in communication style between IMGs and UKGs is highlighted in

Lise’s quote: `The main thing what I felt was their (UKGs) consultation was more conversational

and flowing. Mine was more like asking the questions, getting the answers type of consultation.’

72

3. Lack of exposure to UK General Practice prior to GPST 3 year

Prior to the interview, data was collected regarding prior exposure to UK General

Practice and this demonstrated a lack of exposure (see table below). Three out of

four of these IMGs came from the Indian subcontinent where the discipline of

general practice is relatively poorly developed.

Prior exposure to UK General Practice prior to GPST3 year

Sky 7 months

Obie 6 months

Mulan 6 months

Lise 4 months

4. English language and linguistic skills

The data below highlights difficulties with English language and linguistic skills

around fluency, developing a conversational style using local language and phrases

and giving patients explanations in everyday (UK) English language.

Fluency and conversational style `The main thing what I felt was their (UKGs) consultation was more conversational and flowing. Mine was more like asking the questions, getting the answers type of consultation.’ Lise `I think it's mainly because of the language and they can make a very friendly conversation with the patient rather than making it as more medical. That's the main weakness I found in my consultation. I keep telling them (UKGs) and they didn’t feel that but I felt it myself.’ Lise

Learning the way we talk in the UK `As a group, we thought the communication side of things, and learning that communication is more about what we see in colleagues that are UK based and trained. The way they sort of speak, the way they explain things to the patients, the way they get a patient to talk using everyday language.’ Obie `And they (UK graduates in peer group) were very kind enough to tell me you have said this thing differently, you could have done better or you could have done differently. Or in this country we say it this way. I think these are the kind of things helped me a lot.’ Mulan

73

`Yes. I think it was mostly surrounded around the consultation skills, how to give patient, how to respond to cues and how to say certain things.’ Mulan

Learning local phrases `I just thought, "Well, this is great," (working with a UK graduate) because that's who I thought it would be useful for me to catch up some local phrases.’ Sky `She was an actor and she's from here as well. She used to give lots of good feedback for us, like where I can improve, what sort of phrases I need to use’ Lise

Speaking English as a foreign language not mother tongue `There's lots of things we don't know how to say, we just say it because we've learned English as a language, as a foreign language. It's not something that comes naturally to an international medical graduate because even if we have studied everything in English the main language that I've always used is Urdu.’ Mulan `For me growing up in Nigeria, my everyday English is already formed, and it would literally mean having to learn a second language to speak in that fluency that white person is already used to….sort of use an everyday type language that they are used to.’ Obie

Difficulties with giving explanations `My explanation is … I don't say always, but in most cases it's always very medical. In attempt to try and use everyday language, it all stretches out and I ramble on and on and on.’ Obie

`But when it came to explanation, when it came to explanation in simple words I was struggling.’ Mulan `DH: I think you've probably answered this question already, but did your group learning highlight any areas you needed to work on? Can you give any examples?

Obie: Yes. I think I've shared my side of things, kind of explaining conditions which was a struggle with exams.’

74

5. Do IMGs have a tendency to form their own IMG-only groups? Jamieson et al claimed that IMGs have a tendency to form IMG-only groups but this

data shows a mixed picture26. In the first attempt in sitting the CSA three out of four

of these eventual successful IMGs belonged to mixed groups. However those who

resat the CSA examination may have a greater tendency to form IMG-only groups

(only one out of five groups were a mixed group).

IMG 1st time of sitting CSA (Mixed group/ IMG only group)

2nd time of sitting CSA (Mixed group/ IMG only group)

Sky 1 mixed/ 0 IMG only 0 mixed / 1 IMG only

Mulan 1 mixed / 0 IMG only Already passed

Obie 0 mixed / 1 IMG only Already passed

Lise 1 mixed / 0 IMG only 1 mixed / 3 IMG only groups

6. Do IMGs tend to focus on doctor-centred behaviours rather than patient-

centred behaviours?

Jamieson claimed that IMGs have a tendency to focus on doctor-centred

behaviours6. However this data shows that IMGs have good insight into doctor-

centred behaviours and are aware of their weaknesses (particularly in terms of

eliciting patient’s ideas, concerns and expectations (ICE) and being patient-centred

in clinical management) and understand the need to develop these skills. Obie, who

was part of an IMG-only group, highlights that the peer group feedback did focus on

eliciting patient’s ICE and matching this with patient-centred management decisions.

75

Emphasising doctor-centred behaviours in IMG only groups `I think it's the same thing. I spend a lot of time in data gathering and too doctor-centred in the management.’ Lise `Because the first time I didn't know about the exam(part of mixed peer group)….I didn't ask ICE (patient’s ideas, concerns and expectations), I thought it comes in everything in the conversation but the second time I used to concentrate more on the patient-centred management which I wasn't doing in the first time.’ Lise

`Again she (IMG colleague) was very helpful because it was her fourth (fourth attempt at CSA). She knew like I have to get the ICE and whenever I don't get the ICE she used to tell me like, "That's more important, try to get the ICE as soon as possible." Lise

`We break it down into the explanation, then the managements of treatments, the condition. And again, when we give feedback we - point of it the explanation wasn’t or was too medical, or wasn't used in an everyday language. Also if the patient expectation from the ICE (patient’s ideas, concerns and expectations) from the earlier data gathering not all met and if the patient's was matched when we're sharing the management.’ Obie

`Things like you've not been too doctor led and carrying on with the explanation on the management. Non-verbal clues as well, we picked up on that, and we put that in our management discussions. And getting the patient involved in the management as much as possible. That's the sort of things we feedback on.’ Obie

76

New Emergent Themes

Group Size, Use of Multiple Groups and Frequency of Meeting

There was a spread of peer group size described in the interviews. All group sizes

ranged from two to six group members. Sky suggested three seems to be a good

minimum number to allow an external assessor to give feedback on the doctor-

patient role plays.

The frequency of meeting also varied from every day to once a week, and varied in

time period between two to ten weeks prior to sitting the exam.

One of the surprises from the data was the use of multiple peer groups. This was

partly due to failed examinations but also the motivation of individuals to have as

much input from peer groups as possible.

Size of group Big to small `It was as simple as that in the beginning, but then lots of people decided they don't want to do it, or they want to do it a bit late, they're not prepared. We started at 10, 12 people initially and we came down to about, say, five.’ Mulan 3 the ideal number? `If it gets more numbers and probably it would be a bit more messy but three people probably is a good enough group.’ Sky

`One thing which is lacking in the first group was we’re only two people. There’s nobody else to hear what we’re talking about. We always thought it might have been better with one more person in that group. Whereas the second group, I think it worked out a bit more better because you got a person who were acting as a patient, who is an assessor, who listens from external. That was good. If you were to ask me, just at least three people in a group would work much better.’ Sky

77

Use of multiple peer groups (usually as failed CSA 1st time) `I think I would say that, you should join a peer group. You might end up with a different peer group in the end but keep practising.’ Mulan `Actually, I was a part of two groups, which is quite different’ Sky

`So he (GP teaching tutor) formed a peer group, like he divided us into few groups and then I had -- the second time I have formed a peer group and there was one of this girl who I came to know that she moved to Stockport program….. And again, I think I had another girl with whom I used to practice in between us and she is from Pakistan. And I think she had failed exam first time so I practiced with her.’ Lise

Frequency of meeting Every day for two weeks prior to exam

`I met the older lady from that and we came together, I think, a couple of weeks

before the exam. And did some really intensive, nearly every day, full-on revisions towards exam.’ Obie Every weekend for two months, increasing in frequency nearer exam `One day of the weekend either Saturday or Sunday. We’ll meet up on one of our houses and then just practice for a few hours, yes, every weekend.’ Mulan `Twice a week at least, and by the end of-- for the last few weeks we met probably three times a week’ Mulan Over ten weeks prior to exam `That was since November 2015, but we had a bit of difficulty meeting, initially, because she lives a bit far. She has to travel like 22 miles or something to get here, which didn't mind, but just getting that right time for us but we did practice over about 10 weeks, up until January time.’ Sky

Forming Peer Groups

One of the major themes emerging from the data was the perceived advantage of

having a UK graduate in the peer group preparing for the CSA. This data has been

summarised in diagrammatical form from the data below. Obie particularly talked

about the barriers he experienced in trying to form a mixed group with other UKGs

which have also been displayed in diagrammatical format. Lise described difficulty

getting into a peer group initially and was allocated a peer group by her GPST tutor,

as part of a wider process of group allocation. The idea of self-forming groups

versus allocation of groups will be debated in the discussion.

78

Perceived advantage of having a UKG in group Role Modelling

`It's been drummed into us from day one that it's absolutely vital that you have a

peer group, but make sure it's a mix of internationals, add like British born doctor, because he would come into be almost like a role model you need to aspire to.’ Obie English Language

`As a group, we thought the communication side of things, and learning that

communication is more about what we see in colleagues that are UK based and trained. The way they sort of speak, the way they explain things to the patients, the way they get a patient to talk using every day languages.’ Obie ` how to say certain things.’ Mulan Specific Consultation Skills `They would say that you probably wouldn't realize but the person sitting next to you might be thinking that's slightly rude or you could say it differently in a way the patient might feel that you are taking an active interest. Picking up verbal and non-verbal cues, because there are lots of things that we don't know about the culture.’ Mulan `DH: Identifying your learning needs from the peer group? Did that help?

Mulan: Yes. I think it was mostly surrounded around the consultation skills, how to give patient, how to respond to cues and how to say certain things. How to share management. How to reflect back if patient has got any ideas, concerns and expectations. And how to reflect it back to the patient and then explain your opinion.’ Perceived quality of feedback Mulan: I think first of all I would say that don’t make a group of just international medical graduates, have a good mix. I wouldn’t say all of the people should do like just one of me was IMG and the rest but have a good mix.

DH: Why?

Mulan: Because of the feedback that you will get might be completely different. It’s very important from what reflection are you going to get from other people Acquiring assessment skills ` Though they (IMG only group) couldn't kind of get to know the three domains by that time even after their own. With my knowledge from my first group (mixed group), that was my transferable knowledge or skill, whatever say. I managed to incorporate into that group’. Sky

79

Barriers to forming groups with UK Graduates (UKGs) Personality type and fear of exposing weakness

`One thing that could help, but speaking from my own experience I think probably I

say this early on is the reluctance on our part as international medical graduates to want -- unless you are the outgoing very extroverted type, there is that reluctance in most international medical graduates to sort of not interact or interact very well. Sort of go out, sort of expose our weaknesses, so makes them vulnerable.’ Obie `Subconscious divide’ `Not meant to do that, but I think there’s just that bit of divide subconsciously, maybe I’m over thinking things. I just thought that might just be the reason.’ Obie Feelings of isolation `So, I was pretty much left alone, and the other lady I suspect the same thing happened to them as well.’ Obie `Feeling like an outsider’ `Unfortunately, the problem I had was, I come from Rome to come to Chester and most of the other people from the group are all from the Chester area. So they literally all formed their own small little group for their exams.’ Obie Pre-formed UKG groups `I think, if I'm being absolutely honest here, it's the fact that, in a half group, there's almost like a quick subgroup. A group within a group sort of formed from day one.’ Obie Lack of access to UKG groups ` But also, the other issue I think, that sort of falls back to, brought us together was the fact that we couldn’t really get involved with the other groups.’ Obie Pull factors to form with IMG only groups Feeling deficient compared to UKGs `Yes, that’s how we -- and then during the courses for the CSA exam we saw how deficient we really were compared to the UK based graduates, and that sort of put on additional pressure for us to just come together and practice as much as we can. [laughs] Basically press ahead of -- basically.’ Obie Like attracts like `Because if it's all left to ourselves we'll end up going with our dates and my friends and who we feel comfortable with. Which again sort of not in resistance ways, it's just what we're comfortable with.

I would go with somebody that I grew up with, similar culture, similar experiences that sort of thing.’ Obie Get on better with IMGs `So, we tend to get along better with the internationals in the graduate field better with them. We could say things how we want to say it. So, it was just all natural to have a group with people you feel more relaxed and more natural with.’ Obie

80

Not meant to do that, but I think there’s just that bit of divide subconsciously, maybe I’m over thinking things. I just thought that might just be the reason.

Obligation (previous acquitance) / altruistic `a help’

` The second one, the second group, as I said, it's more of a-- I wouldn't say it's an

obligation or anything, but it is a help, probably, with the other colleague who I known from before.’ Sky Live local to each other

` She's quite local, yes. She's quite local, quite approachable to me. She lives literally two streets away.’ Sky Terror of exams ` DH: How did you make that decision to form together?

Obie: More from terror of the exams than anything else.’

Self-forming versus allocation `During the first time (of sitting first CSA exam), there was a peer group formed by our trainer A (GP teaching tutor). So he formed a peer group, like he divided us into few groups’ Lise `He (GP teaching tutor) allocated and he divided among -- divided into peer groups but I know I was finding it difficult to find a proper group. So finally, it was two girls who -- who was happy to include me in the group.

DH: Can you tell me a little bit more about that, why was it difficult?

Lise: `…. because I was part-time, in my first year of GPST3 I did that, but I took the exam in the second half. So I wasn’t regularly attending the teaching. So my name was missed initially.’

81

Perceived advantage of having a UKG in peer group for IMGs

82

83

Group dynamics

When discussing important factors in successful peer group learning the importance

of peer group dynamics was highlighted by several of the IMGs. Sky’s quote

regarding the group being `practically more approachable’ is an example of

struggling to understanding meaning from the data.

Sky described an event in her IMG only group where a new dominant member was

invited in on one occasion and thus upset the group dynamics. Theory on `open’

versus `closed’ groups to new members describes the potential effect on group

dynamics and ultimately group performance7.

IMGs describing IMG-only groups expressed a range of emotions relating to the

group dynamics from supportive to competitive. Sky used the word `kudos’ and this

gave the impression of an element of power play happening in one of the IMG-only

groups. Over-dominance of a group member in IMG only groups was also described

by Sky and Obie.

In contrast IMGs describing mixed groups described a positive, encouraging group

dynamic that was a supportive environment.

84

Importance `Peer group just choosing the right group which is practically more approachable and you will have the sense in the first one or two meetings whether it's right or wrong.’ Sky Lise: `I think, make sure all the members are compatible and—

DH: What do you mean by that?

Lise: So they get along well.’

Better to have closed than open group: negative impact of one off new group member on group dynamics `Another colleague of my colleague happened to join us. I was quite disturbing to the group activity because that's not routine. Every person is different so how they want to take over the conversation can be difficult to understand and get along with. Probably that's the only situation otherwise it used to be quite nice and gentle. I think you just sticking to a group that you can work with, works out well.’ Sky

IMG only groups – mixed response regarding group dynamics Supportive `Like if one of my colleagues struggling with say, women’s health, we thought we should give her extra support by giving her more.’ Sky `As I mentioned like the IMG and I do because she had done her exams before, she gave me lots of good advice and support.’ Lise

Intense and stressful `I think just the approach is a little bit different (in IMG only group). As I said, it’s quite more of an ambitious feedback just saying, you got to do this, kudos. It can be a bit more like what’s the correct word, I have to say, I don’t know. I’m not getting any words now. [laughs]. It’s quite intense….it is quite intense and you just sometimes, at times it just felt like, I don’t know. It’s stressing me out, that sort of feeling.’ Sky

Overdominant member of group `It wasn't a big mental problem at all; it's just that sometimes one of us could sort of get carried away and take over. The lady that's done high level preparation before she almost took over the educational side of the group.

`At the end we thought that she do very well as a GP trainer because she is quite good. Her knowledge is -- a person who did better than me and the other lady. Was also that she just gets carried away and goes on and on.’ Obie `Every person is different so how they want to take over the conversation can be difficult to understand and get along with.’ Sky

85

Power play / `kudos’ `“No, you have to learn this. You need to know this. You need to revise this. That’s where you didn’t do very well or this is where you need to do some more studies.” I think just the approach is a little bit different. As I said, it’s quite more of an ambitious feedback just saying, you got to do this, kudos. It can be a bit more like what’s the correct word, I have to say, I don’t know. I’m not getting any words now. [laughs]. It’s quite intense.’ Sky

Mixed group of UKGs with IMG Positive and encouraging `That’s sort of a different, that’s one of the difference between the first group (UKG + IMG) and the second group (IMG only group). I’ve noticed the first group is, and I like the way to be honest, the feedback is more talking about the positive things first. Just try and encourage what you’re doing good and it’s not pointing at your limitations rather than just saying what I think this is where you can make a little improvement.’ Sky `And they (UKGs) were very kind enough to tell me you have said this thing in a differently, you could have done better or you could have done differently. Or in this country we say it this way. I think these are the kind of things helped me a lot more than probably I would have helped them. I would say it helped me a lot.’ Mulan `It was the first time and they (mixed group of 2 UKGs and 1 other IMG) encouraged me and tell me `continue with what you're doing’, `I don't think you'll fail’ those sort of things.’ Lise IMG in mixed group overcoming negative emotions

`They (UKGs) just kept telling me, you are doing so well that was always the first

line. It’s just this you could have done a lot better. Eventually I realized that it was not just me we’re making tiny mistakes. We are all missing things and we are all at the end in the same boat doing the same exam. Initially when I was doing it I think I realized that I am the only one in the groups of probably people who are trained here they have had formal education about consultation skill. There was a bit of a setback for me but within weeks I think I just got over it and I thought you know what I have to give it a go. Let’s see what happens.’ Mulan ``I was actually a bit worried when I started preparing with them. I thought you know what they are all going to pass I'll be the one who will fail.’ Mulan

86

Planning learning activities and learning strategies employed

This section of data analysis will consider aspects of peer group processes that

affect group effectiveness, as outlined by Topping and Ehly’s model27 (described in

Chapter 2). Consideration is given to how the different types of groups (IMG-only

groups versus mixed groups) planned their learning, demonstrated organisational

skills and selected learning strategies to employ.

The majority of sets of groups tended to use textbooks and follow a clinical systems

approach to planning their learning. Both groups demonstrated good organisational

skills in terms of setting the agenda and allocating of tasks to group members.

Some IMG-only groups had little or `random’ planning of their learning. Sky reported

that her IMG-only group had an overemphasis on clinical knowledge which she felt

was the wrong approach. One of her quotes recounts a group discussion on a

patient presenting with atrial fibrillation (AF), and details how the focus of the

discussion was on clinical management of AF, rather than the skill dealing with

uncertainty in the consultation.

One of the surprising findings that emerged from the interviews was that several

IMGs hired an actor, experienced in medical simulation scenarios, to practice with.

Feedback from such actors was positively perceived, particularly in relation to use of

language and clarity of explanations.

Sky’s mixed group used a creative method of devising role plays inspired by cases

taken from everyday practice. There was evidence of experiential, reflective

87

learning in the mixed groups. They considered significant events and challenging

and interesting patient encounters in surgery; however there were no such

descriptions in the IMG only groups.

Planning learning Both groups focused on clinical systems approach and use of role plays from textbooks `That's how we formulated the sessions. It was mainly cardiology once, respiratory other, examination sometimes, so systematically went through everything here.’ Mulan (Mixed group) `We never watched videos together so it was mainly role play that we did’ Mulan (mixed group) ` We made a good practice system, practiced different things, one being the doctor and we role-played patients, and the others will observe and give feedbacks at the end of it’ Obie (IMG only) `DH: Let's talk about the first group (mixed group) first of all. How did you about planning what you are going to study?

Lise: There was a book GP, is it MRCP CSA cases book? We used to do cases from there and then we used to read the guidelines ….

DH: Again, your second group (IMGs only), was it the same?

Lise: It's the same. Second group yes it's the same. Like we just used to do cases.’ Random or no planning of learning in IMG only groups `I would say it’s more of random like-- random cases because most of them, the two other colleagues of mine who's already done the exam before, because that was before my first attempt, and so they know how to approach it.’ Sky (IMG only group)

`DH: Would you divide up things (in IMG only group) how you did things or would you plan before you met?

Lise: No.

DH: No? So no planning beforehand?

Lise: No.’

88

Both groups showed organisational skills Setting agenda for next session, assignment of tasks and pre-reading ` And then at the end of the session we planned what to do, the other systems we have to go through next time.’ Obie (IMG only group) `Someone is going to do something completely different from what I was preparing and then because I had read through it already, I can probably judge them better on that case. That's how we formulated the sessions’ Mulan (Mixed group)

Textbook cases as only source for role play cases in IMG only groups

`We went off with, my colleague and she actually book with loads and loads of cases

to practice, and so we'd just sort of go through different systems, practiced different cases. ‘ Obie (IMG only group) `Lise: It's the same. Second group (IMG only group) yes it's the same. Like we just used to do cases….. It was mainly we used to do just cases.

DH: Okay. Apart from the text book did you use any other ways of preparing or planning for what you did?

Lise: No.’ However contrast mixed group below: `Lise: There was a book GP, is it MRCP CSA cases book? We used to do cases from there and then we used to read the guidelines there or someone else would read the guidelines and let us know later…from the books.

DH: Would you have any other strategies or other things that you used to do to prepare or plan?

Lise: No.’ (mixed group) Greater variety of sources for role play cases in mixed group For example self-written cases and cases from clinical encounters `We had some cases that have been collected over the years from previous CSA candidates, or some of them are like self-return from our clinical experience, like challenging cases with hidden agendas, whatever.’ Sky

`We've taken sample sort of cases, we’ve written our own sometimes from our clinical experience or something interesting case that we've seen.’ Sky

89

Greater focus on clinical knowledge in IMG only groups `The difference I’ve noticed from the first group, which is obviously with the local graduate with the second group being with a couple of other IMGs, is that they’re more focused on clinical knowledge even with the second time (of sitting the CSA).’ Sky

`More focused on the clinical. To be honest, to some extent, I don’t know whether I can say this but with my previous experience with the other colleague (UKG), I try to persuade them to some extent this is not the right approach.’ Sky

DH: Why do you say it wasn’t the right approach (in learning strategy)?

Sky: Because it was more of a clinical discussion related to every case. If you say Atrial Fibrillation, “Oh, you should. You know, you should tell this patient you need to have beta-blockers or you know, you have to CHADVASs score them you need to kind of beta-blocker them what if you don’t know about NOACs or whatever. But I’m sure there are some other ways on how you can deal with uncertainty. You could communicate to the patient. Just say, “Well, you know I’m not pretty sure.” There are some other ways of how you can actually safety net or make the discussion a bit more appropriate by checking a colleague, referring to a colleague or whatever or reading upon it and bring them back tomorrow, ring them whatever rather than stressing on the clinical because it’s like a lifelong learning rather than just snapshots.’

Contrast this self-reflection of Lise: `I think theory wise…theory and guidelines wise like I was good because of the exams and the MRCP. I didn't have to spend time on reading things. The main concentration for me was the communication skills so how we discuss and how we speak to the patient and how I can improve on those skills rather than reading.’ Lise

Learning strategies used in IMG only groups not in mixed groups Use of `case cards’ to practice explanations in simple everyday language in IMG only group `There is this case cards, these college case cards that we sort of went through to help us learn how to explain things in simple language, which was a bit funny.’ Obie Paying an actor to be a simulated patient in IMG only groups

` The other thing we did, nearer time to the exam was we got one of the actors, we

paid for an actor to sort of role-play the patient, and also give us feedback mainly more on the communication side of things.’ Obie `The other thing there was an actress in Liverpool and this was in my second attempt. I used to go with IMG girl from Nigeria and there were two other IMGs who used to join us…. She was an actor and she's from here as well. She used to give lots of good feedback for us, like where I can improve, what sort of phrases I need to use and she used to do a mock session as well, like 13 patients. She used to role play as 13 patients.’ Lise

90

Greater emphasis on reflective learning Experiential learning from clinical practice used in some mixed groups `If someone has seen an interesting case, if there has been any significant event that we could discuss, or if someone has got any information about recent cases or what kind of experience anyone else had, so we did, yes. We went through these kind of things.’ Mulan `First of all, we (mixed group) did had like a debrief on what have you done in the last week or so, and interesting cases that we have seen, so we learn from our experience.’ Sky

` Any particular challenging cases, we (mixed group) used to discuss around the

issues and how it can be dealt different, having communicated different and any ethical dilemmas involved in that.’ Sky

Difference in peer group feedback style in different peer groups

Another aspect of peer group process identified by Topping & Ehly is peer feedback

and reinforcement18. A major difference in feedback style between IMG only groups

and mixed groups is that Pendleton’s rules of feedback (see Footnote 9) are

predominantly used in the mixed peer groups92. This tended to make the feedback

more formative and less summative. IMG-only groups tended not to follow

structured feedback processes and were largely more informal and summative in

their feedback. Sky felt the feedback was `ambitious’ in her IMG only group which

seemed to imply it was peer directed and critical. However Lise pointed out that

when working with her IMG colleague (who was on her fourth attempt of sitting the

CSA) in her second group, the feedback was more specific and less falsely

reassuring than when she was working with her mixed group. In contrast Mulan

perceived it as `kindness’ when the UKGs in her group were honest with her and told

her where she could improve. These mixed perceptions of peer feedback and lack

of cohesive pattern from the data will be discussed in the next chapter. 9 Pendleton’s rules of feedback follow the pattern that the index learner should start by sharing

positive reflections on their performance followed by further positive observations made by the group. Then the learner will reflect on areas that could have been improved and possible suggestions to make that improvement. Observers also may add some areas to improve on with the offer of possible alternative approaches.

91

Feedback Style Predominance of Pendleton’s rules used in mixed peer groups

`That’s sort of a different, that’s one of the differences between the first group (mixed

group) and the second group (IMG only group). I’ve noticed the first group is, and I like the way to be honest, the feedback is more talking about the positive things first. Just try and encourage what you’re doing good and it’s not pointing at your limitations rather than just saying what I think this is where you can make a little improvement. It’s more of an encouraging feedback system.’ Sky `And we have the CSA mark sheet kind of paper and then we'll start marking the person according to those-- keep writing the remarks and then at the end we’ll explain what went well, what could have been improved, what was missed’ Mulan (mixed group) `And they (UKGs) were very kind enough to tell me you have set this thing in a differently, you could have done better or you could have done differently. Or in this country we say it this way. I think these are the kind of things helped me a lot more than probably I would have helped them. I would say it helped me a lot.’ Mulan `We initially started with positive feedback and then the negative feedback. We start with doctor to tell us what they think how they did, what was the positive aspects and what were the negative aspects and then the observer would give some feedback again, the positive and the negative aspects.’ Lise (mixed group) `Lise: Then normally we use Pendleton model of feedback.

DH: Is that what you did?

Lise: Yes. ….and I think that was encouraged by our training program.’

More likely to be summative than formative in IMG only peer groups and vice versa in mixed peer groups `Feedback to each of them (IMGs). When we use the cases from GP website or the books, they used to have a feedback indicator at the back of the case. We used to have a little look at them in three domains and see what we have done good and where and what we could incorporate into that. We used to give like points to them. It’s like three and three and three. How many points would you get? If that’s the case, would you pass this station? That’s basically (it).’ Sky `So what we did was, we looked at the three domains. That is backed up in the exam, the history taking, examination, shared management, and also the personal skill side of things. We sort of worked on those three main domains differently.’ Obie

More likely to be formative than summative in mixed groups `And we have the CSA mark sheet kind of paper and then we'll start marking the person according to those-- keep writing the remarks and then at the end we’ll explain what went well, what could have been improved, what was missed’ Mulan

92

`We used to read the scenario before and then the other person was marking us, marking the doctor. It was not a proper marking as such but they would give us some feedback and then normally we use Pendleton model of feedback.’ Lise

Tendency to be more informal in peer feedback style in IMG only group `I think it was more of an informal type (of feedback), or we didn't do this or we did that, but not knowing what to compare it with is just—‘ Obie `I don't know. It's just like we -- it wasn't a group like, it was just the two of us and …. she would let me know where I can improve... There was no particular model or formal feedback than I did with.’ Lise

Peer feedback could become more intense and evidence of power play in one IMG group

`Sky: With the second group (IMG only group), I think it’s more like a ambitious feedback.

DH: What do you mean by `ambitious’?

Sky: Ambitious, it’s like, “No, you have to learn this. You need to know this. You need to revise this. That’s where you didn’t do very well or this is where you need to do some more studies.” I think just the approach is a little bit different. As I said, it’s quite more of an ambitious feedback just saying, you got to do this, kudos. It can be a bit more like what’s the correct word, I have to say, I don’t know. I’m not getting any words now. [laughs]. It’s quite intense. It is quite intense and you just sometimes, at times it just felt like, I don’t know. It’s stressing me out, that sort of feeling.’

Contrast feedback more specific in IMG only groups `The second time when I took the exam and I because I practice with the other girl who had failed (IMG), she was able to pinpoint and things where I can improve and where I need to concentrate as an IMG.’ Lise

93

Peer Assessment Content and Feedback Styles in Different Peer Groups

94

Peer group study as motivation for learning

A positive aspect of peer group study was its motivating effect on learning as

described below. This was also indicated by the frequency, length of meeting and

multiple peer groups used by IMGs. The only demotivating factor outlined was the

perceived criticism given in feedback.

Positive Motivator ` -and my level of motivation up until the time we got together and formed a group wasn't anything compared to the effort they've put in. So they were strong motivating factor for me.‘ Obie `We just wanted to do as much as we could do to pass.’ Obie `Keep practising because you will end up learning something. It's not going to be harmful in any way. You’re not going to lose anything.’ Mulan `I don’t know about the answer to this whether right or wrong but I think this is a very good way of practicing it. I wouldn’t say that practicing on your own can ever match practicing in a peer group.’ Mulan Contrast with criticism demotivates `I think that the feedback should be honest. Again the criticism also should be more positive. They should have some positive aspects as well in the criticism. If it's all negative then again like it demotivates you, so there should be lots of positive aspects in the criticism. Everyone should be friendly with you.’ Lise

Poor self-esteem and self-monitoring

A major issue that arose from the interviews was low self-confidence and poor self-

esteem. This is a large hurdle for IMGs to overcome in reaching self-actualisation

(as described by Topping and Ehly in their theoretical model27). The IMGs did show

good insight into benchmarking compared to their peers as evidenced in their

comments. However, without external assessment and knowledge of their CSA

performance, it is difficult to verify these claims.

95

Low self-confidence and self-esteem ` What we wanted to do was just get enough to pass, just across that line, because we just felt we are never that good enough,’ Obie DH: `When you're sort of self calibrating, how did you assess where you were towards the exam? How were you be able to do that? Was that possible in the peer group? Was it where you wanted to be?’

Mulan: `Initially it took me a little while to gain my confidence because history taking is more or less-we all have the same clinical knowledge-base. But when it came to explanation, when it came to explanation in simple words I was struggling. When it came to shared management I was struggling and both of these weaknesses have been highlighted by my supervisor as well and when I did the mock exam. I had to work really hard on these two areas, yeah. Initially I was like should I be preparing with them? I'm losing my confidence but in the end it was all worth it I think.’

Obie: `It was, but we just didn't feel we would ever get to that level.’

Self-assessment and Self-monitoring `To be honest I didn't do any self-assessment generally during the course of preparation. Without knowing the main thing which I thought was like-- And I see how far I can go, I can improve my consultation to reach them. But to be honest I didn't do any proper self-assessment or anything myself.’ Lise `Eventually I realized that it was not just me we’re making tiny mistakes. We are all missing things and we are all at the end in the same boat doing the same exam.’ Mulan `I thought my communication skills was better, than my colleague (fellow IMGs) -- I think my use of English, understanding of English language was better than theirs. I think that's one area I did have a bit of an edge compared to the rest of my colleague. But in other areas, management and knowledge side of things, we were just almost on the same level.’ Obie

`Obviously my first group, my colleague was quite good. She worked as a Mersey AIT Rep. She’s got quite good communication skills. I would say that I’m probably not as good communicator as she is compared to herself. I know that I’m good enough candidate to pass the CSA exam…..The second group I would say, I probably I’m the good one [laughs].’ Sky

`Initially it took me a little while to gain my confidence because history taking is more or less-we all have the same clinical knowledge-base. But when it came to explanation, when it came to explanation in simple words I was struggling.’ Mulan `I think it's mainly because of the language and they can make a very friendly conversation with the patient rather than making it as more medical. That's the main weakness I found in my consultation. I keep telling them and they didn’t feel that but I felt it myself.’ Lise

96

`This used to feel that I'm below the-- knowledge-wise I used to feel I'm good but in the communication skills and other things, I used to feel not very, below, but less than my other members of the group‘ Lise

The universal essence: `It’s different from where I’m from’

In addressing the research question it is helpful to distil the common emergent

theme from the data. Van Manen referred to this as the `universal essence’83. The

significant statement or phrase that repeatedly kept emerged from the data was ``it’s

different’ or `alien’ from where I come from’. This recurring phrase was identified in

the data and then subcodes were developed from that theme. The areas of

difference highlighted were English language, education system (both teaching of

communication and consultation skills, and the assessments themselves), status of

doctors, doctor-patient interaction and model of healthcare. This has been

summarised in diagrammatical format below. In recognising these differences, IMGs

sense their need to learn new approaches and skills (particularly in peer feedback)

in peer-assisted learning for the CSA.

97

98

Where does peer group learning fit into the context of other learning strategies

for the CSA?

`I don't know if I can say it. I think CSA is not all about the peer group learning. The first thing

my trainer, was a very good resource. She was really helpful for me to get to the CSA level by

video recordings and feeding back to me. The second thing is, the time you spend in your

own surgery, because most of the international medical graduates have, I don't know whether

I can say this, but do have families like myself, and because I've got so many of my

colleagues who are in the same situation, who can only spend little time out of their work. I

use most of my time in the practice as my CSA preparation.’ Sky

Learning Strategy

Hiring actor Role play and mock CSA (small peer group)

GP trainer activities

Joint surgeries Review of videoed consultations Roleplay

GP practice Practice with other GPs Patient Satisfaction Questionnaire (feedback from patients) Reflection on patient encounters

GPST teaching Practice in small groups Mock CSA

Training courses RCGP training course Deanery wide training course

E-learning E-learning resources and websites (www.bradfordvts.co.uk and www.pennine-gp-training.co.uk)

Mulan noted: `in that peer group (GPST teaching peer group) there were people who never really

practice it within the group but they still managed to do quite well. They did pass their exam.’ The

question is raised as to whether peer groups are essential for passing the CSA

examination?

99

Peer group learning is not the preferred learning method of all candidates, as

highlighted by Obie:

`I'll never do it (peer group study) again to be honest, absolutely never do that again. CSA is

over and done with, peer group, no because I don't like it-- In a sense, I think it's still the fact

that it’s not something I'm used to. Preparation for the CSA well because it was CSA, I

wouldn't necessarily go into a peer group learning type thing just for the sake of it or just for

the fun. It is not fun for me.’

However, in contrast Mulan also said: `I would say that practicing on your own can’t ever

match practicing in a peer group.’”

100

101

Chapter 5: Discussion

Introduction

This exploratory study highlights the challenges IMGs face in adapting from their

own culture and educational background, to the mode of study, and focus on patient-

centred communication in the CSA exam. This discussion will highlight what this

study confirms in the literature regarding factors that influence IMG’s performance in

the CSA. Secondly it will discuss new emergent themes from the data in terms of

how peer groups form, function and perform. Differences noted between IMG-only

and mixed peer groups in aspects of peer assisted learning will be discussed.

Discussion

When answering the research question, the repeated significant statement that

emerged from the data was ``it’s different or `alien’ from where I come from’. These

IMGs were used to theory based clinical examinations, with no emphasis on

communication and consultation skills. Peer group learning with reflective feedback

is a new style of learning that contrasts with their previous learning experiences.

Cultural differences in the doctor-patient relationship were also highlighted. For

example, the concept of doctors and patients coming together as equals is alien to

IMGs. The different emphasis on patient-centred care from their own cultural

context, correlates to new consultation skills that need to be acquired, as seen in the

following quote: `I worked there (Pakistan) for a year and there’s no concept of shared

management. As a doctor you only tell your patients what they need to do. You don’t really ask or

share that, “Why don’t we do this? What do you think about that? How do you feel about it? These

are things I had to learn, these never existed.’ Mulan.

102

What does this research confirm around IMG factors affecting performance in

the CSA?

Lack of prior communication and consultation skills teaching

This study does align with Remedios’ work in the lack of communication and

consultation skills teaching and training these BME IMGs have as undergraduates7.

This appears to set them off on the back foot compared to their UK trained peers.

`Initially when I was doing it (peer group study) I think I realized that I am the only one in the groups of

people who are trained here, they have had formal education about consultation skill.’ Mulan

Cultural difference in doctor-patient relationship

Another emergent theme was the difference in the cultural aspects of the doctor-

patient relationship. The reference to doctor as `demi-god’ by Obie, agrees with

findings by Pilotto that IMGs come from cultures where the doctor operates from a

position of considerable power in the community17. The idea of a patient questioning

a doctor is quite alien to many IMGs because, in their home countries, the patient’s

role is one of compliance, trust and cooperation as highlighted in the quote: `if you

don’t do as you’re told, or you come back the next time the problem is all your fault’. There is a

requirement for IMGs to make a cultural shift towards forming more equitable doctor-

patient relationships.

The culturally determined doctor-patient power dynamic corresponds to the variation

in styles of doctor-patient communication. This difference between IMGs and UKGs

is highlighted by Lise: `The main thing what I felt was their (UKGs) consultation was more

conversational and flowing. Mine was more like asking the questions, getting the answers type of

consultation.’ This interview style of communication amongst IMGs, in contrast to a

103

conversation process, agrees with Dahm’s Australian observational study of IMGs

role playing consultations18.

Lack of exposure to UK general practice

As identified in the GMC paper a lack of exposure to good quality general practice is

another significant risk factor for IMGs1. All these IMGs came from outside the

European Economic Area, and from countries where general practice is a poorly

developed speciality. The average exposure to UK general practice prior to starting

their GPST 3 year was 4 to 7 months. This is grossly inadequate in terms of gaining

sufficient understanding of the NHS UK model of healthcare.

English language and Linguistics

IMGs usually speak English as a second language. This is a major obstacle to

overcome in developing a conversational style with patients, and giving explanations

to patients in everyday language as illustrated in the quotes below. One IMG felt so

negatively about this that they said the CSA is `just testing English language and it gets

covered up with communication skills’. These linguistic factors have already been

identified by research carried out by the RCGP and King’s College London12.

`There's lots of things we don't know how to say, we just say it because we've learned English as a

language, as a foreign language. It's not something that comes naturally to an international medical

graduate because even if we have studied everything in English the main language that I've always

used is Urdu.’ Mulan

`…learning that communication is more about what we see in colleagues that are UK based and

trained. The way they sort of speak, the way they explain things to the patients, the way they get a

patient to talk using everyday language.’ Obie

104

New emergent themes from the data on peer group learning

Peer group size, use of peer groups and peer group dynamics

The variation from two to six group members in the peer groups formed

demonstrates the haphazard nature of self-formed peer groups. Many of the groups

described were dyads and this limits the presence of an outside peer observer who

can triangulate peer feedback. For this reason Sky suggested three group

members should be a minimum number as this improves the quality of peer

observation and feedback. Group theory emphasizes once you get beyond six

group members, the easier it is for reticent learners to hide and the more unsafe the

environment feels to practice in93.

Hobma’s postgraduate Dutch study provided evidence that peer group work (with an

outside facilitator) is more effective in improving consultation skills than individual

reflective work51. The greater the number of peer group meetings attended, the

greater the effect on performance. These successful IMGs were very motivated in

terms of the use of multiple peer groups, frequency and longevity of meetings. It

would be interesting to look at these aspects of meetings and compare findings with

unsuccessful IMGs. It should be remembered however, that in contrast to Hobma’s

study51, these were self-formed groups without any outside expert facilitation.

Further research is required to determine the effectiveness of self-formed

postgraduate peer groups in improving consultation and communication skills.

Other issues of group dynamics were mentioned like over-dominant members talking

too much in the peer group sessions. Group facilitation skills are necessary for peer

groups to function well. Awareness of Belbin’s team roles can help understand the

105

different team roles that help a group function94. There was also an example of a

group receiving a new group member that upset the group dynamic and affected the

perceived safety of giving and receiving of peer assessment. Chou described how

longevity of peer relationships in an undergraduate study improved the quality of

corrective and specific feedback and this fits in with this last observation61. Once

peer groups are formed it would be better to keep a `closed’ group rather than an

`open’ group policy.

The issue of formation of IMG-only peer groups

Jamieson noted that IMGs have a tendency to form IMG-only peer groups where

doctor-centred behaviours and the biomedical model were praised, and

recommended IMGs to be supported to form mixed peer groups26. Jamieson worked

with IMGs who had mostly failed the CSA twice. This research showed that three

out of four IMGs used mixed peer groups on first time of sitting the CSA. However

two IMGs who sat the CSA a second time struggled to find mixed groups and

subsequently four out of the five groups utilised were IMG-only groups. This is a

natural consequence of failing the CSA and resitting several times, in that you are

statistically more likely to find other IMGs in the same position who wish to from a

study group. Further quantitative data is needed to verify these early findings and

whether IMGs studying in IMG-only groups are statistically more likely to fail the

CSA.

IMGs do want to form mixed peer groups as they perceive several advantages of

having UKGs in their group as can be seen in Obie’s quote: ` It's been drummed into us

from day one that it's absolutely vital that you have a peer group, but make sure it's a mix of

106

internationals, add like British born doctor, because he would come into be almost like a role model

you need to aspire to.’ As well as role modelling, IMGs recognise the insights UKGs can

give on specific consultation skills, raising cultural awareness of aspects of

communication, understanding some of the psychosocial aspects of the consultation

as well as local English and linguistic skills e.g. phraseology.

These peer groups were self formed, apart from Lise who was allocated a group by

her GPST teaching group tutor. This study reveals possible barriers to IMGs forming

or joining mixed self-forming groups. UKG groups may seem to be pre-formed and

there may be perception of a `subconscious divide’. Obie went on to describe

feelings of being isolated and an outsider: `so, I was pretty much left alone and the other lady

(IMG) I suspect the same thing happened to her.’ There was recognition of personal push

and pull factors as to why IMGs form IMG only groups such as fear of showing

deficiencies to UKGs, introverted personality type, more natural relationships with

IMGs, altruistic reasons in helping IMG colleagues and even a sense of obligation

(when asked by an IMG colleague). Suggested ways of overcoming these barriers

will be discussed in the recommendations.

Emphasis on doctor-centred behaviours not confirmed

In contrast to Jamieson’s findings however, this study shows that these successful

IMG candidates have good insight into their doctor-centred behaviours and are

aware of their weaknesses (particularly in terms of eliciting patient’s ideas, concerns

and expectations (ICE) and being patient-centred in clinical management). Contrary

to what Jamieson suggests, Lise felt the feedback from another IMG colleague who

107

was on her fourth sitting of the CSA was very focused on patient-centred

consultation skills such as eliciting patient ICE early.

However Sky did recognise this tendency in her IMG only group and felt that the

feedback content of her IMG only group was excessively focused on clinical

discussion: `The difference I’ve noticed from the first group, which is obviously with the local

graduate with the second group being with a couple of other IMGs, is that they’re more focused on

clinical knowledge even with the second time (of sitting the CSA)…… I try to persuade them to some

extent this is not the right approach’. It is recognised that these are IMG perceptions from a

small number of interviews and this data would need to be verified with observation

of peer group activity.

Lack of strategic planning and setting of learning goals in IMG-only groups

When considered in the light of Topping and Ehly’s theoretical framework of peer

assisted learning27 there were some differences noted between IMG-only and mixed

groups. Foremost of these was the process of planning learning goals. IMG-only

groups tended to have a lack of self-assessment and planning of learning goals for

group activities. `I think it reflects from our education system. We never used to do these sorts of

things as self-assessments or reflection’ Lise. Mixed groups did carry out some planning of

learning, with the exception of one group. Hanley showed that good self-

assessment and goal setting skills (SAGs) has been shown to improve consultation

skills and those students with poorer skills in SAGs were at higher risk of failing to

grow in their communication skills64. This data confirms Warwick’s finding of IMGs

lack of planning of one’s learning and the majority of IMG trainees struggling with

108

reflective practice24. This highlights the need to improve self-assessment and

strategic planning of learning in IMGs.

More reflective learning methods utilized in mixed peer groups

Closely related to planning learning is learning methods employed in the group

activity. IMG-only groups tended to stick more rigidly to CSA case textbooks. In

contrast mixed groups tended to employ mixed learning methods such as significant

case analysis and reflective practice on recent challenging clinical cases seen.

Given the complexity of the CSA exam and the different skills it is testing, including

the multi-faceted roles of a GP, the multiplicity of learning methods employed

improves the chances of developing these reflective practitioner skills required. One

undergraduate study demonstrated improved reflective skills improved academic

performance95. There was also an example of not adapting learning methods after

failing the CSA the first time in one IMG-only group. One of the skills of self-

regulated learning is the ability to change one’s learning approach when failing to

achieve a learning goal96.

Use of actor for simulation practice in IMG-only groups

One surprising finding in IMG-only groups was the practice of hiring an actor to

perform as a simulated patient in mock CSA stations, and give feedback on

performance in particular communication skills. There is no evidence in the

postgraduate literature supporting the effectiveness of this learning method, and the

undergraduate literature suggests issues with accuracy of assessment when

compared with peer and tutor feedback62. Further research is needed to explore the

quality and accuracy of simulated patient’s feedback in a postgraduate setting.

109

Differences in peer assessment between IMG-only groups and mixed groups

In terms of peer assessment, different themes emerged in terms of the style,

perceived quality and specificity of the feedback given. The use of Pendleton’s rules

was predominant in mixed groups which made the feedback more formal and

formative. IMG-only groups tended to use less structured, informal feedback styles

but contrastingly were more summative in their approach.

The data showed varying emotional responses to peer feedback in the different

types of groups. Sky appreciated the encouraging style of feedback from her UKG

peer but struggled with the `ambitious’ feedback in her IMG-only group, which was

more directive than learner-centred. In contrast Lise described falsely reassuring

feedback in her mixed group and yet found the feedback from another IMG more

directed and specific (particularly in terms of patient-centred consultation skills).

Obie felt the quality of the peer feedback in his IMG-only group was not as high as

that of his GP trainer. This mixed response to peer as compared to tutor feedback,

corresponds to the undergraduate medical literature in terms of perceived quality,

accuracy and validity27. There is a need to develop enhanced feedback skills

amongst all GPSTs and this is particularly relevant to IMGs learning to emphasise a

learner centred approach and focus on relevant, specific patient-centred consultation

skills.

Benchmarking and self-assessment in peer group activity

IMGs were able to benchmark their performances against those of their UK and

other IMG colleagues. They perceived themselves as being similar in their clinical

110

knowledge, but weaker in terms of their English and communication skills. However

caution should be employed here as both undergraduate and postgraduate studies

show the lack of ability of medical students and doctors to accurately assess their

own knowledge and performance compared to their peers97,98.

Low self-esteem and confidence

Self-actualisation, as highlighted in Topping and Ehly’s model27, is necessary for

significant learning to take place in peer groups. Overcoming low self-confidence is

an important aspect of self-actualisation described elsewhere in the literature99. Low

self-esteem and confidence were described in relation to sitting the CSA and working

with UK colleagues in peer groups: `Initially it took me a little while to gain my confidence

because history taking is more or less… we all have the same clinical knowledge base. But when it

came to explanation, when it came to explanation in simple words I was struggling. When it came to

shared management I was struggling and both of these weaknesses have been highlighted by my

supervisor as well and when I did the mock exam. I had to work really hard on these two areas, yeah.

Initially I was like should I be preparing with them? I'm losing my confidence but in the end it was all

worth it, I think.’ Mulan.

Some of this is internal to the IMG, however the external learning environment also

has an effect on the doctor’s learning and confidence. In the author’s opinion this is

why IMGs valued the positive encouragement of other UKGs in their peer group. In

addition, a preference for the Pendleton style of feedback encourages the learner

and group to highlight good performance first, building self-confidence in the learner.

This is in stark contrast to perceived critical feedback described by one IMG in an

IMG-only group. This lowered their self-esteem and a negatively affected motivation

111

for learning. All the IMGs described the necessity for a good group dynamic with

compatible members, thus creating this supportive learning environment.

Peer assisted learning (PAL) in the context of other learning strategies

IMGs had different perceptions as regards the place of PAL alongside other learning

strategies in preparation for the CSA examination. Trainer related learning activities

were highly valued, as were clinical encounters in the surgery as expressed below:

`I don't know if I can say it. I think CSA is not all about the peer group learning. The first thing

my trainer, was a very good resource. She was really helpful for me to get to the CSA level by

video recordings and feeding back to me. The second thing is, the time you spend in your

own surgery, because most of the international medical graduates have, I don't know whether

I can say this, but do have families like myself, and because I've got so many of my

colleagues who are in the same situation, who can only spend little time out of their work. I

use most of my time in the practice as my CSA preparation.’ Sky

The weighting of PAL amongst all other learning strategies (as listed in the previous

chapter) could be further assessed in a focus group. PAL is not everyone’s preferred

style of learning as Obie expressed. Mulan also noticed that some other GPSTs

hardly utilised peer group learning methods yet still went on to be successful in the

CSA. This raises questions about how effective and necessary PAL is for improving

one’s performance and passing the CSA?

However, in contrast Mulan also said: `I would say that practicing on your own can’t ever

match practicing in a peer group.’” This study suggests that IMGs perceive peer group

learning as an important aspect of preparation for the CSA but it is unclear how

crucial it is considered in comparison to other learning strategies.

112

Recommendations

It is recommended for IMGs to consider forming groups of at least three group

members to gain the advantage of an outside peer observer and no bigger than six

as determined by group theory to maintain a perceived safe environment for

rehearsal of skills. If there are problems with peer group dynamics then possible

problem solving skills may include a review of group facilitation skills and Belbin’s

team roles.

IMGs perceive it to be advantageous to have mixed peer groups including UKGs,

when preparing for the CSA. IMGs have described some of the barriers that need to

be overcome if left to self-form such groups. Work by Feichtner and Davis looking at

why some peer groups fail, suggest that heterogeneous groups formed by the tutor

perform better than homogeneous student-selected groups100. This leads to the

question of whether the GPST teaching tutors should allocate peer groups, as seen

in Lise’s experience, to avoid exclusion of IMGs and promote better mixing of peer

groups. Some tutors may argue that this is not their role and GPSTs choosing other

GPSTs to form groups that meet in their own time is simply part of the process of

adult learning. However, the numerous disadvantages IMGs already have when

approaching the CSA, and their risk of lower performance in the CSA could be

considered to be a more overriding concern.

Sky met her UKG whom she practised with through attending a deanery event where

the GPST recruitment scheme was piloted. IMGs need to be proactive in increasing

their professional networks with UKGs, developing friendships and overcoming some

113

of their own negative self-perceptions regarding their `deficiencies’ and feelings

when working with UKGs. The deanery has a role to play in hosting professional

networking opportunities where IMGs can connect with UKGs.

New data from this study reveals the lack of self-assessment and strategic planning

of learning in IMG-only groups. We know IMGs struggle with reflective learning and

need help to develop these skills. Gibbs highlights some useful reflective questions

that can be used to help plan peer group learning sessions101. One recommendation

is the writing of CSA cases (as exemplified in one mixed group); a skill encouraged

by Jamieson in her revision course, as it assists the learner in imagining what is

being tested, thus improving targeted learning (Joubert’s theory - teaching is an

effective way of learning102,103).

This study confirms Jamieson’s findings that some IMG-only groups tended to

overemphasize feedback on clinical knowledge rather than focus on patient-centred

consultation skills. This insight can be gained from having UK graduates in the peer

group. In addition GPST teaching and GP trainers need to raise awareness and

understanding of patient-centred consultation skills, with role modelling of these

patient-centred skills. I would also highlight the need to introduce these concepts at

an earlier stage in GP specialist training than during their GPST3 year.

There was a mixed response amongst IMGs regarding peer feedback in terms of the

style used, its quality and its validity. IMGs generally preferred the more

encouraging Pendleton’s learner-centred style of feedback demonstrated in mixed

groups and encouraged by one GPST group tutor. However this needs to be

114

balanced with specific feedback on patient-centred consultation skills in some

contexts. The development of effective feedback skills is essential for all GPSTs and

whilst there are useful papers on this104, it would be recommended that these skills

should be first developed in facilitated GPST teaching peer groups.

Limitations of this study

The major limitation of this study is that data saturation was not reached in a small

sample size of four. Furthermore as time ran out a focus group was not conducted,

and therefore triangulation of the data could not be achieved. As discussed in the

methodology, it would have been a more useful study if successful versus

unsuccessful IMGs CSA candidates could have been interviewed to help determine

any difference in perceptions and experiences of peer group learning between these

two groups. As previously highlighted some of the English language used by the

IMGs was difficult to understand and derive meaning from. As a result some of the

data had to be ignored and its meaning may have been misinterpreted. The lack of

use of memos in the qualitative data analysis was a further weakness in

methodology. This is a qualitative study looking at IMG perceptions of their peer

group study and there is a need to do more field work through observed peer group

activities which could verify or challenge these perceptions.

Further Research

Suggestions for further research include conducting a focus group in order to

triangulate the data, consider weighting of peer group learning in the context of other

learning strategies and validate a questionnaire. Some of the emerging themes and

115

grounded theories need further testing within a greater sample size in comparison

with IMGs who have failed the CSA. The use of hiring actors for simulated patient

practice needs further exploration in terms of the accuracy of feedback given before

making this a recommendation for IMGs. Finally the whole question should be

visited and given further consideration whether PAL actually improves performance

in the CSA.

116

APPENDIX 1: TABLE SUMMARIZING RESEARCH TRIALS ANALYSED IN BOTH MEDICAL UNDERGRADUATE AND POSTGRADUATE SETTING

Author, Year

Aims of Study Study Design Setting Sample Size Main result Authors Conclusion Critique of study

Systematic Review

Yu 2011

To summarize and critically appraise evidence presented on peer-teaching effectiveness and its impact on objective learning outcomes of medical students.

Systematic review: A literature search was conducted in 4 databases - Medline, Pubmed, Embase and ERIC by 2 reviewers. Modified Kirkpatrick's model used to grade impact of peer-teaching on educational outcomes from years 1990-2010.

Auckland, New Zealand

19 articles found. 15 focused on student-learner and 4 on student-teacher learning outcomes.10 studies randomized allocation. 11 studies provided student-teachers with training.

Overall, results suggest that peer-teaching achieves short-term learner outcomes that are comparable with those produced by faculty-based teaching. Furthermore, peer-teaching has beneficial effects on student-teacher learning outcomes.

Peer-teaching in undergraduate medical programs is comparable to conventional teaching when utilized in selected contexts. There is evidence to suggest that participating student-teachers benefit academically and professionally. Long-term effects of peer-teaching during medical school remain poorly understood.

Clear inclusion and exclusion criteria - avoiding soft measures of self-evaluation and looking for hard measures of learning outcomes. Studies poorly described nature of student-teacher training.

117

Burgess 2014

A systematic review of the literature to establish why and how peer assisted learning (PAL) has been implemented, focussing on training processes, the benefits and the competency of peer tutors. Used a AMEE framework `PAL: a planning and implementation framework' to review aims of studies.

Systematic Review: A literature search was conducted in 3 databases - Medline (Pubmed), Web of Knowledge and ERIC. Original research written in English from years 2002-2012 included.

Sydney, Australia

19 articles found from 8 different countries majority of studies from Germany, UK and Australia.

Many studies found student tutors considered PAL useful for developing professional attributes. 2 studies reported peer assessors more lenient than academic assessors, while 1 study found opposite. Knowledge acquisition by student tutors measured in 2 studies using student tutors’ examination performance, which produced conflicting results. No benefit determined by comm skills exam performance when results of tutors were cf to non-tutors.

The rise in international use of PAL appears to be a consequence of the global increase in medical student intake and limited teaching resources. The mixed results regarding accuracy of peer assessment and feedback needs further research using objective measures. Whilst there are perceived learning benefits for student tutors participating in PAL activities there is no substantial evidence of improvement of one’s own examination performance.

Limited search strategy in terms of only 3 databases used and a narrow time frame. The review admits 4 new relevant articles found in following year 2013-2014. The majority of the data extraction was carried out by 1 independent reviewer (only 20% of the data extracted reviewed by 2

nd

researcher).

Key short hand terms in tables PAL – peer assisted learning comm. = communication OSCE – observed structured clinical examination SP – simulated patient (actor) dr-pt – doctor-patient RCT – randomised control trial cf – compared betw- between

118

Author, Year

Aims of Study Study Design Setting Sample Size Main result Authors Conclusion Critique of study

Scoping Review

Williams 2016

Scoping review to map literature as to how peer assisted learning affects academic performance

Scoping review: 4 databases - Medline (Ovid), Proquest, Cinahl and Embase were used as well as grey literature sites and dissertations from 1996 -2014.

Victoria, Australia

22 articles found (10 mixed methodologies RCT, 1 retrospective trial, 4 controlled trials, 2 RCT crossover, 3 RCTs (prospective), 1 thesis and 1 comparative research design

Positive effect on student teacher performance (confirmed in 5 studies) - greater chance of passing exams. Positive effect on student learner performance (17 articles) - increased OSCE scores and BLS pass rates but not necessarily MCQ scores. No effect / negative impact on performance (7 articles) in OSCE scores.

There is some evidence to suggest that PAL improves academic student performance in an objective manner, similarly equal amount of evidence to it does not. However it is the student teachers that benefit most from this process. More complex comm and clinical skills better taught by more experienced staff.

Single researcher rather than 2. Only 4 databases used in the scoping review and only keywords used (not MeSH terms) in searches. Positively there was searching of the grey literature, dissertations and consultation with an expert in the field.

Randomised Control Study

Hobma 2006

To examine the effectiveness of a learner-centred approach that focuses on actual needs, to improve GPs’ communication with patients.

RCT: GP practices rather than individual GPs randomised (to prevent closely collaborating colleagues). The intervention arm consisted of assessment, selection of topics for improvement, feedback and small group activities with a facilitator over 7/12. Control arm received reading material on dr-pt comm and self-assessment

South Netherlands

100 GPs (49 intervention group, 51 control group)

The mean total scores and scores on the items of the MAAS-global went through a regression analysis and showed a significant improvement from pre to post treatment scores. Participants reported improvement of 5 of 8 self-improvement goals (d-value was 0.66) indicating a moderate to large effect. A regression

The educational approach involving assessment of comm in daily practice, personalised learning activities guided by structured small group work proved to be effective way of improving dr–pt comm (size of the effect was moderate to large). A dose-response effect was seen as the attendance to the intervention with most improvement on

There was a relatively large drop out rate to follow up (5 in intervention group and 9 in control group). 7 GPs decided not to focus on doctor-patient communication skills thus couldn't be included. Both groups also did extra self study CME activities on doctor-patient communication (more in mean hrs in

119

questionnaires only. Performance of GPs assessed by video observation in daily practice using the MAAS-global rating scale before and after intervention.

analysis on % of meetings attended was a significant independent variable on effect of the pre and post scores.

patient-centred items. Need further assessment to see whether the effect is temporary or longer lasting.

control group).

Cave 2007

Aims: (1) Investigate 3 different ways of introducing standard assessment criteria into comm skills teaching, and determine the effect upon performance in an OSCE. (2) Investigate students’ ability to assess own and peers’ comm skills and compare this to assessments of tutors and SPs. (3) Investigate the relationship between students’ performance in the comm skills teaching and the comm skills OSCE.

RCT: Standard arm: Students attended comm skills teaching as usual and signposted to standard assessment criteria on med school website. Intervention A: During comm skills teaching every student given copy of standard assessment criteria to inform discussion and feedback. Intervention B: Teaching sessions run as ‘mini-OSCEs’. Each student interviewed a SP and was marked by peers, tutor and the SP using standard assessment criteria.

London, UK

359 3rd yr med students (Standard group 124, Intervention A group 107, Intervention B group 128)

There was no difference in the OSCE scores betw standard teaching (61.8/80), intervention A (62.9/80), or intervention B (62.3/80). Self-scores significantly correlated with tutor scores and peer scores but not with SP scores. No significant correlations betw students’ performance in OSCE and scores given during teaching by tutors, peers or SPs.

Study demonstrated students have ability to assess their own communication if given right tools and right training. This study did not find any correlation betw students’ performance in teaching and their performance in OSCE. This surprising finding would seem to support opinion that there are discrepancies betw teaching and assessment of comm skills.

Impossible to blind researchers from participants in comm skills teaching and OSCE examination. Only 3 fails in year group (2 didn’t consent to be in study) thus unable to extrapolate data of fails to interventions.

120

Author, Year

Aims of Study Study Design Setting Sample Size

Main result Authors Conclusion Critique of study

Qualitative / Mixed method Studies

Mcmillan 2011

Development and evaluation of generic tool to be used to inform feedback about both GP and pharmacist patient consultations.

Mixed method: 4 GP educationalists used Delphi technique to develop generic consultation assessment tool (leaning towards patient-centredness). Feedback instrument of 24 relevant questions reached through consensus after round table discussion. GPs and pharmacists recruited by convenience sampling agreed to verify the tool using videoed consultations.

West Scotland

6 established GPs, 6 pharmacists

High inter-rater consistency measured for each item in instrument (83% within 1 pt on the Likert scale). Reviewer’s qualitative judgements regarding GPs performed exceptionally well cf those performed less well were statistically different (CI = 1.73 – 2.09, p<0.0001). Evaluation of performance in pharmacist consultation less consistent and scores lower and more widespread.

This pilot study highlights the validity and potential reliability of this tool in providing consistent peer review feedback for GP consultations not pharmacist consultations. Further work required to confirm the overall utility of the instrument particularly for allied health professionals.

Pharmacists currently don't consult patients like GPs and there was minimal adaptation of the tool suited to pharmacist consultations. Furthermore pharmacists being assessed by GPs not their peers. Tool needs further testing in a larger more urban population for validation purposes.

Nestel 2005

To assess 3rd yr med students competence in teaching their 1st yr colleagues in patient-centred interviewing and acceptability of such near peer tutoring.

Mixed method / quasi-randomised trial: All 1st yr students invited to complete written evaluation forms after teaching sessions. 2/12 later randomly selected students invited to conduct interview with SP. Interviews videotaped and later rated by independent assessor. Pt-centred interviewing skills evaluated by comparing skills of students from groups facilitated by peer tutors vs med teachers.

London, UK

299 students divided into 49 groups (feedback on teaching). Analysis of videos 39 med students (20 in peer group, 19 in teacher group).

No difference in pt-centred interviewing skills betw students facilitated by peer tutors vs med teachers confirmed by SPs ratings. 2/12 after greater SP satisfaction with consultations of peer taught group cf with med tutor taught groups. During SP teaching sessions less satisfaction of learning from 1st yr med students in peer led groups vs tutor led groups.

Use of near peer tutoring is potential solution for resource-intensive nature of comm skills teaching. Evaluation shows peer tutoring can be effective for students at start of their education. It is possible that peer tutors provide experiences that med teachers cannot (and vice versa). Better experience of learning in tutor led groups vs peer led groups.

The peer tutors contributed to just 1 of 6 sessions in the comm skills programme so not certain whether this 1 intervention had an effect. Sample size too small to be statistically powerful. There was no description of the teaching given to peer-tutors.

121

Perera 2010

To assess objective structured self-assessment and peer-feedback (OSSP) in enhancing learning of comm skills.

Mixed methods / control trial: OSSP incorporated into small group comm skills (CS) teaching sessions of yr 1 med students learning CS. Control group matched for academic performance, gender and age. Reflective log used for OSSP. Facilitators and SPs provided feedback to students in both groups during CS learning. Student perceptions on OSSP and acceptability explored using questionnaire. CS were assessed in both groups using OSCE (assessors blinded to which group student belonged).

Kuala Lumpar, Malaysia

97 experimental group, 93 control group 1st yr med students

Mean total score at OSCE for experimental group significantly higher (13.30 vs 12.33, p=0.0001)than control group. Students in experimental group scored significantly higher on skills on building rapport with SPs, listening and interview style cf to control group. 86.4% positive comments on usefulness of OSSP for learning CS.

Students exposed to self assessment and peer feedback in small group setting may learn better than students who receive only SP and tutor feedback. Analysis of questionnaire data showed students gained fresh insights into specific areas such as empathy, addressing patients’ concerns and interview style during OSSP which clearly corroborated the specific differences in OSCE scores. OSSP promotes effective CS learning and learner acceptability is high.

OSCE stations were formative and not summative softening the outcome measure. The drop off rate of attendance of 1st vs 2nd OSCE was 84% to 52%.

Cushing 2011

Medical educational intervention of peer tutoring 1) to gain feedback on comm skills and 2) to learn how to provide feedback to peers.

Qualitative Study: Questionnaire + focus group of educational intervention of peer tutors equipped with OSCE marking sheet + teaching workshop on feedback. Likert scale feedback questionnaire and focus group to review peer vs tutor feedback after 2 formative OSCE stations.

London, UK

93 1st year med and nursing students

Thematic results from focus group: Emphasis on status as beginners both as part of experience of being in OSCE and giving feedback. Anxiety on giving -ve or corrective feedback. Presence of peers prompted mixed emotional response, some found it easier to relax, others found it pressurising. Mixed views regarding transferability from OSCE setting to clinical placements.

A practical and acceptable model for learning comm skills and how to give feedback developed. Although novice status acknowledged of med students seen as valuable learning experience. Note greater desire for additional feedback from SP and faculty tutor in 2

nd cohort of

med students.

No statistical difference in evaluation results betw experimental and control groups on questionnaire.

122

Chou 2013

To examine whether longitudinal relationships among peers may increase sense of safety and soften the perceived threat of feedback to allow students to give and receive more effective feedback on comm skills.

Mixed methods: Divided 3rd yr med students into 2 subgroups: (i) Students with prior peer-learning relationships in clinical skills small groups, and (ii) students who hadn't shared small group work in first 2 yrs. Data collected from student surveys and transcripts of videotaped simulated comm skills and peer feedback qualitatively analysed. Outcomes measured were student satisfaction and specificity of peer feedback.

California, USA

163 3rd yr med

students

Questionnaire feedback (Likert scales 1-5, 1=strongly agreed) better specific feedback in intervention group 1.53 vs 1.69 (intervention vs control group). Greater number of constructive comments in group with longer relationships 1.49 vs 1.66 and more corrective feedback seen on videotapes using qualitative and quantative analysis 18.3 (+/- 13.7 variation) vs 10.0 (+/- 8.1 variation) (p=0.014).

Compared with counterparts without prior shared small-group learning experiences, students with prior peer-learning relationships more likely to deliver corrective feedback on comm skills to their peers regarding performance in a simulated setting.

As study highlights likely some of students in newly formed groups knew each other socially from previous 2 yrs so can't say there was no longitudinal relationship in control group.

Lau 2001

To test hypothesis that use of `interpreter' role play is effective tool in teaching comm skills to 1st yr med students compared with standard `observer' role plays.

Control trial: Intervention was a workshop introducing verbal and non-verbal aspects of dr-pt comm. Each pair of tutors ran 2 workshops sequentially, conducting both a new interpreter and routine observer role play session. Every student had chance to play role of interviewer, patient or observer/interpreter. Students completed 16-item Likert survey scale 1-5 (9 items on interpersonal communication skills + 7 items on effectiveness of the training programme).

Hong Kong

160 1st yr med students (groups of 12-26 students supervised by 2 experienced tutors)

Significant pre- to post-effect (F ˆ 73.9 (1,1562 ), p=0.00009) indicating entire class of students reported comm skills improved and programme more effective after workshop. There was no observer/interpreter main effect. There was an interaction between observer/interpreter and pre- and post-effects (F ˆ 4.84 [1,156], P ˆ 0.029); students in observer group showed more of a change in their scores following the workshop than students in interpreter group.

Med students self reported improvement in comm skills following workshop emphasizing verbal and nonverbal aspects of dr-pt comm. Where students were arranged in trios with one the student taking the role of an interpreter this arrangement had some effects in improving their comm skills but not greater than that of students who received training using the standard role play.

This study more interested in role playing interpreters as a learning method in comm skills and less concerned about peer group teaching. Self-reporting Likert scales not good outcome measure and better richness of data would have been achieved through focus groups & interviews.

123

Hulsman 2014

To study characteristics of self-evaluations and peer-feedback annotations of video recorded comm skills. How do characteristics of peer-feedback relate to characteristics of self-evaluations. Features that may bias the assessment of specificity of self-evaluations and peer-feedback.

Mixed method: 2 student groups invited to use VideoFragmentRating (VFR) system (video consultation uploaded to web that can be annotated). Each student video recorded a consultation with SP for formative assessment. The student made some self evaluations and invited peers to review the video and offer feedback. All annotations coded and analysed. The no. of self and peer evaluations calculated and content analysed for specificity using an iterative process judged against the History Taking Assessment Scale (HTAS).

Amster-dam, Holland

25 4th yr med students

Avg number of self assessments 5.64 (2.36 positive valence and 3.28 negative valence). Peer feedback 4.96 (2.08 positive valence and 2.88 negative valence). Main topics `structuring the conversation' (36.2%), medical perspective (22.6%), courteousness and respect (18.8%) and patient perspective (11%). Self-evaluations more specific than peer feedback esp. negative valence. Specificity of peer-feedback is positively assoc with specificity of self-evaluations. Students' self ratings were more negative than peer-ratings. Peers may be biased in a positive direction guided by friendship marking.

Study shows self-evaluation and peer-feedback are highly valued in terms of reflective practice. Teaching students to be more specific in their self-evaluation may stimulate peers to be more specific in their feedback.

4 response patterns in annotations that may bias the quantitative assessment of their specificity. Suggestions are often omitted in annotations with a positive valence.

124

Ahem 2013

To explore learners’ and other stakeholders’ perceptions of shared learning models in general practice, in order to inform policy and training.

Qualitative study: Individual 30-60min semi-structured interviews with GP supervisors, GP registrars, prevocational trainees, medical students and practice managers (PMs) situated in teaching accredited practices, to investigate their perceptions of shared learning (between different levels of medical learners in practice).

New South Wales, Australia

33 participants

in 9 GP training

practices (11 GP

supervisors, 8 GPRs, 2

prevocational trainees, 8

med students, 4

practice managers)

Benefits for learners: Easier/safer to ask questions, active learning as more interaction cf to 1:1, more likely to be structured and planned leading to better learning outcomes, group can learn from knowledge, skills of others in group, discover different approaches to same problem, resources more easily shared, extra qu's are asked that individual didn't think of and everyone challenges each other, encouraging debate and discussion. Learning in group can motivate, allow benchmark against peers improving self-confidence. Benefits for trainer - less repetition of teaching and less chance of burnout due to reduced workload. Benefit for practice - financially beneficial, increased vitality in practice and sustainable employability of GPs.

Shared learning models can be a type of informal community of practice that at best makes learning enjoyable, build social capital, and improve learning outcomes through engendering active learning and tapping into the knowledge and skill of the whole group. However, our participants suggest that 1-to-1 teaching is more suitable in some situations e.g. sensitive topics, remedial training, more junior trainee, shy learner...

Interviewers employees of same health authority potential bias. Good number of interviews and data saturation likely reached from range of stakeholders. Good practice of 2 independent reviewers of data and thematic analysis used.

125

Hanley 2014

To study relationship between medical students’ self-assessment and goal-setting (SAGS) skills and development of interviewing skills.

Prospective cohort study: Assess students’ interviewing skills over 3 pts of time over 17 weeks with SAGS skills at first 2 time points. Students given worksheet checklist for self-assessment. Students showed video segment of SP interview and group gave focused feedback. Quality of students’ SAGS defined by specificity by which able to articulate observations of their interview skills. Research assistant (blind to interviewing performance) rated quality of students’ written SAGS using 3-pt scale (poor, fair, good). A random sample of 31 ‘worksheets’ were re-rated by 2 faculty members. In all 3 interviews, SPs assessed students’ interviewing skills using a 17-item comm skills checklist.

New York, USA

153 1st med students

Higher baseline SAGS skills were assoc with initial decline and then a substantial increase in comm scores. This pattern may be attributable to a phenomenon known as `expertise reversal'. Low-quality SAGS’ ability at baseline that remained poor at 10-weeks later risk factor for failing to sustain growth in interviewing skills. Students who went from having poor SAGS skills to good SAGS skills didn't demonstrate any change in interviewing skills.

This study begins to show he link between goal setting and development of interviewing skills, demonstrating that better SAGS ability is assoc with improvement in performance of interviewing skills in a standardized scenario. Students with poor quality self-assessment skills (40% of our students) did not show this improvement. More research required to understand the longer-term implications of these findings and better understand the constructs being examined for remediation of students.

As the study highlights 12 months may be insufficient time to assess improvement of interviewing skills e.g. those students who went from having poor SAGS skills to good SAGS skills did not demonstrate any change in their interviewing skills but impact of enhanced SAGS skills may not be discernible for some time.

Shield 2011

A description of a programme of comm sessions to foster enhanced comm and care in medical students.

Descriptive study: Mandatory 2-yr curriculum designed to teach medical interview skills, physical examination and professionalism. Schwartz Communication Sessions (SCS) consists of discussions about clinical cases in large and small groups. Use of role plays groups and chance to practice comm strategies in tutor led small groups. Evaluation questionnaires completed on all sessions.

Alpert Medical School, USA

93 1st yr med students and 99 2nd med students

A total of 93% of faculty and 83% of students who completed the evaluations rated the sessions as good, excellent, or exceptional on the 5-pt Likert scale.

Integrating a pre-clerkship comm curriculum may help improve future physicians’ interactions with patients and families

Descriptive study without hard learning outcomes. Med school received grant funds from the Schwartz Center in 2009 to develop and implement ‘‘Schwartz Communication Sessions’’ (SCSs). Lack of balance in terms of reporting student and faculty feedback given.

126

APPENDIX 2: SEMI-STRUCTURED INTERVIEW QUESTIONS

Initial Open-ended questions

1. FORMING: Tell me about how you came to form your small peer group of GPs

whom you worked with to prepare for the MRCGP CSA?

Support questions:

Who was in your group?

How many were in your group?

Which medical school did they qualify from (UKGs or IMGs)?

How did you decide on who would be in your group?

2. PRIOR EXPERIENCE PEER GROUPS: What experiences have you had before

preparing for the CSA of working in peer groups preparing for medical examinations?

3. PRIOR EXPERIENCE COMMUNICATION SKILLS TRAINING: What

experiences have you had before preparing for the CSA in learning communication

and consultation skills?

OR How did your experience of medical education before preparing for the MRCGP

CSA affect how you prepared for the MRCGP CSA? Or didn’t?

4. PLANNING YOUR LEARNING: Could you describe how you planned your learning

as a group?

Support questions:

What did you think as a group was the most important thing to focus on in

preparing for the CSA?

How did you decide on what you were going to do in your group times?

What resources did you use to define cases or areas to look at (the GP

curriculum or other texts)?

How were decisions made as to what you would do in the group process?

5. WHAT LEARNING ACTIVITY TOOK PLACE: Tell me about what you did in your

peer group learning in preparation for the CSA? OR Could you describe a typical

group session that you had?

Support questions:

What different learning methods did you use (role play, video analysis, smart

cards, simulated patients)?

(For those who failed their CSA on 1 or more occasions) Did you change

your learning methods or strategies in anyway after you failed your CSA?

127

6. PEER GROUP FEEDBACK: Could you give me some examples as to how you

shared feedback on each other’s performance in your peer group?

Support questions:

Did you use any assessment tools for feedback?

Did you find your peers’ feedback useful?

Did you find your peers’ feedback helped your understanding of

What strengths and weaknesses in your consultation skills do you feel were

highlighted for you in your peer group feedback?

7. SELF CALIBRATION: How did you calibrate your own performance in the group?

Support question: How did you assess where you were in preparation for the

CSA exam?

8. IDENTIFYING LEARNING NEEDS: Did your group learning highlight areas you

needed to work on? Can you give any examples?

9. REFLECTION ON LEARNING: Looking back do you think your peer group

learning emphasized the right consultation and communication skills required for the

CSA examination?

Support questions: Did you feel there was insufficient understanding in the

group of what the CSA examination was looking for?

10. MOTIVATION: How did your peer group motivate you in your learning and

preparation for the CSA?

Negative or positive? What personal influences at the time do you think

shaped how you worked in the group?

Intermediate Questions

11. THOUGHTS AND FEELINGS: Tell me about your thoughts and feelings about your

peer group learning?

12. EXTERNAL INFLUENCE: Who, if anyone, was involved from outside the group?

What role did they play?

13. STAND OUT EVENTS: As you look back on your peer group learning in preparation

for the CSA, are there any events that stand out in your mind?

Support questions:

Could you describe them? How did this event affect what happened? How did you

respond…..?

14. IMPORTANT LESSONS LEARNT: Could you describe the most important lessons

you learned through experiencing peer group learning in preparation for the CSA?

128

15. PROBLEMS ENCOUNTERED: What problems did you encounter during your peer

group sessions? Tell me the sources of these problems?

16. HELPFUL PEOPLE: Who was the most helpful to you during your preparation for

the CSA? How were they helpful?

Ending Questions

17. UNDERSTANDING OF CSA: What do you think the CSA is testing?

Support question: In particular what type of consultation skills?

18. THOUGHTS ON PREPARING FOR CSA: What do you think are the most important

ways to prepare for the CSA?

19. ADVICE GIVING: After having this experience of working in a peer group preparing

for the CSA what advice would you give to a fellow International Medical Graduate

who is about to form a peer group preparing for the CSA?

20. ANYTHING ELSE: Is there anything else that you might not have thought about

before that occurred to you during this interview?

Support question: Is there anything else you think I should know or understand

better on how peer group prepare for the CSA?

Is there anything else you would like to ask me?

129

APPENDIX 3: INFORMATION SHEET OF STUDY AND CONSENT FORMS

QUALITATIVE INTERVIEWS AND FOCUS GROUP EXPLORING INTERNATIONAL MEDICAL GRADUATES PERCEPTIONS OF THEIR STUDY AND PREPARATION FOR THE MRCGP CLINICAL SKILLS ASSESSMENT EXAMINATION What is the project about? The Northwest Postgraduate Deanery involved in general practitioner specialist trainee (GPST) training has recognised the need to try new approaches for helping struggling GPSTs who fail the CSA examination, and, in particular, international medical graduates (IMGs). A review of the MRCGP CSA examination by the General Medical Council (GMC) highlighted that black and minority ethnic (BME) IMGs are 15 times more likely to fail than their white UK colleagues10. In the Northwest deanery 60.2% of BME IMG candidates were likely to fail the MRCGP CSA (national average of 65.2%) compared with 4.4% of white UK graduates (UKGs) (national average 4.5%)1. There is little known as to how IMGs study and prepare for the CSA examination. You have recently successfully passed or about to sit the CSA examination and that is why we are interested in finding out your thoughts on how you prepared for the CSA examination. We are particularly interested in how you went about studying with your peers in small group work. Therefore we are asking you as a recent successful or soon to be CSA candidate to take part in an interview which we hope will capture the range of views and experiences of IMGs preparing for the CSA. These interviews will be analysed for thematic areas that will then form the basis of discussion of a focus group. Before you decide whether or not to take part it is important for you to understand why the study is being done and what it will involve. Please take time to read and consider the following information carefully and discuss it with others if you wish. If there is anything that is not clear or you would like more information, please contact us using the details at the end of this form. What is the purpose of the Focus Group and Interviews? We need to hear how you studied and prepared for the MRCGP CSA. We are interested in all aspects of your own self directed learning but not necessarily around your GPST teaching group or 1 to 1 training with your trainer although we recognise this may have influenced your own learning. As stated before we are particularly

10

General Medical Council (2013). Independent Review of the Membership of the Royal College of General Practitioners (MRCGP) examination. www.gmc-uk.org/MRCGP_Final_Report__18th_September_2013.pdf_53516840.pdf [accessed on 27/03/2015]

130

interested in how you formed your small peer group learning and how your small groups functioned. This information will help us develop a questionnaire that can be used more widely amongst GP candidates to determine how GP candidates prepare and study for the CSA. Who is organising and funding the study? This project is self-funded by the chief researcher and is in collaboration between University of Central Lancashire (UCLan) and Health Education North West (HENW). The project team who will organise undertake and analyse the focus groups are staff from the University of Central Lancashire (UCLan). Who has reviewed the study? This study has been reviewed and approved by University of Central Lancashire STEMH research ethics committee and the HENW ethics committee. Why have I been approached? You have been approached because you as an international medical graduate have either recently successfully passed the CSA or about to sit the MRCGP CSA. We value your thoughts, views and experiences to help develop further knowledge on how IMGs study and prepare for the CSA that will help develop a knowledge base for future medical educational interventions. What does taking part involve? You are being invited to take part in either a one to one face or telephone interview of about 30-40 minutes. You also might be invited to take part in a focus group will involve a small number of IMGs from a range of backgrounds. The focus group will be facilitated by two members of the project team from UCLan. This will be very like a conversation guided by a number of questions asked by the researchers. The focus group or meeting will last about 45-60 minutes. The initial part of the group will discuss the thematic areas that arise from the interviews. Then there will be further discussion of peer small group learning. During the Focus group we will gather, on a flip chart, key issues considered important by members of the group in response to these areas. These key issues will be verified and prioritised by the group at the end of the meeting. With your consent, the interview or focus group meeting will be audio taped, so that we have a record of what was discussed. What are the possible benefits of taking part? The findings of the interviews and focus group will help develop key thematic areas for future GP candidates to consider in their own study and preparation for the CSA. Will my taking part in this study be kept confidential? As a participant you can be assured that taking part in the study is strictly confidential and will not affect your rights in any way. All data will be kept secure in a lockable filing cabinet at the University of Central Lancashire, or on password protected files on a University computer which only the Project team can access, and will be destroyed at the end of the project. All data will be kept for a minimum of 5 years in line with the University’s ethical guidelines and will then be destroyed.

131

We ask that you do not disclose your name and that of other people during the sessions. We also ask that you do not disclose any identifying information regarding clients, or voice any professional concerns about peers or GP trainers or educators, either internal or external to your organisation. Whilst general themes may be used in the final report produced, these will not be directly attributable to any individual. What are the disadvantages of taking part? This project will focus directly on aspects of your self- study and preparation as well as small group work and will mean that there may be debate around personal issues of self study and group learning. It is possible that you may experience some stress when taking part in the focus group or interview. If so, please alert the member/s of the research team who will assist you. You are free to leave the focus group at any time should you wish. Attendance at the focus group may also mean using time outside your normal working day. What will happen if I report on poor practice? Although it is not the intention behind this study to report on clinical practice issues, if you tell a member of the research team about poor or unethical practice, he/she will be obliged to pass that anonymised information onto the relevant organisation/agency. What will happen to the results of the study? The results of the study will be used to inform a questionnaire that will be sent out more widely amongst GP candidates in the northwest deanery. The overall findings will be reported back to the project team and to the Northwest Postgraduate Medical Deanery. Although direct quotes may be included, no individuals will be identified by name in any report. Do I have to take part? Participation in the project is entirely voluntary. If you are happy to take part, we will ask you to sign the attached consent form and give it to us at interview or focus group and also keep this information leaflet.

Even if you agree to participate you are still free to withdraw from the interview or focus group at any time and without giving a reason. If you withdraw from an interview, you will be given the opportunity to have any data collected destroyed. This will only be possible up until data analysis commences. If you withdraw from the focus group, removal of data will not be possible once the focus group has started because we can’t necessarily identify you due to the nature of the discussions.” A decision to withdraw, or a decision not to take part, will not be recorded or reported within the final report or to your employer. What if I have any complaints or concerns about the project? Please address any concerns or complaints about the conduct of the study or individuals involved to the University Officer for Ethics at the University of Central Lancashire: OfficerForEthics@uclan.ac.uk. Please include the study name or description, the principal researcher, and the substance of your complaint in your email.

132

Further Information If you have any further questions, please don’t hesitate to contact one of the Project team members listed below: David Harniess, General Practitioner with an interest in Medical Education, St Andrew’s Medical Centre, Eccles, Salford, Telephone No: 07542 824348 Email Address: d.harniess@nhs.net Anne Milston, Senior Lecturer, School of Medicine and Dentistry, University of Central Lancashire, Adelphi Street Preston PR1 2HE Telephone No: 01772 895485 Email Address: AMMilston@uclan.ac.uk Paul Milne, Senior Lecturer, School of Medicine and Dentistry, University of Central Lancashire, Adelphi Street Preston PR1 2HE Telephone No: 01772 895426 Email Address: PMilne@uclan.ac.uk

133

CONSENT FORM FOR FACE TO FACE OR TELEPHONE INTERVIEW OF INTERNATIONAL MEDICAL GRADUATES (IMGs) IN RELATION TO THEIR PEER GROUP STUDY AND PREPARATION FOR THE MEMBERSHIP OF THE ROYAL COLLEGE OF GENERAL PRACTITIONER CLINICAL SKILLS ASSESSMENT (CSA) EXAMINATION This research study has been approved by both University of Central Lancashire and Health Education North West. If a face to face interview is to be carried out on your GP premises please seek the permission of your practice manager. Please insert your initials in the boxes provided to indicate ‘YES’ to the following statements:

I have sought and gained permission from my practice manager to go ahead with this interview (if on GP premises)

I have read and understood the information sheet and I have had the opportunity to ask questions, and have these answered to my satisfaction.

I agree to the interview being audio-recorded and/or written notes being undertaken

I understand that I have the right to decline to answer any specific question, and to discontinue the interview at any point. n.b. If you do not want any of the data used, obtained up until the termination of the interview, then please indicate this to the researcher and the audio copy, written notes and any transcribed copies of the interview will be destroyed.

I understand that my participation will be anonymous and any details that might identify me will not be included in reports or other publications produced from the study.

I voluntarily agree to take part in the interview.

Name (PRINT): Date: Signature:

Name of researcher taking consent:

Signature: Date:

1 copy for participant, 1 copy for researcher

134

INTERVIEW INFORMING THE PERCEPTIONS OF INTERNATIONAL MEDICAL GRADUATES (IMGs) IN RELATION TO THEIR PEER GROUP STUDY AND PREPARATION FOR THE MEMBERSHIP OF THE ROYAL COLLEGE OF GENERAL PRACTITIONER CLINICAL SKILLS ASSESSMENT (CSA) EXAMINATION

If you would like a copy of the transcript of our interview please indicate how you would prefer to receive a copy of this document,

i.e. through email or by post (home or work address) and give

your contact details

I would like to receive a copy of the transcript of the interview Yes/No I would like to receive them by Email/Post Contact details:

135

APPENDIX 4: PROPOSED QUESTIONNAIRE OF IMGs PERCEPTIONS OF PEER GROUP STUDY FOR THE CSA EXAMINATION Personal Details Initials: DOB: Age: Sex: Country of graduation: Year graduated: Prior experiences Number of months of UK General Practice prior to starting GPST3 year: What has been your previous undergraduate experience of communication and consultation skills learning? _______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ What has been your previous experience of peer group learning for consultation and communication skills? ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ Group Formation How did you form as members of the same group? Same peer group in GPST teaching Friends previously Deanery event Allocated to groups by GPST tutor Other_________________________________________________________ Make up of group? Number of UKGs____

Number of IMGs ____ How many in peer group? 2 3 4 5 Other How many peer groups were you part of? 1 2 3 Other What were the make up of these group(s)? _________________________________________________________________________________________________________________________________________________________________________________________________________

136

Frequency and longevity of meeting How long did you meet for? (2-3 hours / half a day / full day) ___________________________________________________________________ Frequency of meeting? (once a week / twice a week / more often?) ___________________________________________________________________ How many weeks before the CSA did you meet for?

___________________________________________________________________

Self –assessment and planning learning Did you make a self-assessment and identified your learning needs? _________________________________________________________________________________________________________________________________________________________________________________________________________ How did you plan your learning for your peer group learning times? ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ Peer group activities What activities did you do in your peer group work? (circle as appropriate) Discussion about interesting clinical cases seen Significant event discussion

Role play Use of actor for simulated patient practice Practising explanations Writing own CSA cases Other____________________________________________________________________________________________________________________________ What learning resources did you use? (circle as appropriate) Website / e-learning Revision books Previous trainees Simulated patients Patient encounters Writing your own cases Case cards RCGP curriculum Other_________________________________________________________ During your peer activity which consultation skills did you focus on? _________________________________________________________________________________________________________________________________________________________________________________________________________

137

Prioritise in order what you think the most important thing is to practice in peer groups for the CSA examination?

English language skills Clinical knowledge Timings Interpersonal skills Patient-centred consultation skills (eliciting ICE, shared management) Examination skills

Reflective practitioner skills Other______________________________________________________

Peer assessment and feedback Focus of feedback Clinical knowledge Consultation skills 1 2 3 4 5 6 7 Preferred style of feedback Non-directive Directive 1 2 3 4 5 6 7 Formative Summative Pendleton’s rules Exam Pass / Fail 1 2 3 4 5 6 7 Confidence levels of passing CSA first time Low High 1 2 3 4 5 6 7 Benchmarking Where did you assess your performance in the group? Bottom Mid Top Peer group motivation for learning No motivation High motivation 1 2 3 4 5 6 7 Experience of peer group learning Disliked Enjoyed 1 2 3 4 5 6 7 Importance of peer group in context of other learning strategies Not important (non-essential) Very important (essential) 1 2 3 4 5 6 7

138

Bibliography Britten, N. (1995). Qualitative research: Qualitative interviews in medical research. British Medical Journal, 311, 251. Bell, J. (6th ed.) (2014). Doing your research project: a guide for first-time researchers. Open University Press. Gomm, R., & Woods, P. (2002). Educational research in action. Paul Chapman Publishing. Gallagher, S., & Zahavi, D. (2nd ed.) (2012). The phenomenological mind. Routledge. Easterby-Smith, M. & Thorpe, R., & Jackson, P. (4th ed.) (2013) Management research. Sage. Charmaz, K. (2006). Constructing grounded theory: a practical guide through qualitative analysis. Sage. Flachikov, N. (2001). Learning together: peer tutoring in higher education. Routledge Farmer. Silverman, D. (3rd ed.) (2010). Doing qualitative research. Sage. Miles, M.B., & Huberman, A.M., (2nd ed.) 1994. Qualitative data analysis. Sage. Cohen, L., (7th ed.) (2011). Research methods in education. Routledge. Bryman, A. (3rd ed.) (2008) Social research methods. Oxford University Press. Howling, T. (2007). To validate a pre- and post-attachment tool to assess doctors’ perceptions concerning foundation training attachment in general practice. MSc. Greenhalgh, T., & Taylor, R. (1997). How to read a paper. Papers that go beyond numbers (qualitative research). British Medical Journal, 315,740-3. Pope, C., Ziebland, S., & Mays, N. (2000). Analysing qualitative data. British Medical Journal 320,114-6. Remedios, L., & Deshpande, A., & Harris, M. (2010) Helping international medical graduates to succeed in the nMRCGP. Education for Primary Care 21:143–4. General Medical Council. (2013). Independent Review of the Membership of the Royal College of General Practitioners (MRCGP) examination. Retrieved March 27, 2015, from http://www.gmc-uk.org/MRCGP_Final_Report__18th_September_2013.pdf_53516840.pdf

139

Pilotto, L.S., & Duncan G.F., & Anderson-Wurf J. (2007). Issues for clinicians training international medical graduates: a systematic review. Medical Journal Australia 187(4):225-228. RCGP website. MRCGP CSA. Retrieved Jan 2, 2016 from http://www.rcgp.org.uk/training-exams/mrcgp-exams-overview/mrcgp-clinical-skills-assessment-csa.aspx Roberts, C., & Atkins, S., & Hawthorne, K. (2014) Performance features in clinical skills assessment: linguistic and cultural factors in the MRCGP examination. Centre for Language, Discourse and Communication King’s College London. Dahm, M.R. (2011). Patient centred care - are international medical graduates 'expert novices'? Australian Family Physician 11:40(11):895-900. Topping, K.J., & Ehly, S.W. (2001) Peer Assisted Learning: A Framework for Consultation. Journal of Educational and Psychological Consultation 12(2):113-132. Jamieson, M., & Browne, K. (2011). How to help your international medical graduates pass the CSA. Education for Primary Care 22:178-91. CASP tool for qualitative papers. Retrieved 1 May, 2016 from http://media.wix.com/ugd/dded87_29c5b002d99342f788c6ac670e49f274.pdf. MRCGP Clinical Skills Assessment (CSA). Retrieved 2 August. 2016 from http://www.rcgp.org.uk/training-exams/mrcgp-exams-overview/mrcgp-clinical-skills-assessment-csa.aspx. Evans, J., & Benefield, P. (2001) Systematic reviews of educational research: does the medical model fit? British Educational Research Journal 27(5):27-541. Warwick, C. (2014). How international medical graduates view their learning needs for UK GP training. Education for Primary Care (25):84-90. Hobma, S., & Ram, P., & Muijtjens, A., & van der Vleuten, C., & Grol, R. (2006). Effective improvement of doctor-patient communication: a randomised controlled trial. British Journal of General Practice 56(529):580-586. Hanley, K., & Zabar, S., & Charap, J., & Nicholson, J., & Disney, L., & Kalet, A., et al. (2014). Self-assessment and goal-setting is associated with an improvement in interviewing skills. Medical Education Online 19, 24407.

140

References 1 General Medical Council. Independent Review of the Membership of the Royal College of General

Practitioners (MRCGP) examination. 2013. www.gmc- uk.org/MRCGP_Final_Report__18th_September_2013.pdf_53516840.pdf [accessed on 27/03/2015] 2 Esmail A, Roberts C. Academic performance of ethnic minority candidates and discrimination in the

MRCGP examinations between 2010 and 2012: analysis of data. British Medical Journal 2013;347:1-10. 3 Tyrer SP, Leung W-C, Smalls J, Katona C. The relationship between medical school of training,

age, gender and success in the MRCPsych examinations. Psychiatric Bulletin 2002;26:257-263. 4 Dewhurst NG, McManus C, Mollon J, Dacre JE, Vale AJ. Performance in the MRCP (UK)

Examination 2003-4: analysis of pass rates of UK graduates in relation to self-declared ethnicity and gender. BMC Medicine 2007;5:8. 5 Woolf K, Potts H, McManus I. Ethnicity and academic performance in UK trained doctors and

medical students: systematic review and meta-analysis. British Medical Journal 2011;342:d901. 6 Royal College of General Practitioners (2012) GP Training and Exams, MRCGP Exam Fees. RCGP:

London. www.rcgp.org.uk/training-exams/mrcgp-exams-overview/mrcgp-exam-fees.aspx [accessed on 27/03/2015] 7 Remedios L, Deshpande A and Harris M. Helping international medical graduates to succeed in the

nMRCGP. Education for Primary Care 2010;21:143–4. 8 RCGP website. www.rcgp.org.uk/training-exams/mrcgp-exams-overview.aspx [accessed on

13/10/2015] 9 RCGP website. MRCGP CSA. www.rcgp.org.uk/training-exams/mrcgp-exams-overview/mrcgp-

clinical-skills-assessment-csa.aspx [website accessed on 29.1.2016] 10

Shaw Q. Changes in exam and candidate selection lie behind failures. British Medical Journal 2013;347:f6442. 11

Wakeford R, Denney M, Ludka-Stempien K et al. Cross-comparison or MRCGP & MRCP (UK) in a database linkage study of 2,284 candidates taking both examinations: assessment of validity and differential performance by ethnicity. BMC Medical Education 2015;15:1. 12

Roberts C, Atkins S, Hawthorne K. Performance features in clinical skills assessment: linguistic and cultural factors in the MRCGP examination. Centre for Language, Discourse and Communication King’s College London. 2014. 13

Pope L, Hawkridge A, Simpson R. Performance in the MRCGP CSA by candidates’ gender: differences according to curriculum area. Education for Primary Care 2014;25:186-93. 14

Slowther A, Lewando Hundt GA, Purkis J, Taylor R. Experiences of non-UK-qualified doctors working within the UK regulatory framework: a qualitative study. Journal of Royal Society of Medicine 2012;105(4):157-165. 15

Hall P, Keely E, Dojeiji S, Byszewski A, Marks M. Communication skills, cultural challenges and individual support: challenges of international medical graduates in a Canadian healthcare environment. Medical Teacher 2004;28:120–5. 16

McDonnell L, Usherwood T. International medical graduates: Challenges faced in the Australian training program. Australian Family Physician 2008;37:481–4. 17

Pilotto LS, Duncan GF, Anderson-Wurf J. Issues for clinicians training international medical graduates: a systematic review. Medical Journal Australia 2007;187(4):225-228. 18

Dahm MR. Patient centred care - are international medical graduates 'expert novices'? Australian Family Physician 2011;11:40(11):895-900. 19

Knight RA. Reasons why doctors who perform well as doctors may fail the MRCGP clinical skills assessment exam. British Medical Journal 2013;347:f5662. 20

De la Croix A, Skelton J. The simulation game: an analysis of interactions between students and simulated patients. Medical Education 2013;47:49-58. 21

Kurtz S, Silverman J and Draper J. Teaching and learning communication skills in medicine. Radcliffe Publishing 2005 Second Edition; 82-103. 22

Remedios L, Deshpande A, Harris M. Helping International Medical Graduates to Success in the MRCGP. Clinical Tutor 2010;15:7–9. 23

MRCGP Clinical Skills Assessment (CSA). www.rcgp.org.uk/training-exams/mrcgp-exams-overview/mrcgp-clinical-skills-assessment-csa.aspx [accessed on 04/09/2015] 24

Warwick, C. How international medical graduates view their learning needs for UK GP training. Education for Primary Care 2014;(25):84-90.

141

25

Patterson F, Knight A, Stewart F, et al. How best to assist struggling trainees? Developing an evidence-based framework to guide support interventions. Education for Primary Care 2013;24:330-9. 26

Jamieson M, Browne K. How to help your international medical graduates pass the CSA. Education for Primary Care 2011;22:178-91. 27

Topping KJ, Ehly SW. Peer Assisted Learning: A Framework for Consultation. Journal of Educational and Psychological Consultation 2001 06/01;12(2):113-132. 28

Vygotsky LS. Mind in society: The Development of Higher Psychological Processes. Cambridge, MA, MIT Press. 1978. 29

Topping KJ. Trends in peer learning. Education Psychologist 2005;25:631–645. 30

Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development and well being. American Psychologist 2000;55:68–78. 31

Ten Cate O, Durning S. Dimensions and psychology of peer teaching in medical education. Medical Teacher 2007;29(6):547. 32

Topping K, Ehly S. Peer-Assisted Learning. 1998. 33

Long KK. Transfer form teaching to learning. Journal Education Psychology 1971;62:167–178. 34

Gaynor J, Wolking W. The effectiveness of currently enrolled student proctors in an undergraduate special education course. Journal Applied Behaviour Analysis 1974;7:263–269. 35

Johnson KR, Sulzer-Azaroff B, Maass CA. The effects of internal proctoring on examination performance in a personalized instruction course. Journal of Personalized Instruction 1976;1:113–117. 36

Annis LF. The process and effects of peer tutoring. Human Learning 1983;2:39–47. 37

Cohen L, Manion L, Morrison K. Research methods in education. 7th edition. Routledge 2011, 121.

38 Bryman A. Social research methods. 3rd Edition. Oxford University Press 2008;95.

39 Silverman D. Doing qualitative research: a practical handbook. Sage 2000

40 CASP tool website: www.casp-uk.net [accessed on 01/05/2016]

41 Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P and Stewart LA.

PRISMA-P Group: Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration & explanation. British Medical Journal 2015;349:g7647. 42

Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Annals of Internal Medicine 1997;126(5):376-80. 43

Evans J, Benefield P. Systematic reviews of educational research: does the medical model fit? British Educational Research Journal 2001;27(5):27-541. 44

Sackett DL, Richardson WS, Rosenberg W, Haynes RB. Evidence-based medicine: How to practice and teach EBM. New York: Churchill Livingston 1997. 45

MRCGP Clinical Skills Assessment (CSA). www.rcgp.org.uk/training-exams/mrcgp-exams-overview/mrcgp-clinical-skills-assessment-csa.aspx [accessed on 29.08.2016] 46

Guyatt GH, Sackett DL, Sinclair JC, Hayward R, Cook DJ, Cook RJ. Users' guides to the medical literature. IX. A method for grading health care recommendations. Evidence-Based Medicine Working Group. JAMA 1995;274(22):1800-1804. 47

Kononowicz AA, Woodham L, Georg C, Edelbring S, Stathakarou N, Davies D et al. Virtual patient simulations for health professional education (protocol). Cochrane Database of Systematic Review 2016. Issue 5. 48

CASP tool for qualitative papers: media.wix.com/ugd/dded87_29c5b002d99342f788c6ac670e49f274.pdf [accessed on 01/05/2016] 49

Duncan GF, Gilbey D. Cultural and communication awareness for general practice registrars who are international medical graduates: a project of CoastCityCountry Training. Australian Journal Rural Health 2007;02:15(1):52-58 7p. 50

Ramaswamy R, Williams A, Clark EM, Kelley AS. Communication Skills Curriculum for Foreign Medical Graduates in an Internal Medicine Residency Program. Journal Of The American Geriatrics Society 2014;62(11):2153-2158. 51

Hobma S, Ram P, Muijtjens A, van der Vleuten C, Grol R. Effective improvement of doctor-patient communication: a randomised controlled trial. British Journal of General Practice 2006;56(529):580-586. 52

McMillan R, Cameron N, Power A. Pilot study to develop and test a generic peer feedback instrument to review videotaped consultations in primary care. Quality Primary Care 2011;11:19(6):347-354. 53

Burgess A, McGregor D, Mellis C. Medical students as peer tutors: a systematic review. BMC Medical Educator 2014;14:115.

142

54

Yu T, Wilson NC, Singh PP, Lemanu DP, Hawken SJ, Hill AG. Medical students-as-teachers: a systematic review of peer-assisted teaching during medical school. Advance Medical Educator Practitioner 2011;2:157-172. 55

Cave J, Washer P, Sampson P, Griffin M, Noble L. Explicitly linking teaching and assessment of communication skills. Medical Teacher 2007;05:29(4):317-322. 56

Nestel D, Kidd J. Peer tutoring in patient-centred interviewing skills: experience of a project for first-year students. Medical Teacher 2003;07:25(4):398-403. 57

Perera J, Mohamadou G, Kaur S. The Use of Objective Structured Self-Assessment and Peer-Feedback (OSSP) for Learning Communication Skills: Evaluation Using a Controlled Trial. Advances in Health Sciences Education 2010;15(2):185-193. 58

Williams B, Reddy P. Does peer-assisted learning improve academic performance? A scoping review. Nurse Education Today 2016;07(42):23-29. 59

Burgess A, Black K, Chapman R, Clark T, Mellis C. Clinical teaching skills for medical students: our future educators. Clinical teacher 2012;9:312-316. 60

Cushing A, Abbott S, Lothian D, Hall A, Westwood OMR. Peer feedback as an aid to learning--what do we want? Feedback. When do we want it? Now! Medical Teacher 2011;33(2):e105-e112. 61

Chou CL, Masters DE, Chang A, Kruidering M, Hauer KE. Effects of longitudinal small-group learning on delivery and receipt of communication skills feedback. Medical Educator 2013;11:47(11):1073-1079. 62

Cave J, Washer P, Sampson P, Griffin M, Noble L. Explicitly linking teaching and assessment of communication skills. Medical Teacher 2007;05:29(4):317-322. 63

Hulsman RL, van der Vloodt J. Self-evaluation and peer-feedback of medical students’ communication skills using a web-based video annotation system. Exploring content and specificity. Patient Education Counsellor 2015;98(3):356-363. 64

Hanley K, Zabar S, Charap J, Nicholson J, Disney L, Kalet A, et al. Self-assessment and goal-setting is associated with an improvement in interviewing skills. Medical Educator Online 2014;07/23;19:24407. 65

Perera J, Mohamadou G, Kaur S. The Use of Objective Structured Self-Assessment and Peer-Feedback (OSSP) for Learning Communication Skills: Evaluation Using a Controlled Trial. Advances in Health Sciences Education 2010;15(2):185-193. 66

Rees C, Shepherd M. Students' and assessors' attitudes towards students' self-assessment of their personal and professional behaviours. Medical Education 2005;39(1):30-39. 67

Evans AW, Leeson RMA, Newton-John TRO. The influence of self-deception and impression management on surgeons’ self-assessment scores. Medical Education 2002;36:1095. 68

Rees CE. Self-assessment scores and gender. Medical Education 2003;37:572–3. 69

Fitzgerald JT, White CS, Gruppen LD. A longitudinal study of self-assessment accuracy. Medical Education 2003;37:645–9. 70

Britten N, Jones R, Murphy E, et al. Qualitative research methods in general practice and primary care. Family Practice 1995;12(1):104-114. 71

Denkin NK, Lincoln YS. Handbook of qualitative research. London: Sage,1994. 72

Britten N. Qualitative Research: Qualitative interviews in medical research. British Medical Journal 1995;311:251. 73

Charmaz K. Constructing grounded theory: A practical guide through qualitative analysis. Sage Publications 2006;30-31. 74

Bryman A. Social Research Methods. Oxford University Press 2008 (3rd

edition), 475. 75

Mays N, Pope C. Assessing quality in qualitative research. British Medical Journal 2000;320(7226):50-52. 76

Baker S, Edwards R. How many qualitative interviews is enough? Expert voices and early career reflections on sampling and cases in qualitative research. Review paper. National Centre for Research Methods 2012. 77

Patterson F, Knight A, Stewart F, et al. How best to assist struggling trainees? Developing an evidence-based framework to guide support interventions. Education for Primary Care 2013;24:330-9. 78

Cohen L, Manion L, Morrison K. Research Methods in Education. Routledge 2011;7th Edition,158. 79

Ibid 74, 155. 80

Contact details of programme directors and PCMEs. www.nwpgmd.nhs.uk/content/programme-directors-primary-care-medical-educators [website accessed 31.5.2016] 81

Ibid 74, 198-199. 82

Pope C, Ziebland S, Mays N. Analysing qualitative data. British Medical Journal 2000;320:114-6.

143

83

Van Manen M. Researching lived experience: Human science for an action sensitive pedagogy. Ontario, Canada: University of Western Ontario, 1990. 84

Cresswell J, Hanson W, Clark V. Qualitative Research Designs: Selection and Implementation. The Counselling Psychologist 2007;35:252-255. 85

MaxQDA is qualitative analytical software tool that can be downloaded at www.maxqda.com [website accessed on 9.9.2016] 86

Greenhalgh T, Taylor R. How to read a paper. Papers that go beyond numbers (qualitative research). British Medical Journal 1997;315:740-3. 87

Ibid 74, 51. 88

Data Protection Act, 1998. www.gov.uk/data-protection/the-data-protection-act [website accessed on 22.9.2016] 89

Freedom of Information Act, 2000. www.legislation.gov.uk/ukpga/2000/36/pdfs/ukpga_20000036_en.pdf [website accessed 22.9.2016] 90

GoTranscript is an online transcription service used by researchers and they specialise in interview transcriptions. www.gotranscript.com/transcription-services/interview-transcription [website accessed 7.9.2016] 91

UCLan Research Code of Conduct. www.uclan.ac.uk/students/research/files/Code_of_Conduct_for_Research_2015.pdf [accessed on 22.9.2016] 92

Pendleton D, Schofield T, Tate P. The consultation: an approach to learning and teaching. Oxford University Press, 1984:68–71. 93

Group dynamics – how group size affects function. www.faculty.londondeanery.ac.uk/e-

learning/small-group-teaching/group-dynamics-how-group-size-affects-function [website accessed on 22.10.2016] 94

Belbin M. Team roles at work. Butterworth-Heinemann, 1993. 95

Devi V, Mandal T, Kodidela S, Pallath V. Integrating students' reflection-in-learning and examination performance as a method for providing educational feedback. Journal of Postgraduate Medicine 2012;58;4:270-274. 96

Zimmerman BJ. Self-Regulated Learning and Academic Achievement: An Overview. Educational

Psychologist 1990;25(1):3-17. 97

Rees C, Shepherd M. Students' and assessors' attitudes towards students' self-assessment of their personal and professional behaviours. Medical Education 2005;39(1):30-39. 98

Tracey J, Arroll B, Barham P, Richmond D. The validity of general practitioners’ self-assessment of knowledge: cross sectional study British Medical Journal 1997;315:1426. 99

Dory V, Beaulieu M-D, Pestiaux D, Pouchain D, Gay B, Rocher G, Boucher L. The development of self-efficacy beliefs during general practice vocational training: an exploratory study Medical Teacher 2009;31:1:39-44. 100

Feichtner, SB. Davis, EA. Why some groups fail: a survey of students’ experiences with learning groups. The Organizational Behaviour Review 1985;9(4):58-73. 101

Gibbs G. Learning in teams: a student manual. Oxford: Oxford Brookes University, 1994. 102

Whitman NA. Peer teaching: to teach is to learn twice, ASHE-ERIC Higher Education. Washington DC, ERIC Clearinghiudse on Higher Education. 1988. 103

Weiss V, Needleman R. To teach is to learn twice. Resident teachers learn more. Archiver of Pediatric and Adolescent Medicine 1998 152:190–192. 104

Klaber B. Effective feedback: an essential skill. Postgraduate Medicine Journal 2012;88(1038):187-8.