Educating for the Educating for the Responsible Conduct of Responsible Conduct of Research in Behavioral Research in Behavioral

HealthHealth

Terry DiLorenzoTerry DiLorenzoJill Becker-FeigelesJill Becker-FeigelesMargaret GibelmanMargaret Gibelman

Yeshiva University, New York, NYYeshiva University, New York, NY

Dr. Margie GibelmanDr. Margie GibelmanWurzweiler School of Social Work Wurzweiler School of Social Work

Yeshiva UniversityYeshiva University

Behavioral HealthBehavioral Health

The application of medical, psychiatric, The application of medical, psychiatric, psychological, social work, and psychological, social work, and educational principles and practices to the educational principles and practices to the diagnosis and treatment of mental health diagnosis and treatment of mental health and substance abuse disordersand substance abuse disorders Includes: Counseling, Psychology, Psychiatry, Includes: Counseling, Psychology, Psychiatry,

Psychiatric Nursing, Social WorkPsychiatric Nursing, Social Work

The ProblemThe Problem

Emphasis on educating for RCR has Emphasis on educating for RCR has focused primarily on biomedical sciences;focused primarily on biomedical sciences;

Behavioral health professionals are Behavioral health professionals are increasingly contenders for public grants increasingly contenders for public grants and contracts and producers of research;and contracts and producers of research;

Behavioral health research typical involves Behavioral health research typical involves interactions with vulnerable populations.interactions with vulnerable populations.

The Problem (cont’d)The Problem (cont’d)

Little is known about RCR education in Little is known about RCR education in behavioral health;behavioral health;

Little is known about the use of mentoring, Little is known about the use of mentoring, collaboration, and internal (school-specific) collaboration, and internal (school-specific) oversight to review and monitor research;oversight to review and monitor research;

Even less is known about relationship Even less is known about relationship between behavioral health professional between behavioral health professional schools and University IRBs.schools and University IRBs.

Goals of the ResearchGoals of the Research Describe research review and monitoring Describe research review and monitoring

mechanisms in behavioral health education mechanisms in behavioral health education programs;programs;

Examine perceptions about adequacy of Examine perceptions about adequacy of procedures regarding research review and procedures regarding research review and monitoring;monitoring;

Examine mentoring and other forms of Examine mentoring and other forms of collaboration to promote socialization to research collaboration to promote socialization to research norms;norms;

Determine extent to which RCR content is Determine extent to which RCR content is included in curricula or provided through other included in curricula or provided through other mechanisms.mechanisms.

MethodsMethods

Mixed method approach:Mixed method approach:

Phase 1: SurveyPhase 1: Survey Mail/on-line survey of deans/directorsMail/on-line survey of deans/directors

Phase 2: InterviewPhase 2: Interview Qualitative interviews with subsample of survey Qualitative interviews with subsample of survey

respondentsrespondents

Phase 1: ResultsPhase 1: Results

Response RateResponse Rate

Sent*Sent* Completed % Completed %NursingNursing 96 96 49 49 51.0 51.0PsychologyPsychology 548548 141 141 25.7 25.7PsychiatryPsychiatry 127127 3131 24.4 24.4CounselingCounseling 178178 4040 22.5 22.5Social WorkSocial Work 177177 6666 37.3 37.3TotalTotal 1126 1126 327 327 29.0 29.0

*excludes undeliverable surveys*excludes undeliverable surveys

Is formal education in RCR required of Is formal education in RCR required of those conducting research?those conducting research?

YesYes NoNo NotNot

SureSure

NursingNursing 85.785.7 2.02.0 2.02.0

PsychologyPsychology 87.287.2 9.29.2 0.70.7

PsychiatryPsychiatry 64.564.5 3.23.2 3.23.2

CounselingCounseling 75.075.0 17.517.5 2.52.5

Social WorkSocial Work 77.377.3 13.613.6 1.51.5

What components of RCR are What components of RCR are covered in curricula?covered in curricula?

Confidentiality Informed Human Conflict of Confidentiality Informed Human Conflict of Consent Subjects Consent Subjects

InterestInterest

NursingNursing 81.6 81.6 81.6 81.6 77.677.6 65.3 65.3

Psychology 80.9Psychology 80.9 80.9 80.9 83.0 63.183.0 63.1

PsychiatryPsychiatry 45.2 45.2 54.8 54.8 51.651.6 51.6 51.6

CounselingCounseling 75.0 75.0 72.5 72.5 75.075.0 60.0 60.0

Social Work 81.8Social Work 81.8 75.8 75.8 81.881.8 57.6 57.6

Faculty familiarity with standards of Faculty familiarity with standards of responsible research conductresponsible research conduct

Not at Somewhat Familiar Very Not at Somewhat Familiar Very

All Familiar Familiar All Familiar Familiar FamiliarFamiliar

NursingNursing 0.0 0.0 6.1 6.1 24.5 24.5 57.1 57.1

Psychology 0.0 6.4 27.0 63.8Psychology 0.0 6.4 27.0 63.8

Psychiatry 0.0 6.5 19.4 45.2Psychiatry 0.0 6.5 19.4 45.2

Counseling 0.0 10.0 30.0 55.0Counseling 0.0 10.0 30.0 55.0

Social Work 1.5 9.1 24.2 57.6Social Work 1.5 9.1 24.2 57.6

Student familiarity with standards of Student familiarity with standards of responsible research conductresponsible research conduct

Not at Somewhat Familiar Very Not at Somewhat Familiar Very All Familiar Familiar FamiliarAll Familiar Familiar Familiar

NursingNursing 0.0 0.0 16.3 16.3 40.8 30.6 40.8 30.6Psychology 0.0 17.7 36.9 41.8Psychology 0.0 17.7 36.9 41.8Psychiatry* 14.3 28.6 23.8 23.8Psychiatry* 14.3 28.6 23.8 23.8Counseling 5.0 17.5 50.0 9.0Counseling 5.0 17.5 50.0 9.0Social Work 1.5 27.3 47.0 13.6Social Work 1.5 27.3 47.0 13.6

*Fellows*Fellows

Phase 1: ConclusionsPhase 1: Conclusions

Formal procedures to educate Formal procedures to educate students/faculty in RCR vary considerablystudents/faculty in RCR vary considerably

Most respondents reported that Most respondents reported that procedures to socialize students/faculty to procedures to socialize students/faculty to RCR could be improved RCR could be improved

Phase 2Phase 2

MethodsMethods

Interview developmentInterview development

Based on Phase 1 results, pool of questions Based on Phase 1 results, pool of questions generated by research team;generated by research team;

Additional questions generated by experts in Additional questions generated by experts in selected disciplines;selected disciplines;

Pilot testing and refinement of instrument.Pilot testing and refinement of instrument.

InterviewInterview What procedures are used to review and What procedures are used to review and

monitor faculty and student research?monitor faculty and student research?

Do you have your own internal review Do you have your own internal review procedures?procedures?

Are there procedures in place for ensuring the Are there procedures in place for ensuring the accuracy of research findings prior to accuracy of research findings prior to publication?publication?

What is the level of participation of your faculty What is the level of participation of your faculty on university IRBs?on university IRBs?

Interview (Cont’d)Interview (Cont’d)

How adequate are research review and How adequate are research review and monitoring procedures perceived to be? Have monitoring procedures perceived to be? Have any issues/problems arisen?any issues/problems arisen?

To what extent does the curriculum include RCR To what extent does the curriculum include RCR content? content?

What efforts are made to provide What efforts are made to provide students/junior faculty with mentors to guide students/junior faculty with mentors to guide research development?research development?

Methods (Cont’d)Methods (Cont’d)

SampleSample Random Selection of those agreeing to be Random Selection of those agreeing to be

interviewedinterviewed agreed agreed interviewed* interviewed*

NursingNursing 4 4 11Psychology 11Psychology 11 55PsychiatryPsychiatry 3 3 00CounselingCounseling 4 4 11Social Work 13Social Work 13 99

* Two interviews not included in content analysis* Two interviews not included in content analysis

Content Analysis (Preliminary)Content Analysis (Preliminary)

First level: 150 codes in 10 major First level: 150 codes in 10 major categories (e.g., formal review, categories (e.g., formal review, informal review, advising)informal review, advising)

Pattern Coding: grouping into meta-Pattern Coding: grouping into meta-codes (e.g., advising as informal codes (e.g., advising as informal review)review)

Hypothesis testingHypothesis testing

First level: 150 codes in 10 major First level: 150 codes in 10 major categories (e.g., formal review, categories (e.g., formal review, informal review, advising)informal review, advising)

Pattern Coding: grouping into meta-Pattern Coding: grouping into meta-codes (e.g., advising as informal codes (e.g., advising as informal review)review)

Hypothesis testingHypothesis testing

Results (Preliminary)Results (Preliminary)

Formal Review ProcessesFormal Review Processes

All had IRB or HSCAll had IRB or HSC

IRB review process is “front-loaded”IRB review process is “front-loaded” Rigorous examination of proposal—human subjects, Rigorous examination of proposal—human subjects,

consent, procedures, and less frequently, consent, procedures, and less frequently, methodologymethodology

No monitoring of on-going research, except annual No monitoring of on-going research, except annual progress report where required by IRB/HSCprogress report where required by IRB/HSC

Limited review of accuracy of FindingsLimited review of accuracy of Findings

Formal Review ProcessesFormal Review Processes

Consideration of scienceConsideration of science 3 had a formal assessment of science3 had a formal assessment of science (4 had informal assessment of science)(4 had informal assessment of science)

IRB ConstitutionIRB Constitution 1 had separate panels for behavioral vs. biomedical1 had separate panels for behavioral vs. biomedical 7 always had rep from discipline on IRB7 always had rep from discipline on IRB

Student ResearchStudent Research Doctoral research reviewed by IRB/HSCDoctoral research reviewed by IRB/HSC Not all Master’s projects/research for courses formally reviewedNot all Master’s projects/research for courses formally reviewed

Informal Review ProcessesInformal Review Processes

FacultyFaculty 4 require signatures of head of unit or 4 require signatures of head of unit or

deandean

StudentStudent Doctoral research reviewed by Doctoral research reviewed by

advisors/committeesadvisors/committees Master’s/other research reviewed by advisors Master’s/other research reviewed by advisors

or school’s representative to IRBor school’s representative to IRB

Education/TrainingEducation/Training

Faculty Faculty 7 required formal (University-based) training for faculty, primarily 7 required formal (University-based) training for faculty, primarily

on-line on-line 2 had formal mentoring—tenure committee, associate dean2 had formal mentoring—tenure committee, associate dean

Student Student 7 required formal training (University-based) for all students 7 required formal training (University-based) for all students

conducting research, primarily on-line conducting research, primarily on-line 2 had dedicated RCR courses for doctoral students2 had dedicated RCR courses for doctoral students RCR primarily infused into other courses—Research, Ethics, RCR primarily infused into other courses—Research, Ethics,

Practice Practice

MisconductMisconduct

RCR training relates to allegations RCR training relates to allegations

No allegations of misconduct reported where No allegations of misconduct reported where formal education in RCR is required prior to formal education in RCR is required prior to conducting researchconducting research

2 reports of misconduct and 2 authorship 2 reports of misconduct and 2 authorship disputes in 2 of 9 where formal RCR training disputes in 2 of 9 where formal RCR training is not required prior to conducting researchis not required prior to conducting research

Respondents’ RecommendationsRespondents’ Recommendations

Recommendation for separate ethics course that has Recommendation for separate ethics course that has focus on ethical conduct in researchfocus on ethical conduct in research ““Well I believe in a separate ethics course. I think that this whole Well I believe in a separate ethics course. I think that this whole

business of infusing ethics throughout the curriculum is a bad business of infusing ethics throughout the curriculum is a bad idea…because there would be an expectation that every single idea…because there would be an expectation that every single faculty member knows a lot about the field and that simply isn’t faculty member knows a lot about the field and that simply isn’t true. And if you look in the backgrounds of people who have true. And if you look in the backgrounds of people who have master’s degrees in social work in this country, 90% of them master’s degrees in social work in this country, 90% of them never had an ethics course and wouldn’t know….so I think it’s never had an ethics course and wouldn’t know….so I think it’s serious. I think it’s a hugely serious issue.”serious. I think it’s a hugely serious issue.”

Connecting RCR in classroom to practiceConnecting RCR in classroom to practice ““Bridging classroom to practice conduct together, in the topic of Bridging classroom to practice conduct together, in the topic of

research, or in the topic of clinical work, is a gap that I’m not yet research, or in the topic of clinical work, is a gap that I’m not yet aware of really good models for addressing at a doctoral level.”aware of really good models for addressing at a doctoral level.”

Respondents’ RecommendationsRespondents’ Recommendations

IRB follow-up on protocolsIRB follow-up on protocols ““You know…the IRB receives the proposal, approves certain You know…the IRB receives the proposal, approves certain

procedures, and we just assume that those are being carried procedures, and we just assume that those are being carried out. So [it might not be a bad idea to select people to] out. So [it might not be a bad idea to select people to] demonstrate that [they’re] doing what [they] said [they] were demonstrate that [they’re] doing what [they] said [they] were going to be doing”going to be doing”

“… “… As someone who has reviewed probably 100 IRB As someone who has reviewed probably 100 IRB applications, you don’t ever know what happens. And there are applications, you don’t ever know what happens. And there are some people who, when they get feedback, …take awhile ..to some people who, when they get feedback, …take awhile ..to respond… so you don’t know did something happen? Did they respond… so you don’t know did something happen? Did they just go ahead anyway? Did they misunderstand? You just don’t just go ahead anyway? Did they misunderstand? You just don’t know and even for students, they don’t have to report in with the know and even for students, they don’t have to report in with the IRB when they complete their process.”IRB when they complete their process.”

ConclusionsConclusions Research review primarily at proposal stage; focuses on Research review primarily at proposal stage; focuses on

human subjects, informed consent, methods. human subjects, informed consent, methods.

Limited attention to scientific merit, oversight of on-going Limited attention to scientific merit, oversight of on-going research, and accuracy of findings beyond the review by research, and accuracy of findings beyond the review by advisor/committee.advisor/committee.

IRB constitution varies, some disciplines lack appropriate IRB constitution varies, some disciplines lack appropriate representation.representation.

Conclusions (Cont’d)Conclusions (Cont’d) Many require training of all engaging in research. Many require training of all engaging in research.

Informal review rarely conducted for faculty; informal Informal review rarely conducted for faculty; informal review for doctoral research more extensive than for review for doctoral research more extensive than for master’s projects of classroom research.master’s projects of classroom research.

RCR content included in most curricula, but RCR content included in most curricula, but methods/courses vary.methods/courses vary.