1

Training for Newer Training for Newer Training DirectorsTraining Directors

Wayne Siegel, PhD, ABPPWayne Siegel, PhD, ABPPJim Besyner, Ph.D., ABPPJim Besyner, Ph.D., ABPP

Acknowledgments/DisclaimerAcknowledgments/Disclaimer•• Some materials browed from APPIC and CoASome materials browed from APPIC and CoA’’s training s training

for TDs.for TDs.•• The information presented is from the practical The information presented is from the practical

experiences of the presenters and is not the official voice experiences of the presenters and is not the official voice or opinion APPIC, CoA or the VAPTC.or opinion APPIC, CoA or the VAPTC.

2

ObjectivesObjectives1.1. Participants will become familiar with the major tasks Participants will become familiar with the major tasks

involved in running accredited psychology internship involved in running accredited psychology internship and residency programs.and residency programs.

2.2. Participants will become familiar with the unique Participants will become familiar with the unique aspects of running psychology training programs within aspects of running psychology training programs within the VA system. the VA system.

3.3. Participants will become familiar with resources Participants will become familiar with resources available to help them run accredited psychology available to help them run accredited psychology internship and residency programs.internship and residency programs.

Being a training director can feel overwhelming at times

3

AgendaAgenda–– What do participants wantWhat do participants want……..–– Importance of consultationImportance of consultation–– Serving multiple mastersServing multiple masters–– OAA and VA Specific IssuesOAA and VA Specific Issues–– Accreditation overview and process Accreditation overview and process –– How CoA How CoA

decides?decides?–– Key concepts in accreditationKey concepts in accreditation

»» CoA's G&PCoA's G&P»» Philosophy goals and objectivesPhilosophy goals and objectives»» Evaluation of goals and objectivesEvaluation of goals and objectives»» Minimal levels of achievementMinimal levels of achievement»» Program evaluationProgram evaluation»» How to do planned sequential and cumulativeHow to do planned sequential and cumulative»» Program resourcesProgram resources»» Cultural and individual diversityCultural and individual diversity»» Trainee and staff relationsTrainee and staff relations»» Self assessment and quality enhancementSelf assessment and quality enhancement

–– TD Roles and duties (yearly tasks)TD Roles and duties (yearly tasks)–– Planning the training yearPlanning the training year–– Recruitment and SelectionRecruitment and Selection–– Understanding interns/residents adjustmentUnderstanding interns/residents adjustment–– Due process and GrievanceDue process and Grievance–– Writing a self studyWriting a self study

4

Importance of ConsultationImportance of Consultation•• Mentorship Mentorship

–– APPICAPPIC–– AVAPLAVAPL–– New New –– VAPTCVAPTC

•• ConsultationConsultation–– VAPTC Exec Committee VAPTC Exec Committee

((http://http://vaww.psychologytrainingcouncil.wss.va.gov/defvaww.psychologytrainingcouncil.wss.va.gov/default.aspxault.aspx))

VA TD Mail Groups (VHA Psychology Internship VA TD Mail Groups (VHA Psychology Internship Directors) (VHA Psychology Postdoctoral Directors)Directors) (VHA Psychology Postdoctoral Directors)

–– OAA (OAA (Robert.Zeiss@VA.GOVRobert.Zeiss@VA.GOV))•• Person connection with other TDs (APPIC, AVAPL, Person connection with other TDs (APPIC, AVAPL,

VAPTC membership Meeting)VAPTC membership Meeting)

5

Optimal Clinical Optimal Clinical TrainingTraining

CoA’s G&PVA Rule and Policies

99

OAA and VA Specific IssuesOAA and VA Specific Issues•• Office of Academic Affiliations (OAA)Office of Academic Affiliations (OAA)

–– Robert Zeiss, Ph.D.Robert Zeiss, Ph.D.–– vaww.va.gov/oaavaww.va.gov/oaa//–– http://http://www.psychologytraining.va.govwww.psychologytraining.va.gov//

•• OAA OAA -- Supervision of Associated Health TraineesSupervision of Associated Health Trainees•• Affiliation Agreements (national)Affiliation Agreements (national)•• Allocation of Associated Health Profession TraineesAllocation of Associated Health Profession Trainees•• Privacy and Security issuesPrivacy and Security issues•• Release time for Release time for TDTD’’ss•• VAPTCVAPTC

6

APA Accreditation IssuesAPA Accreditation Issues

7

The Process of AccreditationThe Process of Accreditation1.1. Program application for initial review Program application for initial review OR OR Assignment Assignment

to a review cycle (accredited programs)to a review cycle (accredited programs)2.2. Submission of the SelfSubmission of the Self--Study Study 3.3. CoA Preliminary review CoA Preliminary review -- feedback feedback –– request for request for

additional informationadditional information4.4. Approval for site visit or request for additional Approval for site visit or request for additional

information (new programs can be deemed not ready information (new programs can be deemed not ready for a site visit) for a site visit)

5.5. Selection of site visitorsSelection of site visitors6.6. Site visitSite visit7.7. Site visit reportSite visit report8.8. ProgramProgram’’s response to the site visit reports response to the site visit report9.9. Review and decision by CoAReview and decision by CoA

Accredit 3/5/7 years w/ requests for SS AR, FlagsAccredit 3/5/7 years w/ requests for SS AR, FlagsDFI, DFC DFI, DFC

14

Key Concepts to AccreditationKey Concepts to Accreditation•• What is the programWhat is the program’’s s philosophy and modelphilosophy and model and does and does

this fit within the home institution (Domain A)?this fit within the home institution (Domain A)?

•• How does the program link the How does the program link the science and practicescience and practice of of professional psychology (Domain B)?professional psychology (Domain B)?

•• How does this How does this model lead to goalsmodel lead to goals, objectives and , objectives and measurable competencies (Domain B)?measurable competencies (Domain B)?

8

15

•• How well does the program How well does the program prepare studentsprepare students to achieve to achieve those competencies as reflected through those competencies as reflected through outcome dataoutcome data ? ?

•• How well does the program prepare students to be How well does the program prepare students to be competent in those areas designated by the G&P competent in those areas designated by the G&P ((Intervention/Assessment/Diversity/Ethics/Supervision/CIntervention/Assessment/Diversity/Ethics/Supervision/Consultationonsultation)?)?

•• Does the program have the Does the program have the resourcesresources to achieve its to achieve its goals (Domain C)?goals (Domain C)?

•• Does the program educate and train students to become Does the program educate and train students to become professionals in a professionals in a diverse and multiculturaldiverse and multicultural society (Domain D)? society (Domain D)?

16

•• How does the program (faculty and students) engage in How does the program (faculty and students) engage in reflective reflective selfself--examination and enhancementexamination and enhancement (Domain (Domain F)?F)?

•• Does the program make sure all of its Does the program make sure all of its public information public information is accurateis accurate (Domain G)?(Domain G)?

•• How does the program make sure that all How does the program make sure that all studentsstudents are are informedinformed of the programof the program’’s policies and procedures s policies and procedures (Domains A, E, and G)?(Domains A, E, and G)?

9

17

Domain ADomain AEligibilityEligibility

As a prerequisite for accreditation, the As a prerequisite for accreditation, the programprogram’’s s purposepurpose must be within the scope of the accrediting must be within the scope of the accrediting body and must be pursued in an institutional body and must be pursued in an institutional setting setting

appropriate for the education and training of appropriate for the education and training of professional psychologistsprofessional psychologists..

18

–– A1. Program offers doctoral A1. Program offers doctoral education and training in education and training in professional psychologyprofessional psychology including preparation for including preparation for practice.practice.

–– A2. Program A2. Program sponsored by an institution of higher sponsored by an institution of higher educationeducation that is accredited by a nationally recognized that is accredited by a nationally recognized regional accrediting body in the U.S. regional accrediting body in the U.S. oror is a member in is a member in good standing of the Association of Universities and good standing of the Association of Universities and Colleges of Canada*.Colleges of Canada*.

10

19

•• A3. The program:A3. The program:–– Is an Is an integral part of the missionintegral part of the mission of the academic unit of the academic unit

in which it residesin which it resides

–– Is represented in the institutionIs represented in the institution’’s s budgetbudget

–– Has Has sufficient studentssufficient students and necessary facilities to and necessary facilities to ensure meaningful interaction, support, and ensure meaningful interaction, support, and socializationsocialization

20

(Domain A)(Domain A)

•• Program requires of each intern/PD the Program requires of each intern/PD the equivalent of 1 equivalent of 1 year of fullyear of full--timetime training to be completed in no fewer than training to be completed in no fewer than 12 months (10 months for school psychology) and no 12 months (10 months for school psychology) and no more than 24 months. more than 24 months.

11

21

(Domain A)(Domain A)

•• A5. Program engages in actions that indicate A5. Program engages in actions that indicate respect for respect for and understanding of cultural and individual diversityand understanding of cultural and individual diversity as as reflected in its reflected in its recruitmentrecruitment and and retentionretention policies for policies for faculty and students, faculty and students, curriculumcurriculum and field placements, and field placements, nondiscriminatory policies and operating conditions, and nondiscriminatory policies and operating conditions, and avoidance of actions that restrict program access on avoidance of actions that restrict program access on grounds irrelevant to success. The definition of diversity grounds irrelevant to success. The definition of diversity includes but is not limited to:includes but is not limited to:

AgeAge DisabilitiesDisabilitiesEthnicityEthnicity GenderGenderIdentityIdentity LanguageLanguageNational originNational origin RaceRaceReligionReligion CultureCultureSexual orientationSexual orientation Social economic statusSocial economic status

22

•• A6. Formal written policies are available concerning: A6. Formal written policies are available concerning: –– Selection Selection –– Practicum and academic preparation requirements Practicum and academic preparation requirements –– Financial and administrative assistance Financial and administrative assistance –– Performance evaluation Performance evaluation –– Feedback, advisement, retention, and termination Feedback, advisement, retention, and termination –– Due process and grievance proceduresDue process and grievance procedures

12

CC--11(a). Accreditation Process for Postdoctoral 11(a). Accreditation Process for Postdoctoral ResidenciesResidencies

•• Format #1 Format #1 -- Traditional Practice ProgramsTraditional Practice Programs•• Format #2 Format #2 -- Specialty Practice ProgramsSpecialty Practice Programs•• Format #3 Format #3 -- Integrated Practice Integrated Practice

Clinical NeuropsychologyClinical NeuropsychologyClinical HealthClinical HealthRehabilitation PsychologyRehabilitation PsychologyClinical Child PsychologyClinical Child PsychologyForensic PsychologyForensic PsychologyFamily PsychologyFamily Psychology

24

Domain BDomain BProgram Philosophy, Objectives, and Program Philosophy, Objectives, and

CurriculumCurriculum

The program has a clearly specified The program has a clearly specified philosophyphilosophy of education of education and training, compatible with the mission of the and training, compatible with the mission of the sponsor sponsor institutioninstitution, and , and appropriate to the science appropriate to the science and practice of and practice of

psychology. The programpsychology. The program’’s education and training model and s education and training model and its curriculum plan are consistent with this philosophy.its curriculum plan are consistent with this philosophy.

13

25

•• B1. Program publicly states a B1. Program publicly states a philosophy and model philosophy and model of trainingof training consistent with the sponsoring institutionconsistent with the sponsoring institution’’s s mission that emphasizes:mission that emphasizes:

–– Integration of science and practiceIntegration of science and practice–– Training that is Training that is sequential, cumulative, and sequential, cumulative, and

graded in complexitygraded in complexity

•• B2. Program specifies B2. Program specifies objectives in terms of objectives in terms of competencies expected of graduates consistent withcompetencies expected of graduates consistent with::

–– ProgramProgram’’s philosophy and training models philosophy and training model–– Substantive area of professional psychology that is Substantive area of professional psychology that is

representedrepresented–– An understanding of legal, ethical, and quality An understanding of legal, ethical, and quality

assurance principlesassurance principles

26

•• B3. Internship is an B3. Internship is an organized, structured, and organized, structured, and programmed sequenceprogrammed sequence of supervised training of supervised training experiences of greater depth and intensity than experiences of greater depth and intensity than practicum training that includes the following: practicum training that includes the following: –– Training Training activities are structuredactivities are structured in terms of in terms of

sequence, intensity, duration, and frequency sequence, intensity, duration, and frequency –– Primary training method is Primary training method is experientialexperiential including including

socialization into the profession socialization into the profession –– Supervision is regularly scheduledSupervision is regularly scheduled, sufficient, and , sufficient, and

defined as a minimum defined as a minimum of 4 hours/week, at least 2 of 4 hours/week, at least 2 hourshours of which is individual supervision of which is individual supervision

–– Training activities address the delivery of Training activities address the delivery of psychological services and include professional psychological services and include professional conduct, ethics, and standards conduct, ethics, and standards

–– Administrative structure Administrative structure and process which controls and process which controls and coordinates training activities and processes and coordinates training activities and processes

–– Designated Designated leader(sleader(s) doctoral psychologist licensed in ) doctoral psychologist licensed in the programthe program’’s jurisdiction who is primarily responsible s jurisdiction who is primarily responsible for the program for the program

14

27

•• B4. Program requires that all interns demonstrate B4. Program requires that all interns demonstrate intermediate to advanced skill, competency, and intermediate to advanced skill, competency, and knowledgeknowledge in: in: –– Theories and methods of Theories and methods of assessment/diagnosisassessment/diagnosis and and

effective intervention including empirically supported effective intervention including empirically supported treatments treatments

–– Theories and/or methods of Theories and/or methods of consultation, evaluation, consultation, evaluation, and supervision (IR and supervision (IR --direct experience not required)direct experience not required)

–– Strategies of Strategies of scholarly inquiryscholarly inquiry–– Issues of Issues of cultural and individual diversitycultural and individual diversity relevant to relevant to

all of above all of above

•• Residents (Residents (AdvancedAdvanced knowledge)knowledge)–– Theories and effective methods of psychological Theories and effective methods of psychological

assessment, diagnosis, and interventions assessment, diagnosis, and interventions –– Consultation, program evaluation, supervision, and/or Consultation, program evaluation, supervision, and/or

teaching teaching –– Strategies of scholarly inquiry Strategies of scholarly inquiry –– Organization, management, and administration Organization, management, and administration issues issues

as they relate to service delivery and practice, as they relate to service delivery and practice, training, and research training, and research

–– Professional conductProfessional conduct, ethics, law, and related , ethics, law, and related standards standards

–– Issues of cultural and individual diversity relevant to Issues of cultural and individual diversity relevant to all of the above all of the above

28

15

29

•• B5. Program promotes the B5. Program promotes the integration of practice and integration of practice and scholarly inquiry scholarly inquiry and takes responsibility for the training and takes responsibility for the training experiences of interns including: experiences of interns including: –– Demonstrating that internsDemonstrating that interns’’ service delivery duties are service delivery duties are

primarily learning orientedprimarily learning oriented and that training takes and that training takes precedence over service delivery and revenue precedence over service delivery and revenue generation generation

–– Ensuring that internsEnsuring that interns’’ prior educational and practicum prior educational and practicum experiences are consistent experiences are consistent with the programwith the program’’s training s training model and goals model and goals

30

•• Note:Note: Program is responsible for documenting how Program is responsible for documenting how students achieve knowledge and competence and for students achieve knowledge and competence and for setting setting minimal levels of acceptable achievement minimal levels of acceptable achievement in the in the above areas (B1above areas (B1--4).4).

16

31

Broad and GeneralBroad and General•• Preparation at the doctoral, internship and postdoctoral Preparation at the doctoral, internship and postdoctoral

level should be level should be broad and generalbroad and general. . –– based on the existing and evolving body of based on the existing and evolving body of

knowledge, skills, and competencies that define the knowledge, skills, and competencies that define the declared substantive practice declared substantive practice area(sarea(s) )

–– well integrated with the broad theoretical and well integrated with the broad theoretical and scientific foundations of the discipline and field of scientific foundations of the discipline and field of psychology in general.psychology in general.””

The programThe program’’s model or philosophy identifies the s model or philosophy identifies the ““whywhy”” of its of its training activities.training activities.–– What is considered important and valuable in the What is considered important and valuable in the

preparation of professional psychologists?preparation of professional psychologists?–– Forces one to define its product & competenciesForces one to define its product & competencies–– What principles would be followed in designing and What principles would be followed in designing and

delivering a internship/postdoctoral residency program?delivering a internship/postdoctoral residency program?–– What is thought to be necessary to learn entryWhat is thought to be necessary to learn entry--level skills?level skills?–– How are science and practice interconnected?How are science and practice interconnected?–– How will training be sequential, cumulative, and graded in How will training be sequential, cumulative, and graded in

complexity?complexity?

The Training ModelThe Training Model

17

Types of ModelsTypes of Models

••Scientist Scientist ––PractitionerPractitioner••Practitioner Practitioner ––ScholarScholar••DevelopmentDevelopment--ApprenticeshipApprenticeship••ApprenticeshipApprenticeship--MentorMentor••PractitionerPractitioner--MentorMentor--ScholarScholar••MentorMentor

Goals, Objectives, and Goals, Objectives, and CompetenciesCompetencies

•• Should follow from a combination of:Should follow from a combination of:–– The programThe program’’s own model or philosophy with its values s own model or philosophy with its values

and principles; andand principles; and–– The generally accepted components of The generally accepted components of

internship/postdoctoral training as represented in the internship/postdoctoral training as represented in the G&P.G&P.

•• Translated into knowledge and specific Translated into knowledge and specific competenciescompetencies that that are expected of program graduates.are expected of program graduates.

18

B.2 format B.2 format –– SS instructionsSS instructions

Goal #1:Goal #1:

Objective(sObjective(s) for Goal #1:) for Goal #1:

Competencies Expected:Competencies Expected:

Appendix & Page Number for Evaluation Forms Used for Appendix & Page Number for Evaluation Forms Used for Expected Competencies:Expected Competencies:

How Outcomes are Measured and Minimum Thresholds How Outcomes are Measured and Minimum Thresholds for Achievement for Expected Competencies:for Achievement for Expected Competencies:

Describing Goals, Objectives and Describing Goals, Objectives and Competencies Competencies (using internship example)(using internship example)

GOALSGOALS are general are general aspirationalaspirational statements that follow statements that follow from the programfrom the program’’s model.s model.

OBJECTIVESOBJECTIVES are the defined subare the defined sub--areas within each areas within each goal. goal.

COMPETENCIESCOMPETENCIES are the are the operationalizedoperationalized and and measurable translation of the objectives.measurable translation of the objectives.

for examplefor example……..

19

GOAL # 1:GOAL # 1:To produce new professionals who have the requisite To produce new professionals who have the requisite knowledge and skills for entry into the practice of professionalknowledge and skills for entry into the practice of professionalpsychology. In this regard, we seek to develop the psychology. In this regard, we seek to develop the professional skills of our interns such that each is able to... professional skills of our interns such that each is able to...

Objective A: Objective A: treat individuals with a wide variety of treat individuals with a wide variety of psychological problemspsychological problems

Objective B: Objective B: perform competent assessmentsperform competent assessmentsObjective C: conduct appropriate consultation andObjective C: conduct appropriate consultation and

supervision with others supervision with others Objective D: Objective D: initiate and use researchinitiate and use researchObjective E: Objective E: function in an ethical mannerfunction in an ethical mannerObjective F: appreciate the role of individual and cultural Objective F: appreciate the role of individual and cultural

differences and diversity in psychological differences and diversity in psychological phenomenaphenomena

Collect AGGREGATE OUTCOME DATA on the Collect AGGREGATE OUTCOME DATA on the competency of all interns/postdocs directly linked to the competency of all interns/postdocs directly linked to the goals, objectives and competencies and provide these goals, objectives and competencies and provide these data in Domain F.1(b)data in Domain F.1(b)

20

39

Domain CDomain CProgram ResourcesProgram Resources

The program demonstrates that it has The program demonstrates that it has resourcesresourcesof appropriate of appropriate quantityquantity and and sufficiencysufficiency to achieve to achieve its education and training goals and objectives.its education and training goals and objectives.

40

•• C1. Program has formally designated intern training C1. Program has formally designated intern training supervisorssupervisors who: who: –– Are of appropriate Are of appropriate qualityquality for the programfor the program’’s training s training

model and goals model and goals –– ParticipateParticipate actively in actively in program planning, program planning,

implementation, and evaluation implementation, and evaluation –– Serve as professional Serve as professional role modelsrole models consistent with the consistent with the

programprogram’’s training goals and objectives s training goals and objectives

21

41

•• C2. Program has an identifiable C2. Program has an identifiable body of internsbody of internswho: who: –– Are Are sufficient in numbersufficient in number to ensure meaningful peer to ensure meaningful peer

interaction, support, and socialization interaction, support, and socialization –– Are Are in a doctoral programin a doctoral program in professional psychology in professional psychology

or are certified for or are certified for respecializationrespecialization–– Have Have completedcompleted adequate and appropriate adequate and appropriate

supervised service delivery supervised service delivery practicum trainingpracticum training–– Have interests, aptitudes, and prior educational and Have interests, aptitudes, and prior educational and

practicum practicum experiences appropriate for the internshipexperiences appropriate for the internship’’ssgoals and objectives goals and objectives

–– Understand the programUnderstand the program’’s philosophy, goals, and s philosophy, goals, and training model training model

–– Have meaningful Have meaningful involvement in activities and involvement in activities and decisions that enhance trainingdecisions that enhance training

–– Have an Have an official titleofficial title at the site that denotes their at the site that denotes their psychology intern statuspsychology intern status

42

C3. Program has additional C3. Program has additional resources necessary resources necessary to achieve its goals and objectives including: to achieve its goals and objectives including:

•• FinancialFinancial support for intern stipends, staff, and training support for intern stipends, staff, and training activities activities (see IR C(see IR C--9, Unfunded Internships and Stipend 9, Unfunded Internships and Stipend Equity)Equity)

•• ClericalClerical and and technicaltechnical support support •• Training materials Training materials and equipment and equipment •• Physical facilities Physical facilities and training settings and training settings •• Training Training settingssettings appropriateappropriate to the programto the program’’s training s training

model model

22

43

Domain DDomain DCultural and Individual Cultural and Individual

Differences and DiversityDifferences and DiversityThe program The program recognizes the importance of recognizes the importance of cultural and cultural and

individual differences and diversity in the training of individual differences and diversity in the training of psychologists. These include, but are not limited to, age, psychologists. These include, but are not limited to, age,

disability, ethnicity, gender, gender identity, language, disability, ethnicity, gender, gender identity, language, national origin, race, religion, culture, sexual orientation, national origin, race, religion, culture, sexual orientation,

and social economic status.and social economic status.

44

•• D1. The program:D1. The program:–– Has made Has made systematic and longsystematic and long--term term efforts to efforts to attract attract

and retain interns and staffand retain interns and staff from differing ethnic, from differing ethnic, racial, and personal backgrounds racial, and personal backgrounds

–– Ensures a supportive Ensures a supportive learning environmentlearning environment for for training diverse individualstraining diverse individuals who represent a broad who represent a broad cultural and individual spectrum cultural and individual spectrum

–– Avoids actions that restrict program access on Avoids actions that restrict program access on grounds irrelevant to success in internship training grounds irrelevant to success in internship training (see Footnote 4 for exceptions) (see Footnote 4 for exceptions)

–– D2. Program has a thoughtful and coherent D2. Program has a thoughtful and coherent plan to plan to provide interns with relevant knowledgeprovide interns with relevant knowledge and and experiences about the role of individual and cultural experiences about the role of individual and cultural diversity in psychological science and practice.diversity in psychological science and practice.

23

45

Domain EDomain EStudentStudent--Faculty RelationsFaculty Relations

The program demonstrates that its education, training, The program demonstrates that its education, training, and socialization experiences are characterized by and socialization experiences are characterized by mutual respect and courtesy mutual respect and courtesy between students and between students and

faculty, and that it operates in a manner that facilitates faculty, and that it operates in a manner that facilitates educational experiences.educational experiences.

46

•• E1. The program: E1. The program: –– Recognizes the Recognizes the rights of internsrights of interns and staff to be and staff to be

treated with courtesy, respect, collegiality, and treated with courtesy, respect, collegiality, and ethical sensitivity ethical sensitivity

–– InformsInforms interns of these interns of these principlesprinciples and of their and of their avenues of avenues of recourserecourse should issues arise should issues arise

•• E2. E2. StaffStaff members are members are accessibleaccessible to interns, provide to interns, provide guidanceguidance and supervision, and serve as and supervision, and serve as role modelsrole modelswho who promotepromote internsinterns’’ acquisitionacquisition of relevant of relevant knowledge, skills, and competencies. knowledge, skills, and competencies.

24

47

•• E3. E3. Respect for cultural and individual diversityRespect for cultural and individual diversity is is demonstrated in accordance with Domain A5. demonstrated in accordance with Domain A5.

•• E4. Upon admission interns are E4. Upon admission interns are given written policiesgiven written policies and and procedures regarding program procedures regarding program requirements, expected requirements, expected performance, continuance, and termination performance, continuance, and termination procedures. procedures. Interns receive, Interns receive, at least semiannually, written at least semiannually, written feedbackfeedback on the extent to which they are meeting these on the extent to which they are meeting these requirements and expectations. requirements and expectations. The feedback should The feedback should include: include: –– Timely, written Timely, written notification of any problemsnotification of any problems and and

opportunity to discuss them opportunity to discuss them –– Guidance regarding Guidance regarding steps to remediatesteps to remediate all problems (if all problems (if

remediable) remediable) –– Written feedback on the extent to which corrective Written feedback on the extent to which corrective

actions have or have not been successful in addressing actions have or have not been successful in addressing the issues of concernthe issues of concern

48

•• E5. Program issues E5. Program issues certificate of completioncertificate of completion to interns to interns who successfully complete internship who successfully complete internship (see IR C(see IR C--6[a],Program Names, Labels, and other Public 6[a],Program Names, Labels, and other Public Descriptors)Descriptors)

•• E6. Programs keep E6. Programs keep recordsrecords of all formal of all formal complaints and complaints and grievancesgrievances filed since the last accreditation site visit. filed since the last accreditation site visit.

Note: Note: Programs must adhere to institutional regulations Programs must adhere to institutional regulations and applicable local, state, and federal statutes and applicable local, state, and federal statutes concerning due process and fair treatment. concerning due process and fair treatment.

..

25

49

Domain FDomain FProgram SelfProgram Self--Assessment and Quality Assessment and Quality

EnhancementEnhancement

The program demonstrates a The program demonstrates a commitment to excellence commitment to excellence through selfthrough self--studystudy, which assures that its goals and objectives , which assures that its goals and objectives

are met, enhances the quality of the professional education and are met, enhances the quality of the professional education and training obtained by its students, and contributes to the training obtained by its students, and contributes to the

fulfillment of its sponsor institutionfulfillment of its sponsor institution’’s mission.s mission.

50

•• F1. With appropriate F1. With appropriate involvement of internsinvolvement of interns, the program , the program engages in engages in regular, ongoing selfregular, ongoing self--studiesstudies that address its: that address its: –– Expectations for the quality of Expectations for the quality of internsinterns’’ preparationpreparation

prior to beginning training prior to beginning training –– Effectiveness in achieving goalsEffectiveness in achieving goals and objectives in and objectives in

terms of terms of outcome data while interns are in the outcome data while interns are in the program and after completionprogram and after completion (Proximal and Distal)(Proximal and Distal)

–– Procedures to maintain current achievements or to Procedures to maintain current achievements or to make changes as necessary make changes as necessary

–– Goals, objectives, and Goals, objectives, and outcome dataoutcome data in relation to in relation to local, regional, state, and national standardslocal, regional, state, and national standards of of practice and changes in the knowledge base of the practice and changes in the knowledge base of the professionprofession

26

F1.b F1.b –– linkage to Domain Blinkage to Domain BConsistent with the spirit of the Consistent with the spirit of the G&PG&P, each program , each program defines its goals, objectives and competencies. To defines its goals, objectives and competencies. To reflect the outcomereflect the outcome--oriented nature of the process, those oriented nature of the process, those competencies outlined in Domain B should be linked to:competencies outlined in Domain B should be linked to:

–– Aggregate outcome data on intern/postdoc Aggregate outcome data on intern/postdoc competencies while in the program; andcompetencies while in the program; and

–– Aggregate outcome data gathered from program Aggregate outcome data gathered from program graduates graduates

Thus, Thus, for each competency stated in Domain Bfor each competency stated in Domain B, there , there should be some aggregate data on intern/postdoc should be some aggregate data on intern/postdoc success with that competency. success with that competency.

52

•• F2. Program provides resources and/or opportunities for F2. Program provides resources and/or opportunities for the continuing professional the continuing professional development of its training development of its training staffstaff. .

•• F3. Program and its F3. Program and its host institutionhost institution value and value and recognize recognize in tangible waysin tangible ways the internship training and the staffsthe internship training and the staffs’’supervisory efforts. supervisory efforts.

27

53

Domain GDomain GPublic DisclosurePublic Disclosure

The program demonstrates its The program demonstrates its commitment to public commitment to public disclosure disclosure by providing written materials and other by providing written materials and other

communications that appropriately represent it to the communications that appropriately represent it to the relevant publics.relevant publics.

54

•• G1. Program G1. Program describes itself accuratelydescribes itself accurately and completely and completely in documents available to current and prospective in documents available to current and prospective interns and other interns and other ““publicspublics”” including: including:

–– Its Its goalsgoals, objectives, and training model; , objectives, and training model; selectionselectionprocedures and procedures and completion requirementscompletion requirements; training ; training staffstaff, , internsinterns, , facilitiesfacilities, and other resources; and , and other resources; and administrative administrative policies and procedurespolicies and procedures

–– Its Its accreditation statusaccreditation status including the name, address, including the name, address, and telephone number of the CoA.and telephone number of the CoA.

28

55

Domain HDomain HRelationship with Accrediting Relationship with Accrediting

BodyBody

The program demonstrates its commitment to The program demonstrates its commitment to the accreditation process by the accreditation process by fulfilling its fulfilling its

responsibilities to the accrediting body responsibilities to the accrediting body from from which its accredited status is granted.which its accredited status is granted.

56

•• H1. Internship H1. Internship abidesabides by the by the CoACoA’’ss published published policiespolicies and and procedures. procedures.

•• H2. Internship H2. Internship informs the CoAinforms the CoA in a timely manner of in a timely manner of changeschanges in its environment, plans, resources, and in its environment, plans, resources, and operations that could operations that could affect program qualityaffect program quality. . –– model, policies/procedures, administrative structure, model, policies/procedures, administrative structure,

faculty resources, supervision resources, area of faculty resources, supervision resources, area of emphases, or tracks/rotations.emphases, or tracks/rotations.

•• H3. Internship pays necessary H3. Internship pays necessary feesfees to maintain accredited to maintain accredited status.status.

29

Major Yearly TasksMajor Yearly Tasks•• OrientationOrientation•• Rotations and evaluation cyclesRotations and evaluation cycles•• Selection (interns and postdocs)Selection (interns and postdocs)•• Maintaining recordsMaintaining records•• Training meeting minutesTraining meeting minutes•• Seminar schedulesSeminar schedules•• APPIC and APA DuesAPPIC and APA Dues•• Self studiesSelf studies•• Update materials (brochure, APPIC listing, manual)Update materials (brochure, APPIC listing, manual)

Orientation WeekOrientation Week•• Orient new Interns and Residents to evaluation data base system Orient new Interns and Residents to evaluation data base system •• Interns and Residents complete baseline self evaluationsInterns and Residents complete baseline self evaluations•• Review Criteria to complete Internship and ResidencyReview Criteria to complete Internship and Residency•• Have new Interns complete Seminar Topic Ranking sheetHave new Interns complete Seminar Topic Ranking sheet•• Review training manuals/memosReview training manuals/memos•• Interns rank mentorsInterns rank mentors•• Interns rank order training activities (form)Interns rank order training activities (form)•• Develop training plan for Interns and make scheduleDevelop training plan for Interns and make schedule•• Assign intern mentorsAssign intern mentors•• Enter new Intern usernames in Evaluation DB (change default yearEnter new Intern usernames in Evaluation DB (change default year))•• Set up tabbed intern and Residents files for the yearSet up tabbed intern and Residents files for the year•• Pagers for Interns and ResidentsPagers for Interns and Residents•• New employee orientation/CPRS trainingNew employee orientation/CPRS training•• Assign phone numbers/ pagers to Interns and ResidentsAssign phone numbers/ pagers to Interns and Residents•• Clinic numbers to ResidentsClinic numbers to Residents

30

SeptemberSeptember**•• Check outs for interviews, MMPICheck outs for interviews, MMPI--2, WAIS2, WAIS--IIIIII•• Mailing/email to recruit InternsMailing/email to recruit Interns•• Remind Interns and Residents of "Important Dates" to reserveRemind Interns and Residents of "Important Dates" to reserve•• End year letter and evaluations to last years' Interns DCTEnd year letter and evaluations to last years' Interns DCT•• Letter to new Interns' TD about rotation plansLetter to new Interns' TD about rotation plans•• Develop training plans for ResidentsDevelop training plans for Residents•• Letters and evaluations to graduate program Training Directors fLetters and evaluations to graduate program Training Directors for or

the previous intern classthe previous intern class•• Interns and Residents to complete Self EvaluationsInterns and Residents to complete Self Evaluations•• Make sure rotation supervisors and Interns and Residents completMake sure rotation supervisors and Interns and Residents complete e

the Levels of Responsibly Form (Kept with supervisor for the the Levels of Responsibly Form (Kept with supervisor for the duration of supervision and then placed in Interns and Residentsduration of supervision and then placed in Interns and Residents’’files)files)

•• Check EEC Auditorium for Ethics and Diversity Day TrainingsCheck EEC Auditorium for Ethics and Diversity Day Trainings•• Update info (database) on past Residents and InternsUpdate info (database) on past Residents and Interns•• Pay APA Dues for training programs (due in Dec.)Pay APA Dues for training programs (due in Dec.)•• Supervisors complete the Levels of Responsibly FormSupervisors complete the Levels of Responsibly Form

OctoberOctober•• Form intern selection committee (update database)Form intern selection committee (update database)•• Diversity Day Diversity Day -- arrange room and refreshments arrange room and refreshments •• Check with Residents about scholarly project (time lines) Check with Residents about scholarly project (time lines) •• Mailings to recruit ResidentsMailings to recruit Residents•• Announcement/posting for InternshipAnnouncement/posting for Internship•• Check in lunch or breakfast with InternsCheck in lunch or breakfast with Interns•• Make sure Interns and Residents are logging hoursMake sure Interns and Residents are logging hours•• Remind Interns and Residents about Diversity Day (end Remind Interns and Residents about Diversity Day (end

of Oct)of Oct)•• Enter intern application info into database as it come inEnter intern application info into database as it come in

31

NovemberNovember•• Set times for Interns Selection Committee to meetSet times for Interns Selection Committee to meet•• Intern application deadline 11/1Intern application deadline 11/1•• Update Intern Selection files and interview formsUpdate Intern Selection files and interview forms•• Email Intern applicants for interviewsEmail Intern applicants for interviews•• Get Seminar rankings back from Interns and ResidentsGet Seminar rankings back from Interns and Residents•• Arrange Seminars for the remainder of the yearArrange Seminars for the remainder of the year•• Form Resident selection committeeForm Resident selection committee•• Announcement/posting for postdocsAnnouncement/posting for postdocs•• Talk to interns about Residency plans etc.Talk to interns about Residency plans etc.

November November -- ContinuedContinued–– Get interview dates from staff Get interview dates from staff –– Get Interns to reserve blocks to meet w/ inter Get Interns to reserve blocks to meet w/ inter

applicants for interview weekapplicants for interview week–– Reminder to staff to set aside times for intern Reminder to staff to set aside times for intern

interview weekinterview week–– Enter staff interview block in databaseEnter staff interview block in database–– Make sure rooms are available for interview weekMake sure rooms are available for interview week–– Make sure intern and Resident hours worked log is up Make sure intern and Resident hours worked log is up

to dateto date–– Enter intern application info into databaseEnter intern application info into database

32

DecemberDecember**•• Speak to current Interns re Residency program (informed consent)Speak to current Interns re Residency program (informed consent)•• Update prac webpage (send to graduate program TDs)Update prac webpage (send to graduate program TDs)•• EE--mail announcement/posting for Residencymail announcement/posting for Residency•• 12/15 Letters to all completed applicants (not interviewed)12/15 Letters to all completed applicants (not interviewed)•• Intern Intern -- interviewees hotel and map letterinterviewees hotel and map letter•• Update Seminars based on Intern and Resident suggestionsUpdate Seminars based on Intern and Resident suggestions•• CheckinsCheckins with Interns and Residentswith Interns and Residents

–– Job plans including Resident positions, Review evaluationsJob plans including Resident positions, Review evaluations•• Set up interview week (data base etc.)Set up interview week (data base etc.)•• Arrange refreshments for intern interview weekArrange refreshments for intern interview week•• Reminder of trimester end evaluations Reminder of trimester end evaluations

–– Supervisor/ Training Experience EvaluationSupervisor/ Training Experience Evaluation•• Check evaluation DB who has not completed evaluations and remindCheck evaluation DB who has not completed evaluations and remind

themthem•• Schedule meetings with Residents and Interns to review evaluatioSchedule meetings with Residents and Interns to review evaluations ns

with TDswith TDs•• Email interns and supervisors about upcoming rotation changesEmail interns and supervisors about upcoming rotation changes•• Supervisors complete the Levels of Responsibly FormSupervisors complete the Levels of Responsibly Form

JanuaryJanuary•• Interview week Interview week -- 1st week of the month1st week of the month•• Clarify number of prac students next year Clarify number of prac students next year •• Announcement/posting for ResidentAnnouncement/posting for Resident•• U of M Prac breakfast 3rd week of monthU of M Prac breakfast 3rd week of month•• Contact local grad programs Contact local grad programs -- practicum and application practicum and application

processprocess•• Update Residency Selection files and interview formsUpdate Residency Selection files and interview forms•• Residency Application Deadline 1/1Residency Application Deadline 1/1•• Complete OAA Trainee Allocation Request form Complete OAA Trainee Allocation Request form –– coordinate coordinate

with Education Officerwith Education Officer•• Survey staff Survey staff –– who can take a prac student for the following who can take a prac student for the following

yearyear•• Enter intern interview evaluation data into database as forms Enter intern interview evaluation data into database as forms

are completedare completed•• Organize practicum applications/ files and enter in to the data Organize practicum applications/ files and enter in to the data

base as they come inbase as they come in

33

FebruaryFebruary**•• 2/5 Deadline for MATCH rank order lists2/5 Deadline for MATCH rank order lists•• Notification of Match results (3rd Monday, clear day)Notification of Match results (3rd Monday, clear day)•• Review prac applications and possible supervisor matchReview prac applications and possible supervisor match•• Merge next years Interns in Evaluation DBMerge next years Interns in Evaluation DB•• Clean up erroneous records in Evaluation DBClean up erroneous records in Evaluation DB•• Schedule meetings to review evaluations with TDsSchedule meetings to review evaluations with TDs•• Acceptance letter to Interns Acceptance letter to Interns •• Personnel letter for accepted InternsPersonnel letter for accepted Interns•• Letter to new InternsLetter to new Interns’’ graduate program training directorsgraduate program training directors•• Residency Match or UND 2/29Residency Match or UND 2/29•• CheckinsCheckins with Interns and Residentswith Interns and Residents

–– Residents Residents –– updates on research and administrative updates on research and administrative projectsprojects

–– Job plans including Residency positionsJob plans including Residency positions–– Review evaluationsReview evaluations

•• Match prac student applicants to supervisors (interview as Match prac student applicants to supervisors (interview as needed) (Asst. TD coordinates practicum)needed) (Asst. TD coordinates practicum)

February February -- ContinuedContinued•• Mid year letter to current internsMid year letter to current interns’’ TDs about progress TDs about progress

including copies of rotation and adjunctive evaluationsincluding copies of rotation and adjunctive evaluations•• Reminder of midReminder of mid--year trainee self, program and adjunctive year trainee self, program and adjunctive

trainee evaluations to supervisors and Interns and Residentstrainee evaluations to supervisors and Interns and Residents–– Training Program Evaluation (Intern and Residents)Training Program Evaluation (Intern and Residents)–– SelfSelf--Evaluations (Interns and Residents)Evaluations (Interns and Residents)–– Supervisor/Training Experience Evaluations Supervisor/Training Experience Evaluations –– Assessment Clinic EvaluationsAssessment Clinic Evaluations–– Adjunctive Training Experiences EvaluationsAdjunctive Training Experiences Evaluations

•• Check DB for who has not completed evaluations and remind Check DB for who has not completed evaluations and remind them them

•• Schedule meetings with Residents and Interns to review midSchedule meetings with Residents and Interns to review mid--year and adjunctive evaluations with TDsyear and adjunctive evaluations with TDs

•• Organize practicum applicationsOrganize practicum applications•• Copies of rotation and adjunctive evaluations to be sent to Copies of rotation and adjunctive evaluations to be sent to

current internscurrent interns’’ TDsTDs

34

MarchMarch

•• Supervision workshop Supervision workshop -- rooms and refreshmentsrooms and refreshments•• Merge next years Residents into the evaluation Merge next years Residents into the evaluation

databasedatabase•• Rejection letters to Residency applicantsRejection letters to Residency applicants•• Acceptance letters to accepted Residency Acceptance letters to accepted Residency

applicantsapplicants•• Begin privileging for ResidentsBegin privileging for Residents•• Get consent from incoming Intern and Resident Get consent from incoming Intern and Resident

classes to share emails among the classesclasses to share emails among the classes

AprilApril **•• Start revising Website content Start revising Website content •• Update trainee manuals for next yearUpdate trainee manuals for next year•• Update APPIC Intern and Residency websitesUpdate APPIC Intern and Residency websites•• CheckinsCheckins with Interns and Residentswith Interns and Residents

–– Job plans, Review evaluationsJob plans, Review evaluations–– Residents Residents –– research and administrative projectsresearch and administrative projects

•• Reminder of trimester end evaluations Reminder of trimester end evaluations •• Trainee Trimester EvaluationTrainee Trimester Evaluation

–– Supervisor/ Training Experience EvaluationSupervisor/ Training Experience Evaluation•• Check evaluation DB who has not completed evaluations and remindCheck evaluation DB who has not completed evaluations and remind

themthem•• Clean up erroneous records in Evaluation DBClean up erroneous records in Evaluation DB•• Supervisors complete the Levels of Responsibly FormSupervisors complete the Levels of Responsibly Form•• Schedule meetings with Residents and Interns to review evaluatioSchedule meetings with Residents and Interns to review evaluations ns

with TDswith TDs•• Check EEC Auditorium for MAPPIC Ethics/Supervision TrainingCheck EEC Auditorium for MAPPIC Ethics/Supervision Training•• Reminder to Interns and Residents RE MAPPIC Ethics Day Reminder to Interns and Residents RE MAPPIC Ethics Day •• Coordinate Human Resources processing of new Interns, Residents Coordinate Human Resources processing of new Interns, Residents and and

prac students including OAA Residency appointment forms.prac students including OAA Residency appointment forms.

35

MayMay•• NoneNone•• Get caught up on everything elseGet caught up on everything else

JuneJune•• Web site rotations/training experiences revisions (email Web site rotations/training experiences revisions (email

to staff)to staff)•• Staff vita revisions (email to staff)Staff vita revisions (email to staff)•• Revise/update Trainee Handbook (review for needed Revise/update Trainee Handbook (review for needed

changes)changes)•• Get dates from PSL Director's secretary for Grand Get dates from PSL Director's secretary for Grand

Rounds for next year's 4 Residents and 8 Interns and Rounds for next year's 4 Residents and 8 Interns and schedule them. Residents go first starting Octoberschedule them. Residents go first starting October

•• Check in with Coordinate Human Resources regarding Check in with Coordinate Human Resources regarding processing of new Interns, Residents and prac students processing of new Interns, Residents and prac students including OAA Residency appointment forms.including OAA Residency appointment forms.

36

JulyJuly•• CheckinCheckin with incoming Intern and Resident classeswith incoming Intern and Resident classes•• Plan trainee Graduation PartyPlan trainee Graduation Party•• End year exit interviews with TDs and Chief End year exit interviews with TDs and Chief •• Develop orientation schedule Develop orientation schedule •• Clear conference room for Intern/Resident OrientationClear conference room for Intern/Resident Orientation•• Set up Practicum student orientation with CPRS trainingSet up Practicum student orientation with CPRS training•• Revise/update Trainee HandbookRevise/update Trainee Handbook•• Resident mentors to contact incoming Residents and Resident mentors to contact incoming Residents and

discuss opportunities and possible research projectsdiscuss opportunities and possible research projects

AugustAugust•• New trainee party New trainee party •• Update Evaluation DB with new Interns and ResidentsUpdate Evaluation DB with new Interns and Residents•• Change default training year in Evaluation DBChange default training year in Evaluation DB•• Finalize Grad PartyFinalize Grad Party•• Set up rotation datesSet up rotation dates•• Welcome party for new Interns Welcome party for new Interns •• Reminder all to complete evaluations (check evaluation DB who haReminder all to complete evaluations (check evaluation DB who has not s not

completed)completed)•• Levels of Responsibly Forms placed in filesLevels of Responsibly Forms placed in files•• Make sure files are complete and up to dateMake sure files are complete and up to date•• Return APPIC MATCH numbers materialsReturn APPIC MATCH numbers materials•• APPIC duesAPPIC dues•• Orientation Binders for Interns and Residents Orientation Binders for Interns and Residents •• Clean up erroneous records in Evaluation DBClean up erroneous records in Evaluation DB•• Schedule New Employee Orientation and CPRS TrainingSchedule New Employee Orientation and CPRS Training•• Clinic number for ResidentsClinic number for Residents•• Dictation codes for Interns and ResidentsDictation codes for Interns and Residents•• Practicum orientation dayPracticum orientation day•• Collect followCollect follow--up info on Residents and Interns and enter into DBup info on Residents and Interns and enter into DB•• Business/Appointment card for Interns and ResidentsBusiness/Appointment card for Interns and Residents

37

•• OAA Web Portal (Program Brochures) OAA Web Portal (Program Brochures) ((http://http://www.psychologytraining.va.govwww.psychologytraining.va.gov//

•• APPIC Directory APPIC Directory http://appic.org/directory/4_1_directory_online.asphttp://appic.org/directory/4_1_directory_online.asp

•• Paper mailingsPaper mailings•• Email advertisements (Email advertisements (DCTsDCTs, Minority lists, Division lists , Minority lists, Division lists

etc, APPIC lists)etc, APPIC lists)•• Be clear on criteria you list for applicant on APPIC Be clear on criteria you list for applicant on APPIC

directory and website (hours, preferred programs, directory and website (hours, preferred programs, required experience, etc)required experience, etc)

RecruitmentRecruitment

SelectionSelection•• Online AAPIOnline AAPI•• ProcessProcess

–– Section CommitteeSection Committee–– InterviewsInterviews–– Strategies to manage data rankingsStrategies to manage data rankings–– How do you evaluate files and interviewsHow do you evaluate files and interviews–– Rank list Rank list –– how to decide?how to decide?

•• Understanding the MATCHUnderstanding the MATCH•• Need to maximize ability to MATCHNeed to maximize ability to MATCH

–– Understanding the applicants stressUnderstanding the applicants stress–– Postdoc selection issuesPostdoc selection issues

38

Helping Interns Cope

The word ‘servant’ is so degrading. Why don’t we just call you ‘intern’.

1.1. Trust vs. MistrustTrust vs. Mistrust

2.2. Autonomy vs. DoubtAutonomy vs. Doubt

3.3. Initiative vs. GuiltInitiative vs. Guilt

4.4. Industry vs. InferiorityIndustry vs. Inferiority

5.5. Identity vs. Role DiffusionIdentity vs. Role Diffusion

6.6. GernerativityGernerativity vs. Stagnationvs. Stagnation

EricksonErickson’’s Lifespan Theory: A metaphor s Lifespan Theory: A metaphor for Conceptualizing the Internship Yearfor Conceptualizing the Internship Year

Guinee, J. (1988) Professional Psych

39

DUE PROCESS DUE PROCESS -- A procedure employed when a A procedure employed when a program has a concern or problem with an intern.program has a concern or problem with an intern.

GRIEVANCE GRIEVANCE -- A procedure employed when an intern A procedure employed when an intern has a complaint or problem with an internship has a complaint or problem with an internship program. program.

Due Process & Grievance Due Process & Grievance ProceduresProcedures

Sample Problematic BehaviorsSample Problematic Behaviors•• Intern does not acknowledge, understand, or address the Intern does not acknowledge, understand, or address the

problem when it is identified.problem when it is identified.•• The problem is not merely a reflection of a skill deficit The problem is not merely a reflection of a skill deficit

which can be easily rectified by training.which can be easily rectified by training.•• The quality of services delivered by the intern is sufficiently The quality of services delivered by the intern is sufficiently

negatively affected.negatively affected.•• The traineeThe trainee’’s behavior does not change as a function of s behavior does not change as a function of

feedback, remediation efforts, &/or time.feedback, remediation efforts, &/or time.•• The problematic behavior has potential for ethical or legal The problematic behavior has potential for ethical or legal

ramifications if not addressedramifications if not addressed..

40

Elements of Due ProcessElements of Due Process1.1. NOTICENOTICE: An intern must be formally notified that the : An intern must be formally notified that the

program has identified problematic behavior and plans to program has identified problematic behavior and plans to address the behavior.address the behavior.

2.2. HEARINGHEARING: A meeting of internship faculty with the intern : A meeting of internship faculty with the intern is held in which the problematic behavior is articulated is held in which the problematic behavior is articulated and in which the intern has the opportunity to present and in which the intern has the opportunity to present information regarding the problematic behavior. information regarding the problematic behavior.

3.3. APPEALAPPEAL: Provides the intern with an opportunity to : Provides the intern with an opportunity to appeal the decision made after the Hearing . The intern appeal the decision made after the Hearing . The intern should have the opportunity to appeal to a person above should have the opportunity to appeal to a person above the Training Director.the Training Director.

Sample Grievance IssuesSample Grievance Issues

••Agency has misrepresented itself in public documentsAgency has misrepresented itself in public documents••Supervision is lacking or insufficientSupervision is lacking or insufficient••Boundary violationsBoundary violations••Disagreement with evaluationsDisagreement with evaluations••HarassmentHarassment

41

The SelfThe Self--StudyStudy•• Slightly altered each year Slightly altered each year –– use the ones for the year of use the ones for the year of

your scheduled (reyour scheduled (re--accredaccred) or anticipated (applicant) site ) or anticipated (applicant) site visitvisit

•• Use them as a Use them as a template template (download in Word format)(download in Word format)•• *Read the 3*Read the 3--5 pages of additional instructions and 5 pages of additional instructions and

information preceding the transmittal page* information preceding the transmittal page* •• Use the Use the bold prompts bold prompts to focus your responsesto focus your responses•• Read the instructions and footnotes for each TableRead the instructions and footnotes for each Table•• Read the relevant Read the relevant Implementing RegulationsImplementing Regulations (IR(IR’’s) when s) when

they are referenced throughout the instructionsthey are referenced throughout the instructions

** ** NOTE:NOTE: Programs who do not use the correct outline will Programs who do not use the correct outline will be required to be required to reviserevise and and resubmit resubmit the entire selfthe entire self--study study (IR D.2(IR D.2--3).3).

42

OrganizationOrganization•• Provide a completed Summary Check Sheet for Documents and Provide a completed Summary Check Sheet for Documents and

Appendices Appendices •• Number pages whenever possible Number pages whenever possible

–– Read instructions for specific requirementsRead instructions for specific requirements–– Handwritten numbers are better than no numbers!Handwritten numbers are better than no numbers!

•• Label all documents and Appendices clearlyLabel all documents and Appendices clearly–– How will someone unfamiliar with your program and selfHow will someone unfamiliar with your program and self--study find the study find the

relevant material?relevant material?•• Be SPECIFIC when referencing policies, evaluation forms, etcBe SPECIFIC when referencing policies, evaluation forms, etc

–– DonDon’’t just cite the general Appendix or document (e.g., Handbook) t just cite the general Appendix or document (e.g., Handbook) –– how how will the reviewers find what you are referring to?will the reviewers find what you are referring to?

•• Physical submission:Physical submission:–– All selfAll self--studies will be put in identical accordion folders upon submissistudies will be put in identical accordion folders upon submission, so on, so

no need to submit them in bindersno need to submit them in binders–– Forms of binding permitted but should be able to be easily removForms of binding permitted but should be able to be easily removeded–– Tabs, dividers, colored pages, folders, clips, rubber bands Tabs, dividers, colored pages, folders, clips, rubber bands –– OKOK–– Remember the idea is to make things simple and easy to findRemember the idea is to make things simple and easy to find

Areas of Common ConfusionAreas of Common Confusion

DOMAIN BDOMAIN BGoals, Objectives, Competencies Goals, Objectives, Competencies ––•• Minimal levels of achievementMinimal levels of achievement not identified in relation to not identified in relation to

the programthe program’’s goals and objectives and/or s goals and objectives and/or curricular/content area requirements curricular/content area requirements

•• Goals/objectives/competenciesGoals/objectives/competencies are too general and not are too general and not specific enough in order to later review outcome data in specific enough in order to later review outcome data in Domain F for interns/residents in the program and after Domain F for interns/residents in the program and after program completionprogram completion

•• (If using an evaluation form to assess competence) (If using an evaluation form to assess competence) –– does does not specify not specify wherewhere in the evaluation form the specific areas in the evaluation form the specific areas are reviewedare reviewed

43

Curriculum/Content Areas Curriculum/Content Areas ––

•• B.4/B.3 (internships/postdocs) B.4/B.3 (internships/postdocs) –– not describing not describing in detailin detail the the training activities used to gain knowledge in each content training activities used to gain knowledge in each content areaarea

•• Internships Internships --–– not using the required table to provide the required not using the required table to provide the required

training activity, competencies expected, how outcomes training activity, competencies expected, how outcomes are measured, minimum thresholds for achievement, are measured, minimum thresholds for achievement, and the appendix and page number for the evaluation and the appendix and page number for the evaluation form used to assess intern competence in each form used to assess intern competence in each competency areacompetency area

–– not specifying which items on the evaluation form not specifying which items on the evaluation form correspond to expected competencies for each content correspond to expected competencies for each content areaarea

•• Programs do not address how interns are exposed to the Programs do not address how interns are exposed to the current body of knowledge of current body of knowledge of theories and/or methodstheories and/or methods of of consultation and supervision as described by consultation and supervision as described by IR CIR C--11 (note (note that that providingproviding consultation or supervision is NOT required consultation or supervision is NOT required at the intern level)at the intern level)

•• Internship programs misinterpret the definition of Internship programs misinterpret the definition of ““evaluationevaluation”” required by B.4b:required by B.4b:–– IR CIR C--1:1: ““Refers to such activities as program Refers to such activities as program

evaluation or evaluation of an intervention at the evaluation or evaluation of an intervention at the individual or group level individual or group level –– not the psychological not the psychological assessment of an individual personassessment of an individual person””

44

DOMAIN CDOMAIN C•• No justification of No justification of sufficiencysufficiency of supervisors of supervisors •• Providing clarification of the capacity of the supervisory Providing clarification of the capacity of the supervisory

roles of training supervisors roles of training supervisors –– who is providing what kind who is providing what kind of supervision? of supervision?

•• Describing APPIC Match process rather than how it Describing APPIC Match process rather than how it determines that intern interests, aptitudes, and prior determines that intern interests, aptitudes, and prior experiences are a good fit for the goals/objectives experiences are a good fit for the goals/objectives specificspecificto that internship programto that internship program

DOMAIN DDOMAIN D•• Describing outcomes of diverse recruitment, rather than Describing outcomes of diverse recruitment, rather than

efforts efforts –– effortsefforts are in G&P; outcomes are notare in G&P; outcomes are not•• Education/experiences in issues of diversity not articulated Education/experiences in issues of diversity not articulated

in a thoughtful and coherent planin a thoughtful and coherent plan•• Remember broad definition of diversity Remember broad definition of diversity –– Domain A.5Domain A.5

DOMAIN EDOMAIN E•• Not describing Not describing how interns/residents arehow interns/residents are awareaware of their due of their due

process and grievance proceduresprocess and grievance procedures•• Feedback/evaluation process (for students, interns, Feedback/evaluation process (for students, interns,

residents) unclear:residents) unclear:–– How often How often formal, written formal, written feedback is provided: at least feedback is provided: at least

semisemi--annually (internships and postdocs)annually (internships and postdocs)–– Does it include ALL elements of Domain E.4aDoes it include ALL elements of Domain E.4a--c (intern) c (intern)

or Domain E.4aor Domain E.4a--d (postdocs)?d (postdocs)?•• Including remediation steps, and written feedback if Including remediation steps, and written feedback if

remediation is unsuccessful?remediation is unsuccessful?•• Whether or not formal complaints/grievances have been Whether or not formal complaints/grievances have been

filed, program needs to discuss its process in keeping filed, program needs to discuss its process in keeping records confidential should it occurrecords confidential should it occur (IR C(IR C--12)12)

45

DOMAIN FDOMAIN F•• Unclear how interns/residents are involved in selfUnclear how interns/residents are involved in self--

assessment processassessment process•• Failure in Domain F to evaluate program goals, objectives, Failure in Domain F to evaluate program goals, objectives,

and expected intern/resident competencies in a manner and expected intern/resident competencies in a manner which is linked to its evaluation benchmarks in an which is linked to its evaluation benchmarks in an aggregate aggregate manner, so that the programsmanner, so that the programs’’ success may be success may be measured proximally and distally on those parametersmeasured proximally and distally on those parameters

•• Lack of Lack of aggregate outcome dataaggregate outcome data* (F.1b):* (F.1b):–– Not providing Not providing proximalproximal outcome data that demonstrates how the outcome data that demonstrates how the

program evaluates intern/resident achievement of competencies program evaluates intern/resident achievement of competencies while they progress through the programwhile they progress through the program

–– Not providing data on intern/resident views or assessments of thNot providing data on intern/resident views or assessments of the e programprogram

–– Not providing outcome data of its program graduates and how Not providing outcome data of its program graduates and how these these distaldistal measurements demonstrate the programmeasurements demonstrate the program’’s s effectiveness in achieving its goals and objectiveseffectiveness in achieving its goals and objectives

* required by U.S. Department of Education* required by U.S. Department of Education

•• No discussion of No discussion of how data have been usedhow data have been used to make to make changes or adjustments to programchanges or adjustments to program

•• Failure to discuss any negative aspects of program Failure to discuss any negative aspects of program graduate survey responses about the programgraduate survey responses about the program

•• Not answering Not answering eacheach component of Domains F.2/F.3 component of Domains F.2/F.3 (internships/post(internships/post--docs)docs)•• These areas tend to be glazed over after the data portion has beThese areas tend to be glazed over after the data portion has been en

discusseddiscussed

•• Read over the CoARead over the CoA’’s s last decision letterlast decision letter (re(re--accredaccredprograms)! If any areas have been asked to be addressed programs)! If any areas have been asked to be addressed ““in the next selfin the next self--study,study,”” make sure these items have been make sure these items have been specifically discussed either within the related domain specifically discussed either within the related domain and/or Domain F (2009 instructions) or at the end of each and/or Domain F (2009 instructions) or at the end of each domain (2010 instructions)domain (2010 instructions)•• Areas not discussed may be identified as Areas not discussed may be identified as continuing problematic continuing problematic

issuesissues with the programwith the program

46

DOMAIN GDOMAIN G•• Providing general references to large documents (e.g., Providing general references to large documents (e.g.,

Handbook, website) with regard to each of the required Handbook, website) with regard to each of the required public disclosure componentspublic disclosure components–– Please specify Please specify wherewhere each piece of information can be found!each piece of information can be found!

•• Information presented in the selfInformation presented in the self--study is study is differentdifferent than the than the information provided in the programinformation provided in the program’’s public documents s public documents (i.e., curriculum, goals and objectives, rotations, (i.e., curriculum, goals and objectives, rotations, faculty/staff)faculty/staff)–– Interested applicants should be able to make an informed decisioInterested applicants should be able to make an informed decision n

about entering the programabout entering the program•• Failure to update program web pages, or links not workingFailure to update program web pages, or links not working•• Not providing the CoA contact information in all public Not providing the CoA contact information in all public

documents which cite the programdocuments which cite the program’’s accredited status s accredited status (including telephone number: (including telephone number: 202202--336336--5979)5979)–– Tell us Tell us wherewhere that info is located within the document!that info is located within the document!

DOMAIN HDOMAIN H•• Program does not discuss correspondence with the CoA Program does not discuss correspondence with the CoA

regarding programmatic changesregarding programmatic changes–– Know the requirements of Know the requirements of IR CIR C--1919

47

Curriculum vitae Curriculum vitae ––•• Not provided for ALL required individuals listed on staff Not provided for ALL required individuals listed on staff

tables (as explained by footnotes)tables (as explained by footnotes)•• Not in required template (abbreviated)Not in required template (abbreviated)•• More than 2 pages in lengthMore than 2 pages in length•• Missing required information from templateMissing required information from template•• CVCV’’s provided for people not listed on the staff tables with s provided for people not listed on the staff tables with

no explanation as to their role in the programno explanation as to their role in the program•• Listing doctoral, internship, and/or postListing doctoral, internship, and/or post--doc programs doc programs

attended as APAattended as APA--accredited when they are not (especially accredited when they are not (especially ones clearly outside the scope of APA accreditation; ones clearly outside the scope of APA accreditation; e.ge.g, , doctoral program in Developmental Psychology)doctoral program in Developmental Psychology)

•• Primary Professional Appointment listed on CV does not Primary Professional Appointment listed on CV does not appear consistent with information on staff table and time appear consistent with information on staff table and time devoted to the program under reviewdevoted to the program under review

AccuracyAccuracy

Tables Tables ––•• When calculating amount of time devoted to the program, When calculating amount of time devoted to the program,

it should only be it should only be to the program under reviewto the program under review–– Should NOT include time spent in research; Should NOT include time spent in research;

administrative duties; private practice; etcadministrative duties; private practice; etc•• Use the definitions provided for each staff category to Use the definitions provided for each staff category to

determine the best placement for each contributing staff determine the best placement for each contributing staff member, to the best of your judgmentmember, to the best of your judgment

•• All intern data provided for the last All intern data provided for the last 7 7 years years –– despite the despite the date of the last site visitdate of the last site visit

•• Double check the consistency of numbers across all staff Double check the consistency of numbers across all staff and intern/resident tables, including demographics tablesand intern/resident tables, including demographics tables

48

CoherenceCoherence•• ““The big pictureThe big picture””•• Do the domains relate to one another?Do the domains relate to one another?•• Pay particular attention to Pay particular attention to ““BB--F CorrespondenceF Correspondence”” ––

goals, objectives, competencies in Domain B are directly goals, objectives, competencies in Domain B are directly related to the evaluation mechanisms and outcome data related to the evaluation mechanisms and outcome data discussed in Domain Fdiscussed in Domain F

•• Public and other written information about the program Public and other written information about the program (Domain G) is related to what is reported elsewhere in (Domain G) is related to what is reported elsewhere in the selfthe self--studystudy

Clarity and PresentationClarity and Presentation•• Be Be judiciousjudicious with Appendices with Appendices –– no extraneous materialno extraneous material

–– Ensure all materials referenced in selfEnsure all materials referenced in self--study are in study are in Appendices, and viceAppendices, and vice--versaversa

•• Ensure Appendices and page numbers referenced in Ensure Appendices and page numbers referenced in narrative are narrative are correctcorrect–– Check all of them again Check all of them again –– PROOFREAD!PROOFREAD!–– Is it organized in a way that makes it easy to find Is it organized in a way that makes it easy to find

materials? materials? –– Can it be easily read by someone who does NOT Can it be easily read by someone who does NOT

know your program or selfknow your program or self--study? Get an outside study? Get an outside reader to page through and make sure things can be reader to page through and make sure things can be found easily.found easily.

•• Last step Last step –– signed transmittal page: signed transmittal page: -- indicates the selfindicates the self--study has been study has been read read and and approved, approved, and serves as the and serves as the invitation to conduct a site visitinvitation to conduct a site visit

49

Guidelines and Principles (G&P)

Implementing Regulations (IR’s)•check regularly for updates

CoA / OfficeContact Information

FAQ’s on self-studysubmission and guidelines

Self-Study Instructions•Based on year ofexpected site visit•Change each year