Embed Size (px)
Citation preview
University of Alabama at Birmingham School of Education
Department of Human Studies
Counseling Program
2017 – 2018 State of the Program Report
Affiliated Faculty Sean Hall, Ph.D., Clinic Director, Assistant Professor
Larry Tyson, Ph.D., Associate Professor, School Counseling Coordinator Shannon McCarthy, Ph.D., Assistant Professor
Dayna Watson, Ph.D., Assistant Professor, Program Coordinator
THIS PAGE IS INTENTIONALLY LEFT BLANK
Executive Summary Introduction The UAB Counseling Program (CP) relies on data from various sources to inform programmatic decisions. These data can be divided into two categories including: enrollment data and learning outcomes. Enrollment data is used to document trends in the characteristics of applicants, current students, and alumni. Learning outcomes are used to assess whether students are indeed acquiring and demonstrating the knowledge, skills and abilities targeted by the current curriculum. Interested readers may review a detailed outline regarding the CP’s assessment procedures documented in the UAB Program Assessment and Evaluation Manual. The purpose of the current report is to summarize program data in an effort to guide decision making and program development. Enrollment Data Enrollment data is collected via three data sources: Admission/Application Data, Enrollment or Headcount Data, and Specialty Track Data.
Enrollment Data Semester Count
Fall 2011 92 Spring 2012 82 Summer 2012 74 Fall 2012 95 Spring 2013 86 Summer 2013 65 Fall 2013 92 Spring 2014 87 Summer 2014 75 Fall 2014 93 Spring 2015 89 Summer 2015 70 Fall 2015 81 Spring 2016 87 Summer 2016 76 Fall 2016 91 Spring 2017 98 Summer 2017 64 Fall 2017 111 Spring 2018 120 Summer 2018 102
From the above data (Median= 87, Mean=87.14 [SD=14.104], and Mode=92), we can infer that enrollment trends have remained relatively stable. However, in the past year, we’ve seen a rapid increase which was likely due to the Marriage, Couples, and Family Counseling (MCFC) certificate. The following graph highlights this improvement.
9282
74
9586
65
92 8775
93 89
7081 87
7691
98
64
111120
102
0.00
20.00
40.00
60.00
80.00
100.00
120.00
140.00
Fall 20
11
Spring 2
012
Summer 20
12
Fall 20
12
Spring 2
013
Summer 20
13
Fall 20
13
Spring 2
014
Summer 20
14
Fall 20
14
Spring 2
015
Summer 20
15
Fall 20
15
Spring 2
016
Summer 20
16
Fall 20
16
Spring 2
017
Summer 20
17
Fall 20
17
Spring 2
018
Summer 20
18
Overall CP Enrollment 2011-2018
To further unpack headcount trends, the CP tracks enrollment numbers within each specialty. A review of these data appear to demonstrate that the School Counseling specialty track has remained stable, with only slight fluctuations in enrollment numbers since 2011. Readers may observe substantial declines in enrollment during the summer semester. These temporary drops occur because SC students no longer participate in clinical coursework during the summer. This program policy was implemented due to an increasing number of districts transitioning summer school programs to an online format. As such, opportunities for obtaining direct hours became less reliable. Readers may also observe that the enrollment in Clinical Mental Health Counseling (CMHC) has steadily increased since Summer 2013. This observation must be interpreted in context of the MCFC specialty and certification programs. The number of students enrolled in the MCFC certificate has increased rapidly since Fall 2017. However, the number of students enrolled in the MCFC specialty program has grown more slowly. This discrepancy is likely due to students, who are primarily interested in MCFC, but also want extra training in CMHC, choosing to simultaneously enroll in the CMHC specialty and the MCFC certification program. Feedback from students appears to indicate that the desire to appear more marketable upon graduation supersedes their primary interest in MCFC. In other words, enrollment in the MCFC specialty program is undermined by the MCFC certificate option. To overcome this imbalance, the counseling program faculty have introduced a CMHC certificate option. This option will allow students, who are primarily interested in MCFC, to still specialize in their desired area, while also allowing them to pursue additional training in clinical mental health. We anticipate that, once implemented, enrollment in the CMHC program will decrease slightly and level off, while enrollment in the MCFC specialty will begin to steadily increase.
AGYC CMHC RHC SC MCFC MCFC Cert. Fall 2011 62 0 3 23 0 0 Spring 2012 62 0 2 17 0 0 Summer 2012 55 0 2 16 0 0 Fall 2012 73 0 1 19 0 0 Spring 2013 63 3 1 18 0 0 Summer 2013 42 3 1 19 0 0 Fall 2013 48 21 0 23 0 0 Spring 2014 36 27 0 24 0 0 Summer 2014 27 30 0 18 0 0 Fall 2014 27 39 0 27 0 0 Spring 2015 17 43 0 29 0 0 Summer 2015 6 47 0 17 0 0 Fall 2015 4 50 0 27 0 0 Spring 2016 1 56 0 30 0 0 Summer 2016 0 53 0 23 0 0 Fall 2016 0 59 0 28 1 3 Spring 2017 0 63 0 31 2 2 Summer 2017 0 44 0 15 1 4 Fall 2017 0 60 0 25 5 20 Spring 2018 0 66 0 25 6 23 Summer 2018 0 56 0 12 8 26 AGYC: Agency Counseling; CMHC: Clinical Mental Health Counseling; RHC: Rehabilitation Counseling; SC: School Counseling; MCFC: Marriage Couples and Family Counseling; MCFC Cert.: MCFC Certificate NOTE: AGYC was replaced with CMHC. The RHC program was closed during Fall 2012. The MCFC and MCFC Cert programs began Fall 2016.
0 0 0 03 3
21
2730
3943
4750
5653
5963
44
60
66
56
2317 16
19 18 1923 24
18
27 29
17
2730
2328
31
15
25 25
12
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 15 6 8
0 0 0 0 0 0 0 0 0 0 0 0 0 0 03 2 4
2023 26
0
10
20
30
40
50
60
70
Fall 20
11
Spring
2012
Summer
2012
Fall 20
12
Spring
2013
Summer
2013
Fall 20
13
Spring
2014
Summer
2014
Fall 20
14
Spring
2015
Summer
2015
Fall 20
15
Spring
2016
Summer
2016
Fall 20
16
Spring
2017
Summer
2017
Fall 20
17
Spring
2018
Summer
2018
Enrollment by Specialty (2011-2018)CMHC SC MCFC MCFC Cert
CPCE Examination Results Summary 2014 – 2018
Results from the aforementioned plot, provide a rough estimate for how UAB counseling students score on the CPCE (between 2014 and 2018) when compared to other counseling students across the nation. From these data it appears that UAB counseling students, on average, score above the national mean across all domains. UAB Counseling students appear to consistently demonstrate strong performances in Group, Career, and Professional Ethics. Conversely, UAB student scores tend to fall closer to the national means on diversity, Helping Relationships, and Research. This plot was constructed from the means and standard deviations, for each subscale, on non-exit CPCE exam scores. Then, we compared UAB students to the normative sample. The above plot represents the average difference statistic between the mean subscale scores for UAB students and the norming sample accounting for each administration of the CPCE between 2014 and 2018.
0.89
0.26
0.751.04
1.60
0.980.82
1.33
-2.50
-2.00
-1.50
-1.00
-0.50
0.00
0.50
1.00
1.50
2.00
2.50
Development Diversity HelpingRelationships
Group Career Appraisal Research Ethics
Average Deviation from National CPCE Scores (Non-exit) 2014-2018
Lower Upper AVERAGE
To better understand how UAB students compare to other counseling graduate students across the country, we plotted the average difference statistic between UAB students and the norming sample, across each domain, for the past 5 academic years.
An examination of these plots permit multiple interpretations. Among the most notable are: • UAB counseling students have consistently scored significantly better than national test takers in Human
Development.
2.03 2.08
0.30
2.08 2.15
0.00
0.50
1.00
1.50
2.00
2.50
2014 2015 2016 2017 2018
Development
0.59 0.490.21
0.77
-0.78-1.00
-0.50
0.00
0.50
1.00
2014 2015 2016 2017 2018
Diversity
-0.35-0.02
0.44
1.45
2.23
-0.500.000.501.001.502.002.50
2014 2015 2016 2017 2018
Helping Relationships
0.891.33
0.91
2.30
-0.25-0.500.000.501.001.502.002.50
2014 2015 2016 2017 2018
Groups
1.301.77
2.34
1.081.49
0.00
0.50
1.00
1.50
2.00
2.50
2014 2015 2016 2017 2018
Career
0.13
0.931.18
0.80
1.87
0.00
0.50
1.00
1.50
2.00
2014 2015 2016 2017 2018
Appraisal
0.29
1.15
0.79
1.43
0.42
0.00
0.50
1.00
1.50
2.00
2014 2015 2016 2017 2018
Research
0.881.42
1.09
3.15
0.110.000.501.001.502.002.503.003.50
2014 2015 2016 2017 2018
Ethics
• Since the UAB Counseling program has recruited and retained a stable set of core faculty who have worked to revise and advance the curriculum for both Theories and Skills, UAB students performed significantly better than national test takers in 2018.
• UAB students consistently outperform national test takers on Career, Research, and Appraisal. • In 2018, UAB students scored closer to the national mean on Diversity, Groups, and Professional Ethics. While
consistent with the performance of other counseling students across the nation, these mean performance declines were unexpected. It is worth noting that during UAB’s CACREP reaccreditation process, various adjunct faculty were recruited as instructors to allow program faculty sufficient time to complete the required self-study. It is possible that the intensive reaccreditation process had an indirect effect on student learning outcomes.
See Appendix I for additional plots comparing UAB test takers to counseling graduate students across the country. Please note that values falling below the lower or upper bounds (dotted line) represent statistically significant differences.
NCE Examination Results 2011 – 2018
We evaluated National Counselor Examination (NCE) results across a 7-year period, from 2011 through 2018. Notably, the CP prefers to emphasize results from the CPCE as a key performance indicator because the NCE is elective. As such, a number of students may opt out of the exam during any given academic year. Because of this, NCE results are regarded as supporting data used to either corroborate or contradict findings from the CPCE. Results from the NCE appear to generally support the findings discussed above. The most notable finding is that no deviations were statistically significant, suggesting that, on average, differences between UAB students and national test takers are trivial. For instance, UAB CP students appear to score above national test takers from CACREP accredited programs in Helping Relationships, Career, Appraisal, Research, and Ethics. However, the average national test taker (NTT) appears to score above CP students on Diversity. On average, CP students and NTTs are indistinguishable on Human Development and Group.
0.06-0.07
0.43
0.04
0.760.50 0.47
0.32
-2.50
-2.00
-1.50
-1.00
-0.50
0.00
0.50
1.00
1.50
2.00
2.50
Development Diversity HelpingRelationships
Group Career Appraisal Research Ethics
Average Deviation from NCE Scores (2011 -2018)
To better understand how UAB students compare to other counseling graduate students on the NCE, differences between UAB students and the national sample were plotted across each domain, for the past 7 academic years.
1.39 1.33
-0.76
0.29
0.90
0.29 0.21
-2.46
-0.87
0.47
-2.40
-1.68
0.46
0.96
-0.34
-1.71
-0.02-0.34
-0.13
-3.00-2.50-2.00-1.50-1.00-0.500.000.501.001.502.002.50
10/1/
11
10/1/
124/1
/13
10/1/
134/1
/14
10/1/
14
4/15 (
CMH)
4/15 (
SC)
10/15
(CM
H)
10/15
(SC)
4/16 (
CMH)
4/16 (
SC)
10/16
(CM
H)
10/16
(SC)
4/17 (
CMH)
4/17 (
SC)
10/17
(CM
H)
4/18 (
CMH)
4/18 (
SC)
Human Growth and Development (N=139)
-1.59
1.17
0.25
-0.43-0.21
-0.66
0.55
-1.45
0.550.27
-1.69
-0.07
1.51
0.71
1.36
-0.80
0.72
0.100.07
-2.50
-2.00
-1.50
-1.00
-0.50
0.00
0.50
1.00
1.50
2.00
2.50
10/1/
11
10/1/
124/1
/13
10/1/
134/1
/14
10/1/
14
4/15 (
CMH)
4/15 (
SC)
10/15
(CM
H)
10/15
(SC)
4/16 (
CMH)
4/16 (
SC)
10/16
(CM
H)
10/16
(SC)
4/17 (
CMH)
4/17 (
SC)
10/17
(CM
H)
4/18 (
CMH)
4/18 (
SC)
Social and Cultural Diversity (N=139)
1.55
2.67
-0.16
4.70
0.22-0.40
4.50
-1.94
0.39 0.39
-1.24
-2.33
1.23
-0.22
0.97
-1.74
0.67 0.780.56
-3.00
-2.00
-1.00
0.00
1.00
2.00
3.00
4.00
5.00
6.00
10/1/
11
10/1/
124/1
/13
10/1/
134/1
/14
10/1/
14
4/15 (
CMH)
4/15 (
SC)
10/15
(CMH)
10/15
(SC)
4/16 (
CMH)
4/16 (
SC)
10/16
(CMH)
10/16
(SC)
4/17 (
CMH)
4/17 (
SC)
10/17
(CMH)
4/18 (
CMH)
4/18 (
SC)
Helping Relationships (N=139)
1.291.84
-0.79
2.20
0.85
-0.90
0.65
-2.89
-0.170.42
-0.61
-2.67
1.55
-0.83
0.60
-2.13
-0.24 -0.48
0.31
-4.00
-3.00
-2.00
-1.00
0.00
1.00
2.00
3.00
10/1/
11
10/1/
124/1
/13
10/1/
134/1
/14
10/1/
14
4/15 (
CMH)
4/15 (
SC)
10/15
(CM
H)
10/15
(SC)
4/16 (
CMH)
4/16 (
SC)
10/16
(CM
H)
10/16
(SC)
4/17 (
CMH)
4/17 (
SC)
10/17
(CM
H)
4/18 (
CMH)
4/18 (
SC)
Group Work (N=139)
0.80
2.46
0.55
-0.16
0.580.73
1.73
-0.43
1.72
0.40
-1.28
0.95 0.99
0.18
1.95
-0.33
0.88
0.00 0.01
-2.50
-2.00
-1.50
-1.00
-0.50
0.00
0.50
1.00
1.50
2.00
2.50
3.00
10/1/
11
10/1/
124/1
/13
10/1/
134/1
/14
10/1/
14
4/15 (
CMH)
4/15 (
SC)
10/15
(CM
H)
10/15
(SC)
4/16 (
CMH)
4/16 (
SC)
10/16
(CM
H)
10/16
(SC)
4/17 (
CMH)
4/17 (
SC)
10/17
(CM
H)
4/18 (
CMH)
4/18 (
SC)
Career Counseling (N=139)
0.08
2.66
0.26
-0.42
0.55
0.08
1.77
-0.37
0.671.00
-0.95
-0.27
0.79
-0.49 -0.38
-1.45
1.50 1.40
0.30
-3.00
-2.00
-1.00
0.00
1.00
2.00
3.00
10/1/
11
10/1/
124/1
/13
10/1/
134/1
/14
10/1/
14
4/15 (
CMH)
4/15 (
SC)
10/15
(CMH)
10/15
(SC)
4/16 (
CMH)
4/16 (
SC)
10/16
(CMH)
10/16
(SC)
4/17 (
CMH)
4/17 (
SC)
10/17
(CMH)
4/18 (
CMH)
4/18 (
SC)
Appraisal (N=139)
Notably, UAB counseling students rarely score significantly higher (or lower) than the national mean across all domains measured by the NCE. This observation may indicate an area of improvement for the counseling program.
2.07
-0.03
-1.56
1.91
0.56
0.04 0.20
-1.17
0.66
2.30
-0.36
-0.95
1.65
1.090.68
-0.65
0.03
0.670.57
-2.50
-2.00
-1.50
-1.00
-0.50
0.00
0.50
1.00
1.50
2.00
2.50
3.00
10/1/
11
10/1/
124/1
/13
10/1/
134/1
/14
10/1/
14
4/15 (
CMH)
4/15 (
SC)
10/15
(CM
H)
10/15
(SC)
4/16 (
CMH)
4/16 (
SC)
10/16
(CM
H)
10/16
(SC)
4/17 (
CMH)
4/17 (
SC)
10/17
(CM
H)
4/18 (
CMH)
4/18 (
SC)
Research (N=139)
1.09
1.77
0.00
1.14
0.56
-0.64
1.13
0.16
0.66 0.66
-0.48
-2.26
1.36
-0.71
-0.19
-1.53
-0.51
1.21
-0.44
-2.50
-2.00
-1.50
-1.00
-0.50
0.00
0.50
1.00
1.50
2.00
2.50
10/1/
11
10/1/
124/1
/13
10/1/
134/1
/14
10/1/
14
4/15 (
CMH)
4/15 (
SC)
10/15
(CMH)
10/15
(SC)
4/16 (
CMH)
4/16 (
SC)
10/16
(CMH)
10/16
(SC)
4/17 (
CMH)
4/17 (
SC)
10/17
(CMH)
4/18 (
CMH)
4/18 (
SC)
Ethics (N=139)
Next, we compared whether CMHC and SC produced different scores on the NCE between 2015 – 2017. NCE did not provide data disaggregated by specialty/concentration before 2015 or after 2017. The following plots were produced:
0.410.97
1.752.32
1.031.38
0.55
1.94
-2.5-2
-1.5-1
-0.50
0.51
1.52
2.53
Develop
ment
Diversi
ty
Helping
Rela
tionships
Group
Career
Appraisal
Researc
hEthi
cs
Spring 2015 (CMH compared to SC)
0.43
1.17
2.11 2.28
1.301.63
0.53
2.09
-2.5-2
-1.5-1
-0.50
0.51
1.52
2.5
Develop
ment
Diversi
ty
Helping
Rela
tionships
Group
Career
Appraisal
Researc
hEthi
cs
Octoer 2015 (CMH compared to SC)
0.28
1.05
1.90
3.26
1.071.51
0.82
2.35
-3
-2
-1
0
1
2
3
4
Development Diversity HelpingRelationships
Group Career Appraisal Research Ethics
April 2016 (CMH compared to SC)
0.46
1.14
2.06 2.15
1.291.59
0.49
1.99
-2.5-2
-1.5-1
-0.50
0.51
1.52
2.5
Develop
ment
Diversi
ty
Helping
Rela
tionships
Group
Career
Appraisal
Researc
hEthi
cs
October 2016 (CMH compared to SC)
To interpret these plots, positive numbers indicate higher scores for CMH and negative numbers indicate higher scores for SC. As such, these plots demonstrate that CMH students, on average, score higher across all domains measured by the NCE. Also, CMH students are more likely to score significantly better than SC students in Helping Relationships, Group, and Ethics. As such, these domains may represent areas that need additional attention in the SC curriculum. In the CMH curriculum, such topics are more likely to receive additional and deeper coverage beyond the CACREP core. For instance, these topics are more likely to be encountered by CMH students in more depth during both Advanced techniques and Practicum/Internship. As such, SC students may benefit from additional coverage of these topics at later points in the curriculum.
0.41
1.13
2.06 2.15
1.291.59
0.49
1.99
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
Develop
ment
Diversi
ty
Helping
Rela
tionships
Group
Career
Appraisal
Researc
hEthi
cs
April 2017 (CMH compared to SC)
Counselor Competency Scale
To measure student competencies in clinical practice, program faculty ask site supervisors to administer the Counselor Competency Scale (CCS) as a developmental (mid-term and final) evaluation during ECG 695 (Practicum II), ECG 697 A (Internship I), and ECG 697 B (Internship II). To visualize student progress, we plotted mean scores for each subscale of the CCS across each phase of clinical training.
The difference in mean scores on the Counseling Skills subscale was not significantly different between ECG 695 and ECG 697 A. However, student scores obtained during ECG 697 B, were significantly lower than scores observed during either ECG 695 or ECG 697 B.
The difference in mean scores on the Professional Dispositions subscale was not significantly different between ECG 695 and ECG 697 A. However, student scores obtained during ECG 697 B were significantly lower than scores observed during either ECG 695 or ECG 697 B.
The difference in mean scores on the Professional Behaviors subscale was not significantly different between scores obtained during ECG 695, ECG 697 A, and ECG 697 B. Overall, it is evident that clinical site supervisors perceive student growth between ECG 695 and ECG 697A, but these findings are not statistically significant. Notably, supervisors tend to document that student competencies were significantly lower than previous observations on the Counseling Skills and Professional Dispositions subscales during ECG ECG 697B. In other words, at graduation, site supervisors appeared to rate student performance on these domains as significantly worse than when students started and midway through their internship experience. Student competency neither significantly increased or decreased on Professional Behaviors subscale as students progressed during their internship experience, seemingly indicating that supervisors didn’t observe any significant improvement or decrement during this period. To interpret these findings, it is important to acknowledge several considerations. First, students must achieve a minimum score to be considered minimally competent to progress in their training Therefore, to ensure minimum competency is achieved, each item restricts the passing range of student scores to the higher end of the 8 point scale (between 6 and 8). It should be noted that the scaling configuration progresses evenly as 2, 4, 6, and 8. While the vast majority of scores tend to fall on even values such as six or eight, some supervisors were observed to enter odd values such as three or 7. Second, faculty supervisors evaluate students against a progressively higher standard. For example, when students reach ECG 697A, they are evaluated against a higher standard than required for ECG 695 but at a lower level of mastery than expected during ECG 697B. Moreover, a score of 6 during ECG 695 entails a lower level of mastery than the same value during ECG 697A. As such, these issues create potential problems when estimating item difficulty and person ability. Based on how the instrument was constructed, supervisors are not able to use the scaling configuration in such a way to elicit meaningful interpretations. More specifically, higher scores should indicate that a higher level of mastery is needed, but for this scale the vast majority of students are scoring in the higher categories across all items, indicating poor discrimination between students with lower or higher levels of mastery. Essentially, the scale is operating as a binary indicator of competent or not. This, coupled with poor sensitivity of scores to student developmental level, ultimately yields indicators of student performance which are difficult to interpret. For instance, we attempted to perform for advanced statistical analyses capable of identifying items which tend to be more difficult for UAB Counseling interns. These values would then have been useful for targeting areas of improvement that could be addressed by increasing coverage of these topics in the curriculum. Given the issues with how supervisors are using the scaling configuration, such analyses were not possible as the data do not sufficiently conform to the requisite assumptions. It is recommended that the UAB Counseling program faculty reevaluate scoring instructions for the CCS document and provide more focused training for supervisors, so as to ensure evaluators are using the scale consistently across students and time. For example, the minimum cutoffs might be removed and supervisors should be instructed to evaluate based on mastery rather than developmental level. Once the scoring issues are addressed, further analysis on item performance is necessary to establish which items provide the most useful information and which items may be discarded from future analyses.
Appendix I CPCE Scores
0.171790561
-0.34372469
-0.648351573
1.37280605
-0.1737255480.081952311
0.8525208641.052914873
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
Development Diversity HelpingRelationships
Group Career Appraisal Research Ethics
CPCE 2/26/14
0.249927235
1.341725613
0.3095420210.566409808
2.433217539
-0.62358289-0.457583858
0.813365579
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
3
Development Diversity HelpingRelationships
Group Career Appraisal Research Ethics
CPCE 6/22/14
-0.463942837
0.769752403
-0.702090361
0.740412848
1.635831428
0.93389179
0.463095298
0.784940596
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
Development Diversity HelpingRelationships
Group Career Appraisal Research Ethics
CPCE 10/22/14
3.06599929
1.982332996 2.100256395
4.6142965554.315885618
1.394647
3.5485983433.250396801
-3
-2
-1
0
1
2
3
4
5
Development Diversity HelpingRelationships
Group Career Appraisal Research Ethics
CPCE 3/2/15
0.670748737
-0.239360028
-1.18338429
-1.741125521
-0.696420921
0.289219425
-0.622697487
0.16520122
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
Development Diversity HelpingRelationships
Group Career Appraisal Research Ethics
CPCE 7/1/15
-0.309237057 -0.270105506
-0.977099439
1.113557058
1.689936409
1.113995526
0.538671028
0.85442351
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
Development Diversity HelpingRelationships
Group Career Appraisal Research Ethics
CPCE 10/14/15
4.20
1.221.42
2.22
3.96
1.81 1.941.52
-3
-2
-1
0
1
2
3
4
5
Development Diversity HelpingRelationships
Group Career Appraisal Research Ethics
CPCE 2/24/16
-0.53-0.71 -0.64 -0.65
0.22 0.24
-1.40
-0.47
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
Development Diversity HelpingRelationships
Group Career Appraisal Research Ethics
CPCE 6/22/16
1.95
0.12
0.55
1.17
2.85
1.481.82
2.21
-3
-2
-1
0
1
2
3
4
Development Diversity HelpingRelationships
Group Career Appraisal Research Ethics
CPCE 10/19/16
2.37
1.812.18
2.90
2.43
0.39
1.74
4.70
-3
-2
-1
0
1
2
3
4
5
6
Development Diversity HelpingRelationships
Group Career Appraisal Research Ethics
CPCE 2/22/17
1.85
0.87
2.822.60
0.99
1.921.69
3.68
-3
-2
-1
0
1
2
3
4
Development Diversity HelpingRelationships
Group Career Appraisal Research Ethics
CPCE 6/21/17
1.85
0.87
2.822.60
0.99
1.921.69
3.68
-3
-2
-1
0
1
2
3
4
Development Diversity HelpingRelationships
Group Career Appraisal Research Ethics
CPCE 6/21/17
0.932678858
-0.107562103
-2.577491417
0.655729553
2.183664967
0.808002226
1.536865729
0.37747714
-3
-2
-1
0
1
2
3
Development Diversity HelpingRelationships
Group Career Appraisal Research Ethics
CPCE 2/22/18
-1.033491809
-1.57344366
-2.941198452
-1.09827572
0.703820546
-1.52596322
0.009526475
-1.081919997
-4
-3
-2
-1
0
1
2
3
Development Diversity HelpingRelationships
Group Career Appraisal Research Ethics
CPCE 6/12/18
0.097619612
-0.657420006
-0.913834322
-0.297666653
1.579662805
0.716390939
-0.293655265
1.044646057
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
Development Diversity HelpingRelationships
Group Career Appraisal Research Ethics
CPCE 10/19/18
