View
0
Download
0
Category
Preview:
Citation preview
Running head: STUDENT AFFECT 1
Student Affect During an HBCU Summer Research Program
Avis D. Jackson, Eric Pickering Boorman,
Farin Kamangar, Christine F. Hohmann
Morgan State University
Avis D. Jackson, Research Associate, ASCEND Center for Biomedical Research,
Morgan State University; Eric Pickering Boorman, Psychometrics Program, Department of
Psychology, Morgan State University; Farin Kamangar, Principal Investigator, ASCEND Center
for Biomedical Research, and Professor and Associate Dean for Research, School of Computer,
Mathematical, and Natural Sciences, Morgan State University; Christine F. Hohmann, Research
Enrichment Core Director, ASCEND Center for Biomedical Research, and Professor of Biology,
School of Computer, Mathematical, and Natural Sciences, Morgan State University.
Acknowledgements: This research is supported by funding from the NIH Diversity
Program Consortium BUILD Award (UL1MD009605/RL5MD009590/TL4MD009635). We
would like to thank Ms. Gillian Silver for her help with editing this manuscript.
Correspondence concerning this article should be addressed to Avis Jackson, Ph.D.,
ASCEND Center for Biomedical Research, Morgan State University, Carnegie Hall G64,1700
East Cold Spring Lane, Baltimore, Maryland, 21251. Email: Avis.Jackson@Morgan.edu
Running head: STUDENT AFFECT 2
Abstract
The popularity and effectiveness of intensive summer research programs to increase student self-efficacy is known. The Summer Research Institute (SRI) training experience, as part of undergraduate student training in Morgan State University’s NIH BUILD program, uses an entrepreneurial approach to prepare students for careers in health-related research. Bandura’s self-efficacy theory’s (1977) four antecedents are represented in the SRI curriculum, which provides multiple opportunities for mastery experiences and for moments of roused feelings. These occurrences are accompanied by extensive multilayered mentoring, where the mentors provide verbal encouragement, and facilitate various modes of academic, psychosocial and institutional support. To our knowledge, student affect over time has not been tested to assess impact or program effectiveness in a summer research training program. This study is based on the qualitative assessment of bi-weekly journals of 28 students in the SRI. The practice of students consistently writing journals is aligned with the scaffolded knowledge integration framework proposed by Linn (1995). The journals were reviewed for their cumulative affective content and change over time. The students responded as expected with positive and negative affect throughout the program and ended with overwhelmingly positive affect with their concluding presentations. Using Linguistic Inquiry Word Count (LIWC), a text analysis program, we matched the fluctuations in affect to activities during the program and interpreted the changes for program assessment. This type of analysis opens a window into student affective responses to training components that, to our knowledge, have not been widely used for research training programs of this kind.
Keywords: research training programs, NIH BUILD, emotion, LIWC, cooperative learning, self-efficacy
Running head: STUDENT AFFECT 3
Introduction
Intensive summer research internship programs, sometimes described as research “boot
camps,” have become a mainstay in the landscape of undergraduate student research training
(Ashley, Cooper, Cala, & Brownel, 2017; Ghee, Keels, Collins, Neal-Spence, & Baker, 2016;
Lopato, 2007). These experiences have been shown to be quite effective in increasing student
science interest, persistence, identity, and self-efficacy, which are all key components of
increasing student success, college retention and completion (Ashley et al. 2017; Ghee, et al.
2016; Lopato, 2007). These findings across analyses of many projects have led to their adoption
across a spectrum of training programs. The common denominator of research “boot camps” is
that they are residential programs, expecting students to work with each other and on their
research experiences exclusively for a continuous stretch of 8 to 10 weeks, and they usually end
with a capstone experience such as an oral or poster presentation of the summer’s research.
The ASCEND Summer Research Institute (SRI) at Morgan State University (MSU) is the
initial component of a two-year scholarship program sponsored by the National Institutes of
Health Building Infrastructure Building Infrastructure Leading to Diversity (NIH BUILD)
program at MSU (Kamangar et al., 2017). Approximately 30 sophomore and junior STEM and
social/behavioral sciences students enter the SRI each summer, and 20 of these students are then
selected to become ASCEND Scholars, which is a two-year program that starts in the fall
subsequent to the SRI. The ASCEND SRI, like other internship programs referenced above, is a
residential, research-intensive training model. Unlike most other programs, however, the SRI
uses a group-based, cooperative learning model to acquire research skills in health sciences.
Instead of training in individual faculty laboratories, students proceed through a “backward
Running head: STUDENT AFFECT 4
design” curriculum that teaches them research principles, provides hands-on experience with
research techniques, and culminates in students designing group-generated research proposals
(Handelsman et al., 2004). These research proposals are a high-stakes capstone experience for
the students, because they factor into their selection for the ASCEND Scholars program.
In STEM research programs for underrepresented minorities (URMs), science identity
increases and positively influences persistence (Estrada-Hollenbeck et al. 2011; Linn et al. 2015;
Robinson et al. 2016). The students’ belief in their capacity to respond well and perform over
time, or self-efficacy, is formed from their experiences. It strengthens their response to, and
mediates, anxiety while matriculating through their majors and programs (Bandura, 1977;
Estrada-Hollenbeck et al. 2011; Robinson et al. 2016). STEM self-efficacy positively influences
persistence in STEM (Chemers et al. 2011). Further, self-efficacy and science identity builds as
students experience successes, failures, and positive mentored feedback while constructing skills
(Estrada-Hollenbeck et al. 2011; Lent, 2007; Robinson et al. 2016).
Bandura’s self-efficacy theory (1977) posits four antecedents for self-efficacy:
performance accomplishments (mastery experiences), vicarious experience (such as role models
or mentors), verbal persuasion (encouragement), and emotional arousal (positive or negative
affect). Self-efficacy is widely referenced in research programs as integral to student success and
to continuance in STEM fields through graduate school and into careers (Chemers et al. 2011;
Estrada-Hollenbeck et al. 2011; Frantz et al. 2017). The four origins of self-efficacy undergird
social cognitive career theory (SCCT; Lent, Brown, & Hackett, 1994). SCCT is also used as the
theoretical framework to explain URM STEM persistence (Estrada-Hollenbeck et al. 2011;
Frantz et al. 2017).
Running head: STUDENT AFFECT 5
The four antecedents to self-efficacy can be interpreted as specifically represented and
subsequently witnessed in the curriculum of the ASCEND SRI. The “boot camp” format
provides multiple opportunities for mastery experiences and for moments of roused feelings.
These occurrences are accompanied by extensive multilayered mentoring, where the mentors
provide verbal encouragement and facilitate various modes of academic, psychosocial and
institutional support. The students respond with positive and negative affect throughout the
program and end with what we believe is overwhelmingly positive affect in their concluding
presentations.
Journaling allows students to consider their experiences: what they have learned, and
how these thoughts fit into their understanding of how the world functions (Linn et al. 2015).
Linn further notes that the practice of students writing journals is aligned with a scaffolded
knowledge integration framework (Linn,1995), which provides a structure to assist students with
developing inquiry skills, and to help them refine, clarify and weigh acquired knowledge to
advance in and integrate into their field (Linn et al. 2015). Thus, analyses of student journals can
provide a window into the students’ affectual experience as they master the science content.
The purpose of this study was to examine changes in emotional arousal (positive and
negative affect) during the course of ASCEND SRI, measured through qualitative assessment of
required bi-weekly journal entries that the students are asked to maintain throughout the SRI.
These journals are reviewed for their cumulative affective content and change over time. We
match the fluctuations in affect to activities during the program and interpret the changes for
program assessment. We hypothesized that positive and negative changes in student affect would
relate to program activities and self-efficacy. To our knowledge, student affect over time has not
Running head: STUDENT AFFECT 6
been tested through a summer research training program to assess either its impact or program
effectiveness.
Methods
IRB approval was included under the overall ASCEND NIH BUILD grant. Twenty-eight
students completed journals reflecting on their experiences in the SRI 2017 program. The journal
entries were open-ended, allowing students the freedom to discuss any topics the students felt
were relevant to their experiences in the program, building on previous journal entries. Journals
were completed and submitted electronically, two times per week, with the exceptions of the first
and final weeks during which only one journal entry was completed.
Data were entered into Linguistic Inquiry Word Count (LIWC), a computer program
designed to quantify various aspects of written text such as emotional content and cognitive
content. LIWC analyzes the individual words participants wrote and codes the words based upon
the LIWC internal dictionary. The LIWC internal dictionary determines the proportion of
emotional content, cognitive content, and other linguistic categories (Pennebaker, Boyd, Jordan,
& Blackburn, 2015). LIWC variables selected for further inquiry in this study comprised: Total
Affect, Anger, Anxiety, Negative Emotion, Positive Emotion, and Sadness. These variables
represent all of the emotional content variables that LIWC produces, and all of these variables
were chosen for the study. By analyzing them, we can determine particular emotional feelings or
trends within the SRI program.
Individual journal entries were extracted into Microsoft Word files without alteration of
the original documents. Documents were named to reflect a participant identification number as
Running head: STUDENT AFFECT 7
well as a code to represent the journal entry number. These data were then entered into LIWC,
which analyzed each individual journal entry that a student produced. All students’ LIWC scores
were combined into a single data file and arranged by the file codenames such that each entry
corresponds to one student’s LIWC scores across the 14 weeks. Data were screened to identify
any participants with outlier scores on any of the six LIWC emotion variables in any of the 14
journal entries. Outliers were removed so as to not skew any individual or combined results.
Results were similar regardless of whether outliers were included or removed.
Results
A series of Repeated Measures ANOVA analyses was conducted to determine how each
emotional content variable changed over the 14 journal entries. As shown in Table 1 and Table 2,
most variables (Total Affect, Anxiety, Negative Emotion, and Positive Emotion) displayed
statistically significant changes across time. These trends are best conceptualized by their orders
or the sum of the exponents (linear is x1, quadratic is x2, cubic is x3, 4th order is x4, etc.). These
exponents reflect how many turning points, or points where the curve changes from positive to
negative or from negative to positive, are seen. The number of turning points can be determined
by subtracting the order of the trend by one (e.g., a linear trend has zero turning points, a
quadratic trend has one turning point, a cubic trend has two turning points, a 4th order trend has
three turning points). While many changes were lower order (e.g., linear, quadratic), the trends
with the highest effect sizes are higher order trends (e.g., 5th order, 6th order, 7th order, 8th order,
and 10th order trends). These higher order trends alternated between positive and negative values
repeatedly, as determined by the number of turning points, throughout the eight-week period.
Some variables did not display significant trends (Anger, Sadness); rather, these qualities of the
Running head: STUDENT AFFECT 8
written text stayed at a near-zero level throughout the SRI program. Results of this analysis are
illustrated in Figure 1.
When comparing these non-linear trends to the timeline of the SRI program, we
discovered that peaks and valleys corresponded closely to specific training modules in the SRI
curriculum, suggesting that these training experiences precipitated the affective changes seen
(See Figure 1). Verbal indicators of negative emotion and anxiety peaked around week five of
the eight-week experience. This point in the training represents the journal entry after the first,
longest and most complicated wet-lab module, which includes multiple revisions to their group
reports, group presentations, and multiple feedback sessions. The introduction of the second lab-
module, with an all-day seminar that includes presentations from multiple researcher/professors,
followed closely on that experience, and on its heels was an all-day trip to a partner institution
that included lab visits and presentations and interaction with students from other programs. The
following three journal records (See Figure 1 records 6, 7, 8) show decreasing levels of both
negative emotions and anxiety, and these emotions remain relatively low throughout the
remainder of the program. We interpret these trends as indication of the students’ acclimation to
the program and its requirements, while simultaneously recognizing their capabilities and
consequently responding with lower anxiety and negativity in their written expression. These
results are the type of responses one might expect based on Bandura’s Self-Efficacy theory
(1977).
The indicator of positive emotion drops to its lowest level after the third journal entry,
which is at the beginning of the first module discussed above, and modulates up and down after
this point. It maintains a largely positive trajectory, reaching its highest level at the end of the
SRI, at the time that students were preparing their capstone research proposals. We interpret this
Running head: STUDENT AFFECT 9
result as further evidence of students’ increasing self-efficacy in their abilities to appreciate and
begin to incorporate the skills they have acquired while successfully meeting the challenges
provided by the program.
We have added to our LIWC journal analyses another set of data generated in the
evaluation of our ASCEND SRI program that provides complimentary information. The
Attitudes Toward Research Scale (ATR; Papanastasiou, 2005) is designed to measure student
attitudes towards a basic research methods course. The 30-question scale uses a five-point self-
report Likert scale with 1=Strongly Disagree, 2=Disagree, 3=Neutral, 4=Agree, 5=Strongly
Agree. Sample items include “I enjoy research” and “I use research in my daily life.” The ATR
contains five subscales including Research Anxiety and Positive Attitudes Toward Research.
Comparison of pre-post responses of two of these ATR subscales provides convergent
validity to our interpretations of students’ journal entries. The Research Anxiety subscale pre-
post paired t-test results show significant reduction in anxiety (t(26)= 2.109, p<.05). Further, the
Positive Attitudes Toward Research subscale showed consistent high levels (pre M=4.12,
SD=.53; post M=4.19, SD=.56; see Figure 2).
Discussion and Conclusion
The SRI is a high-intensity and very challenging experience for participants. We
anticipated students’ experience and expression of anxiety resulting sometimes in negative
emotions during the course of training. We were surprised that the highest levels of such
negative affect correlated with wet-bench research experiences, because students in previous
cohorts have asked us for more of such training experiences. In this particular cohort, that may
Running head: STUDENT AFFECT 10
have been the first time in their academic careers that many of the students were introduced to
wet-bench research. The high level of positive emotions and low level of anxiety in latter weeks
of the SRI, as participants define research questions and subsequently prepare their selected
group proposals, indicate that students have gained a sense of mastery. The following verbatim
quotes from students illustrate this:
It is amazing to see how far everyone of us have come from the first day of
not knowing how to properly achieve literature review to forming our own
research proposal. (Student 1)
I appreciate all the support and contributions to proposal development
and the effort that everyone puts in to make the program beneficial to everyone….
I appreciate being able to step outside of my comfort zone a little. As our final
presentation date approaches, I have become anxious yet excited. (Student 2)
The instructors have trained us to start thinking for ourselves, and to push
our ideas. … This summer showed me that I can be more than I am right now. It
showed me that I don’t have to be scared to speak up and make myself visible.
I’ve learned to not be scared to take risks, and to stop withdrawing. (Student 3)
Overall, the training experience leaves students with a mentored and
supported sense of accomplishment in the preparation and public presentation of
their formal, high-stakes group presentations. As anticipated, this analysis of the
SRI demonstrates the fourth antecedent—emotional arousal—of Bandura’s Self-
Efficacy theory (1977).
Running head: STUDENT AFFECT 11
To our knowledge, this is the first time a qualitative word count analysis has been used to
assess student experiences of positive and negative affect in a summer research training program
within the framework of Bandura’s Self-Efficacy theory. Future studies will subject the journal
entries to more fine-grained, manually-coded analysis of student responses to explore the four
antecedents for self-efficacy and other specific concepts of attitudes towards research that have
been raised in the literature. Important to the Understanding Interventions community is that our
data suggest that this may be a good approach to gauge student responses to training
components. As a summative review of program components, the assessment is valuable for
insight into specific affective impacts across time that may be obscured with simple pre-post
assessment design.
Running head: STUDENT AFFECT 12
References
Ashley, M., Cooper, K.M., Cala, J.M. and Brownel, S.E. (2017) Building Better Bridges into STEM: A Synthesis of 25 Years of Literature on STEM Summer Bridge Programs. CBE Life Science Education, 16(3):1-13.
Bandura, A. (1977). Self-efficacy: Toward a Unifying Theory of Behavioral Change.
Psychological Review 84, 191-215. Chemers, M. M., Zurbriggen, E. L., Syed, M., Goza, B. K., & Bearman, S. (2011). The Role of
Efficacy and Identity in Science Career Commitment Among Underrepresented Minority Students. Journal of Social Issues, 67(3), 469-491. doi:10.1111/j.1540-4560.2011.01710.x
Estrada-Hollenbeck, M., Woodcock, A., Hernandez, P. R., & Schultz, P. W. (2011). Toward a
Model of Social Influence that Explains Minority Student Integration into the Scientific Community. J Educ Psychol, 103(1), 206-222. doi:10.1037/a0020743
Frantz, K. J., Demetrikopoulos, M. K., Britner, S. L., Carruth, L. L., Williams, B. A., Pecore, J.
L., . . . Goode, C. T. (2017). A Comparison of Internal Dispositions and Career Trajectories after Collaborative versus Apprenticed Research Experiences for Undergraduates. CBE Life Sci Educ, 16(1). doi:10.1187/cbe.16-06-0206
Ghee, M., Keels, M., Collins, D., Neal-Spence, C. and Baker, E. (2016), Fine-Tuning Summer
Research Programs to Promote Underrepresented Students’ Persistence in the STEM Pathway. CBE Life Science Education, 15(28): 1-11.
Handelsman, J., Ebert-May, D., Beichner, R., Bruns, P., Chang, A., DeHaan, R., . . . Wood, W.
B. (2004). Education. Scientific teaching. Science, 304(5670), 521-522. doi:10.1126/science.1096022
Kamangar, F., Silver, G., Hohmann, C., Hughes-Darden, C., Turner-Musa, J., Haines, R. T.,
Jackson A., Aguila H. N., Sheikhattari, P. (2017). An entrepreneurial training model to enhance undergraduate training in biomedical research. BMC proceedings,11(Suppl 12), 69-77.
Running head: STUDENT AFFECT 13
Lent, R. W., Brown, S. D., & Hackett, G. (1994). Toward a unifying social cognitive theory of career and academic interest, choice, and performance. Journal of Vocational Behavior, 45(1), 79-122.
Linn, M. C. (1995). Designing computer learning environments for engineering and computer science: The scaffolded knowledge integration framework. Journal of Science Education and Technology, 4(2), 103-126. doi:10.1007/bf02214052
Linn, M. C., Palmer, E., Baranger, A., Gerard, E., & Stone, E. (2015). Education. Undergraduate
research experiences: impacts and opportunities. Science, 347(6222), 1261757. doi:10.1126/science.1261757
Lopato, D., (2007), Undergraduate research experiences support science career decisions and
active learning. CBE Life Science Education, 6 (4): 297-306. Pennebaker, J.W., Boyd, R.L., Jordan, K., & Blackburn, K. (2015). The development and
psychometric properties of LIWC 2015. Austin, TX: University of Texas at Austin. DOI: 10.15781/T29G6Z
Richardson, J. T. (2011). Eta squared and partial eta squared as measures of effect size in
educational research. Educational Research Review, 6(2), 135-147. doi: 10.1016/j.edurev.2010.12.001
Robinson, S., Neergaard, H., Tanggaard, L., & Krueger, N. F. (2016). New horizons in
entrepreneurship education: from teacher-led to student-centered learning. Education + Training, 58(7/8), 661-683. doi:10.1108/ET-03-2016-0048
Figure 1. Linguistic Inquiry Word Count (LIWC) variations in emotional variables across 14
journal entries. Twice weekly journal entries started at the end of the 1st week (entry 1) in the
SRI and concluded, after students completed their research proposals (14). Text boxes and
arrows mark key SRI activities that coincide with changes in affect.
Figure 2. Pre- and post-test results of two Attitudes Toward Research Scale (ATR)
(Papanastasiou, 2005) subscales indicated significantly reduced scores in the research anxiety
subscale and improved scores in the positive attitudes toward research subscale.
*p < .05
Table 1
Table of Repeated Measures ANOVA Results
LIWC Variable
Statistical Test
Degrees of Freedom Effect
Degrees of Freedom Error
F Partial eta square
Observed power
Total Affect Sphericity Assumed
13 78 6.09*
0.50 1.00
Anger Sphericity Assumed
13 65 0.72
0.13 0.39
Anxiety Sphericity Assumed
13 91 9.32***
0.57 1.00
Negative Emotion
Sphericity Assumed
12 91 6.74***
0.49 1.00
Positive Emotion
Sphericity Assumed
13 91 4.78***
0.41 1.00
Sad Sphericity Assumed
13 91 0.85
0.11 0.48
Note. * indicated p < .05. ** indicated p < .01. *** indicated p < .001.
Table 2
Table of Linear and Nonlinear Trends across LIWC Emotion Variables
LIWC Variable Trend Degrees of Freedom Effect
Degrees of Freedom Error
F Partial eta square
Total Affect Cubic 1 6 7.97*
0.57
4th order 1 6 8.02*
0.57
5th order 1 6 7.50*
0.55
8th order 1 6 37.97***
0.86
10th order 1 6 8.27*
0.58
Anxiety Linear 1 7 9.33*
0.57
Quadratic 1 7 25.55***
0.79
Cubic 1 7 14.28**
0.67
5th order 1 7 28.40***
0.80
6th order 1 7 12.58**
0.64
7th order 1 7 13.80**
0.66
8th order 1 7 9.00*
0.56
Negative Emotion
Linear 1 7 5.96*
0.46
Quadratic 1 7 18.50**
0.73
5th order 1 7 31.50***
0.82
7th order 1 7 9.80*
0.58
8th order 1 7 13.45**
0.66
Positive Emotion
Quadratic 1 7 11.92*
0.63
Cubic 1 7 10.52*
0.60
4th order 1 7 20.62**
0.75
8th order 1 7 14.27**
0.67
10th order 1 7 6.84*
0.49
Note. Sad and anger not included as these trends were not significant. * indicated p < .05. ** indicated p < .01. *** indicated p < .001.
Recommended