Upload
phamkien
View
213
Download
0
Embed Size (px)
Citation preview
Running Head: ACADEMIC ADVISING LEARNING OUTCOMES 1
Academic Advising Learning Outcomes
Megan Tucker
CTCH 826: Short Paper
George Mason University
ACADEMIC ADVISING LEARNING OUTCOMES 2
Academic Advising Learning Outcomes
Assessment is a way for universities and colleges to systematically gather, analyze and
interpret evidence to determine how student learning matches the school's expectations (Suskie,
2009). Specifically, outcomes-based assessment has become a prevalent area of research with
most colleges and universities assessing the impact of the college experience on student learning
and growth in the respective majors, general education programs, as well as in individual courses
(Bresciani, 2006; Driscoll & Wood, 2007; Smith, Szelest & Downey, 2004; Terenzini & Wright,
1987). However, research has begun to shift focus to other academic structures such as academic
advising (Haney & McClellan, 2009; Hurt, 2007; McClellan, 2011; Smith, Szelest & Downey,
2004). Though the research in this area has expanded, researchers have approached assessment
of academic advising in very different ways. The goal of this paper will be to identify the
prevalent methods of assessment for undergraduate academic advising and use this information
to create a proposed protocol for general academic advising assessment with the focus on
establishing student learning outcomes.
This research explores existing literature on learning outcomes related to academic
advising. Furthermore, the paper examines a variety of ways to assess these learning outcomes
using different theories and models. Through an analysis of literature on outcomes-based
assessment, learning outcomes in academic advising and pre-existing models of assessment, a
new option for assessing academic advising is introduced. The proposed assessment for
academic advising includes a mission and established goals, learning outcomes based in Bloom's
(1956) taxonomy, and a three-pronged assessment protocol encompassing progress and degree
completion statistics, student focus groups and an exit survey. Finally, implications for further
research and implementation are discussed.
ACADEMIC ADVISING LEARNING OUTCOMES 3
Background and Literature
Outcomes-Based Assessment
A plethora of literature exists on outcomes-based assessment as it pertains to the program
level of academia. Bresciani's (2006) text explores ways of showing how to conduct assessment
through sharing best practices of institutions that focus on improvements in teaching/learning,
research and service. Specifically the text focuses on good practices in outcomes-based
assessment programs in an effort to transform institutional decision making regarding student
learning and development (Bresciani, 2006). Likewise, Driscoll and Wood (2007) approach the
how-to process of outcomes-based assessment through the themes of intrinsic motivation and
promoting faculty trust through constructive dialogue and collaboration.
Terenzini and Wright (1987) utilize an ex post facto survey system for students to assess
their academic development. This study models the sources of influence on students' reported
academic development over a four-year period, focusing on institutionally controllable
influences on student growth and assessing how those influences vary from one year to another.
The surveys address the following points: 1 . Gaining factual knowledge; 2. Evaluating ideas,
materials, and methods; 3. Applying abstractions or principles in problem solving; 4. Learning
principles, generalizations, and theories; 5. Recognizing general principles in specific
events/conditions; 6. Understanding a particular discipline's research methods; 7. Understanding
a particular discipline's various schools of thought; 8. Developing a sense of the interrelatedness
of different disciplines. This survey approach is utilized in the assessment protocol introduced in
this paper via exit surveys.
ACADEMIC ADVISING LEARNING OUTCOMES 4
Finally, Smith, Szelest and Downey (2004) utilize Palomba and Banta's (1999) six
essentials of successful assessment that provide a guide to performing outcomes assessment and
apply it to academic support structures, such as academic advising. First, there should be an
agreement regarding the goals and objectives for learning, followed by a concrete plan of
assessment to measure outcomes. The third essential requires representation of students, faculty,
and staff in the process. The fourth component regards the selection of instruments used to
measure the outcomes of objectives. The last two essentials speak to the improvement loop
whereby results are used to improve practice and to assess the assessment (Smith, Szelest &
Downey, 2004). According to the authors, this framework, which has been successful in the
assessment of academic majors and the assessment of general education, can also be applied to
academic support units (Smith, Szelest & Downey, 2004). The last two essentials here of the
improvement loop and continual assessment are incorporated into the proposed model of
assessment in this paper (See Appendix D)
Models for Assessment
Research on assessment of academic advising has used a variety of models and
approaches. Haney and McClellan's (2009) research in assessment of student learning within
student affairs evolved through three transformative initiatives: (1) the institution’s decision to
shape itself into a learning college; (2) the revision of accreditation standards, placing increased
emphasis on institutional assessment and assessment of student learning; and (3) a statewide
initiative to develop a model for the assessment of student learning within student affairs units at
community colleges. These initiatives may be adapted to fit a larger university structure.
Likewise, Smith, Szelest, and Downey (2004) presents the development and
implementation of a program of outcomes assessment at a 4-year public university in the
ACADEMIC ADVISING LEARNING OUTCOMES 5
northeast. Results indicate that students come to college with varying expectations of the role of
academic advisors and range on a continuum of academic and social preparation to meet the
demands of higher education. A majority of students are meeting the stated learning outcomes.
Feedback from the assessment model is being utilized to improve the quality of services
provided to successive cohorts of students at the institution (See Appendix B).
Erlich and Russ-Eft's (2011) review of social cognitive theory constructs of self-efficacy
and self-regulated learning is applied to academic advising for the purposes of assessing student
learning. They incorporate the development of a model for assessing student learning outcomes
in academic advising using these theoretical constructs first established by Bandura (1986). The
use of a theoretical framework compliments their incorporation of the model. In future research
related to this proposed protocol, a similar framework would be utilized.
Metzner (1989) collected data from 1,033 freshmen at a public urban university and
examined the effect of the perceived quality of academic advising on student attrition in a model
of the student attrition process (See Appendix A). Metzner found that high-quality advising
negatively influenced attrition through effects on GPA, satisfaction in the role of a student, the
value of a college education for future employment, and intent to leave the university. Low-
quality advising was related to greater attrition than was high-quality advising, but, on the other
hand, low-quality advising was associated with less attrition than no advising at all. The focus on
quality of academic advising contributes to this study because of the focus on attrition and
establishing greater satisfaction in advising. The goal of the proposed assessment model is to
foster high quality advising and discourage attrition, which is the same goal for the model
developed here.
Learning Outcomes for Academic Advising
ACADEMIC ADVISING LEARNING OUTCOMES 6
Establishing learning outcomes for academic advising may need to broader in language
than outcomes for a specific program would be. The reason for this is the interdisciplinary focus
on academic advising and the attempt at creating a universal set of outcomes that can be applied
in most any academic program by advisors. Hurt (2007) discusses possible learning outcomes for
academic advising based on Bloom's taxonomy of educational outcomes:
1. List the classes student can take to fulfill general-education critical-thinking
requirements (knowledge)
2. Explain in their own words the importance of general education in their degree
(comprehension)
3. Calculate their grade point average (application)
4. Compare and contrast career options based on choice of major (analysis)
5. Develop a plan to graduate within 4 years (synthesis)
6. Decide which job offer to accept (evaluation)
These outcomes address a broad spectrum of concerns students many have without being too
specific to one program or college. I have utilized this sequence of outcomes as a baseline for
this paper's recommendations.
McClellan (2011) discusses student learning outcomes and the idea of using a balanced
scorecard for academic advising. The proposed the use of the balanced scorecard concept to
advising program assessment and evaluation (See Appendix C for balanced scorecard for
advising). There are five suggested areas of focus: learning outcomes, student perspective,
advisor perspective, program perspective, and budget perspective (McClellan, 2011). These areas
of focus became part of the structure of the proposed mission and goals statement for this
project.
ACADEMIC ADVISING LEARNING OUTCOMES 7
Similarly, Campbell and Nutt (2008) note that while there are many ways to support
student immersion and engagement in learning, one strategy that is progressively more
acknowledged for its potential in this regard is academic advising. According to the authors,
outcomes-based assessment for academic advising demands the development of an assessment
plan through which "learning opportunities are developed, benchmarks for success are identified,
and multiple measure are used to gather evidence to inform improvement and demonstrate
program success" (Campbell & Nutt, 2009, p.7). The focus here is on learning outcomes in
regards to academic advising. The use of benchmarks here are recycled in the application of the
proposed assessment in this paper. The benchmarks in this case are identified through the
learning outcomes established below.
Application
Based on the literature there may be a variety of ways to implement and assess learning
outcomes for academic advising. One way to assess learning outcomes for academic advising
would be through a self-study (Bresciani, p. 24). Bresciani (2006) notes the following goals for
learning outcomes: (1) Program mission and goals, (2) Major learning outcomes, (3) Assessment
protocol for each MLO, and (4) Student learning.
Keeping these goals in mind I first developed a mission and goals for an academic
advising program. For the purpose of this paper the intended mission and goals are similar to
those in place for an advising program for which I work at George Mason University, as well as
drawing from the balanced scorecard approach that McClellan (2011) discusses. However, in
order to fit the needs of this paper they have been broadened for a general application to a variety
of programs and colleges, and the budget aspect has been removed to maintain focus on the
advisor and advisee and less on the university. First and foremost the mission of the academic
ACADEMIC ADVISING LEARNING OUTCOMES 8
advising program is to assist students in their progress to degree completion. Focusing on student
development and learning, the advisor will engage in a professional, interpersonal relationship
with the student in order to identify academic interests and strengths, career goals, and any
barriers or conflicts the students face during their college tenure. Finally, the goal of the advisor
is to offer enough advice and information for the student to feel confident in decision-making
and their own self-efficacy as they reach the end of their degree and look towards career
development.
The next step would be developing major learning outcomes that keep these goals and the
mission in mind (Bresciani, 2006). Using Hurt's (2007) suggested learning objectives, I have
altered the six levels of outcomes using Bloom's (1956) taxonomy as well as my own
undergraduate academic advising experience and use of best practices (Tucker & Wright, 2013):
1. Knowledge: Identify general education and major course options with the student; If
undeclared, open up the discourse of interests, strengths and long-term goals to help
narrow in on major options.
The first step may be the most important for new college students. At this point (within
year one), the student may not have a major area of study yet, but they still will have a series of
general education courses to begin taking. Also, at this point the advisor should start a dialogue
with the student about their future interests to help them narrow in on a major path to graduation.
The student may take up to two years before deciding on a major course of study, so the earlier
the advisor begins that conversation, the more opportunity the student has to explore options
early-on.
ACADEMIC ADVISING LEARNING OUTCOMES 9
2. Comprehension: Explain the process of degree completion in four years to the student,
as well as registration procedures and campus services.
The focus of this step is to ensure that the student understands their own accountability in
reaching degree completion, and following guidelines set forth by the university and in their
catalog year.
3. Application: Calculate their degree progress and GPA each semester or year based on
need; Identify possible barriers to timely degree completion.
The third step is crucial as a reoccurring process. Each semester when a student comes
for advising, the advisor should take the time to calculate their degree progress and monitor
changes in their GPA. The student could identify issues they have faced and address them with
their advisor. During this step, the first benchmark of barrier identification is incorporated.
4. Analysis: Discuss with student the possible career options based on major choices, as
well as opportunity for complimentary minors or certifications that fit into degree
progress.
By the midpoint of degree-seeking (around 60 credit hours completed) the student should
have a major chosen and most general education courses completed. It is at this point that the
advisor should steer meetings towards the focus of the major and options following graduation.
This step identifies the next benchmark of choosing a major and/or minor.
5. Synthesis: Integrate internship, study abroad, student work and community service
options/information into the advising session based on student's major and expressed
interests.
ACADEMIC ADVISING LEARNING OUTCOMES 10
By the students' junior year, offering options for resume' building and experience may
further assist the student in reaching degree completion with confidence in establishing a career
soon after. Thus, discussing internship options, study abroad opportunities and community
service projects as a compliment to their coursework and extracurricular activities may prove
beneficial to the student. The synthesis step offers the final benchmark of career preparation and
resume'-building.
6. Evaluation: Follow up on student's reflections of courses taken, grades earned and
prior advising sessions in order to identify paths towards moving forward to successful
degree completion.
By the end of the students' senior year, the exit survey can be taken regarding the
students' thoughts on their journey to degree completion. This survey should allow for the
student to provide reflection on their course choices, grade changes and advising sessions
throughout their tenure at college to identify successes and areas of weakness. This is a crucial
outcome to meet because of the cyclical nature of assessment. Weaknesses addressed here by
graduating students may have an impact on assessment changes for future students.
The third step would be developing an assessment protocol for each major learning outcome
(Bresciani, 2006). This would include focus groups of students while they are enrolled (ideally at
the midpoint of degree completion, or around 60 credit hours). The focus groups would
incorporate questions drawn from the first three established learning outcomes. Also, at the end
of the student's academic career they would take part in an exit survey created with the learning
objectives in mind (Smith, 2003). The exit survey would be comprised of questions regarding
each learning outcome, but would focus more on outcomes 4, 5 and 6.
ACADEMIC ADVISING LEARNING OUTCOMES 11
Finally, the last step in the process would be assessing student learning through
incorporation of the assessment protocols. The proposed protocol involves three main methods
of assessment: focus groups, exit surveys and degree progress and completion information (See
Appendix D for Proposed Model of Assessment). Degree progress and completion information
would include a student's academic transcript and transfer credit evaluation (if valid), as well as
recorded date of enrollment and date of degree conferral.
Conclusion
Though the literature on assessment of academic advising can be scattered, the focus on
quality and student satisfaction remains at the core of the research. This paper sought to find in-
depth literature and a variety of methods to assess in order to create a model and plan for
academic advising assessment at a large public university in the Mid-Atlantic region. The
struggle with any assessment is making sure it is a proper fit for the program or organization
being assessed. In this case, the assessment tool and model of outcomes were made to be broad
and very generalizable to appeal to a wide array of academic advising for different programs.
However, this is not to say that this proposed protocol would fit every advising department's
needs or address every issue or outcome it could. This is merely a starting point to a much bigger
goal of creating a current, general assessment protocol for academic advising in regards to
student development and degree completion.
Though this is only a proposed model of academic advising assessment, the framework
for further research is present. Similar to what has been addressed in our assessment course,
finding the best "fit" for assessment at the university is by taking some pieces of successful
models and applying them to the needs of the university being assessed. I focused on creating
learning outcomes based in Bloom's (1956) taxonomy due to the success in using these outcome
ACADEMIC ADVISING LEARNING OUTCOMES 12
structures in past research. Using Hurt's (2007) outcomes as a guide, I pulled in my own
experience as an academic advisor and make changes accordingly. I also, separated the learning
outcomes by time to make this an ongoing cycle of assessment and not just every 4 years.
Student needs are constant, and assessing the structure of advising is an on-going process. Next,
I chose the three assessment strategies of progress and completion rates, focus groups and exit
surveys for a few reasons. First, there has been heavy emphasis in the literature on using exit
surveys for graduating college students as a way to use final assessment. Likewise, focus groups
were utilized, but usually early on in the student's college career. I chose to use focus groups at
the midpoint as a way to gauge what changes needed to be made as the student progress, and not
wait until after the fact to make changes. Finally, I chose to focus on statistics of degree progress
and completion for means of triangulating these three protocols. Perhaps there is a link between
them that influences the other. A full-scale research project would need to be initiated to be
certain. Regardless, the movement to assess academic advising is a much needed step in ensuring
student success.
ACADEMIC ADVISING LEARNING OUTCOMES 13
Annotated Bibliography
Bloom, B. (1956). Taxonomy of educational objectives, handbook 1: Cognitive domain. New York: Addison Wesley Publishing Company.
Bloom's text identifies three domains of learning for creating learning outcomes: cognitive, affective and psychomotor. The author created levels for both the cognitive and affective domains.
Bresciani, M. (2006) Outcomes-based Academic and Co-curricular Program Review. Sterling: Stylus Publishing.
Focuses on best practices for outcomes-based assessment and program reviews. Offers examples of assessment models, learning outcomes and criteria for good practices.
Campbell, S., & Nutt, C. (2008). Academic advising in the new global century: Supporting student engagement and learning outcomes achievement. Peer Review, 10, 4-7.
While there are many ways to support student immersion and engagement in learning, onestrategy that is increasingly being acknowledged for its potential in this regard is academic advising.
Driscoll, A., & Wood, S. (2007) Developing Outcomes-based Assessment for Learner-centered Education. Sterling: Stylus Publishing.
A narrative-based text of insights and shared expertise on outcomes-based education and assessment models. Describes best practices and the focus of a learner-centered approach.
Erlich, R., & Russ-Eft, D. (2011). Applying social cognitive theory to academic advising to assess student learning outcomes. NACADA Journal, 31, 5-15.
Review of social cognitive theory constructs of self-efficacy and self-regulated learning is applied to academic advising for the purposes of assessing student learning. A brief overview of the history of student learning outcomes in higher education is followed by an explanation of self-efficacy and self-regulated learning constructs and how they can be applied to academic advising. The article concludes with the development of a model for assessing student learning outcomes in academic advising using these theoretical constructs.
Getty, L., Young, D., & Whitaker-Lea, L. (2008). Casting the assessment net wide: Capturing all student learning. About Campus, 10-16. DOI: 10.1002/abc.247.
North Georgia College and State University has joined a national effort to move away from the isolated approach to measuring student learning. Educators from three divisions offer their perspectives. The challenge for any institution is to use the energy from an impetus event to build momentum. The culture of assessment can become normal operating procedure as long as it is prevented from becoming so routine that it becomes stagnant again. The involvement of
ACADEMIC ADVISING LEARNING OUTCOMES 14
many groups across campus can revitalize the entire community and make the goal of measuring outcomes across the entire student experience a reality.
Haney, R., & McClellan, D. (2009). Community college strategies: Building a model for assessment within student affairs. Assessment Update, 21, 10-12. DOI 10.1002/au
Frederick Community College’s assessment of student learning within student affairs has evolved as a result of the confluence of three transformative initiatives: (1) the institution’s decision to shape itself into a learning college; (2) the revision of accreditation standards, placing increased emphasis on institutional assessment and assessment of student learning; and (3) a statewide initiative to develop a model for the assessment of student learning within student affairs units at Maryland community colleges.
Hurt, R. (2007). Advising as teaching: Establishing outcomes, developing tools, and assessing student learning. NACADA Journal, 27, 36-40.
Using Bloom’s taxonomy of educational outcomes, Hurt explains how to develop learning objectives within advising contexts. The article also suggests commonly available educational materials, such as university catalogs, as content delivery mechanisms for students; in addition, it offers ideas such as reading guides and on-line lectures. Finally, Hurt explains how to assess student learning, that is, how to determine the extent to which students' learning outcomes mirror the objectives established by the advisor.
McClellan, J. (2011). Beyond student learning outcomes: Developing comprehensive, strategic assessment plans for advising programmes. Journal of Higher Education Policy and Management, 33, 641-652.
The article suggest an extended framework for understanding and engaging in comprehensive assessment of advising programs. Focuses on expanding learning outcomes to include a balanced scorecard approached to advising. The scorecard may be helpful in creating a list of learning outcomes.
Metzner, B. (1989). Perceived quality of academic advising: The effect on freshman attrition. American Educational Research Journal, 26, 422-442.
Using data from 1,033 freshmen at a public urban university, this study examined the effect of the perceived quality of academic advising on student attrition in a model of the student attrition process. High-quality advising negatively influenced attrition through effects on GPA, satisfaction in the role of a student, the value of a college education for future employment, and intent to leave the university.
Smith, J., Szelest, B., & Downey, J. (2004). Implementing outcomes assessment in an academic affairs support unit. Research in Higher Education, 45, 405-427.
This article presents the development and implementation of a program of outcomes assessment at a 4-year public university in the northeast. Results indicate that students come to college with
ACADEMIC ADVISING LEARNING OUTCOMES 15
varying expectations of the role of academic advisors and range on a continuum of academic and social preparation to meet the demands of higher education. A majority of students are meeting the stated learning outcomes. Feedback from this assessment model is being used to improve the quality of services provided to successive cohorts of students at this institution.
Smith, J. (2003). Developing a survey to assess student learning outcomes in academic advisement. Assessment Update, 15, 6-15.
The article describes the process of developing an instrument to assess student learning objectives associated with academic advising at the University of Albany. The instrument is one of three components of a comprehensive model of assessment with the emphasis on an exit survey. This will be incorporated into my application section.
Suskie, L. (2009). Assessing Student Learning: A Common Sense Guide. San Francisco: Jossey-Bass.
A general reference guide for using assessment and understating how it changes as well as ways to promote assessment culture. Definitions and characteristics of good assessment are used in this paper.
Terenzini, P., & Wright, T. (1987). Influences on students' academic growth during four years of college. Research in Higher Education, 26, 161-179.
This study modeled the sources of influence on students' reported academic development over a four-year period, focusing on institutionally controllable influences on student growth and assessing how those influences vary from one year to another. Social integration was influential in students' reported academic growth only (but prominently) in the junior and senior years. The nature and strength of the influences varied over time, however, with academic and social integration.
Tucker, M. & Wright, C. (2013) Department of Communication: Academic Advising Training Protocol, Proceedings from NACADA Drive-In Conference, Fairfax, VA.
Training protocol provides detailed job description of academic advisor as well as protocol for meetings and communicating with students about degree progress.
ACADEMIC ADVISING LEARNING OUTCOMES 16
Appendix A
Metzner's Model of the Attrition Process
ACADEMIC ADVISING LEARNING OUTCOMES 17
Appendix B
Smith, Szelest, and Downey (2004) Model of Outcomes Assessment
ACADEMIC ADVISING LEARNING OUTCOMES 18
Appendix C
McClellan's Components of a Balanced Advising Scorecard
ACADEMIC ADVISING LEARNING OUTCOMES 19
Appendix D
Proposed Model of Assessment