The Five Essential Roles of Assessment Practitioners · The Five Essential Roles of Assessment Practitioners Natasha A. Jankowski, Ruth C. Slotnick Journal of Assessment and Institutional

The Five Essential Roles of Assessment Practitioners Natasha A. Jankowski, Ruth C. Slotnick

Journal of Assessment and Institutional Effectiveness, Volume 5, Number1, 2015, pp. 78-100 (Article)

Published by Penn State University Press

For additional information about this article

Access provided by your subscribing institution. (1 Oct 2018 18:18 GMT)

https://muse.jhu.edu/article/605247

https://muse.jhu.edu/article/605247

journal of assessment and institutional effectiveness, Vol. 5, No. 1, 2015Copyright © 2015 The Pennsylvania State University, University Park, PA

The Five Essential Roles of Assessment Practitioners

natasha a. jankowski and ruth c. slotnick

abstract

Assessment practitioners are tasked with a range of responsibilities from enhanc-

ing teaching and learning to improving institutional effectiveness and providing

quality assurance, yet little is known about the roles and related skill sets needed

to undertake these tasks. Through an examination of job postings coupled with a

review of the current literature, one-on-one interviews with four leaders in the field

of assessment and an exploration of our own professional experience, this paper

proposes a framework of five essential roles for assessment practitioners including

assessment/method expert, narrator/translator, facilitator/guide, political naviga-

tor and visionary/believer. Keywords: Assessment director, assessment practitioner

roles, higher education leadership, institutional research

The Five Essential Roles of Assessment Practitioners

While the field of assessment includes copious literature on classroom-based, program, and institutional assessment (see, for instance, Banta, Jones, and Black 2009; Maki 2010; Suskie 2009a; Walvoord 2010), there is a lack of literature on the various roles the assessment practitioner plays within a range of institutional types, specifically regarding the critical conversations an assessment professional must fluently master when

Essential Roles of Assessment Practitioners 79

interacting with faculty, administrators, state and national assessment colleagues, accreditors, and boards of higher education. Further, with increased pressure from accreditation agencies seeking longitudinal data that clearly document student learning, paired with an increase in state governments leveraging college completion by using performance-based funding, the function of assessment director is both critical and political and in need of deeper analysis. In an attempt to better understand assessment practitioners and how they can be most beneficial to enhancing student learning, it is useful to develop a conceptual framework of the roles and responsibilities of this unusually nebulous position. This article serves as a starting point to define the roles, skill set, and potential theoretical framework to conceptualize the position of assessment practitioner.

For the purposes of this article, we define assessment practitioners as individuals who are responsible for coordinating and leading institution-wide, program- and course-based assessment efforts and who contribute to reporting assessment activities and results to a variety of internal and external stakeholders. A cursory view of the position of assessment practitioner shows it to be characterized by a scope of attributes: Linda Suskie (2009b) asserts that assessment is a field into which one enters from many paths and does not necessarily require formal structured training in assessment. Peggy Maki (2010) underscores that the role is increasingly in flux in relation to the changing nature of higher education and the perceived value of a college degree. In addition to having various points of entry into a role that shifts with accountability discourses, the position requires assessment practitioners to fulfill a wide range of responsibilities to enhance teaching and learning, improve institutional effectiveness, provide quality assurance, stay current in the field of assessment, be cognizant of the ever-changing higher education landscape and traverse the oftentimes bumpy political landscape of assessment.

Moreover, the position, though not well understood, is one that is experiencing much growth in the field of higher education. In the last year, a search of posted positions with responsibility for learning outcomes assessment returned nearly 100 advertised job opportunities (mostly at the director level, followed by coordinator and assessment specialist)—and such a search does not capture internal appointments or positions created where a faculty member is assigned assessment duties or where institutional research subsumes the responsibility for assessment in a broadening office of institutional effectiveness.

80 natasha a. jankowski and ruth c. slotnick

Yet, with the position’s growth encompassing a varied role and responsibility list alongside connections to external accountability and accreditation efforts, surprisingly little is known about the assessment practitioner. Lacking is a requisite job classification nomenclature to fully describe the actual, on-the-ground knowledge and skill sets for an assess-ment practitioner to successfully meet duties and expectations of the role.

With so many different points of entry and ways assessment practitioner positions are structured and assigned within institutions, along with the lack of literature related to their roles, we sought a variety of sources of evidence including examination of job postings, exploration of literature on the changing roles and responsibilities of related offices, the counsel of those with a longstanding view of the field, and exploration of our own professional experience. Looking across the various inputs we propose a framework on the essential roles of assessment practitioners.

Assessment Practitioners and Their Roles within the Literature

Marilee Bresciani (2012) writes that “being an assessment coordina-tor is not a job for the faint of heart” (10) and most individuals cur-rently serving in such a role would agree. Assessment practitioners are engaged in assessing program quality and student learning as well as championing a range of assessment related initiatives across the institu-tion. Sharlene Sayegh (2013) notes that assessment directors encourage faculty to reflect on their practices, encouraging them to see the system-atic connections between their work in classrooms and larger institu-tional goals. In addition to fostering different ways of knowing in faculty, Sayegh adds that directors highlight the strengths of what is occurring while simultaneously asking departments and programs to be more con-scious and intentional in their assessment practices. Creating safe spaces in which to engage faculty in reflection on their own practices is compli-cated by the reporting function played by assessment practitioners who are required to communicate effectively with a variety of audiences on the assessment practices and results of student learning (Fulcher, Swain, and Orem 2012).

Thus, assessment practitioners are sometimes seen as at odds with or against improving teaching and learning and instead more focused on meeting external accountability demands. A recent national survey by Slotnick and Nicholas (2014) confirmed this finding when examining the


possibility of tenure for assessment directors. Both those for and against tenure-based assessment positions shared the concern of walking a fine line between gaining faculty trust and meeting accountability demands. With accreditation a main driver of assessment activities for institutions (Kuh et al. 2014), this balancing act is very real.

In trying to capture a model that might encompass the diverse roles of assessment practitioners, Debra Smith (2013) proposed utilizing Kouzes and Posner’s five practices of exemplary leadership (2002) as a starting point. Smith posits that in addition to trying to “create believers” and “finding confidence in our own voice as assessment leaders to then engage the institution in a discussion around assessment,” assessment directors within an institution guide others through the steps of assessment by “talking assessment, believing in assessment, and working at assessment” (6). The structure presented by Smith includes Kouzes and Posner’s “modeling the way, inspiring a shared vision, challenging the process, enabling others to act, and encouraging the heart” (6). Based upon this model, Smith conducted interviews with assessment colleagues at eleven colleges and universities; 73 percent of those interviewed claimed that the model encompassed most of the roles of the assessment leader. Ultimately, Smith claims that assessment directors “lead assessment through listening, fostering collaboration, building trust, and strengthening others” (7).

With this picture, one can begin to see why Bresciani (2012) proclaims that leading assessment is not for the faint of heart. Bresciani writes of the struggles of the assessment practitioner who can find the work incredibly rewarding, inspiring, and self-fulfilling when things are working well, but also requiring the ability to navigate tensions within the organization. She points to the difficulty in engaging with colleagues who do not see the value of a position responsible for overseeing assessment or who see no value in taking part in assessment itself. To those doing the work, she stresses that it requires patience, a willingness to be calm, the ability to listen carefully, and the understanding that one represents to other individuals a challenge to their status quo. Yet she declares that assessment practitioners are not in a position to judge academic quality or evaluate personnel; they “facilitate the discovery, and aid in the articulation and documentation of the learning” (10). Thus while assessment professionals provide support to enhance student learning, colleagues may have different conceptions of assessment work or reporting that limits their impression of the value of assessment related work.


From this multifaceted picture of assessment practitioners, it appears that they are at the forefront of championing an approach that can improve teaching and learning; but it is one that is also beholden to accountability efforts, forcing them to manage a contested middle ground. The strained relationship between improvement and accountability is not solely a result of the position, but may be related to assessment itself, as Peter Ewell (2009) has written of the longstanding tension between assessment under-taken for accountability purposes and that for continuous improvement. Further, the roles described above define the assessment practitioner as a leader, a systems thinker, someone who is building trust and space to allow individuals within the institution to actively reflect upon and discuss their practices—a reporter to internal and external audiences. To do each of these tasks requires the ability to expertly navigate within and across an institution while working with people from an array of positions and disci-plinary backgrounds.

A strongly similar field and position, as well as one that has been traditionally involved in assessment efforts, is that of institutional research (IR). We explore next what we might learn from this position to better understand the assessment practitioner’s role.

Institutional Research: A Suitable Model for Assessment Practitioner’s Role?

Much has been written in the field of institutional research (IR) on the changing nature of the IR role and position. Regardless of whether assessment is housed in an IR office or in its own office, IR and assessment both serve the function of providing data and evidence for fostering insti-tutional improvement and external reporting. Moreover, a survey of IR professionals found that 64 percent of offices reported being intricately involved in campus assessment efforts (Volkwein 2011).

While those few pieces of literature on assessment practitioners that do exist mention balancing skill sets related to passion, listening, engaging and patience, IR literature has tended to map the areas of responsibility focusing on the reporting and accountability functions. For instance, Patrick Terenzini (1993) outlined three areas in which IR professionals are subsumed under the heading of organizational intelligence. The first tier is technical and analytical intelligence, the second includes issues intelligence, and the third involves contextual intelligence. The first tier is focused on the reporting function of IR, while the second has more to


do with being aware of the issues and data needs of decision-makers and proactively meeting those needs. The third involves understanding the placement of the institution in the larger external environment to be sure that there are data collected to address not just current, but future needs as well. In this sense, the IR office serves as a gateway point, providing offices and personnel with data that are available to enhance decision-making, a function that involves partnering with assessment offices (Bers 2008). Beyond the reporting function, IR offices may be involved in developing curriculum or instructional design (Wehlburg 2006), creating buy-in to systematic processes (Reynolds-Sundet and Adam 2014), engaging in a social process of knowledge construction (Woodley 1999) and supporting a culture of inquiry (Brittingham, O’Brien, and Alig 2008). Further, IR has been related to program design (Delaney 2009), strategic planning efforts (Voorhees 2008), problem-solving knowledge workers (Calderon and Mathies 2013), expert witnesses (Fincher 1997) and information architects (Matier and Sidle 1995). These supporting roles require knowledge of organizational behavior (Schmidtlein 1999) in addition to an understanding of faculty work (Middaugh, Kelly, and Walters 2008).

To meet these varied roles, IR professionals need a skill set that involves knowing how other campus units operate, responding to their data needs, and being able to “talk like, think like, and interact with varied professions without being one” (Calderon and Mathies 2013, 86). Frederick Volkwein (2011) states that, while “IR serves at times as a home for theory-driven social science research it is more often a practice-oriented detective agency” (10) searching through data to answer questions that may be less than clear at the time. Thus, there is some consistency between the IR literature and assessment practitioners’ roles in having a diverse skill set, in the functions performed, and in what is involved to achieve those functions. Volkwein (1999) writes of the Janus-like nature of the IR professional who must navigate between internal and external, academic versus administrative cultures, and institutional versus professional roles. Yet, the roles presented within IR do not directly address the specific position of the assessment practitioner, as much as they may be aligned.

While these IR roles and functions resonate with those required of assessment practitioners, the IR literature is so vast and multilayered that it is difficult to piece together what would and would not relate to assessment to test in the field. In addition, the vocalization of passion and belief that was presented in the assessment literature by assessment practitioners is not found in the IR literature. Institutional research is not typically


viewed as attempting to shift an entire institutional culture to believe that assessment can fundamentally alter the way we think and operate within our institutions, all points raised by practitioners as fundamental to succeeding in the assessment work. To do so requires a potentially different skill set and position. Further, there is not a consensus around a framework that encompasses the roles and functions of IR that could inform our examination of assessment practitioners. Thus, while IR provides an informative place from which to begin, it does not provide an adequate model of the essential roles applicable for the unique requirements of assessment practitioners.

Exploration of the Role of Assessment Practitioners

To examine the role of assessment practitioner, three approaches were undertaken, including job description analysis, storytelling (through selected interviews with four assessment scholars), and the researcher reflective journal.

Methods

The first method, analyzing job descriptions, was used to identify advertised roles and responsibilities of assessment practitioners. Job advertisements posted on employment websites and listservs, including the Chronicle

of Higher Education, Higher Ed Jobs, Inside Higher Education, the New England Educational Assessment Network (NEEAN), and the ASSESS listserv, were collected over the course of a year. Each job advertisement was analyzed for title, position type (i.e., faculty, administration, consultant), reporting lines (i.e., position reports to the VP of Academic Affairs, VP of Institutional Effectiveness, or the president), required skills, disciplinary background, and any additional qualifications. Salary, if included, was analyzed in relationship to the required job duties. The findings were compared to the proposed essential roles and responsibilities of the assessment practitioner.

The second method used was storytelling. Jankowski (2012) asserts that storytelling allows for reflective recounting of experiences through the eyes of the involved party, including institutional context and history as cultural underpinnings to the storyline. The selection of the interviewees was purposeful. Given the lack of availability of oral history within the field of


assessment, we chose a small subset of assessment scholars who have had a major impact on the assessment field through their scholarly publications, national reputations, and unwavering commitment to deepening and expanding the definition of the assessment practitioner. The assessment scholars interviewed included Trudy Banta, Peggy Maki, Linda Suskie, and Barbara Walvoord.

Three interviews were conducted via phone in March 2014 and one interview took place in person in February 2014. Each interview lasted approximately 70 minutes. Interviewees were invited to share their stories around their personal professional journey in assessment, their perceptions on the role of the assessment practitioner and how that role has changed over the years, any potential or actual barriers for assessment practitioners as well as successes, and what they were worried about and hopeful for in the work of assessment practitioners. Following the data analysis process, each interviewee provided commentary and feedback on the proposed roles.

Through the telling of their assessment story, our assessment scholars shared the varied roles that they have held over time, such as graduate student, doctoral researcher, tenured faculty, campus administrator, accreditation reviewer, higher education assessment consultant, public speaker, and published scholar, as well as how they have navigated the shifting terrain of their career and current position in the field. In addition, storytelling provided a vehicle through which to discuss how each of the assessment scholars engaged with assessment practitioners at different institutions and how they have seen those roles shift over time.

The third method, the researcher reflective journal, was applied as a heuristic tool and technique to provoke the researcher to an active role in examining perceptions from multiple points of view (Slotnick and Janesick 2011). We used an electronic journal to reflect on the collected job advertisements, email correspondence, and the interview transcripts. The journal also served as a repository for compiling notes on the literature review, inspired the creation of the semi-structured interview protocol, and provided a platform to invite commentary from other researchers in the field to validate our findings.

Findings: A Framework for the Assessment Practitioner Role

In our conversations with the four assessment scholars, we heard much that converged with job descriptions and aligned with the literature on


assessment practitioners. Our examination and analysis of the aggregate findings yielded a natural grouping of roles; therefore, we offer five concep-tualizations of the essential roles of assessment practitioners that combine a series of skill sets with related roles and responsibilities. We offer these as a framework to serve as a point of departure for future research around the role of assessment practitioners.

While we present these as five separate categories, they are interrelated and build on each other. Further, while an assessment practitioner may invoke all five of these roles, at different points in time one or more may come to the forefront in any given situation and more often than not a bal-ancing act between the roles will take place. Figure 1 attempts to illustrate this relationship.

Assessment/Method Expert

Our assessment scholars spoke of needing to have a strong background in a range of methods and an extensive knowledge of assessment practice. Assessment practitioners need to know how to formulate measurable questions and subsequently gather and analyze data. In order to address

fig. 1 Representation of assessment practitioner roles.


the varied questions of interest that faculty and institutions raise in relation to student learning, assessment practitioners need to be well versed in qualitative and quantitative methodological approaches along with applying differing scholarly approaches to collecting and reporting data results. Not only do assessment professionals facilitate conversations and guide campus stakeholders through examining questions of interest via assessment, they also provide professional development to faculty to undertake assessment work. As Banta asserts, they need to understand and be involved in instructional design, and know from experience what it means to have a “teaching focus.” The network created from collaborating with others may facilitate this role, which also involves reporting to accreditors and other interested parties on the results of assessment activities.

The task of aligning measures of student learning with outcomes and ensuring that assessment methods match the inquiry or question at hand is one that is difficult to carry out in order to integrate both relevant and realistic strategies for providing information that can be integrated into departmental feedback and curriculum design (Jankowski 2013). This requires connecting with faculty and administration on measures of institutional effectiveness and student learning (Volkwein 2008). While the literature that exists on assessment practitioners minimizes the role as assessment/methods expert, it is implicitly present throughout.

Job postings targeting this skill called for qualitative or quantitative expertise at the director, coordinator, specialist and assistant or consultancy level. Job titles that included the words “evaluation,” “analyst,” “policy,” “planning,” and “accreditation” tended to focus more heavily on quanti-tative skills, data analytics, and expertise in statistics. Three institutions emphasized assessing high-impact practices while other postings stressed skills in assessment management systems and using e-portfolios for data collection. Some positions called for expertise as an interpreter-analyzer, a communicator of data, or an individual fluent in business analytics who is able to clean, code and interpret survey data. One institution wanted “best practices and using data to improve student learning.” Several job postings required assessment practitioners with backgrounds in science, engineering, medical, or educational fields. Surprisingly, only one institu-tion directly stated that it wanted a candidate to guide faculty in their use of data to “close the loop.”

Assessment professionals are increasingly asked to assist faculty in clarifying questions related to the assessment of student learning and to design projects and studies as they relate to learning acquisition. As an


assessment/method expert, the role of consultant rises to the forefront in terms of helping faculty and other individuals within the institution determine how best to address their questions on students’ learning. Assessment by its nature is an iterative process that generates more questions than answers. It is a dialogic process, one that is “nested in discussions, decisions, and actions” (Maki 2010, 4). Varied stakeholders play a role in assessment across a campus and each may have preferred methods to address particular scholarly or pedagogical issues related to student learning. The ability to draw from the requisite method to answer questions of learning, with skill and disciplinary expertise, is crucial for the assessment practitioner role (Ewell 2011).

Narrator/Translator

An assessment practitioner operating as a narrator/translator is able to bridge competing demands and frame issues with a focus on improving student learning. Focusing on narrating the students’ lived experience through evidence-based storytelling allows for internal sharing of what good assessment practice entails for a specific institution, but also fosters external understandings of the ways that a specific institution may help enhance its students’ learning (Jankowski and Cain 2015). In addition, assessment professionals need to understand the inherent assumptions that institutions and those within them hold regarding students. Exploration of assumptions about learning fosters a reflective stance regarding why learning is structured and organized in the way that it is, for the students served, at this time and place.

Walvoord asserts that assessment practitioners “look at the native culture called academe and describe it for outside audiences by sharing the kinds of assessment already going on there.” The focus on “explaining in everyday terms” and “narrating the story” of the institution to others resonated across our interviews. Assessment practitioners not only facilitate meaning-making of results and assessment data, but they help, as Maki indicates, “faculty find their voice” and “communicate to different audiences.” In addition, as Suskie spoke of translating assessment-related information to various audiences, assessment practitioners also “create a safe place for faculty to talk about teaching” and “work to protect faculty time and data.” Further, Walvoord emphasizes that the role is one of an ethnographer who nurtures and highlights what is already happening, within the classrooms and with faculty, to better advance student learning. The focus on narrating


what is occurring is reflected in Smith’s (2013) conversation of leading assessment through listening, Sayegh’s (2013) focus on highlighting the strengths in what is already occurring and Bresciani’s (2012) writing of aiding in the articulation and documentation of learning.

Job postings targeting the narrator role sought candidates adept at developing reports for multiple stakeholders, assisting faculty with translating data results for annual assessment reports, and crafting reports for accreditation visits. Many postings also called for an ability to network across campus, participate in and lead multidisciplinary teams, create professional development workshops for faculty, and collaborate with faculty from a range of disciplines. Where community colleges tended to stress the nomenclature of required knowledge in transfer and articulation of learning outcomes, four-year institutions required evidence of research and scholarship through peer-reviewed publications and presenting at national assessment conferences.

Both community colleges and four-year institutions alike required teaching experience as a necessary skill to connect with faculty on issues of pedagogy. For example, one large community college wanted a director who could “consult and collaborate with program heads and departmental faculty to develop and maintain ongoing assessment plans, including student learning outcomes, measures, and target achievement levels” while a large Midwestern university wanted an assessment director who could translate assessment data into actionable change. Only a few institutions were looking for research design skills for grant-funded projects and assisting faculty with translating both direct and indirect assessment data results for program improvement. While many of the job advertisements stressed the narrator/translator role, none of the postings directly required a person who could easily locate methods and frameworks used by other institutions to build capacity and knowledge of assessment, an important skill when working with faculty at the program level (Ewell, Paulson, and Kinzie 2011, 17). Interestingly enough, one student-affairs position required a storyteller of data. Overall, what appears to be lacking in the job advertisements is the crafting role, one person who can embroider “a promising pattern of using results for curricular and program improvement” (Ewell et al. 2011, 20).

Facilitator/Guide

In addition to helping structure methodologically sound assessment practices and reporting on those practices, assessment practitioners have


the role of helping others through listening, motivating, and mentoring. Walvoord spoke of “listening to how people think and do their work” and then helping to “facilitate insights to better their [the faculty’s] own goals.” Maki spoke of fostering conversations and dialogue around issues of student learning, whereas Walvoord noted the importance of taking a “trusting stance” and Suskie added the skill of the “listener role” requiring openness to new ideas and flexibility in thought. Guiding others in assessment also meant helping faculty think reflectively through focusing on their “wondering” and “innate curiosity.” Facilitating involved engaging others in conversations and collaborating with a network of colleagues across the institution including institutional research, centers for teaching and learning, and student affairs professionals. Banta noted that the facilitation of larger campus conversations was key. She stated, “You cannot do this work alone and do it well.” Suskie further asserted that an assessment practitioner must “build trust to partake in questioning” for an ongoing dialogue about teaching and learning. The focus on questioning— or a “questioning stance” as a participant observer who builds from faculty interests through listening to questions and concerns—highlights the nature of the facilitator/guide role. The role aligns with Bresciani’s (2012) discussion of listening to the interests of faculty, of Sayegh’s (2013) focus on helping faculty see through the eyes of students and reflect on practices and Smith’s (2013) writing of helping to guide others in the process of assessment through discussions.

It is important to note that the assessment practitioner’s role of facilitator/guide is one of assisting others to undertake assessment—they are not responsible for doing assessment themselves. Thus, they support others in answering questions related to student learning and they focus assessment efforts on areas of interest that emerge from within institutions as opposed to external forces alone.

Job advertisements targeting this quality required candidates who could serve as faculty collaborators with the ability to work across disci-plines for continuous program improvement. For example, one insti-tution required candidates with a “talent for listening carefully and communicating tactfully,” while another sought someone to “help faculty meaningfully relate assessment to curricular and pedagogical needs.” A private institution specifically asked for expertise in “understanding fac-ulty needs in a research/student centered institution” and “knowledge and experience with educational publications” as well as “scholarship of teach-ing and learning as it applies to various disciplines.” Another institution


went a step further by requiring the candidate to be able to work with graduate students, conduct grant project evaluations, lead focus groups and one-on-one interviews, implement sound survey design, and possess a strong working knowledge of IRB protocols related to assessment. Job postings overall, however, showed limited focus on facilitating/guiding faculty interest in program assessment by connecting assessment practices to their disciplinary research and scholarly interests.

To serve as a facilitator/guide requires an understanding of various campus cultures, an interdisciplinary stance for engaging with faculty, and the ability to build assessment literacy among faculty and staff. Thus, assessment practitioners are required to facilitate conversations, locate leverage points within the system to advance student learning, and help others unearth the questions that drive assessment processes. Creating space and opportunities for various stakeholders to interact together, to make meaning and to reach consensus requires an assessment practitioner who can navigate between faculty and upper-level administration.

Political Navigator

To be effective in the role of advancing student learning, assessment practitioners need to understand internal issues of language, culture, and power as well as multiple ways of framing and identifying problems. Deborah Stone (2001) writes of the various ways in which members of an organization or political entity understand and interact with policy-related problems through her work in her book, Policy Paradox. She stresses that the issues of how information is viewed, understood and constructed as well as how issues are framed have political implications throughout an organization. Assessment practitioners review a variety of data, compose reports, present findings, and otherwise frame assessment and results of assessment within the institution and externally. They frame problems and possible solutions. In some instances, as our interviewees discussed, there is a fear of negative results and issues of power and occurrences of data suppression that must be overcome to engage in dialogue as to what the results may mean for students and the institution. This claim is supported by a recent national survey that asked assessment directors to comment on the possible protections of tenured-faculty status for the assessment professional. Those in favor of tenure noted that tenure status would potentially protect them from presenting data that might fly in the face of institutional progress in student learning (Slotnick and Nicholas 2014).


Navigating multiple cultures, professional roles, disciplinary differences, competing interests, and varied value structures is a skill that requires a well-developed cognitive schema aligned with that of Pamela Eddy (2010), who writes about leadership models in higher education. Eddy states the importance of how one’s cognitive schema directly informs leadership decisions (14). She asserts that leaders are multidimensional and multifac-eted, relying on different skills and perspectives to address the complexity of their leadership challenges.

Eddy’s model draws on Mezirow’s (2000) typology of transformational leadership, which incorporates constant reflection and evaluation of the self in the process of leading. In Mezirow’s view, for transformation to occur, one must expand one’s current perspective and be able to adopt multiple perspectives. This is where the balancing act is similar to Ewell, Paulson, and Kinzie (2011), who emphasize the importance of concurrently taking the pulse of faculty at the ground level and navigating perceptions of upper-level administrators regarding campus-wide assessment efforts.

Job postings targeting this skill required strong interpersonal skills and an ability to work effectively with a range of stakeholders (including liaising with national, regional, state and local organizations) and collaborating with non-academic units. One institution mirrored the complexity of this role by stating that the assessment director would “serve as a liaison between the internal and external departments, groups, agencies, and individuals to share and disseminate information regarding articulation and curriculum, such as transfer equivalence of courses, programs, degrees and certificates, and advising.” One community college asked for “skills in effective business and interpersonal communication” while another community college stressed strategic leadership. For one assessment specialist position, the institution requested a “politically sensitive, trustworthy, and reliable person who can assure absolute confidentiality, and possess a high work ethic with exceptional interpersonal skills.” A specialized college required the assessment director to have a “high level of sophistication in social and professional settings.” At a large for-profit institution, the job candidate was expected to “effectively communicate with a wide range of individuals and constituencies in a diverse multi-ethnic and multi-cultural community.” A university in the Midwest wanted a job candidate with “tact, diplomacy, and patience.” Even a part-time staff position in assessment called for a strong “ability to interact productively and congenially with a diverse population of colleagues, faculty, students, staff and other stakeholders.”


In all the above positions, the assessment practitioner was required to work closely with academic departments, programs, faculty, and deans, as well as upper-level administrators such as the chief academic officer and president. This indicates to us that an assessment practitioner must have the ability to navigate the campus’s political terrain and the inherent courage to point out possible assessment inefficiencies or problematic data results with solutions in hand at the same time. Of equal importance, the assessment practitioner has to be aware of and sensitive to the existing campus culture, be able to function inside of both unionized and nonunion-ized environments, and operate internally to the institutional structure and norms as well as externally to various stakeholders.

Visionary/Believer

Each of our scholars recounted stories of how assessment changed their lives and outlined that to do this work requires passion and strong belief in assessment and what it can do, and they used language such as “ prophets” in helping others within the institution see the value of assessment. They spoke of assessment in spiritual tones and of believing in the power of assessment. They stated that a visionary/believer sees the larger educational system and national trends, allowing practitioners to work within what is valued by an institution with an eye toward a greater cause than an individual course and another eye on the lived experience of students. Maki spoke of “advancing lives and student learning,” while Suskie mentioned the “power of assessment to change how people think and not just their behavior”—words that align well with the writings of Smith (2013), Sayegh (2013), and Bresciani (2012) in terms of their understanding of assessment directors.

To provide vision, assessment practitioners need to believe in the ability of assessment to fundamentally enhance student learning, align and alter educational systems to better foster student learning, and engage in conversations with various stakeholders internal and external to institutions about the benefits and value of higher education. Such a focus asks assessment practitioners to be champions of assessment but also innovative in their approach to undertaking and engaging others in understanding assessment processes and practices. It also places the role of assessment as one of improving student learning, with external accountability mandates being met through focusing on students and their learning internally.


Thus, while assessment practitioners may report externally, they are driven by internal desires to improve and enhance educational quality.

We found that job advertisements required candidates with the visionary/believer role to be adept at building a sustainable culture of institutional assessment. One posting focused on candidates as innovators at the crossroads of technology, pedagogy, and curriculum through online learning. While many job postings indicated needing a campus champion who could lead cultural change, most postings tended to downplay the visionary/believer role over the more pressing, compliance-driven need for interpreter or methods analyst and reporter of data results for accreditation purposes. Thus as a visionary/believer, assessment practitioners must bring to the forefront the students’ lived experience of the curriculum and institution, strive for alignment and coherence across various entities within and outside the institution, clearly delineate for various audiences the value and purpose of engaging in assessment, and balance competing accountability demands internally and externally. In contrast, though, job descriptions do not yet focus on helping others see the value in assessment work or making connections to larger systems.

Implications for Accountability

The five essential roles provide the skill sets required of assessment practitioners to foster a meaningful, consequential, and institutional engagement with assessment. The focus of the assessment scholars was upon aligning the role of assessment practitioners with fostering teaching and learning with faculty, less so on the side of navigating the institution to do this work and engaging with administration, a theme that emerged in the literature and in the reflective journaling. There was an underlying current that if the assessment practitioner can fulfill the roles of assessment/method expert, narrator/translator, facilitator/guide, political navigator, and visionary/believer, then accountability was addressed and no separate system was needed.

As an increasing number of external audiences grow interested in assessment evidence, assessment practitioners must work to balance issues of external comparability and standardization of assessment results, increasing bureaucracy of the position as a reporting function involving data management, and meeting the demands for information without turning assessment into a mechanical reporting process. A focus solely on external reporting positions the assessment practitioner as an unwelcomed


campus compliance entity and not one associated with the elements of faculty development or teaching and learning. The outlined five essential roles squarely position the assessment practitioner as critical to develop-ing a culture of improvement embedded within teaching and learning as opposed to creating a separate reporting structure divorced from students and their learning. As Banta noted, compared to its earlier iterations three decades ago, assessment is a faculty development role.

By fostering and growing a culture of assessment and inquiry around questions of interest in addition to documenting through ethnography what is happening within a campus, the assessment practitioner would be well positioned to respond to the demands of accountability with a variety of examples and information. Yet, to actually undertake and navigate these roles, applying a balanced and blended skill set is required, one that is not readily apparent in national job postings for assessment practitioners. Thus, what assessment practitioners may be tasked to do and the roles and skill sets required to meet those tasks may be highly misaligned. Further, it is rather unlikely that a single assessment practitioner will successfully embody or demonstrate mastery of all five roles and related skill sets equally. It may be that this framework of role differential may be useful to current practitioners to reflect on the strengths they bring to their position, the type of role they have, and areas where there are opportunities for continued professional development. It may also speak to how to craft an assessment office, staffed with different professionals who can work together as a team to address the five roles as a collective.

Concluding Observations

While the assessment practitioner roles presented in this article are overarching and inclusive of roles that may occur throughout the tenure of the position with an institution, the different roles and responsibilities an assessment practitioner demonstrates may depend on where an institution is in its assessment efforts and, as such, the roles and barriers might be different and institutionally specific. In addition, the operationalization of the roles may change if undertaken by a faculty member, administrator, or office of assessment, but regardless the practitioner must possess a fluid nature of moving between or in and out of the roles and knowing when to engage in which. Where the office and position is housed (whether academic or professional) and reporting lines have implications for


the ability of assessment professionals to conduct and be successful in assessment related work (Kinzie and Jankowski 2015). While assessment practitioners may exist in a space in between academic and professional roles, they do exist within institutions and the broader national higher education environment. It is increasingly important on a national scale for assessment practitioners to be aware of what to say, when to say it, and whom to say it to as a key-holder and narrator/translator of institutional knowledge related to student learning.

The assessment scholars we spoke with stressed the need to balance conflicting pressures. As Barbara Walvoord stated, “leading assessment and leading change sometimes require different personalities,” but it is clear that regardless of personalities there are realities in the field of higher education at large which may impact the outlined roles. Access to evidence of student learning is increasingly crucial in day-to-day institutional decision-making and determining when to provide information that is actionable and pertinent to a variety of decision-makers is an increasingly complex exercise. Knowing how assessment practitioners balance these varied roles and potentially contested terrains is vital to helping institutions answer questions of the value they provide for the cost and why higher education is “worth it” for students to attend and complete. Further, they provide support and mechanisms to enhance teaching and improve student learning.

To fulfill each of the roles, assessment practitioners need to know how to navigate the formal and informal organization. James Bess and Jay Dee (2008) state that informal organizations are emergent, complementary to the formal structure, but are largely unseen, much like an iceberg. The tip of the iceberg, they argue, represents the formal organizational elements such as organizational structure, job titles, and so on, while the rest of the iceberg represents patterns and relationships, perceptions, norms and value systems, all of which may not be readily apparent but may sink the assess-ment boat. Assessment practitioners must be aware of both. They must move from presenting the value and benefit of assessment to translating assessment in a variety of alternative approaches, all while operating within the institutional norms and administrative structures of the institution.

Thus, assessment practitioners are much like blended professionals (Whitchurch 2009) whose appointment spans the professional and academic domains. Like assessment practitioners, blended professionals have academic credentials and research interests, but may not have an academic appointment. Blended professionals speak of belonging and


not belonging, of traveling and navigating multiple realities, or being in between and existing in contested space. Blended professionals find that they need to create networks and alliances, and build authority through day-to-day interactions as opposed to being seen as having expertise or authority due to their position within the organizational chart or their specialist knowledge, a route assessment professionals may need to take. In some ways, an assessment professional who is able to master the roles and responsibilities outlined in this paper would be well prepared and suited for senior-level administrative positions. Assessment practitioners would bring with them a heightened institutional focus on students and learning, yet would lead with a critical eye to support faculty needs and efforts in this evidence-based approach.

We invite assessment interested scholars to explore and enter into dialogue around the fit of the framework with the changing nature and roles of assessment over time and for current practitioners to consider the alignment of their work with the five areas that we have outlined as essential. Depending on the immediate and long-term goals of the institution, which role will take precedent will require careful reflection on the part of the assessment practitioner and associated institutional leadership.

References

Banta, T., E. Jones, and K. Black. 2009. Designing Effective Assessment:

Principles and Profiles of Good Assessment Practice. San Francisco, CA: Jossey-Bass.

Bers, T. H. 2008. “The Role of Institutional Assessment in Assessment Student Learning Outcomes.” New Directions for Higher Education 141:31–39. doi:10.1002/he.291.

Bess, J. L., and J. R. Dee. 2008. Understanding College and University

Organization: Theories for Effective Policy and Practice. Sterling, VA: Stylus Publishing.

Bresciani, M. 2012. “Adding Humor to the Position Descriptions of Assessment Coordinators.” Assessment Update 24 (3): 9–10.

Brittingham, B., P. M. O’Brien, and J. L. Alig. 2008. “Accreditation and Institutional Research: The Traditional Role and New Dimensions.” New Directions for Higher Education 141:69–76.

Calderon, A., and C. Mathies. 2013. “Institutional Research in the Future: Challenges within Higher Education and the Need for Excellence


in Professional Practice.” New Directions for Institutional Research 157:77–90.

Delaney, A. M. 2009. “Institutional Researcher’s Expanding Roles: Policy, Planning, Program Evaluation, Assessment, and New Research Methodologies.” New Directions for Institutional Research 143:29– 41.

Eddy, P. 2010. Community College Leadership: A Multidimensional Model for

Leading Change. Sterling, VA: Stylus Publishing, LLC.Ewell, P. T. 2009. Assessment, Accountability, and Improvement: Revisiting

the Tension. NILOA Occasional Paper no. 1. Urbana: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment.

———. 2011. The New Ecology for Higher Education: Challenges to Community

College Accreditation. Alameda, CA: Western Association of Schools and Colleges.

Ewell, P., K. Paulson, and J. Kinzie. 2011. Down and In: Assessment Practices

at the Program Level. Urbana: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment.

Fincher, C. L. 1997. “Researchers as Consultants and Expert Witnesses.” New Directions for Institutional Research 96:51–59.

Fulcher, K. H., M. Swain, and C. D. Orem. 2012. “Expectations for Assessment Reports: A Descriptive Analysis.” Assessment Update 24 (1): 1–2, 14–16.

Jankowski, N. 2012. “Mapping the Topography of the Evidence Use Terrain in the Assessment of U.S. Higher Education: A Multiple Case Study Approach.” PhD diss., University of Illinois Urbana-Champaign.

Jankowski, N. A. 2013. “Assessment for Learning Research Methods: A Multi-Faceted Terrain.” Presentation for the Higher Education Academy Conference on Social Sciences, Liverpool, UK, May.

Jankowski, N. A., and T. R. Cain. 2015. “From Compliance Reporting sto Effective Communication: Assessment and Transparency.” In G. D. Kuh, S. O. Ikenberry, N. A. Jankowski, T. R. Cain, P. T. Ewell, P. Hutchings, and J. Kinzie, Using Evidence of Student Learning to

Improve Higher Education, 201–19, San Francisco, CA: Jossey-Bass.Kinzie, J., and N. A. Jankowski. 2015. “Making Assessment Consequential:

Organizing to Yield Results.” In G. D. Kuh, S. O. Ikenberry, N. A. Jankowski, T. R. Cain, P. T. Ewell, P. Hutchings, and J. Kinzie, Using Evidence of Student Learning to Improve Higher Education, 73–94, San Francisco, CA: Jossey-Bass.


Kouzes, J. M., and B. Z. Posner. 2002. The Leadership Challenge. 3rd ed. San Francisco, CA: Jossey-Bass.

Kuh, G. D., N. Jankowski, S. O. Ikenberry, and J. Kinzie. 2014. Know-

ing What Students Know and Can Do: The Current State of

Student Learning Outcomes Assessment in US Colleges and

Universities. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assess-ment (NILOA).

Maki, P. 2010. Assessing for Learning: Building a Sustainable Institutional

Commitment Across the Institution. 2nd ed. Sterling, VA: Stylus Publishing.

Matier, M. W., and C. C. Sidle. 1995. “Institutional Researchers’ Roles in the Twenty-First Century.” New Directions for Institutional Research 85:75–85.

Mezirow, J. 2000. “Learning to Think Like an Adult: Core Concepts of Adult Learning Theory.” In J. Mezirow and Associates, Learning as

Transformation: Critical Perspectives on a Theory in Progress, 3–34. San Francisco: Jossey-Bass.

Middaugh, M. F., H. A. Kelly, and A. M. Walters. 2008. “The Role of Institu-tional Research in Understanding and Describing Faculty Work.” New Directions for Higher Education 141:41–56.

Reynolds-Sundet, R., and A. J. Adam. 2014. “Community College Strategies.” Assessment Update 26 (1): 12–14.

Sayegh, S. 2013. “Aligning Outcomes Throughout the Curriculum.” Assessment Update 25 (6): 8–9.

Schmidtlein, F. A. 1999. “Emerging Perspectives on Organizational Behavior: Implications for Institutional Researchers.” New

Directions for Institutional Research 104:61–72.Slotnick, R. C., and V. J. Janesick. 2011. “Conversations on Method:

Deconstructing Policy Through the Researcher Reflective Journal.” The Qualitative Report 16 (5): 1352-1360.

Slotnick, R. C. and M. Nicholas. (2014, October). Tenure and the Position of

the Director of Assessment. Paper session presented at the IUPUI Assessment Institute, Indianapolis, IN.

Smith, D. K. 2013. “Working at Assessment.” Assessment Update 25(3): 6–13.Stone, D. 2001. Policy Paradox: The Art of Political Decision Making.

New York: W. W. Norton.Suskie, L. 2009a. Assessing Student Learning: A Common Sense Approach.

2nd ed. San Francisco, CA: Jossey Bass.


Suskie, L. 2009b. “What Are the Biggest Issues with Assessment These Days?” A Common Sense Approach to Assessment and Accreditation blog, September 19. Retrieved from http://home.comcast .net/~linda.suskie/site/?/blog/view/9/.

Terenzini, P. 1993. “On the Nature of Institutional Research and the Knowledge and Skills It Requires.” Research in Higher Education 34 (1): 1–10.

Volkwein, J. F. 1999. “The Four Faces of Institutional Research.” New

Directions for Institutional Research 104:9–19.Volkwein, J. F. 2008. “The Foundations and Evolution of Institutional

Research.” In Institutional Research: More Than Just Data, ed. D. G. Terkla. New Directions for Higher Education 2008 (141): 5–20.

Volkwein, J. F. 2011. Gaining Ground: The Role of Institutional Research

in Assessing Student Outcomes and Demonstrating Institutional

Effectiveness. Occasional Paper No. 11. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment.

Voorhees, R. A. 2008. “Institutional Research’s Role in Strategic Planning.” New Directions for Higher Education 131:77–85.

Walvoord, B. 2010. Assessment Clear and Simple: A Practical Guide

for Institutions, Departments, and General Education. 2nd ed. San Francisco, CA: Jossey Bass.

Wehlburg, C. 2006. Meaningful Course Revision: Enhancing Academic

Engagement Using Student Learning Data. San Francisco, CA: Jossey-Bass.

Whitchurch, C. 2009. “The Rise of the Blended Professional in Higher Education: A Comparison between the United Kingdom, Australia and the United States.” Higher Education 58 (3): 407–18.

Woodley, A. 1999. “Doing Institutional Research: The Role of the Partisan Guerrilla.” Open Learning 14 (2): 52–58.

Documents

The Five Essential Roles of Assessment Practitioners · The Five Essential Roles of Assessment Practitioners Natasha A. Jankowski, Ruth C. Slotnick Journal of Assessment and Institutional