SECTION VI: ASSESSMENT OF STUDENT LEARNINGmidstate/pdf/sectionVI.pdf · SECTION VI: ASSESSMENT OF STUDENT LEARNING ... piloting the CLA for the assessment of student learning

SECTION VI: ASSESSMENT OF STUDENT LEARNING (STANDARD 14)

Overview............................................................................................................................1 The National Context ..................................................................................................2 Background .................................................................................................................3 A Renewed Focus........................................................................................................4

University Educational Assessment Initiatives: An Update..........................................5 Office of Institutional Research and Academic Planning............................................5

Assessment of Academic Programs .....................................................................5 Assessment of Incoming Students’ Preparation for College ................................6 Assessment Surveys and Special Studies .............................................................6

Center for Teaching Advancement and Assessment Research....................................8 Student Instructional Ratings Survey ...................................................................8 Faculty Development and Assessment .................................................................8

Center for Organizational Development and Leadership ............................................9 Support for Planning, Assessement, and Improvement .............................................11 Office of Instructional and Research Technology .....................................................11

Assessing Student Learning Outcomes .........................................................................12 General Education .....................................................................................................12

New Brunswick ..................................................................................................12 Newark ...............................................................................................................17 Camden ..............................................................................................................18

Programs and Departments........................................................................................18 Campus Writing Programs .................................................................................19 Departmental and School-Based Assessment Practices .....................................21 Rutgers University Libraries ..............................................................................34

Graduate Education ...................................................................................................38 Moving Forward.............................................................................................................39

Building a Universitywide Structure and Plan ..........................................................40 Recommendations...........................................................................................................46 Websites Referenced in Section VI................................................................................47

OVERVIEW

There is a growing desire nationally⎯among students, parents, elected officials, and the general public⎯ for greater clarity and more thorough assessment and documentation of the results of higher education. This task has traditionally consisted of evaluating the quality of the learning opportunities, resources, and expertise offered to students. More recently, the emphasis has broadened to include assessing the benefits and learning outcomes derived from the specific educational programs and services we offer, especially through the undergraduate experience.

Educators increasingly recognize the need to articulate more clearly our educational goals, more systematically assess our success in achieving them, and more effectively communicate these results to students and the multiple constituencies whom we serve. At the same time there is a keen awareness of the complexity and challenges of meaningful and valid outcomes assessment as a guide for genuine improvement in the quality of the teaching-learning process. This is particularly so at the large, decentralized, multipurpose universities where the range of programs and services that could be a focus for assessment and documentation is so vast. Like many other institutions, Rutgers is addressing this challenge and is committed to responding to the importance of enhancing our assessment initiatives, vigorously pursuing new assessment goals in a variety of ways.

Building on our long and successful history of institutional assessment, Rutgers is now engaged in a comprehensive program to develop and incorporate realistic and effective student learning outcomes assessments into the daily business of all of our academic units. The university is actively involved in a wide range of assessment activities—including pilot testing some of the national instruments, such as NSSE and CLA; devising our own assessment tools; strengthening our own institutional resources to bring concepts into practice; and encouraging the development of a widespread understanding of the functions of assessment universitywide. The transformation of our undergraduate core curricula is also stimulating assessment initiatives; changes in general education and in education in the undergraduate major are all being designed with significant built-in assessment components. Over the next few years these features will become more deeply embedded in Rutgers’ culture. We have already evaluated all of our academic departments to determine what kinds of learning outcomes assessment they now employ. University offices responsible for assessment and institutional research are taking on more responsibility for encouraging rapid development of effective assessment measures and realistic timelines are in place. The university’s central administration is strongly committed to this sea change; budget and planning decisions—previously loosely coupled with assessment data—are now increasingly dependent on these data. This change is supported by all-funds budgeting, a data-driven budgeting process that allows deans to more closely integrate institutional resources with accomplishment of their schools’ academic priorities. The relationship between resources and assessment will be strengthened as the culture of assessment increasingly pervades the way we do business at Rutgers. We have established a number of key universitywide strategic planning committees, including an Executive Committee on Assessment, to oversee, coordinate, and support the development and integration of thoughtful and useful assessment practices, and we are committed to going forward.

Section VI provides an update on university educational assessment initiatives since the last Middle States review, an overview of ongoing institutional assessment and student learning assessment efforts, a summary of our most recent initiatives on student learning assessment, and our plans going forward.

MSCHE Reaccreditation 2008 VI: Assessment of Student Learning Rutgers, The State University of New Jersey 1

THE NATIONAL CONTEXT

With great interest and concern, Rutgers has followed the national discussion over accountability and assessment that has emerged with the Spellings Commission report on the future of higher education and the debate over the reauthorization of the Higher Education Act. The recent public dialogue reflects the uncertainty that has developed as scrutiny of the higher education enterprise has intensified over the last few years. Concerns over costs and affordability for students, challenges to academic freedom, the continued underfunding of higher education on both state and national levels, and ethical lapses by higher education administrators and researchers have placed the nation’s colleges and universities on the defensive. At the same time, various stakeholders are asking for documentation about the worth of higher education and posing the kinds of questions about student learning that are being asked of the nation’s primary and secondary schools.

In this climate, faculty and administrators at Rutgers are joining their colleagues in the national discussion about the importance of implementing more rigorous and data grounded standards of accountability and assessment in higher education. Recent initiatives at Rutgers include:

• Collaboration with our peers in the Association of American Universities, through the AAU Data Exchange, to develop a set of comparative indicators for both undergraduate and graduate education. The development of these indicators will be used by the AAU and its member schools not only to facilitate internal assessment of institutional effectiveness, but to provide critical information about institutional performance to external stakeholders. Many of these indicators will be ready for use by the individual AAU institutions in spring 2008.

• Piloting the Collegiate Assessment of Academic Proficiency (CAAP) exam, the standardized assessment program developed by ACT, to determine if such standardized testing instruments are useful tools for the university in its effort to assess and evaluate outcomes of our general education program.

• Periodic administration of the National Survey of Student Engagement (NSSE). Rutgers piloted NSSE on the New Brunswick campus in 2005 and is expanding the survey to all three campuses in spring 2008. It is expected that we will administer NSSE every two or three years. The university will continue to administer its own suite of surveys to undergraduates in non-NSSE years.

• The university, in concert with its peer institutions in the Association of American Universities, is presently evaluating its participation in the VSA and is piloting the CLA for the assessment of student learning.

• National leadership in collecting both institutional and comparative data about graduate education, in addition to applying the traditional mechanisms of assessment associated with academic graduate degrees. The Graduate School−New Brunswick has been an active participant and national leader in the development of these measures.

• Establishment of assessment advisory committees in each school at Rutgers, establishment of the Assessment Council to provide oversight and coordination of the work of the assessment committees, and establishment of the Executive Council on Assessment to provide strategic planning for the Assessment Council and to ensure that a feedback loop brings the results of assessment back to the school and program level for improvement.


• Development of a broad set of dashboard indicators as a management productivity tool in helping to guide the university in its pursuit of excellence and accountability.

In undertaking these activities, the university is seeking to determine the best mix of instruments and methods for comprehensive institutional effectiveness and student learning outcomes assessment programs. The goal is to select, among the vast array of national, commercial, and homegrown programs, those that provide Rutgers with the most useful information for planning and program development within each of our programs and schools.

BACKGROUND

Educational assessment has always been a critical component of good instruction. At Rutgers, as at other excellent higher education institutions, evaluation of the curriculum and evaluation of student knowledge and competencies have long been integral to the teaching process. What has changed is a greatly enhanced awareness of the importance of assessment, a desire to integrate this awareness into the culture of the institution (create a “culture of assessment”), a determination to develop practical and valid learning outcomes assessment processes throughout the university, a commitment to use the results of assessment in a regular process of improvement and renewal, and a mechanism to infuse these practices into the academic enterprise. Our present efforts derive from a long-standing institutional commitment to systematic review, feedback, and renewal.

In moving forward to address these challenges, Rutgers has a long and solid tradition of assessment and renewal upon which to build. These institutional strengths were highlighted in the Middle States Periodic Review Report, 2003, which commended the university on a number of specific initiatives that promote institutional program review and assessment, inform planning and resource allocation, and contribute to institutional effectiveness. The periodic review report also noted the support for assessment within the university leadership, and cited several centers and initiatives focused on program review, evaluation, and improvement.

Assessment activities are increasingly part of all of our educational, cocurricular, and student services programs. The Transforming Undergraduate Education (TUE) initiative, developed in response to the findings of the New Brunswick Task Force on Undergraduate Education, provides the most dramatic example of change in organization and curriculum since the last Middle States review. The Report of the Task Force, issued in 2005, included far-reaching recommendations pertaining to all aspects of the undergraduate experience, coupled with a strong commitment to the assessment of student progress and learning outcomes. The university community considered these recommendations in a series of 39 meetings with various constituencies. On March 10, 2006, the Board of Governors approved President McCormick’s recommendations and the university is now implementing the proposed changes.

Since the last Middle States review, Camden and Newark faculty, staff, and students have also undertaken initiatives to review, evaluate, and modify undergraduate education. Under the leadership of the Camden Faculty of Arts and Sciences, a Task Force was formed in 2006 to identify ways to enhance the undergraduate experience to achieve the campus goal of becoming a top-ranked small urban public research university. The Camden Task Force Report included an analysis of undergraduate education and a set of recommendations for improvement. Specifically, the report focused on attracting high-quality undergraduate students, providing additional opportunities for their intellectual growth, strengthening the honors program, and encouraging the development and expansion of dual- and joint-degree programs.


http://oirap.rutgers.edu/reports/MSA2008/Self-Study-Reports/MSAPRRA.pdf

http://ur.rutgers.edu/transform_ru/index.shtml

http://urwebsrv.rutgers.edu/ugtaskforce/report.shtml

http://oirap.rutgers.edu/reports/MSA2008/Self-Study-Reports/Final-Deans-Task-Force-Report.pdf

Rutgers−Newark also engaged in a planning process that included periodic reassessments of the campus mission, and reaffirmation of the commitment to provide a first-rate undergraduate education accessible to a student body on one of America’s most diverse campuses. In academic year 2005−2006, the administration and Faculty of Arts and Sciences undertook a comprehensive analysis aimed at liberal arts programs and services. The Report of the Committee on Assessment of Undergraduate Programs included recommendations and plans to address their implementation. The Newark Campus is now undertaking initiatives to upgrade its physical facilities and its enhance its academic programs.

A RENEWED FOCUS

In the years since the last Middle States review, Rutgers has expanded its assessment efforts and developed a number of initiatives designed to assess the quality of the educational experience both in and beyond the classroom. With a heightened national focus on assessment, and increased institutional recognition of the importance of systematizing assessment efforts, we have committed ourselves to expanding, intensifying, and coordinating assessment approaches at all levels and across all units.

As we pursue this agenda, overarching fundamental questions—each of which defines a number of daunting challenges—guide our work.

• How do we create a universitywide culture of assessment? • How do we provide for consistency and high standards while at the same time

recognizing the great breadth and heterogeneity of our academic units, student life programs, and general services for students, as well as myriad other programs offered across the university?

• How do we incorporate the existing individual assessment efforts at the university—some self-initiated, some dictated by professional degree programs and accrediting bodies, and some in nonacademic areas guided by national association standards and practices—into a cohesive program with distinctive institutional flavor that draws upon and contributes to our mission, heritage, strengths, and location?

• What are the appropriate structures and incentives that will effectively sustain a universitywide commitment to a culture of assessment linked to planning and improvement?

• How do we ensure that there will be systematic and consistent use of the results of assessment to revise, improve, and renew our programs?

Full and meaningful engagement, discussion, and debate within the university community in considering each of these issues continue to be important goals. The university has benefited from consultation with leaders in the field of assessment and from visits to peer institutions that are addressing many of these same issues. The university has already put in place a variety of new mechanisms to ensure ongoing attention and response to these issues. We recognize, however, that while we have already accomplished a great deal, the effort is a work in progress.

In the subsections that follow we first provide a summary of institutional units and initiatives available to support assessment activities, we then give case studies from across the university on established assessment protocols in individual programs, and finally we describe the universitywide, more highly integrated and consistent assessment program we are implementing as a framework for all our educational activities.


http://oirap.rutgers.edu/reports/MSA2008/Self-Study-Reports/UAC090606.pdf


UNIVERSITY EDUCATIONAL ASSESSMENT INITIATIVES: AN UPDATE

Rutgers has made considerable progress since its last reaccreditation in developing and promoting assessment and improvement. And we have more broadly integrated educational assessment practices into university culture. The following discussion highlights initiatives launched by several key universitywide offices.

OFFICE OF INSTITUTIONAL RESEARCH AND ACADEMIC PLANNING

The Office of Institutional Research and Academic Planning (OIRAP) provides support to both undergraduate and graduate programs in surveying and assessing student learning and progress. OIRAP collects and analyzes data, and provides assessment, planning, and reporting services to inform planning and policy decisions in a wide range of academic and administrative areas. It is the lead office for academic program development and assessment, strategic planning, testing and placement, institutional research data warehouse development and web applications, and survey development. OIRAP provides reporting, assessment, benchmarking, planning, and public information services to support institutional effectiveness and to respond to the needs of the university community, the citizens of New Jersey, and to national higher education agencies and governmental organizations.

Assessment of Academic Programs

The quality of individual academic programs is of paramount importance to the university. Rutgers’ program approval procedures are designed to ensure that consistent and coordinated decisions about program development and resource allocation are made with full and open discussion among all relevant parties at the departmental, college/school, campus, university, and state levels. In the last ten years, Rutgers has established 21 new bachelor’s degrees: four in Camden, six in Newark, and 11 in New Brunswick. See the List of Academic Developments since academic year 1997–1998.

In academic year 2006−2007, the academic degree program approval process was revised to include additional requirements concerning student learning outcomes and assessment. The following student assessment components must be clearly stated in the plans for new academic programs.

• Statement of student learning outcomes: the knowledge, skills, and competencies that students are expected to exhibit upon successful completion of the proposed academic program

• Statement of the measures, tools, and strategies that will be used to assess students’ achievement of the program’s learning outcomes

• Assessment activities, includes projected costs of measures and tools to be used in the evaluation of the achievement of the program’s learning outcomes

• Statement of how assessment results will be used for renewal, revision, and improvement.

External reviews of academic programs are another important means of assessing program quality. From the early 1980s until 2005, with staff support from OIRAP, the university conducted periodic comprehensive external reviews of departments/units on all three campuses. A committee of distinguished senior faculty participated in framing the questions for the external reviewers and then used the reviewers’ reports along with additional information from university sources, to make its own assessment of each progam. The faculty committee made recommendations for program changes, including curricular revision, changes in program size, organization, reporting structure, and financial support.


http://oirap.rutgers.edu/

http://oirap.rutgers.edu/reports/MSA2008/Self-Study-Reports/SignificantAcademicDevelopments1997-2007.pdf

Over time the cyclical review process became unwieldy because of the large number of programs eligible for review and the resources each review required. Instituted in 2005, a new system calls for strategic reviews. The choice of programs to be reviewed and the questions to be asked, are designed to provide information in furtherance of strategic goals. The faculty committee, now known as the Committee on Academic Planning and Review, has been restructured with an expanded mission and includes selected advisory functions in the realm of academic affairs as well as program reviews.

As part of this strategic approach, the university now reviews and evaluates academic programs, either singularly or as part of a broader cluster review of departments/units that have a common or related mission. The intention is to assist in determining how university resources can be most effectively used in supporting a specific disciplinary area or how the contribution of related units taken together can be greater than the sum of their individual contributions.

To date, cluster assessment reviews of the university’s business programs, psychology-related programs, and computing-related fields have taken place. Each review produced recommendations regarding faculty quality, improving graduate and undergraduate programs, resources, etc. While some of the recommendations will take years to implement (e.g., organizational changes and faculty growth), Rutgers has already begun encouraging and supporting faculty collaboration across related units for cluster reviews. Individual units may request external reviews in order to address specific issues, as needed. Strategic cluster and department reviews of both undergraduate and graduate programs are ongoing.

Assessment of Incoming Students’ Preparation for College

Rutgers has been testing and assessing students’ verbal and mathematical skills for more than 30 years in order to place them in appropriate courses in their first year of matriculation. In close consultation with decanal units and academic departments, OIRAP administers the testing of all incoming students to identify their readiness for Rutgers courses. One of the mainstays of the program is its inherent process of reevaluation of students who have completed remedial coursework. Students can move on to take nonremedial, required courses in mathematics and English only after their successful completion of the remedial courses. OIRAP works closely with the Departments of Mathematics and English on each campus to ensure that students are appropriated placed into remedial courses and given good opportunities to succeed academically.

OIRAP regularly reports placement and remedial statistics to the state of New Jersey and other interested parties. The Office also tracks the academic progress of remedial students through graduation and compares success with nonremedial students. The office also works closely with individual units to evaluate the relationship between testing and placement and eventual academic success and plans for change, as needed. The overhaul of the Writing Program in Newark, for example, began because an analysis of student placements indicated a continuing problem. OIRAP, Rutgers–Newark administrators, and outside consultants successfully developed an improved program. For a more comprehensive description of this example, see Section VII.

Assessment Surveys and Special Studies

OIRAP conducts surveys to assess the academic experiences of Rutgers undergraduates, their evaluation of academic and student services, their academic and career goals, and their reasons for attending and, where relevant, leaving Rutgers. Results from these surveys provide the university with benchmarks to assess the success of its academic and cocurricular programs in the intellectual, social, and personal development of Rutgers undergraduates.

An important measure of institutional effectiveness is the value-added that an institution provides its students during their undergraduate years. Surveys, administered at different points during the


academic careers of undergraduates, provide insight into the contribution a Rutgers education makes to the academic, intellectual, and developmental growth of its undergraduates. Students complete the Cooperative Institutional Research Program (CIRP) survey during orientation, a questionnaire that asks entering students about their academic and personal aspirations and their expectations for achieving these goals while at Rutgers. Students answer similar questions in other surveys as they progress towards their degrees. For example, the First Year Experience Student Opinion Survey evaluates students’ transition from high school to college as they complete their first year at Rutgers. This survey was first administered to Rutgers College students and is now being utilized on the Camden Campus. The Upper Division Student Opinion Survey, which evaluates undergraduates’ transition to upper-division work, is administered in most years. The Graduating Student Opinion Survey is administered to all graduating seniors. Graduate student surveys, including the Ph.D. Exit Survey, are also used by graduate programs as critical assessment tools.

The university also participates periodically in the National Study of Student Engagement (NSSE), and will do so again in spring 2008, and every two or three years thereafter. NSSE focuses on students’ interactions with faculty, staff, and other students and assesses the academic effort by these students in relation to the extent to which they are engaged with the university. This survey also complements Rutgers’ institutionally based assessment surveys by offering comparison data from other schools in the Association of American Universities. The results of these and other surveys have informed institutional decision-making activities, including the TUE initiative, student life and student services improvements, the design of new residence halls, and the proposed revision of the student Academic Integrity Policy.

In addition to providing data, OIRAP engages in a broad array of analytical and empirical studies to understand and advance educational outcomes, inform and guide institutional decision-making, and report on institutional performance. Institutional Research has worked collaboratively with other administrative and academic offices such as budgeting, undergraduate admissions, undergraduate education, and deans of academic units—as well as independently—to study issues pertaining to enrollment growth, academic advising, campus climate, academic integrity, student retention, student learning, and institutional accountability and effectiveness.

OIRAP is also a key resource in various university administrative decisions providing the data, infrastructure, and analyses to inform decisions. For example, OIRAP reports to the executive vice president of academic affairs and the vice president for budgeting to support the all-funds budgeting process by tracking enrollments and course credit data. Similarly, through its data warehouse, OIRAP, in concert with the Office of Budget and Resource Studies, provides deans with enrollment and course credit data to help them with their planning processes. In order to further aid senior university administrators in their planning and assessment efforts, OIRAP has recently developed dashboard indicators that provide universitywide and campus-level benchmarks of institutional effectiveness and efficiency.

OIRAP also works with colleagues at peer institutions to develop measures that assist in comprehensive institutional assessment. This function is largely met through participation in the Association of American Universities Data Exchange (AAUDE). A recent AAUDE activity included the development of a set of comparative indicators for both undergraduate and graduate education. The undergraduate data collection is designed to be responsive to external calls for increasing accountability and improved information about costs and program performance characteristics, as well as to provide analytically useful information for university administrators. The graduate program data collection recommendations are focused on Ph.D. programs and are intended to provide better inter-institutional comparative data on program performance, as well as provide improved information to prospective students and other external constituencies.


http://www.pb.uillinois.edu/AAUDE/

CENTER FOR TEACHING ADVANCEMENT AND ASSESSMENT RESEARCH

Teaching Excellence Centers (TEC) at New Brunswick, Newark, and Camden were established in the early 1990s to assist departments and schools in the evaluation of teaching, to provide resources to faculty members wishing to develop new teaching methods and curricula, and to provide a place on each campus for ongoing discussion of teaching and learning. The TEC in Camden has evolved into the Teaching Matters program, with a focus on faculty support for teaching and faculty development. The TEC in Newark has evolved into the Office of Academic Technology, with a focus on support for the Blackboard course management system and technologically enhanced classrooms. The TEC in New Brunswick (TEC–NB) has been expanded and reconfigured, and is now the Center for Teaching Advancement and Assessment Research (CTAAR) with universitywide responsibilities. CTAAR and its predecessor, TEC−NB, have developed a broad range of teaching improvement and assessment programs. Several are briefly described below.

Student Instructional Ratings Survey

TEC−NB developed the Student Instructional Rating System (SIRS), which is designed to provide instructors and departments with information to be used in improving teaching and curricula. SIRS results are available to the Rutgers faculty, staff, and students at a password-protected site. This information is also used for tenure and promotion processes.

Faculty Development and Assessment

TEC−NB created a number of methods to assist departments and faculty in making full use of assessment results to improve teaching-learning outcomes.

• Workshops and seminars: In its early years, the center offered a variety of workshops and seminars for instructors in New Brunswick on the latest and most effective pedagogical methods. Since 1994, the center has offered workshops on instructional technology, especially the use of the internet as a teaching tool. The center offers about 140 workshops on instructional and information technologies each year, for a total of 1,200 to 1,400 participants. In addition, the office offers techniques to assist faculty with classroom assessment techniques and teaching development. Each year there are about 15 pedagogical workshops for groups of 10 to 120 faculty including programs for new junior faculty.

• Mid-course correction. The mid-course correction system was designed as a non-evaluative, whole-class interviewing technique that gives the instructor a view of the course from the student’s perspective, as well as an opportunity to respond to the student’s comments and concerns. This technique has been used effectively in individual classes and has been adopted for general use by several large departments (e.g. mathematics and economics).

• Teaching improvement assistance. Instructors seeking general information about how to improve their teaching have always been invited to contact the TEC–NB. Departments and schools also consult with the center for assistance in designing assessment tools, preparing accreditation self-studies, etc. Videotaping is also offered as a self-assessment tool.

• Teaching fellows. From 1994 to 1998 the TEC–NB used a Lilly Endowment Inc. Teaching Fellows grant to create the RU Teaching Fellows Program. The program also served as the basis for the successful GE Learning Excellence Grant codirected by the TEC–NB and Rutgers Business School−Newark and New Brunswick. Both of these funded programs assisted faculty with design and development of instructional technology tools to support their teaching. Many of


http://teachingmatters.camden.rutgers.edu/

http://oat.newark.rutgers.edu/

http://ctaar.rutgers.edu/


the Lilly Fellows were early adopters of instructional technology, transforming entire departments and units through their leadership. For example, the School of Arts and Sciences Department of Geography moved to using many online resources in courses and provided training in the tools to graduate and undergraduate students in their own labs. The GE Learning Excellence Grant generated a similar core of technology-savvy business faculty who moved their entire unit to online support for teaching using the Blackboard course management system.

• Teaching portfolios: Since the mid-1990s, many faculty members and graduate students embraced the teaching portfolio, a documented statement of an instructor’s teaching responsibilities, philosophy, goals, and accomplishments. As a result of a faculty initiative on the Camden Campus from 1995–1997, all faculty there now create their own portfolios. The School of Environmental and Biological Sciences recommends the use of such portfolios for all personnel decisions. The University Senate, in response to a Faculty Affairs and Personnel Committee report on the Best Practices in the Assessment of Teaching, recommended that all faculty create teaching portfolios.

• The Office of Staff Computer Literacy and Consulting: A new office within the TEC−NB was created in 2000 with additional full-time resources to respond to the increasing need for technical support to train and support information technology. Each semester, over 5,000 laptops, over 2,000 DVDs or video players, and over 20 video slide projects are used in the 15 large lecture halls on the New Brunswick campus. Technicians support the classrooms with an average of 20 calls a day, and six classroom visits per day.

TEC−NB has also provided substantial technological support of teaching and learning based on an assessment of faculty needs. For example, the center created Digiclass, a course management system, as a collaborative website designed to complement other course delivery systems, and designed to provide an easy-to-update and access repository of instructional materials. The system is in use in support of all language instruction in New Brunswick, two language programs on the Newark Campus (Spanish and Portuguese), and the School of Management and Labor Relations.

CENTER FOR ORGANIZATIONAL DEVELOPMENT AND LEADERSHIP

The Center for Organizational Development and Leadership (ODL), formerly known as the Quality and Communication Improvement initiative, identifies, develops, tests, and implements new approaches to assessment, planning, and improvement for Rutgers and for the higher education community more generally. The center was created to translate successful organizational practices from education, healthcare, and business to promote improvements in effectiveness within colleges and universities. Rutgers showcased the ODL programs as a “selected topic” in the 1998 Middle States periodic review report; the university was subsequently commended by MSCHE for its leadership in this area.

The center has undertaken a number of major assessment initiatives in the years since the periodic, including:

• Rutgers 2003: A Progress Report – the ’Red Tape Report’ Revisited, 10-year retrospective examination of operational enhancements within the university since the center was established


http://senate.rutgers.edu/s0109.html

http://digiclass.rutgers.edu/

http://www.odl.rutgers.edu/

http://oirap.rutgers.edu/reports/MSA2008/Self-Study-Reports/tenyears.pdf

• Pursuing Excellence in the Undergraduate Student Experience – Assessing and Improving Advising, a detailed analysis of undergraduate advising in New Brunswick, and

• Advancing Community Engagement, an assessment of collaborations between the university and the community.

ODL’s Excellence in Higher Education (EHE) Program has been especially valuable in facilitating self-assessment and improvement activities on all three campuses. EHE is a higher-education adaptation of the Malcolm Baldrige National Quality program—developed and instituted by the National Institute for Standards and Technology. The goals of the Baldrige program include defining essential elements of organizational excellence, identifying organizations that demonstrate these characteristics, promoting information sharing among exemplary organizations, and encouraging the adoption of effective organizational principles and practices.

The EHE framework provides a guide for individual academic, student life, administrative, and service programs and departments within colleges and universities to engage in an active process of organizational assessment, action planning and improvement. (See Table 6.1.)

TABLE 6-1. EXCELLENCE IN HIGHER EDUCATION FRAMEWORK

Category

Leadership Assesses how leaders and leadership practices encourage excellence, innovation, and attention to the needs of individuals, groups, and/or organizations that benefit from the programs and services of the institution, department, or program; and how leadership practices are reviewed and improved.

Purposes and Plans Considers how the mission, vision, and values of the institution, school, department, or program are developed and communicated; how they are translated into goals and plans; and how faculty and staff are engaged in those activities. Also considered are the ways in which goals and plans are translated into action and coordinated.

Beneficiaries and Constituencies

Focuses on how the unit assesses the needs, perceptions, and priorities of those groups, and how that information is used to enhance the organization’s effectiveness in addressing the needs and expectations of these groups, and in building strong relationships with those constituencies.

Programs and Services

Considers the programs and services offered by the institution, department, or program under review and how their quality and effectiveness are assessed. The most important operational and support services are also reviewed.

Faculty/Staff and Workplace

Assesses how the program, department, or institution being reviewed recruits and retains faculty and staff, encourages excellence and engagement, creates and maintains a positive workplace culture and climate, and promotes and facilitates personal and professional development.

Assessment and Information Use

Focuses on how the program, department, or institution assesses its efforts to fulfill its mission and aspirations and the effectiveness of its programs and services. Also considered is how assessment information is used for improving programs and services, day-to-day decision making, and the quality of the program, department, or institution, more generally.

Outcomes and Achievements

Seeks information and evidence to document or demonstrate the quality and effectiveness of the program, department, or institution.


http://oirap.rutgers.edu/reports/MSA2008/Self-Study-Reports/AdvisingReport4-29-final.pdf

http://oirap.rutgers.edu/reports/MSA2008/Self-Study-Reports/AdvisingReport4-29-final.pdf

http://oirap.rutgers.edu/reports/MSA2008/Self-Study-Reports/ODL-Outreach-Report.pdf

http://www.odl.rutgers.edu/core_progs/org_assess.htm#excellence

As of December 2007, approximately 45 Rutgers units across all three campuses have participated in the process, including 20 academic units and 25 student life and administrative offices.

SUPPORT FOR PLANNING, ASSESSEMENT, AND IMPROVEMENT

Figure 1 describes the roles of the three major centers within the university that provide support for planning, assessment and improvement. The Office of Institutional Research and Planning provides institutional-level support; the Center for Organizational Development and Leadership a resource for organizational review, assessment and improvement efforts in academic, student life, administrative and services; and the Center for Teaching and Assessment Research support and coordination of teaching and learning assessment and improvement activities throughout the university.

The directors of these three organizations serve on the Executive Council for Assessment (ECA), the strategic planning body for learning outcome assessment in the university, which meets periodically to set policy; coordinate efforts among the offices; integrate approaches to information gathering, dissemination and use; and more generally, enhance the culture of, and linkages among planning, assessment and continuous improvement throughout the university.

FIGURE 6.1. SUPPORT FOR PLANNING, ASSESSMENT, AND IMPROVEMENT

OFFICE OF INSTRUCTIONAL AND RESEARCH TECHNOLOGY

From 2000 to 2005, several significant projects facilitated the assessment and advancement of educational support technology. During this time, the university’s Instructional Technology Initiative provided a total of $1.5 million in funding for projects in 12 departments. Each project was designed to transform a department or a course by using new instructional technologies.

With the Mellon Foundation grant, Cost-Effective Uses of Technology in Teaching, the TEC–NB supported rigorous learning outcome assessments while considering the cost-effectiveness of new technologies. The grant enhanced Rutgers’ analysis of the effectiveness of the Instructional



Technology Initiative, introduced a pilot project to improve undergraduate education in 17 courses taught by five different departments. The grant also enabled the university to identify and assess the cost-effectiveness of a variety of support tools. This information has informed policies implemented throughout academic departments, and has led to the creation of a new central office within the Oof Information Technology: the

ffice Office of Instructional and Research Technology (OIRT). OIRT

supports the technological infrastructure faculty members use daily to reach their students,support for course management systems. Over time, OIRT will provide the technological

including

infr best.

lished for

provide programs and services that support these goals as they relate to undergraduate education.

of e

g each

se and

s and college-level assessment practices, illustrated with examples from different academic units.

nd

astructure for new teaching methods as the faculty decide what tools suit their students

As a result of these programs and the work of universitywide committees, such as the Instructional Technology Faculty Support Committee, universitywide goals have been estabeducational support technology. These goals include access to and benefits from access to information, to people, technological tools, training and support in the use of technology, and administrative tools and information. Universitywide offices and projects specific to each campus

ASSESSING STUDENT LEARNING OUTCOMES As with much at Rutgers, student learning assessment has been largely decentralized; the methodsevaluating learning, the comprehensiveness of the programs to assess learning, and the use of thresults of assessment, have mostly been the purview of individual programs, departments, and schools. Nevertheless, these varied programs have had several common elements, providinwith meaningful measures of student learning. Programs have used evidence from student performance in courses (grades and written work), evaluations from standardized student courinstructor evaluations, surveys of graduating seniors, measures of academic achievements of graduates, reports on graduates’ success in post-baccalaureate education and the labor market, and success in national competitions. In the following, we provide an overview of campu

GENERAL EDUCATION

General education requirements for arts and sciences students are established at the campus level aby individual academic units, including professional schools with requirements set by specialized accrediting bodies. All programs are responsive to Rutgers’ learning goals, a set of standards that are purposefully constructed to enable the campuses, academic units, and faculties to integrate ththeir unique educational missions while supporting an overall institutional mandate. But the variations in focus and approach have created significant differences across the units (see the current general education requirements for Rutgers undergraduate schools in the Rutgers

em into

catalogs). Over the s, there has been increasing interest in developing common indicators and a more

and

. e general

edu

last several year

New Brunswick

Until 2006, each of the liberal arts colleges in New Brunswick had its own general educationgraduation requirements. This meant that two students enrolled in different colleges in New Brunswick had different requirements for graduation–even if they both completed the same majorThis arrangement was not only confusing to the students, it made it difficult to assess th

cation outcomes and difficult to plan for change in general education requirements.

http://oirt.rutgers.edu/

http://oirap.rutgers.edu/reports/MSA2008/Self-Study-Reports/LearningGoals.pdf

http://catalogs.rutgers.edu/


The college structure in New Brunswick provided four alternative home bases for arts and sciences undergraduates (Rutgers, Douglass, Livingston, and University Colleges), but the Faculty oArts and Sciences were united in one entity, w

f ithout clear responsibilities to the four colleges. Over

tim y little

007, the libe

of

e RU Learning Goals form the foundation for the Interim Core Curriculum and its assessment. The interim core curriculum was designed to ma

6.2. INTER HOOL O ES

e, faculty engagement in the liberal arts colleges declined and most faculty had relativelinvolvement with general education issues.

The TUE initiative was designed to address this issue and to reengage the faculty with undergraduate education, especially general education and its assessment. As of July 1, 2

ral arts colleges were officially combined into the School of Arts and Sciences and an interim general education curriculum was established for the class entering in September 2007.

An Interim Curriculum Committee, a working group of faculty and academic advisers from the liberal arts colleges and the professional schools, developed this curriculum by reconciling the Douglass, Livingston, Rutgers, and University Colleges’ education/core curricula into a single setrequirements for students in the School of Arts and Sciences and for the undergraduates in the professional schools, who take components of that curriculum. Th

p to the RU Learning Goals, as seen in Table 6.2.

TABLE IM CORE CURRICULUM, SC F ARTS AND SCIENC

Rutgers University Learning Goals

School of Arts and Sciences Liberal Arts Distribution Requirements

Other Graduation Requirements

I. Intellectual and Communication Skills

A. Critical thinking: Students wdevelop their abil

ill ity to engage in

logical thinking and complex critical analysis. cal

gies

ty course

subject through completion of a major by all students.

All students must successfully complete a course in which formal analytical reasoning and/or advanced mathematireasoning are employed. All students must successfully complete courses across disciplines as required by the SAS distribution requirements and the requirement that all students complete a major and a minor. This experience with competing disciplinary perspectives and methodolopromotes complex critical analysis. In particular, all students must successfully complete one diversiwhich juxtaposes two or more visions or methods.

In-depth concentration in a particular

B. Communication: Students wildevelop their skills in expr

l essing

complex ideas through written and oral communication.

tudents are placed in

remedial courses where necessary.

involve some combination of written and oral presentations.

Successful completion of writing requirements: English 101 and one upper level writing intensive course. Writing placement test is administered to incomingfreshmen; s

Departmental capstone requirements which generally

TABLE 6.2. CONTINUED INTERIM CORE CURRICULUM, SCHOOL OF ARTS AND SCIENCES

Rutgers University Learning Goals



I. Intellectual and Communication Skills Cont.

C. Mathematical reasoning and analysis: Students will develop their skills in analyzing and interpreting numerical data, and in reasoning and problem solving through mathematical processes.

Quantitative requirement: all students must successfully complete one math course that requires Algebra II as a prerequisite. All students must successfully complete a second course in which formal analytical reasoning and/or advanced mathematical processes are employed. Math placement test is administered to incoming freshmen; students are placed in remedial courses if necessary.

Varies by major, although nearly all majors address questions of discipline- specific methodologies

D. Scientific inquiry: Students will develop their understanding of scientific methods of inquiry, including the use of observation and experimentation to answer questions and generate new knowledge.

All students must successfully complete two natural science courses.

Varies by major, although nearly all majors address questions of discipline- specific methodologies.

E. Information and computer literacy: Students will develop their skills in gathering, accessing, analyzing, and interpreting information, in part through using the tools of modern computer technology.

IT literacy skills are developed in subject specific courses throughout the curriculum. Students are required to use online systems to manage their undergraduate career and many courses use online course management systems.

Disciplinary specific IT literacy skills incorporated in major coursework requirements and senior capstone projects/theses

II. Understanding of Human Behavior, Society, and the Natural Environment

A. Historical understanding: Students will develop their understanding of the historical bases of the societies and world in which we live.

Each student is required to successfully complete four courses in the social sciences, humanities, and/or interdisciplinary studies, many/most of which engage students in longitudinal analysis.


TABLE 6.2. CONTINUED INTERIM CORE CURRICULUM, SCHOOL OF ARTS AND SCIENCES




II. Understanding of Human Behavior, Society, and the Natural Environment Cont.

B. Multicultural and international understanding: Students will understand the multicultural aspects and international dimensions of the societies and world in which we live.

Each student is required to successfully complete one diversity course which engages students in theoretical issues and political debates pertaining to questions of "diversity," namely race, ethnicity, language, migration and diasporas, gender, and sexualities. Each student is required to successfully complete one global awareness course which deepens area based knowledge and encourages analysis of global or transnational processes.

C. Understanding of literary and artistic expression: Students will develop their understanding of and appreciation of the various creative literary and artistic endeavors.

All students are required to successfully complete at least one course in the humanities disciplines.

D. Understanding the bases of individual and social behavior: Students will develop their understanding of the nature of human behavior.

All students are required to successfully complete at least one course in the social sciences.

The School of Arts and Sciences has established a support structure for its students’ academic efforts, offering special assistance to students during the transition from four undergraduate liberal arts colleges to one new School of Arts and Sciences, through the new Office of Academic Services. The new SAS Office of Undergraduate Education, charged with overseeing the SAS curriculum, departmental curricula, and instruction, has been fully operational since September 2007.

Assessment and the Interim Core Curriculum The unification of the core curriculum, even on an interim basis, has provided the university with the ability to introduce systematic assessment of student progress toward successful completion of the distribution requirements. Assessment of learning outcomes in the interim curriculum involves the following:

• Faculty in the departments assess student work in honors courses, capstone programs, and advanced written course work, involving oral presentations, poster sessions, or other evidence.

• SAS is initiating a rubric-based assessment of student learning outcomes for each major university learning goal on a rotating basis.


http://sasundergrad.rutgers.edu/

http://sas.rutgers.edu/index.php?option=com_content&task=view&id=537&Itemid=251

• In academic year 2007−2008, SAS is conducting direct longitudinal assessment of the two components of the intellectual and communications skills learning goal, critical thinking and communications.

• Student placement essays and the same student’s writing projects from an identified capstone course will be assessed with a specific evaluative rubric for 200-300 students.

• In academic year 2008−2009, an assessment will be conducted in a similar way for components of the university learning goal, understanding human behavior, society, and the natural environment.

• Experience gained from these activities will guide SAS as it establishes a three-year cycle of learning outcome assessment for general education requirements under the new curriculum, which will be implemented in fall 2009.

• These findings will be used for revision and improvement of the curriculum, as necessary, through a new standing core curriculum committee.

Students, faculty, and advisers may monitor progress toward completion of the liberal arts distribution requirements, as well as the completion of major and minor requirements, in real-time, through the online Degree Navigator.

The professional schools in New Brunswick also have established general education requirements, although learning goals for undergraduates in professional programs, such as the schools of Engineering, Nursing, Pharmacy, and Social Work, tend to be more closely integrated with discipline-based missions. Through the implementation of the TUE model for undergraduate education in New Brunswick, this integration will become more visible. Students, faculty, and staff will all have a clearer understanding of assessment programs and general education requirements.

A New Curriculum for a New Era With the creation of the School of Arts and Sciences, responsibility for general education is housed in a single administrative unit. With the interim curriculum in force, faculty can now go on to advance the goals of the TUE initiative through the creation of a wholly new core curriculum. The SAS Core Curriculum Committee (CCC) has been charged with devising this curriculum by formulating the goals for Rutgers’ program of general education and specifying in broad outline the means by which those goals are to be pursued.

This effort is directly responsive to the recommendations of the 2005 Report on Transforming Undergraduate Education (TUE), which call for:

• A single core curriculum built around a single set of expectations. • A shift in the emphasis in the distribution requirements from distributing student

enrollments across disciplines to engendering engagement with the core mission of a research university.

• Providing undergraduates during the first two years with a solid foundation in the fundamental areas that make academic success and academic research possible: writing, reasoning and information competence, quantitative thinking, and scientific inquiry.

The Core Curriculum Committee put forward two additional principles: • That each student’s education should also have ”a vertical dimension extending

from the first year of study until graduation because what lies at the core of the university’s work is the development of increasingly nuanced levels of understanding,” and


http://nbdn.rutgers.edu/

• That one of the purposes of the core curriculum should be “ . . . creating a signature undergraduate experience that will assist in building community.”

The CCC is charged with specifying processes for ongoing assessment of the core curriculum and for developing plans for feedback mechanisms to ensure that knowledge gained from the assessment is fed back to improve the curriculum. In the committee’s view, the principal function of the assessment of current practice is to inform future practice; the committee must structure the process so that the results of assessment are used to improve our performance. For this reason, the CCC will work closely with the CTAAR and the associate vice president of academic affairs for teaching and assessment research as it proceeds with its work.

Newark

The Newark Committee on Assessment of Undergraduate Programs (CAUP) was convened in fall 2005 and issued its Report in fall 2006. Four CAUP subcommittees addressed general education and curriculum, technology and instruction, campus life, and faculty engagement.The general education and curriculum subcommittee developed a set of student learning outcomes, all consistent with the university’s learning goals:

• Intellectual skills. Be competent in written and oral communication, critical and creative thinking, quantitative literacy, and information literacy.

• Knowledge. Possess a broad understanding of the major disciplines of science, social science, mathematics, humanities, and the arts.

• Individual and social responsibility. Demonstrate an appreciation of civic responsibility and engagement, ethical issues, and diversity of culture.

These broad expectations have guided subsequent discussion on campus, including discussions of the Committee on the Future of Undergraduate Education, convened by the Newark provost in December 2006. He charged the committee to consider the recommendations contained in the spring 2006 report of the Faculty of Arts and Science’s Committee on Assessment of Undergraduate Programs, since many of these recommendations had clear implications for the entire campus. In particular, he asked the Committee to assess the general education core curriculum in light of the increasingly preprofessional nature of the undergraduate student body, over half of whom now graduate with professional majors. And more broadly, he asked the group to develop a comprehensive vision for undergraduate education that reflects the distinctive character and unique strengths of the Newark Campus, in particular its urban location, the diversity of the student body, and the preprofessional aspirations of so many of its undergraduates.

The CFUE considered recommendations regarding a coordinated first-year experience, the strengthening of learning objectives, a greater emphasis on academic major and away from general education requirements, the possibility of requiring students to select two fields of concentration, capstone courses, and five year baccalaureate/master’s programs for outstanding students.

The Committee issued its final report to the provost in the fall of 2007. It is anticipated that the provost will consult with the deans and appoint a task force to continue the important work the CFUE began. This will include ongoing work on refining the vision for the Newark Campus, recommendations for general education, and specification of the kinds of outcomes departments must have as part of the majors they sponsor – including competencies in writing, ethics, critical thinking, as well as clearly defined and measurable learning goals and assessment criteria.

The Faculty of Arts and Science–Newark (FASN) has also focused attention on assessment. In spring 2007 its Undergraduate Council produced an assessment report that describes requirements for NCAS and UC students and identifies introductory courses designed to satisfy the general



curriculum. FASN will assess student learning outcomes for general education, thus providing a new structure for assessment outside of the individual arts and sciences departments. The report concludes that the liberal arts tradition of FASN and UC−N will be strengthened through the articulation of specific learning outcomes for general education.

Camden

In Camden, the Faculty Senate has created an Academic Policy Committee, charged with developing a framework for the assessment of general education and a means of evaluating and modifying requirements. The plan will include a set of collegiate learning goals, a statement of student learning outcomes and the methods to assess them, a schedule of assessment practices, plans to report the results from these practices, and a method for application of those results to effect change. The APC is especially interested in the development and use of web-based student learning portfolios as a means of direct assessment of student learning outcomes.

In a related effort, the dean of the Camden College of Arts and Sciences (CCAS) appointed an Ad Hoc Advisory Committee for Student Learning Assessment (AHCSL) with representation from CCAS, University College−Camden, the Honors College, the Graduate School–Camden, the Faculty Senate, the Academic Policy Committee, and three additional faculty members. The AHCSL is responsible for reviewing the various assessment plans for general education and departmental curricula and recommending their approval, modification, or rejection. In conjunction with the Teaching Matters Program, the committee is also providing information to the faculty about assessment strategies and practices. The AHCSL’s three-year student learning outcomes review cycle will include direct and indirect methods of assessment. The AHCSL is a temporary body; its functions and responsibilities integrated into existing faculty and administrative structures in as timely a manner as possible.

Plans are well underway, as indicated in the following timetable.

• Fall 2007 − the Academic Policy Committee and individual departments begin to develop appropriate assessment plans.

• March and April 2008 – assessment plans are submitted to the dean’s AHCSL committee or the appropriate Faculty Senate committee for approval.

• August 2008 –The AHCSL or the appropriate Faculty Senate committee provides approval or guidance for modification of assessment plans.

• Academic year 2008−2009 – first scheduled assessments of student learning outcomes begin; start of three-year cycle.

• Academic year 2009−2010 – assessments of learning outcomes continues; second year of three-year cycle.

• Academic year 2010−2011 – assessments of learning outcomes continues; third year of three-year cycle.

• April 2011 – new assessment plans are submitted to the dean’s AHCSL committee (or its successor organization) for review.

PROGRAMS AND DEPARTMENTS

In fall 2006, the university surveyed academic departments throughout Rutgers to document program learning goals and current methods used to assess student learning outcomes. The wide variety of approaches are documented and compiled in an Assessment Inventory . Most departments and


http://ctaar.rutgers.edu/assessment/WGIVAssesMethbyUnits.pdf

programs are doing a thorough and careful job of assessing student learning outcomes, but some are not yet meeting those standards and are targeted for improvement.

The examples described below—writing programs on each campus, arts and sciences programs, professional school programs, and programs developed by Rutgers University Libraries—illustrate the range of existing assessment plans and methods

Campus Writing Programs

Writing programs help undergraduates in Camden, Newark, and New Brunswick learn to write clearly and effectively, to read critically and to learn to use sources appropriately. Programs on all three campuses also offer remedial services to enable students to succeed in college-level writing courses. The centers focus on ongoing learning assessments; they make expectations clear and regularly assess students’ progress. The writing programs also regularly assess the effectiveness of their own courses in meeting learning objectives.

Camden The Writing Program at Rutgers−Camden includes seminar courses with a focus on writing skills that liberal arts students will need in upper level courses. Each course features a theme and a series of crossdisciplinary readings. Students use this theme in a sequence of assignments that build skills and allow ample opportunity for revision and incorporation of peer and instructor feedback. Continual revision, as well as a combination of informal and formal writing, allows instructors to continually assess student progress and determine whether students have mastered editorial and critical reading skills. Courses are designed to develop the student’s autonomy in selecting writing topics, drafting and revising, and conducting research. Early in the semester, instructors set goals for each student; throughout the course, they provide students with detailed written and oral responses on their work.

In 2002, the Camden Faculty Senate passed a resolution requiring students to complete a writing-intensive course, taken after the composition sequence. Students typically complete this requirement through a course in their major program.

Newark The Newark Writing Program emphasizes the importance of writing accurately, analytically, and in accordance with the conventions of standard English. Students are expected to demonstrate competence in critical thinking based on careful analysis of texts, develop writing strategies appropriate to given academic tasks, utilize appropriate grammar and mechanics of standard English, and strengthen their research skills. These goals constitute the basis for the assessment indicators for this program.

Writing Program courses require students to strengthen their reading skills, write effectively about what they read, and learn to edit and revise their work. Essays are based on text assignments and are evaluated for their insight, development, and language use. The Grading Guide, used by Writing Program instructors, is available to all writing students either as a classroom handout or on Blackboard. In some courses students are asked to keep a portfolio and, at the end of the semester, to evaluate their own work. For a more comprehensive discussion of the Newark Writing Program, see Section VII.

New Brunswick The Writing Program provides instruction to more than 11,000 students every year in courses ranging from Composition Skills for beginning writers, to Expository Writing required for graduation, to Writing for Business and Professions, an elective course for students wishing to hone their business writing skills. Courses are taught by full-time instructors, part-time lecturers, teaching assistants from disciplines across the Arts and Sciences, and the Writing Program directors. The program offers more


than 500 courses a year and provides tutoring in writing centers located on the College Avenue, Livingston, and Douglass campuses.

The teaching faculty and directors of the Writing Program have designed an elaborate internal assessment plan to make certain that all students receive the same pedagogical support and that all student writing produced in courses is assessed according to the same rigorous standards. Thus, for example, all students in Expository Writing read and respond to student papers, work in peer groups, and receive sustained instruction in the revision process. They write and revise six essays over the course of the semester; their papers are assessed according to the same grading criteria, which are clearly described on the Gradatorium website.

The directors of the Writing Program have developed a curriculum centered on student writing that addresses the university’s goal of providing all Rutgers undergraduates with the support necessary to acquire and perfect skills that are highly valued in the university. Thus, regardless of the level of the writing course, there is a structure for reading, discussing, and revising student essays. Students learn to distinguish a compelling interpretation from a fanciful or flippant response; how to provide evidence that advances an argument; and how to generate writing that articulates a position consistent with the ideas and issues raised in the assigned readings. Peer review of student work alongside the work of successful, published writers, provides students with the opportunity to participate in the arts of interpretation and critical assessment.

Table 6.3. matches the learning outcomes objectives of the Writing Program in New Brunswick with the university’s learning goals.

TABLE 6.3. NEW BRUNSWICK WRITING PROGRAM LEARNING GOALS

University Learning Goals Writing Program Learning Goals

Reasoning and Analysis Students will develop their skills in analyzing and interpreting text, and in reasoning and problem solving through critical reading processes

Students are guided in the creative reading process for developing and then better understanding their own ideas as they relate to the ideas of others through readings and a series of writing assignments which undergo multiple reviews and drafts.

Critical Thinking Students will develop their ability to engage in logical thinking and complex critical analysis.

Students learn writing as a way to investigate and then evaluate a variety of perspectives on a specific theme or idea.

Communication Students will develop their skills in expressing complex ideas through written and oral communication about their work, work of peers and selected model authors

Students: • demonstrate competency in oral and written

communication skills based on well-documented criteria for grading (see Gradatorium)

• work effectively in groups for peer review of their own work and the work of others.


http://wp.rutgers.edu/courses/101/gradatorium/introduction.html

TABLE 6.3. CONTINUED NEW BRUNSWICK WRITING PROGRAM LEARNING GOALS

University Learning Goals Writing Program Learning Goals

Reasoning and Analysis Students will develop their skills in analyzing and interpreting texts, and in reasoning and problem solving through the use of evidence and engaging styles of writing.

Students learn to: • pay particular attention to moments when the

writer is citing someone else • understand how the writer's mind works on a

problem • know what the writer thinks counts as evidence • follow the model for how the writer engages

with other writers.

Social and Ethical Awareness Students will develop their skills in analyzing and interpreting texts, and in reasoning and problem solving through the use of evidence and engaging styles of writing.

Students learn: • how to be committed to the highest ethical and

professional standards • that plagiarism means understanding the

boundaries between your ideas and the ideas of others, knowing where your ideas end and theirs start.

Information and Computer Literacy Students will develop their skills in gathering, accessing, analyzing, and interpreting information, in part through using the tools of modern computer technology.

Students learn • to utilize library and online information

sources and evaluate the reliability of obtained information.

The Writing Program also provides an extensive support system for underprepared students, with

non-credit courses designed to prepare them for demanding college-level work. The program carefully monitors the achieved grades in all of its offerings, including developmental courses. For example, a recent study indicated that those students who completed the developmental course, English 098, in fall 2005 actually had a passing rate that was higher than all other students in English 100 in spring 2006 (82 percent for students who completed English 098 versus 80 percent for graduates of English 100). Students who completed English 100R successfully had a completion rate of 90.9 percent when they took English 101 the following semester. They did better than those English 100 students who were not required to take English 099 (89.7 percent), and also better than students who were not required to take any prerequisites (86.7 percent). The program makes curricular adjustments in response to statistically significant changes in aggregate student performance.

Departmental and School-Based Assessment Practices

Departments and schools within Rutgers have a wide range of assessment plans and use a broad array of assessment tools. Several examples are provided in this section.

Major in Environmental Sciences The undergraduate program in environmental sciences provides a good illustration of a well-developed assessment program. This major is offered by the Department of Environmental Sciences in the School of Environmental and Biological Sciences which also offers bachelor of science degrees in meteorology and bioenvironmental engineering, and master’s and doctoral degrees in


environmental sciences and atmospheric science. Goals and outcomes for each program are approved by the departmental faculty. Table 6.4. identifies goals and objectives for the environmental sciences program and shows their correspondence to university learning goals.

TABLE 6.4. ENVIRONMENTAL SCIENCES UNDERGRADUATE MAJOR

University Learning Goals and Objectives Environmental Sciences Learning Goals and Objectives

Critical thinking Students will develop their ability to engage in logical thinking and complex critical analysis.

Students will be able to evaluate environmental problems, assess environmental risk, and propose appropriate remediation actions.

Information and computer literacy Students will develop their skills in gathering, accessing, analyzing, and interpreting information, in part through using the tools of modern computer technology.

Students will be able to utilize library and online information sources and evaluate the reliability of obtained information.

Communication Students will develop their skills in expressing complex ideas through written and oral communication.

Students will demonstrate competency in oral and written communication skills.

Mathematical reasoning and analysis Students will develop their skills in analyzing and interpreting numerical data, and in reasoning and problem solving through mathematical processes.

Students will demonstrate competency in computational and data analysis procedures.

Communication Students will develop their skills in expressing complex ideas through written and oral communication.

Students will work effectively in group projects.

Communication; scientific inquiry Students will develop their skills in expressing complex ideas through written and oral communication; their understanding of scientific methods of inquiry, including the use of observation and experimentation to answer questions and generate new knowledge.

Students will be able to present individual and group research in graphical and other visual formats; students will demonstrate an understanding of scientific methods of inquiry.


Students will demonstrate the skills and commitment necessary for lifelong learning.

Information and computer literacy Students will develop their skills in gathering, accessing, analyzing, and interpreting information, in part through using the tools of modern computer technology.

Students will demonstrate the skills in current technologies and their underlying principles to adapt to changing technical and educational demands of their profession.


TABLE 6.4. CONTINUED ENVIRONMENTAL SCIENCES UNDERGRADUATE MAJOR

University Learning Goals and Objectives Environmental Sciences Learning Goals and Objectives

Social and ethical awareness Students will develop their ability to recognize and assess ethical questions, and to make reasoned judgments about alternative solutions to those issues

Students will be committed to the highest ethical and professional standards.


Students will understand the basic concepts underlying the function of ecosystems; the species, populations and community dynamics of the system; and the flux of nutrients and energy controlling ecosystems development and sustainability. Students will understand the biological, chemical, and physical processes controlling the function of damaged/remediated ecosystem and the environmental risk and human health impacts of such systems.

Mathematical reasoning and analysis; scientific inquiry: Students will develop their skills in analyzing and interpreting numerical data, and in reasoning and problem solving through mathematical processes; their understanding of scientific methods of inquiry, including the use of observation and experimentation to answer questions and generate new knowledge.

Students will understand the principles of experimental design, data collection, application of analytical and numerical techniques, and data interpretation.

Critical thinking; scientific inquiry Students will develop their ability to engage in logical thinking and complex critical analysis; their understanding of scientific methods of inquiry, including the use of observation and experimentation to answer questions and generate new knowledge.

Students will understand the basic principles of general and organic chemistry as they apply to assessment and solution of environmental problems.

Critical thinking; scientific inquiry Students will develop their ability to engage in logical thinking and complex critical analysis; their understanding of scientific methods of inquiry, including the use of observation and experimentation to answer questions and generate knowledge.

Students will understand the concepts underlying chemical and biological analysis of air, soil and water samples, including a working knowledge of currently used analytical procedures and techniques.


The following list identifies more specifically the skills that graduates of the environmental sciences program are expected to develop. They will be able to:

1. Evaluate environmental problems, assess environmental risk and propose appropriate remediation; they will be able to apply basic concepts of biological, chemical, and physical processes to understanding damaged environments, and risks to human and environmental health

2. Apply principles of chemical and biological analysis to evaluating contaminants in air, water, and soil/sediment samples including knowledge of current analytical methods and techniques

3. Apply principles of experimental design, analytical, numerical, and statistical techniques to data collection and data interpretation

4. Work effectively in group projects; demonstrate competency in oral and written communication; and develop, interpret, and present data clearly in graphical or tabular form

5. Utilize library and primary source literature competently, be able to evaluate online information resources critically, demonstrate skills for lifelong learning, adapt to changing technical and educational needs of their profession, and demonstrate high ethical and professional standards

6. Obtain employment upon graduation in the environmental sciences or related fields or enter a graduate program in environmental sciences or a related field.

Faculty members review and track student performance by reviewing student achievement in courses; mastery of skills demonstrated in posters, team projects, and capstone courses are all used to measure the effectiveness of learning activities. These data are used by the faculty advisers and curriculum committee for program assessment. Faculty continually review outcomes and measurement goals in an ongoing and dynamic process that is discussed frequently at faculty meetings.

For learning outcomes 1, 2, and 3, completion of the required courses demonstrates the individual’s success at mastering the principles, content, and understanding of the subject matter. In addition to hands-on experimental labs in the required basic chemistry, biology, and physics courses, laboratory courses specific to chemical, physical, and microbiological analyses of environmental samples are part of the curriculum, and directly address outcomes 2 and 3 in detail. All students also take a capstone course in their junior or senior year. Learning outcomes 4 and 5 are measured in all required courses to a large extent and in particular in the required core courses. For example, in Chemical Principles of Environmental Sciences, students do all their calculations and problem solving in groups both in and out of class; in Biological Principles of Environmental Sciences, posters are presented by students working in pairs and posters are evaluated and critiqued by the other students in the class. This is a particularly useful skill in the environmental sciences since problem solving in the environmental consulting industry is always through interactions with a multidisciplinary team. Outcome 6 is measured by advisers and the curriculum coordinator. A new project in the department to identify and establish contact with alumni is an outcome of this approach.


Programs in Food Science The Bachelor of Science program in food science, offered by the School of Environmental and Biological Sciences, has options in food biological technologies, food chemistry, and food operations/management. The Institute of Food Technologists (IFT), which accredits these programs, has specific requirements regarding assessment of the programs and of student learning outcomes. In addition to information about the faculty and the facilities, the accrediting agency requires a comprehensive description of the curriculum. For each required course, in each option, the department must specify which of the IFT core competencies is covered. For example, for the option in food chemistry, one of the core competencies is defined as “Understand how molecular structure influences the properties and chemical reactions of the chemical components of foods.” The department specifies whether this competency is a major or a minor emphasis or is not covered for each of the 12 courses included in the core for that option. For each required food science course, data are reported in a table that contains:

• Specific learning outcomes (if not yet developed for all courses, plans in place to accomplish this, including how the course addresses the core competencies)

• Tools used to assess learning outcomes (portfolios, oral presentations, papers, reports, projects, academic journals, quizzes and exams, etc.). These are described using a taxonomy that distinguishes three types of learning: cognitive, affective, and psychomotor

• Brief narrative summary of assessment results to date.

Table 6.5. illustrates the grid used to specify which core competencies are included in each course required for the major.


TABLE 6.5. EXAMPLE OF CORE COMPETENCIES IN FOOD SCIENCE

The Department of Food Science has accomplished the analysis illustrated in Table 6.5. for each of the courses required for the major. In addition, the Department has described the expected outcomes and assessment in each of the three options within the major. This analysis includes a list of the specific outcomes of the undergraduate food science program at Rutgers, the student learning assessment philosophy, an assessment of instructional quality, an assessment of food science undergraduate program outcomes, and a report on an external review of the department.

The food science faculty use information from the assessment process to revise and update curricula, based on discussions of course content and teaching methods within the program and with members of their industry advisory council. For example, the food engineering faculty recently evaluated course content for the two food engineering courses to ensure that it is current and relevant to the needs and requirements of the food industry. The undergraduate chemistry and biology faculty


have initiated a similar process for the food chemistry and food biology courses. For more information about the assessment work of the Department of Food Sciences, see the Food Science IFT Report.

In evaluating curricula, faculty are keenly aware of the needs of their own students. For example, they noted that homework assignments and exam scores indicated that food science students generally do not have the technological background for the food engineering courses and have difficulty in learning engineering concepts. But students do need to learn this material to meet the needs of the food industry. To respond to these observations, the faculty recently modified the curriculum. A two course modular structure that includes Food Processing Technology and Food Engineering Fundamentals has replaced the previous two food engineering courses focused on heat and mass transfer calculations. The first new course emphasizes technology and covers the principles of operation and design of industrial equipment used in the processing, storage and packaging of foods; this descriptive and qualitative approach provides students with background on the engineering process and the impact of these process on food product quality. Examples utilizing different food commodities are incorporated to ensure that the students gain an understanding of the relationship between commodities and processes. This course is available for non-food science majors (students from chemistry, landscape architecture, and chemical engineering have already taken this course) and is included as the engineering component in the new joint Department of Agricultural, Food, and Resource Economics/Food Science option in Food Science and Management Economics. The second course builds on the first by providing a traditional quantitative approach to food engineering.

Major in Economics The Department of Economics within the School of Arts and Sciences, like the Department of Food Science in SEBS, became concerned because of its analysis of student learning outcomes and undertook an extensive process of curriculum revision. Economics is among the popular social sciences disciplines, typically with around 1,000 majors and 11,000 course registrations per year. The department examined the paths its majors took through its curriculum and as a result made significant changes. First faculty observed what seemed to be a pattern—students were taking economics courses in an illogical sequence; in an attempt to avoid taking the difficult core classes, students were taking the economics major “backwards” by taking electives first and later taking required core classes (such as intermediate microeconomics, intermediate macroeconomics, and econometrics). The department then substantiated these anecdotal observations with a study of its prerequisite structure and the actual registration habits of its students. They leaerned that in academic year 2002–2003, 69 percent of all the sections of elective courses in economics required only principles of economics, and the required econometrics course was a prerequisite for only one elective course, advanced econometrics.

Using information provided by the Registrar’s Office, faculty found that 77 percent of economics majors took econometrics in their senior year, 46 percent of the majors took intermediate macro in their senior year, and 31 percent of the majors took intermediate micro in their senior year. Ideally, these three courses should be completed in the sophomore or junior year. Consequently, the department adopted strategic principles to develop a new prerequisite structure for the major, determined a simple rule based on those principles to provide incentive for their majors to take the core requirements appropriately, and applied that rule to determine the details of how upper level courses fit in the new structure. The changes were approved by a vote of the entire department, reviewed and approved by the arts and sciences curriculum committee, and approved by a full vote of the arts and sciences faculty in May of 2004. The department offered additional (and larger) sections of the core courses to meet the transitional demand. The combination of econometrics and intermediate theory courses as prerequisites for the electives meant that instructors could now teach the elective at a higher level of sophistication, and it gave students the incentive to take the core requirements early rather than postponing them until their last semesters.


http://oirap.rutgers.edu/reports/MSA2008/Self-Study-Reports/FoodScienceIFTReview.pdf

http://oirap.rutgers.edu/reports/MSA2008/Self-Study-Reports/FoodScienceIFTReview.pdf

School of Engineering The School of Engineering (SOE) combines cutting edge research, high-quality instructional programs that make use of innovative educational philosophies and advanced technical tools, and provides outreach and service to our constituencies. The school seeks to integrate these facets of its mission into an educational environment of recognizable excellence. The SOE offers the bachelor of science degree for nine different majors: applied sciences in engineering, biomedical engineering, bioenvironmental engineering, chemical and biochemical engineering, civil and environmental engineering, electrical and computer engineering, industrial and systems engineering, materials science and engineering, and mechanical and aerospace engineering.

Learning objectives and assessment plans are consistent with the guidelines of the Accreditation Board for Engineering and Technology (ABET), the national accrediting agency. The formal assessment program at SOE began in 1998 when the school made preparations for its fall 2000 ABET accreditation visit. Until that time, assessment efforts had been decentralized, with each of the school’s professional programs designing and implementing its own assessment procedures. Based on the experience of the 2000 accreditation review, a more standardized approach was developed, and common procedures were established. The assessment process now includes establishing discipline-based educational objectives with faculty, students, alumni, and an industrial advisory board (IAB); defining outcomes to achieve these objectives; establishing curricula to achieve outcomes; developing indicators and mechanisms to evaluate objectives; delivering instruction and activities; and evaluating objectives and assessing outcomes, using this information for improvement, and for clarifying, as necessary, educational objectives and intended teaching/learning outcomes.

Each course in the engineering curriculum has been designed to deliver specific learning outcomes—information that is included in each course syllabus. The year-long senior capstone design sequence demonstrates achievement of all learning outcomes. Learning outcomes are measured using course grades, course-instructor evaluations, senior exit surveys, alumni surveys, employer surveys, industrial advisory board surveys, performance on external exams and national/regional competitions, and placement data of graduates. The associate dean for academic affairs convenes a schoolwide assessment committee. While learning outcomes are still defined and assessed by each individual program rather than at the school level, centralized schoolwide procedures are under development.

Table 6.6. below shows how the mission, goals, objectives, and outcomes of the mechanical engineering program are consonant with those of the university and the School of Engineering. They are also very similar to the standards required by ABET.


TABLE 6.6. SCHOOL OF ENGINEERING MISSION, GOALS, OBJECTIVES, AND OUTCOMES

University Mission Rutgers is New Jersey’s comprehensive public university: a learning community dedicated to excellence in creating knowledge through research and scholarly inquiry; preparing students with the competencies needed for personal enrichment, career development and lifelong learning; and employing knowledge for the common good and to address the needs of a changing society.

School of Engineering Goals

The sound technical and cultural education of the student and the advancement of knowledge through research. The emphasis is on a thorough understanding of fundamental principles and engineering methods of analysis and reasoning. All curricula are sufficiently comprehensive to form a foundation for more advanced scientific and technical research, more specialized professional engineering fields, or business and management opportunities in industry.

Mechanical Engineering Program Educational Objectives

• To educate and train students in Mechanical Engineering in a technically sound, challenging and professional manner

• To prepare students to enter careers ready to make positive contributions to their profession and society, or to continue on to successful graduate research and education

• To inculcate in students the responsibilities and rewards associated with an engineering career and life-long service to the profession.

Educational Outcomes for Mechanical Engineering Graduates

Ability to: • apply knowledge of mathematics, science, and engineering • design and conduct experiments; analyze and interpret data • design a component or a system • be a team player and solve problems as a group • identify, solve and formulate mechanical engineering problems • assess ethical responsibility and have an understanding of

professional responsibilities • communicate verbally; make an effective technical presentation • understand the impact of engineering solutions in a global and

societal context • appreciate the importance of life-long learning • address contemporary issues confronting engineering • bring together techniques and tools to solve engineering problems • use computational technology • think logically

Cell Biology and Neuroscience In order to jump start the development of an assessment plan, the chair of the Department of Cell Biology and Neuroscience, in the School of Arts and Sciences, appointed an Ad Hoc Assessment Committee (AHAC). Committee members developed a common understanding of the key elements of successful assessment of student learning and submitted a set of recommendations in November 2007. The Committee noted that to be a meaningful exercise, program assessment must be accepted


as a real commitment to an ongoing departmental activity, rather than a one-time event. In its simplest form, program assessment should follow a set of basic principles. The program faculty needs to: (1) identify the knowledge and/or skills its students should acquire; (2) develop a mechanism to measure the extent to which this knowledge and/or skill has been acquired; and (3) use the information to make appropriate changes to its educational structure to improve student learning. The AHAC believes that to attain an effective program assessment plan, the members of the department need to buy into the basic premise of assessment and accept the responsibility for developing the details of the plan.

Committee members reviewed the commonly used student learning assessment tools, including surveys and interviews, focus groups, capstone courses, program reviews, student portfolios, admission to graduate/professional schools, and employment. In order to begin an effective process within the department, the committee recommended the use of two types of assessment. First, they recommended the capstone course – with specified cognitive, affective, psychomotor learning modalities and specified course expectations. The AHAC noted that this form, when appropriately modified, is a powerful existing assessment tool. Several courses, including honors research, honors seminar, topics, and research already serve as capstone experiences and are well suited to the assessment process. Second, AHAC recommended using surveys—entry-level (sophomore) and graduating-level (senior), as well as alumni surveys. The department plans to develop a searchable database with the results of these and other surveys to aid in assessment, curriculum review, and planning.

It is anticipated that the CBN Curriculum and Assessment Committee will be charged with the ongoing task of curriculum assessment, based not only on the indicators mentioned above but also on the results of course-level assessments carried out by faculty. Moreover, once the database is established, CBN curricular elements may be compared with those at other institutions with highly-ranked undergraduate cell biology/neurobiology programs. The high level of engagement in assessment planning within the Department of Cell Biology and Neuroscience will ensure that the tools developed are effective measures of student learning and that the results of the assessments are fed back into the department for program development and improvement.

Teacher Education Program−Newark The Newark Faculty of Arts and Sciences Teacher Education Program is in the process of meeting the requirements for accreditation by the Teacher Education Accreditation Council (TEAC). Faculty have developed program level assessment guidelines and have articulated expected outcomes as a series of goals for program graduates. These goals are linked both to TEAC principles of quality and to state licensure requirements. Faculty also identified a series of empirical methods to determine the extent that the program meets these goals. In compliance with the accreditation standards, the program submits to TEAC faculty reviews of student journals and selected notebooks, additional data drawn from entry and exit surveys, results of Praxis II® subject assessments offered by Educational Testing Service, and student GPAs overall and in the major.

The program is continuing to improve the tools to measure student learning. For example, the questions on the student exit survey are being revised so that they are in line with the stated program objectives. Similarly, the extensive clinical evaluation instrument used by program staff, Rutgers faculty members, and cooperating teachers from the local school system is being substantially redesigned. Notable differences between elementary and secondary school teacher candidates in at least two areas on the clinical evaluations have led to two immediate programmatic changes. First, in response to comparatively lower evaluations of secondary school teacher candidates in the area of


culturally responsive pedagogy, the program has hired a new director of placement who has a sound background in urban education and who will teach Issues in Secondary Education as needed. Second, in response to the comparatively lower scores in instructional design and core curriculum among secondary education teacher candidates, the faculty has replaced the previously undifferentiated Methods of Teaching Secondary School with three discipline-specific curriculum and instruction courses—each one tailored specifically for science, mathematics, or social studies and English teacher candidates. Both of these changes were implemented in the fall 2007 semester.

College of Nursing The Rutgers College of Nursing (CoN) offers baccalaureate, master’s and doctoral degree programs. The mission and goals of each program are congruent with those of the university, as indicated in Table 6.7.

TABLE 6.7. COLLEGE OF NURSING MISSION AND GOALS

Rutgers University Mission and Goals

College of Nursing Mission and Goals Congruence Statement

Mission Instruction, research and service

Mission Educate students and advance the discipline of nursing through research. Respond to the health care needs of the public and demonstrate local, national, and international leadership for the profession.

Mission The CoN mission is congruent with the university mission through its three degree granting programs, its research and its service to the public.

Goals Improve the already high quality of the undergraduate experience. Increase the number of areas of graduate education, research, and scholarship of national and international renown.

Goals Increase enrollment and completion of well-qualified and diverse students. Recruit and retain well-prepared faculty members who share the vision of the CoN.

Goals The CoN recruitment plan focuses on high quality student enrollment. An aggressive national search process for faculty is ongoing. Evaluation and enhancement of the graduate programs is an ongoing process.

Develop and improve programs to serve society’s needs for broadly educated citizenry and well educated and competent professionals.

Identify, evaluate, and shape the marketplace demands and ability for the CoN to respond in a timely manner.

The CoN faculty monitor societal health care trends and update curricula to meet current needs

Serve the needs of the state of New Jersey.

Establish and sustain a health partnership with the community.

CoN faculty have forged partnerships with many members of its external communities and provide direct services to those communities.

Continue development of the university as a national and international resource.

Be a national and international resource for research.

International partnerships for graduate students and faculty research are ongoing.


Baccalaureate Program. The overall goals of the undergraduate program are to prepare graduates who possess a broad-based liberal arts and science education; have specialized education in professional nursing that encompasses knowledge from relevant disciplines, have an understanding of the issues and problems confronted by professional nursing in an ever-changing diverse society, and possess a comprehensive knowledge base to deal with the issues and problems of the people they serve and the complexities of the health care delivery system.

The undergraduate program is structured to meet these goals, which are consonant with those of the university. Rutgers seeks to prepare graduates to meet the needs of a changing society and to encourage their personal and professional development. The following baccalaureate program objectives are designed to convey the mission and philosophy of the College of Nursing. The Rutgers College of Nursing baccalaureate graduate is prepared to:

• Think critically, synthesizing theory and research-based knowledge from nursing, the humanities, and sociocultural and biobehavioral sciences to understand self, others, and societies as the basis for professional nursing practice

• Communicate effectively in a variety of written, verbal, and electronic formats with clients, peers, and health care professionals

• Implement the nursing practice roles of provider of care, care manager/coordinator, and member of the nursing profession to promote quality health care

• Analyze, interpret, and utilize health data and research findings for nursing practice

• Make decisions and take action using personal and professional values that incorporate ethical principles and the law and that respect diverse values and beliefs

• Provide culturally sensitive nursing care that demonstrates knowledge and respect for variation among diverse population groups

• Apply relevant knowledge regarding, social, public health, political, ethical, economic, and historical issues to the analysis of societal, professional, and health care service issues

• Pursue advanced nursing education.

Master’s Program. The master’s program prepares advanced practice nurses, who meet the certification requirements for specialized areas or fields of nursing. The program is structured to include general core courses, a scientific core, and specialty courses; the latter are specific to the students’ areas of specialization and practice. In the general and science core courses, advanced nursing knowledge needed by all students is presented. The knowledge transmitted in the specialized courses builds on and extends that of the core courses. The goals of the program are consonant with those of the university; graduates possess in-depth knowledge of the discipline and profession of which they are a part and the specialty in which they practice. Their practice is research based, focused on outcomes, and accomplished with due consideration to the cultural context of the clinical site and the special needs of the clients being served. The following master’s program objectives are designed to convey the mission and philosophy of the College of Nursing. Graduates of the master’s program will be prepared to:

• Apply knowledge from nursing and related disciplines as the basis for advanced nursing practice


• Manage independently and collaboratively, the health care problems of clients in a

• Analyze values and beliefs as the basis for a personal philosophy of advanced nursing practice

• Use research findings to provide high quality health care, initiate change, and improve nursing practice

• Design and provide quality, cost-effective care in health care systems; • Provide culturally competent care • Participate in regulatory, legislative, and professional policy to promote

healthy communities • Use information systems for the storage, analysis, and retrieval of data for

clinical practice and research • Evaluate the standards of practice and consensus or evidence-based practice

guidelines applicable to a particular population or area of practice • Employ educational strategies (using instructional theories/research) with clients,

families, staff, and others • Use management theories and research to optimize health system functioning • Pursue doctoral study.

Outcomes Assessment Tables, prepared by the Assessment Committee of the College of Nursing, lists the types of assessments in place for students. These include Clinical Evaluation Tools, NCLEX-RN results, MS Certification Exam Results, and NLN Diagnostic Tests. For faculty, assessments include teaching evaluations, the university’s annual faculty survey, and annual accountability reports.

Rutgers School of Business−Camden In fall 2008, the Rutgers School of Business−Camden (RSBC) will welcome its inaugural class of first-year students, marking the transition from a two-year upper-level program to a full four-year program for undergraduates. RSBC will continue to enroll students who transfer from county colleges, but the new program will provide high school graduates with a full business immersion experience from their first day as college students.

As the school completes the transition to a full-fledged four-year undergraduate professional school, it is reviewing and modifying its undergraduate curriculum, This process includes the creation of a first-year Business Dynamics course. The school already has a capstone course, Business Policy and Strategy, which will be integrated into the new four-year program. The planning process will also include a reexamination of learning goals for each course, and the implementation of additional direct and indirect assessment methods.

The Association to Advance Collegiate Schools of Business (AACSB) accredits RSBC and provides a framework for the curricula in its undergraduate and graduate degree programs. The accrediting agency’s criteria address issues of student selection, curriculum content/review, survey instrument results, and demonstration of performance. The school collects course-specific outcomes for specific undergraduate and M.B.A. courses. Additionally, course syllabi files are maintained and periodically reviewed for course content and objectives. Since its last reaccreditation, the school has revised the M.B.A. curriculum. The school also collects and analyzes placement statistics, internship placement data and employer evaluations of interns, and samples of student work products and administers and reviews surveys as students complete their programs. In addition, the Career Center conducts employer surveys to assess employers’ views of students’ academic preparation for their


http://ctaar.rutgers.edu/assessment/Nursing-OutcomesAssessmentTablesdraft.pdf

careers. Table 6.8. lists the indices used to assess student learning at Rutgers School of Business−Camden.

TABLE 6.8. ASSESSMENT OF STUDENT ACQUISITION OF AACSB REQUIRED KNOWLEDGE AND ABILITIES

Selection Criteria BS MBA

Admissions requirements ensure that students have appropriate general knowledge and abilities as a condition for entry

x

Basic skills courses are required for entry to make up deficiencies in general knowledge x

Curriculum Content

General education requirements contribute to student acquisition of general knowledge and abilities

x

The course matrix documents contributions of individual School of Business courses to student acquisition of general knowledge and abilities

x x

The course matrix documents contributions of individual School of Business courses to student acquisition of management-specific knowledge and abilities.

x x

Articulation agreements are used to ensure that credits transferred in fulfillment of degree requirements match the quality of the coursework given at RSB−C

x

Course syllabi are regularly collected x x

Survey Instruments

Students complete Graduating Student Exit Survey x x

Employers are regularly surveyed x x

Employers rate the work of student interns x

Demonstration of Performance

Placement Statistics demonstrate quality of the program x x

Samples of student work products demonstrate student performance x x

Rutgers University Libraries

The mission of the Rutgers University Libraries (RUL) is to support and enrich the instructional, research, and public service missions of the university through the stewardship of scholarly information and the delivery of information services. The libraries aspire to provide outstanding information resources and services that advance research and learning, support the university’s goal to be among the top public AAU institutions in the country, and serve as an essential information resource for the state and beyond.

RUL has been an institutional leader in assessment and improvement efforts. Its current five-year strategic plan, for example, responds to information gathered from Rutgers students, faculty, staff, and administrators through a year-long planning process that included the participation in the Center


for Organizational Development and Leadership (ODL) Excellence in Higher Education (EHE) assessment program LibQual+TM survey, a communications audit, numerous focus group discussions, and departmental surveys.

A key objective in the Libraries Strategic Plan, 2005-2011, which was developed after broad consultation with the Rutgers community, is to “address information competency standards for students through information literacy materials, services, and programs in partnership with the teaching faculty.” This objective supports the strategic goal to “improve the quality of scholarly resources and information services that support the advancement of academic excellence at Rutgers.” The libraries ascribe to the information literacy competency standards developed by the Association of College and Research Libraries, the premier division of the American Library Association for libraries serving higher education. The standards are supported by a number of higher education organizations, including the Middle States Commission on Higher Education. The five standards specify that the information literate student:

• Determines the nature and extent of the information needed. • Accesses needed information effectively and efficiently. • Evaluates information and its sources critically and incorporates selected

information into his or her knowledge base and value system. • Uses information effectively, individually or as a member of a group, to

accomplish a specific purpose. • Understands many of the economic, legal, and social issues surrounding the use

of information and accesses and uses information ethically and legally.

During academic year 2003−2004, the Libraries Instructional Services Committee met with a representative group of teaching faculty from across the university for two in-depth discussions about information literacy. These sessions were held in conjunction with Information Literacy and Student Learning at Rutgers: Standards, Competencies, and the Search for Strategies, a symposium developed by the libraries and cosponsored by the executive vice president for academic affairs, the provosts of the Camden and Newark campuses, the vice president for continuous education and outreach, the vice president for undergraduate education, the Faculty of Arts and Sciences−New Brunswick, and the libraries. Faculty heard nationally recognized speakers address the meaning of information literacy in the context of Middle States Commission on Higher Education criteria for excellence and a practical application of how a complex, multicampus institution is infusing its curriculum with information competence. The results of this information literacy initiative are presented in the Committee’s final report.

In spring 2005, the libraries released an online information literacy tutorial for undergraduates called Searchpath. This interactive web-based tutorial is designed to teach students basic library and research skills and covers the research process from initial topic selection to citation styles and the issues of copyright and plagiarism. Each of the six modules—Starting Smart, Choosing a Topic, Using IRIS, Finding Articles, Using the Web, and Citing Sources—includes learning outcomes towards building information literacy skills for effectively searching, selecting, and evaluating information sources. The tutorial is intended as a stand-alone tool and as a supplemental tool for library research instruction sessions. Preliminary quantitative and qualitative assessment of the Searchpath tutorial has been underway since its introduction, primarily in the form of written feedback from students, module quiz results, and interviews with students. A number of instructors in New Brunswick Writing Program courses have incorporated the tutorial into their sections. The Libraries Instructional Services Committee oversees Searchpath, which also has a reports function from which analyses are developed for tutorial improvement and for informing instruction planning


http://www.libraries.rutgers.edu/rul/staff/groups/instructional_services/reports/infolit-init-planning-sp04.shtml

http://www.libraries.rutgers.edu/rul/staff/groups/instructional_services/reports/infolit-init-planning-sp04.shtml

http://searchpath.libraries.rutgers.edu/

discussions with academic faculty. Individual student quiz scores and system profiles are evaluated and used as the basis for improvement in the manner in which the program is promoted and used.

During academic year 2005−2006, RUL participated along with 69 other academic institutions in the development of the Project for the Standardized Assessment of Information Literacy Skills (Project SAILS). While the pilot included only a limited number of students, participation provided valuable experience for understanding and playing a part in the development of a national measure, as well as in the analysis of and thinking through possible follow-ups of gaps in learning. (See the RUL Final Project Report.) Based on these efforts, a series of literacy goals and outcome expectations have been articulated, as shown in Table 6.9.

TABLE 6.9. RUTGERS UNIVERSITY LIBRARIES INFORMATION LITERACY OUTCOME EXPECTATIONS

General Goal Department Goals and Objectives

Students will determine the nature and extent of the information needed.

Define and articulate the need for information. • Identify key concepts and terms that describe the information need. Identify a variety of types and formats of potential sources for information. • Know how information is formally and informally produced,

organized and disseminated. • Recognize that knowledge can be organized into disciplines that

influence the way information is accessed. • Identify the value and differences of potential resources in a variety

of formats (e.g., multimedia, database, website, data set, audio/visual, books).

Reevaluate the nature and extent of the information need. • Review the initial information need to clarify, revise, or refine the

question.

Students will access needed information effectively and efficiently.

Select the most appropriate investigative methods or information retrieval systems for accessing the needed information. • Identify appropriate investigative methods (e.g. laboratory

experiment, simulation, fieldwork). Construct and implement effectively-designed search strategies. • Develop a research plan appropriate to the investigative method. • Identify keywords, synonyms and related terms for the information

needed. • Select controlled vocabulary specific to the discipline or information

retrieval source. Construct a search strategy using appropriate commands for the information retrieval system selected (e.g., Boolean operators, truncations, and proximity for search engines; internal organizers such as indexes for books).


http://www.libraries.rutgers.edu/rul/staff/pub_serv/reports/SAILS_final_report.pdf

http://www.libraries.rutgers.edu/rul/staff/pub_serv/reports/SAILS_final_report.pdf

TABLE 6.9. CONTINUED RUTGERS UNIVERSITY LIBRARIES INFORMATION LITERACY OUTCOME EXPECTATIONS

General Goal Department Goals and Objectives

Students will access needed information effectively and efficiently. Cont.’d

Retrieve information online or in person using a variety of methods. • Use specialized online or in person services available at the

institution to retrieve information needed (e.g., interlibrary loan/document delivery, professional associations, institutional research offices, communication resources, experts and practitioners).

Refine the search strategy if necessary. • Assess the quantity, quality, and relevance of the search result to

determine whether alternative information retrieval systems or investigative methods should be utilized.

• Identify gaps in the information retrieved and determine if the search strategy should be revised.

• Repeat the search using the revised strategy as necessary. Extract, record, and manage the information and its sources. • Differentiate among the types of sources cited and understand

elements and correct syntax of a citation for a wide range of resources.

• Record all pertinent citation information for future reference.

Students will evaluate information and its sources critically and incorporate selected information into their knowledge base and value system.

Articulate and apply initial criteria for evaluating both the information and its sources. Determine whether the initial query should be revised. • Determine if original information need has been satisfied or if

additional information is needed. • Review search strategy and incorporate additional concepts as

necessary. • Review information retrieval sources used and expand to include

others as needed.

Students will, individually or as members of a group, use information effectively to accomplish a specific purpose.

[This standard is primarily the province of the teaching faculty. Librarians can assist with this effort, but not address it independently with existing programs.]

Students will understand many of the economic, legal, and social issues surrounding the use of information and will access and use information ethically and legally.

Understand many of the ethical, legal and socio-economic issues surrounding information and information technology. • Demonstrate an understanding of intellectual property, copyright,

and fair use of copyrighted material. Acknowledge the use of information sources in communicating the product or performance. • Select an appropriate documentation style and use it consistently

to cite sources.


As we move forward in consolidating and coordinating our assessment efforts, the libraries, with units located on all campuses, will be instrumental in working with other offices to enhance our programs. The current work that is being done by the faculty on the review and reframing of the undergraduate curriculum throughout the university offers opportunities to achieve these goals. Many of the performance indicators and outcomes derived from the Association of College and Research Libraries standards are within the province of the teaching faculty, so a collaboration of teaching and library faculty is essential. The general and departmental goals listed in Table 6.9. include just those standards, indicators, and outcomes that pertain to independent library teaching, and because of this, RUL has recommended that the university undertake the development of an information literacy assessment plan that is integrally linked to the new curriculum and that includes detailed learning outcomes expectations, assessment measures and criteria, as well as an assessment schedule.

GRADUATE EDUCATION

The Graduate School−New Brunswick has been among the national leaders in collecting both institutional and comparative data about graduate education, in addition to applying the traditional mechanisms of assessment associated with academic graduate degrees.

Graduate programs throughout Rutgers, as in all U.S. universities, have always had formal processes of assessment. Master’s degrees have a culminating experience, in almost all cases a comprehensive examination, usually conducted either as a formal written examination based on the master’s curriculum or as an oral examination in conjunction with the defense of the master’s thesis. Receipt of the degree depends on success in this examination. In recent years the traditional forms of the examination have evolved in programs based in some professional areas, particularly engineering fields, so that defense of a culminating paper or project may be substituted. In all cases, however, success requires approval of at least three members of the faculty and cannot be based simply on achieving satisfactory grades in coursework.

In Ph.D. programs at Rutgers the requirements are parallel but significantly more demanding. The student must pass a qualifying examination (written or oral) conducted by at least four members of the program faculty. In some fields the student may submit extended essays, sometimes called “field statements,” in lieu of a more traditional examination, but in all cases success at this stage is assessed by at least four members of the program faculty and is required before the student may advance to candidacy for the Ph.D. and formally begin his or her dissertation research. The content of these examinations or statements typically goes beyond prior coursework and includes more general knowledge of the literature of the field, of experimental methods needed for further research, and, often, includes defense of a formal proposal for the dissertation. Upon completion of the dissertation the student must defend it before a committee of at least four faculty members, at least one of whom comes from outside the student’s program of study. Generally, the graduate programs requires that the dissertation work will be either already published in part or at least potentially publishable. This, in effect, broadens the basis of assessment, since it entails the expectation that peer reviewers for scholarly journals or books will also find the work meritorious and worthy of broader dissemination.

At Rutgers, most Ph.D. students are also expected to teach during the course of their training and the university’s universal program of teaching assessment provides feedback from undergraduate students regarding their effectiveness. Many programs are in place to assist graduate students with their teaching skills and each of these in turn provides assessments of progress. Graduate students


typically are encouraged to assemble teaching portfolios to demonstrate their teaching experience and effectiveness for the assessment of prospective employers and in many disciplines these are expected.

The ultimate test of the quality and effectiveness of our Ph.D. programs is the quality of graduates’ initial professional positions and subsequent careers. Each of our degree programs collects information about this and programs are encouraged to make this information is available to prospective students and others. The Graduate School−New Brunswick has a data collection site to which programs are asked to post the results of their placements and as much information about subsequent careers as they can assemble. This presents the opportunity to review placement success. The university has had growing success in student placement.

In addition, Rutgers administers an exit survey to all Ph.D. students at the time they file for their degrees. The Rutgers instrument was developed independently, although it employed items from existing surveys at other AAU schools. It became the basis of a jointly adopted survey by a small group of schools and then, in a somewhat revised form, by the AAU Data Exchange, in a version based at MIT. Selected portions of that survey have now been recommended by AAU as common items for all member institutions and an AAU implementation task force is currently working on the final details. Results of the survey are documenting what students perceive is and is not effective in our programs. These results, once sufficient responses accumulate, are being discussed with the program faculty as a way of seeking improvements in program quality.

There has been a national movement for the systematic collection of data about graduate education for some years now. After an abortive effort by the AAU in the 80s, the first continuing effort to collect basic data that is comparative at the program level across institutions was initiated by a group of six AAU universities, of which Rutgers was one. Initially led by Indiana University, the effort migrated to Rutgers after the retirement of its originator. The above mentioned survey was one aspect of this effort, and other institutional data were also collected at the program level and displayed comparatively. These included data to assess completion rates and time-to-degree, data that has become very useful in our discussions of ways to improve program effectiveness and efficiency.

With the new efforts by the National Research Council to collect these kinds of data across the country, the AAU and its Data Exchange have taken responsibility for collecting comparable data and a process is underway to refine and generalize this effort, an effort in which Rutgers has been an active participant. Parallel to this process the Council of Graduate Schools has pursued its Ph.D. Completion Project, in which the Graduate School−New Brunswick also participates, and this too has collected completion and attrition data and assessed the elements of doctoral programs that contribute to better completion rates. All of these data form the basis of regular consultations with directors of graduate programs on potential improvements, based on prior performance and accumulating knowledge of best practices.

Assessment, then, occurs in very concrete ways for individual students but also in important and increasingly effective ways for the programs in which they study.

MOVING FORWARD

The changing nature of expectations of higher education coupled with the availability of new technologies that bring vast quantities and varieties of information to everyone is changing the standards that we are expected to meet with regard to measures of the academy’s success. These new expectations necessitate new institutional structures within higher education and within the university


that better document and make transparent the success of our students. More than ever, we recognize the importance of being able to codify our learning goals, prove the extent to which we achieve them, and demonstrate how matriculation at Rutgers makes a difference for those who earn a Rutgers degree.

As noted above, the Assessment Inventory made clear that some central review of assessment methods and results of those methods would be very useful, especially in disseminating best practices across the schools and campuses in the university, and in encouraging and supporting departments and programs to use assessment results to redesign curricula and instructional methods to improve learning outcomes. A more cohesive and structured institutional program of assessment would benefit the university and its students. That program must have clearly defined expectations that can be applied across the university, with its array of disciplines and different modes of instruction. And that program needs to recognize our commitment to academic freedom.

To this end, the university has reorganized The Center for Teaching Advancement and Assessment Research (CTAAR), previously known as the Center for the Advancement of Teaching, with an expanded mission to invigorate and coordinate assessment activities and to bring energy, resources, and visibility to new and innovative assessment activities throughout the university. CTAAR will provide the means for the university to communicate with the state and the nation through publications, websites, and open workshops and presentations on topics in assessment. Through the center’s programs, the university will cooperate with other institutions of higher education throughout New Jersey to learn from each other about best practices, and to share the results of pilot projects and new initiatives. Rutgers is approaching learning outcomes assessment, for both general education and for departments and programs, as a continuous and long-term process, based on sound research principles that can be a model for institutions in the state of New Jersey and the nation. As an administratively sanctioned, highly visible, respected, and experienced central resource to promote and implement effective assessment strategies across the university, CTAAR will be a key resource in transforming faculty interest and commitment to assessment.

Over the next five years, CTAAR’s broad mandate includes a review of assessment practices in all facets of university function that affect student life and learning. CTAAR’s leader, who holds the new title of associate vice president of academic affairs for teaching and assessment, will bring a unified vision to assessment practices while still providing for the diversity necessary to enable varied disciplines and operations to design effectiveness measures that meet their specific missions. The university’s new assessment mechanisms will utilize and build on the many successful practices and programs on which it has relied for years.

BUILDING A UNIVERSITYWIDE STRUCTURE AND PLAN

Rutgers has devised and is now implementing a systematic universitywide process that will encourage more widespread use of assessment measures to improve instruction and student learning and will generate comparable data to assist in planning at the department, school, campus and university level.

The university is committed to building an effective universitywide structure to coordinate and support school- and department-based assessment planning. This effort builds directly on the strong faculty commitment to gauging learning outcomes as a critical component of excellence in teaching. Schools and departments have had their own, sometimes informal, assessment structures for many years; the new universitywide support infrastructure will make these individual efforts more formal





and more visible. At the same time, by systematizing and disseminating effective assessment tools, and by including a feedback loop to inform future considerations about curricula and course content, the efforts of individual units can also be generalized to improve instruction and student learning in other units.

The university administration is committed to the enhancement of assessment, comprehensive data analysis, and clear and consistent use of assessment results as feedback to improve programs and students learning at Rutgers. In October 2006 the executive vice president for academic affairs asked all deans to work with their faculty to create an Assessment Advisory Committee (AAC) for their school. This process was completed by the first day of classes in January 2007. The AACs have been charged with the development of schoolwide plans to assess student learning outcomes and coordinate the learning outcome assessment plans for each department and program within the school.

To coordinate the activities of the AACs and to oversee the learning outcome assessment process universitywide, the associate vice president of academic affairs for teaching and assessment created an Assessment Council for the university. The chairs of the AACs are members of this council. The Assessment Council will foster a collaborative, universitywide approach for assessment of the student experience, especially student-learning outcomes and it will encourage the use of unit-specific plans, within a common, well-understood framework. It will also ensure that data from disparate approaches are collected, analyzed, and that results are fed back into the system to make changes, as needed.

The Assessment Council has been functioning since summer 2007 and expects to meet approximately four times per year. At its first in-person meeting in October 2007,

• The Council decided to focus on restructuring the university’s learning goals. A small group is refining the goals and plans to carry them through to the approval process.

• CTAAR announced three workshops: Developing Learning Goals and Objectives, Methods of Assessment, and Closing the Loop.

• CTAAR announced an assessment reporting system, which is now available on the CTAAR website.

• Units provided reports on their progress in pilot programs. For example, the Graduate School of Education, Career Services, Douglass, and First Year Undergraduate Programs are all developing student portfolio programs.

• The Council discussed some serious challenges, primarily the need for a closer link between learning assessment in general education and in departmental learning outcomes in the School of Arts and Sciences. Some units are making great strides, but progress in SAS is uneven.

The Center for Teaching Advancement and Assessment Research is serving as a central resource in these assessment efforts. We now have a good understanding of our existing programs, as documented in the Assessment Inventory. CTAAR is now well positioned to work with individual units and schools to document student outcomes in a systematic fashion within and between decanal units. The timing of this endeavor is particularly fortuitous in New Brunswick, given the current implementation of the new undergraduate organizational model and curricular offerings; we fully expect that Newark and Camden will reap the benefits of this timing as well.



CTAAR is overseeing a wide range of resources to assist faculty and administrators. Between September 2006 and June 2007, CTAAR held workshops and a conference on undergraduate teaching, sponsored webinars, brought in national experts, and sent representatives to learn from successful programs elsewhere.

In order to coordinate the university’s efforts, a four-year assessment cycle has been developed. Many departments already use learning outcome assessments extensively; the new cycle will provide a structure for communicating the results of those activities and it will encourage all units to participate actively in the process. The well publicized cycle will help the Assessment Council and the CTAAR to provide assistance, such as expert speakers, consultation, workshops, seminars and programs about assessment to departments as they need them, and will improve communications across the university, keeping alive the conversation and awareness of learning outcome assessment activities.

In any given year, one quarter of all departments will be revising their learning outcome assessment plans based on previous assessment results, one quarter will submit the revised plans for review by the Assessment Council, one quarter will be conducting assessments, and one quarter will report assessment results to the Assessment Council for review. See Table 6.10.

TABLE 6.10. UNIVERSITYWIDE ASSESSMENT CYCLE

Fall Spring

Year 1 Submit revised learning outcome assessment plan to Council on Assessment.

Department plan returned with comments and suggestions.

Year 2 Department implements plan and conducts learning outcome assessment.

Department implements plan and conducts learning outcome assessment.

Year 3 Department reports assessment results to the Council, with planned actions for change based on the results.

Council reviews action plan based on assessment results, and gives comments, suggestions to the department. The CTAAR will work to support the department’s efforts.

Year 4 Department implements the changes in teaching and learning based on assessment results.

Department revises assessment strategy in light of the changes made and new information collected.

CTAAR will support the work of the departments as they implement change based on assessment results. The center will continuously monitor the results of the changes implemented. Therefore, in year four, when a change in curriculum or teaching method is instituted, data and materials will be collected to ensure that there is an informed revision of the department assessment plan when the cycle begins again.

To get the process started, a group of departments has been selected to enter the four-year cycle at each level in academic year 2007−2008. For example, a group of departments will enter the cycle at year three, reporting assessment results for their programs. In the School of Arts and Sciences these are Africana studies, anthropology, economics, French, genetics, Jewish studies, linguistics, physics, psychology, Spanish and Portuguese, women’s studies, and the Writing Program. Several schools will also enter the cycle at year three, including the Ernest Mario School of Pharmacy; the School of Communication, Information and Library Studies; and the School of Social Work.


The Assessment Council will augment the unit-specific programs with selective use of pilot tests used on other campuses. Information gleaned from Rutgers’ participation in standardized national tests will help to inform future developments of universitywide and discipline-specific outcomes assessment tools.

The activities of the Assessment Council will be monitored by an Executive Council for Assessment (ECA), the strategic planning body for learning outcome assessment in the university. ECA, chaired by the executive vice president for academic affairs, includes the associate vice president of academic affairs for teaching and assessment research, the director of institutional research, representatives from the offices of the vice presidents for undergraduate and graduate education, and teaching has one representative each from the offices of the provost in Newark and Camden, the vice president for undergraduate education, the vice president for research and graduate and professional education, plus two deans’ appointees. This group provides a forum for senior university leadership to develop the university’s assessment strategy and to determine to what extent Rutgers should participate in national comparative programs, such as the Voluntary System of Accountability, recently endorsed by the National Association of State Universities and Land-Grant Colleges and American Association of State Colleges and Universities. For example, the university is now piloting the Collegiate Learning Assessment test to determine whether it will further our student learning outcomes assessment goals.

Additionally, CTAAR and the Assessment Council are working with appropriate governing bodies to refine and review the university’s learning goals as the new School of Arts and Sciences develops a general education curriculum in collaboration with the professional schools in New Brunswick. Following is a brief list of assessment initiatives planned for academic year 2007−2008.

• The School of Arts and Sciences will do a direct longitudinal assessment of the communications learning goal. Student placement essays and their writing projects from an identified capstone course will be assessed with a specific evaluative rubric for 200-300 students.

• Rutgers University Libraries will work with the School of Arts and Sciences to create and implement a rubric that will identify the extent to which students are meeting the learning goals pertaining to information literacy.

• The School of Environmental and Biological Sciences is designing and will administer an assessment tool for seniors to identify what students found most useful about their education and to learn the extent to which students believe they learned what the faculty intended them to learn.

• Using the Sakai Information Management System, several units are collaborating on the development of e-portfolios. Student learning portfolios will be instituted at the Graduate School of Education, the School of Communications, Information and Library Studies, the New Brunswick Honors Program, the Edward J. Bloustein School of Policy and Planning, and the Douglass Residential College. Each student portfolio will include a grid matrix with nine skill categories (communication-oral, communication-written, creativity, critical and analytical thinking, leadership, project development, research, social responsibility, and technology) and five experiential categories (coursework, internships and jobs, life experiences, memberships and activities, and service and volunteer work). Students will have the option to include samples of their work and accomplishments in each “box” of their grid and also reflect on the skills that they have developed. During academic year 2007-2008 selected undergraduate and graduate students will participate in a pilot portfolio project.


Experiences at Mason Gross School of the Arts, where learning portfolios have been used for several years, will help with benchmarking.

• Continuing its efforts to research useful assessment tools, the CTAAR will pilot the Collegiate Learning Assessment (CLA) to at least 100 first year students in fall 2008 and 100 seniors in spring 2009. The CLA is designed to assess learning outcomes within the intellectual and communications skills area of our university learning goals, including critical thinking, communication, reasoning, and problem solving. The School of Environmental and Biological Sciences has expressed interest in piloting the CLA. Other schools on campus will also be given the opportunity to participate in the pilot project.

• Beginning with a pilot group of first year students and seniors, CTAAR and Institutional Research have administered the American College Testing’s College Assessment of Academic Proficiency (CAAP) Examination. The goal is to determine whether this type of testing helps us better understand the value-added in general education areas such as critical thinking and enhance our ability to document student performance on a longitudinal basis.

• The Office of Institutional Research is administering the National Survey of Student Engagement.

• To complement our existing utilization of the National Survey of Student Engagement, the Office of Student Affairs is implementing Student Life Transcripts to help assess the effect of student participation in cocurricular and student life activities on student learning outcomes.

The creation of CTAAR has signaled to the entire university community a significant institutional commitment to building a culture of assessment. The national focus on student learning outcomes, accountability, and accreditation issues has underscored the importance of our faculty fully understanding the necessity for this major initiative at Rutgers at this time. Through participation in multiple on- and off-campus programs, faculty have been involved as we collaboratively codify and enhance our existing assessment efforts while adopting new practices to meet the needs of our dynamic student population and curriculum.

The task now is to build on our strong foundation and long history of decentralized assessment to create a coordinated effort that yields timely and accurate data regarding student learning outcomes. For the core curriculum and for each major, we will need to devise an assessment system, consisting of goals, measures, and plans for the use of results. Such an effort will require a sustained institutional commitment led by the talented individuals we now have in place; educators with the vision and dynamism to realize the multiple assessment goals we have established.

The outline of our plan of action is as follows: • Creation of the Assessment Council (AC): The AC was created in the summer of

2007 to provide leadership for the continued development of learning outcome assessment at the university, monitor the four-year assessment cycle described above, and ensure that results of assessments are put to work to create change. It will report its findings upon review of learning outcome assessment plans directly to the appropriate dean and provost and the executive vice president for academic affairs, and will follow the progress of the implementation of change through the four-year assessment review cycle.

• Creation of the Executive Council on Assessment (ECA): The ECA, created in fall 2007, provides strategic planning for learning outcome assessment across the


university, ensuring the coordination of the departmental and school-based plans and activities with the university’s learning goals. Specifically:

• Benchmarking: The ECA will work with the AAC, ODL, OIRAP and the CTAAR, to develop benchmarking tools to give guidance and direction to the development of learning outcome assessment practices.

• Data resources: The ECA will coordinate the creation of data resources within the university so that good assessment practices are shared within the university community.

• Alumni outcomes: A longitudinal outcomes database for alumni, separate from the Rutgers Foundation or development office records, will ascertain the professional attainment and employment status of our graduates.

• Communication with employers: The ECA will plan and oversee the implementation of a web-based system of communication with the state’s employers about their perceptions of the preparation of our graduates for the world of work after graduation.

• Creation of assessment resource site: A comprehensive assessment resource website, CTAAR, is now available to the university community. The site includes a schedule of presentations and seminars offered by CTAAR, OIRAP, New Brunswick Faculty Council and others; guidelines for departmental assessments; descriptions of pilot projects; lists of members of key assessment committees; an online form for reporting assessment activities; links to school and departmental websites containing the learning outcome coals of the respective academic units; research reports on learning outcomes assessment; and links to other external resources.

• Reporting and discussion of learning outcome assessment. The assessment resource site will be the gateway for all learning outcome assessment activities; it will publish an online newsletter on the activities of the AAC, with a focus on the general strengths and weaknesses of various assessment plans on campus and it will offer reviews of best practices in learning outcome assessment from within and without the university. It is already publishing CTAAR Newsbeam, a website with links to informative, just published articles about pedagogy and assessment and links to many of the university’s communications venues, such as the in house online newsletter, Rutgers Focus.



RECOMMENDATIONS

1. Continue to develop and implement new assessment structures to improve data collection and analyses to support the assessment of student learning outcomes at all levels: institutional, school, and program.

Primary responsibility: Associate vice president for academic affairs—teaching and assessment

Assessment: Develop a comprehensive portfolio of measures to monitor student learning outcomes and progress, and also to evaluate the assessment system.

2. Develop and implement plans for enhancing support for and coordination of assessment efforts throughout the university.

Primary responsibility: Executive Council on Assessment

Assessment: Document and monitor progress relative to plans.


WEBSITES REFERENCED IN SECTION VI

Middle States Periodic Review Report, 2003 http://oirap.rutgers.edu/reports/MSA2008/Self-Study-Reports/MSAPRRA.pdf

Transforming Undergraduate Education Website http://ur.rutgers.edu/transform_ru/index.shtml

Transforming Undergraduate Education - Report of the Task Force on Undergraduate Education http://ur.rutgers.edu/ugtaskforce/report.shtml

Realizing the Vision for Arts and Sciences at Rutgers - Camden http://oirap.rutgers.edu/reports/MSA2008/Self-Study-Reports/Final-Deans-Task-Force-Report.pdf

Committee on Assessment of Undergraduate Programs-Faculty of Arts and Sciences 2005-2006 http://oirap.rutgers.edu/reports/MSA2008/Self-Study-Reports/UAC090606.pdf

Office of Institutional Research and Academic Planning http://oirap.rutgers.edu/

Significant Academic Developments 1997-1998 to 2006-2007 http://oirap.rutgers.edu/reports/MSA2008/Self-Study-Reports/SignificantAcademicDevelopments1997-2007.pdf

Association of American Universities Data Exchange http://www.pb.uillinois.edu/AAUDE/

Teaching Matters Program http://teachingmatters.camden.rutgers.edu/

Office of Academic Technology http://oat.newark.rutgers.edu/

Center for Teaching Advancement and Assessment Research http://ctaar.rutgers.edu/

Report of the Senate Faculty Affairs and Personnel Committee on Charge S-0109 Best Practices in Assessment of Teaching

http://senate.rutgers.edu/s0109.html

[email protected] http://digiclass.rutgers.edu/index.html

Office of Instructional and Research Technology http://oirt.rutgers.edu/

Center for Organizational Development and Leadership http://www.odl.rutgers.edu/

Rutgers 2003: A Progress Report - The “Red Tape Report” Revisited http://oirap.rutgers.edu/reports/MSA2008/Self-Study-Reports/tenyears.pdf

Pursuing Excellence in the Undergraduate Student Experience - Assessing and Improving Advising http://oirap.rutgers.edu/reports/MSA2008/Self-Study-Reports/AdvisingReport4-29-final.pdf

Advancing Community Engagement http://oirap.rutgers.edu/reports/MSA2008/Self-Study-Reports/ODL-Outreach-Report.pdf

Rutgers University Learning Goals http://oirap.rutgers.edu/reports/MSA2008/Self-Study-Reports/LearningGoals.pdf



Catalogs Home Webpage http://catalogs.rutgers.edu/

Office of Academic Services http://sasundergrad.rutgers.edu/

School of Arts and Sciences - Office of Undergraduate Education http://sas.rutgers.edu/index.php?option=com_content&task=view&id=537&Itemid=251

Degree Navigator http://nbdn.rutgers.edu/

Assessment Inventory of Academic Units http://ctaar.rutgers.edu/assessment/WGIVAssesMethbyUnits.pdf

Rutgers Writing Program - Gradatorium http://wp.rutgers.edu/courses/101/gradatorium/introduction.html

Department of Food Science - accreditation report submitted to the Institute of Food Technologists http://oirap.rutgers.edu/reports/MSA2008/Self-Study-Reports/FoodScienceIFTReview.pdf

College of Nursing - Outcomes Assessment Tables http://ctaar.rutgers.edu/assessment/Nursing-OutcomesAssessmentTablesdraft.pdf

Rutgers Universities Libraries - Information Literacy Initiative Planning - Spring Semester, 2004 http://www.libraries.rutgers.edu/rul/staff/groups/instructional_services/reports/infolit-init-planning-sp04.shtml

Rutgers Universities Libraries – Searchpath http://searchpath.libraries.rutgers.edu/

Documents

SECTION VI: ASSESSMENT OF STUDENT LEARNINGmidstate/pdf/sectionVI.pdf · SECTION VI: ASSESSMENT OF STUDENT LEARNING ... piloting the CLA for the assessment of student learning