July 19, 2004 - Pace University in New York | PACE … · Web viewWriting Program Mission Statements, Objectives, and Expected Outcomes 29 (with an Example for a Hypothetical Bachelor

September 27, 2004

Learning Outcomes Assessment Planfor

The School of Computer Science and Information Systems

Dr. Susan M. Merritt, Dean

Prepared byAllen Stix, CSIS Assessment Director

Judith E. Sullivan, Assessment/Research Analyst

September 27, 2004



1. General Understandings Underlying Learning Outcomes Assessment 1

2. Scope of Assessment in CSIS 2A. The 16 Programs to be Assessed -- Management and Responsibility 2B. Programs Not Slated for Focused Assessment by CSIS until 2006 5

3. Planning for Assessment -- Building on Our Solid History 7A. The Three Underlying Reasons for Learning Outcomes Assessment 7B. The CSIS Assessment Plan is to Continue and Extend Past Practice 9C. Assessment to Date 10D. High Priority Needs 11

4. CSIS Assessment 12(as per pages 10 and 11 of the Pace University Assessment Plan for Continuous Improvement, May 1, 2004)A. Preliminary Tasks to be Completed by Assessment Leaders 12B. Routine Activities to be Completed by Assessment Leaders 15C. Preliminary Tasks to be Completed by the Assessment/Research Analyst 16D. Routine Activities to be Completed by the Assessment/Research Analyst 16

Assessment Leaders' Calendar of Due Dates for Preliminary and Ongoing Tasks 19

ii

Appendices

1. Index of Assessment Tools Used in CSIS 21

2. For Reference: A Compilation of Mission Statements 22Pace University's Mission 22Mission of the General Education Core Curriculum 23Mission of the School of CS & IS 24Objectives of CSIS Programs 25Mission of the Bachelor of Science in Computer Science 25Mission of the Bachelor of Arts in Computer Science 26Mission of the Bachelor of Science in Information Systems 26Mission of the Bachelor of Science in Technology Systems 27Mission of the B.S. in Professional Computer Studies 27Mission of the B.S. in Professional Technology Studies 28

3. Writing Program Mission Statements, Objectives, and Expected Outcomes 29(with an Example for a Hypothetical Bachelor of Science in Computer Science)

4. Template for Reporting upon Improvements Originating from Assessments 38

5. Course Portfolios: Keith Barker's Views on Inclusions 39

6. A Rubric for Evaluating Prose Written by Students 40

7. University-Wide "Assessment Plan Procedures" 42(Pages 10-11 from the Pace University Assessment Plan for Continuous Improvement)

iii



Embraced within a Document Describing the CSIS Posture on Assessment

Accompanied by anAppendix on Missions and Program Objectives

1. General Understandings Underlying Learning Outcomes Assessment

All programs offered by CSIS provide the theoretical background as well as the applied abilities and skills needed by graduates to practice in the profession as well as to do graduate study. Learning outcomes assessment in CSIS is used to document and develop the effectiveness of its programs. It involves monitoring and improving the program's instructional content, students' mastery of the knowledge and skills within the curriculum, the success experienced by new graduates in the job market, and the professional success of alumni.

Assessments of a program are rooted in its educational objectives and associated outcomes. Assessments are processes involving the identification, collection, and preparation of data to evaluate the achievement of objectives and outcomes. Sometimes the evidence is quantitative; sometimes it is qualitative. Often it results from a short-answer questionnaire or a test; but it may come from more open-ended surveys, reviews of course portfolios, or observation. The transcending requirement is that the evidence must be trustworthy. The evaluation part of the process interprets the data and evidence accumulated through assessment practices; this includes determining the extent to which program outcomes or objectives are being achieved and results in decisions and actions to improve the program.

The term assessment is currently the referent for a number of different enterprises. In the context of this document, it refers to large-scale learning outcomes. It does not refer to mini-tests (e.g. "one minute assessments") or other edumetric indicators that might be used by individual instructors to determine how well a point, a lecture, or a section of study was received.

1

2. Scope of Assessment in CSIS

A. The 16 Programs to be Assessed -- Management and Responsibility

Sixteen different programs in CSIS require assessment including CIS101, the Learning Communities in which CSIS participates, the writing enhanced courses, and other contributions to education at Pace.

Assessment effectiveness requires good instruments and techniques as well as timetables for activities and procedures. Efficiency is promoted when good assessment instruments and techniques are shared across programs. This can be accomplished by developing and administering program assessments in accordance with similarities in programs' objectives and similarities in the needs of the students (i.e. similarities in program missions). For instance, all baccalaureate programs incorporate objectives fulfilled by the University's Core Curriculum. These objectives, however, are not shared by programs at the master's level.

The management of each program's assessment will be performed through the efforts of the individuals specified below.

The CSIS Assessment/Research Analyst, Judith Sullivan, reporting to Deans Merritt and Houle and working with Allen Stix, will oversee all assessment activity and assist. Department and program chairpeople are the CSIS assessment leaders. They are entrusted with ensuring that the assessments of programs under their aegis are completed fully, properly, and on schedule. This includes reviewing and using the results. The plan and functioning of each program's assessment will be overseen and approved by Dean Merritt.

The Pace University Assessment Plan for Continuous Improvement [Draft Eleven, page 10, item 6.a.] specifies that assessment plans include the names of those responsible for assessment.

The buy-in and involvement of faculty (and students) is critical. Support is needed for proactive cooperation in compiling course portfolios (e.g. sample projects and exams), in constructing and embracing standardized course syllabi, in the progressive maintenance of program content, in writing and scoring "rising junior" and "graduating senior" achievement tests, and in identifying creative means for improvement followed by the energy to make them happen.

2

Department chairpeople, or program directors, are the individuals with primary responsibility for each program's assessment. CSIS looks to them for assessment leadership.

The Pace University Assessment Plan for Continuous Improvement [Draft Eleven, page 7] emphatically,through repetition, calls for faculty participation, with chairperson support, in all aspects of assessment.The goal is that assessment will become an integral part of departmental culture, with faculty using the results "to improve the curriculum" as well as "to foster improvement" in student learning.

In an earlier section (page 4), this document states that "The University is committed to developing a sense of faculty, staff, and student ownership of the assessment plan in the hope that ownership will translate into a commitment to use assessment results in improving courses and programs."

Programs leading to theAssociate in Science Degree in Applied Information Technology

CLOUT - Computers-Literacy-Opportunity-University-TechnologyRepresented by Bernice Houle, Nancy Hale, and Lynne Larkin

NACTEL - National Coalition for Telecommunications and LearningRepresented by David Sachs and Nancy Hale

Other Applied Information Technology ProgramsNancy Hale

3

Programs leading to theBaccalaureate Degree

Bachelor of Arts in Computer ScienceBachelor of Arts in Information Systems

Bachelor of Science in Computer ScienceBachelor of Science in Information SystemsBachelor of Science in Technology Systems

Bachelor of Science in Professional Computer StudiesBachelor of Science in Professional Technology Studies

Represented by Departmental Chairpeople:Narayan Murthy and Sotiris Skevoulis (CS)Daniel Farkas and John Molluzzo (IS)Nancy Hale (TS)

and Program Curriculum Committee Chairpeople:Mary Courtney (CS) and Constance Knapp (IS)

Programs leading to theMaster of Science Degree

Master of Science in Computer ScienceMaster of Science in Information SystemsMaster of Science in Internet Technology for e-CommerceMaster of Science in Software Design and DevelopmentMaster of Science in Telecommunications

Represented by Departmental Chairpeople:Narayan Murthy and Sotiris Skevoulis (CS)Daniel Farkas and John Molluzzo (IS)Nancy Hale (TS)

and by Program Curriculum Committee Chairpeople:Mary Courtney (CS) and Constance Knapp (IS)

Doctor of Professional Studies in Computing

DPS Program Chair Fred Grossman and Associate Chair Charles Tappertand the DPS Executive Committee (Susan Merritt, David Sachs, Allen Stix)

4

Service Learning

CIS101 (Nancy Hale, Jonathan Hill)Learning Communities (Allen Stix)Writing Enhanced Courses (Allen Stix)

B. Programs Not Slated for Focused Assessment by CSIS until 2006

CSIS assessment efforts are expended upon programs serving the greatest number of students and upon programs identified for development either because they are new or because they are undergoing substantial rejuvenation. The programs offered or supported by CSIS but deferred from early concern are listed below. Among the supported programs are the Bachelor of Business Administration in Information Systems, the Master of Business Administration in Information Systems, and the Master of Science in Accounting Information Systems. These programs are run by the Lubin School of Business and assessed by Lubin. CSIS has, and will, assist attentively in these assessments by supplying course materials, results from course evaluation surveys, pertinent staffing information, and other documentation and cooperation to enable the fulfillment of Lubin's requirements.

Although the following programs are not slated for early CSIS assessment, they will benefit from more imminent assessment activities inasmuch as they are built from the same staple of courses, taught by the same faculty members, and situated within the same academic climate.

Baccalaureate Programs with Heavy CSIS Support

BBA in Information Systems (Lubin program)

Combined Degree Programs within CSIS

BA in Computer Science and MS in Information SystemsBA in Computer Science and MS in TelecommunicationsBS in Technology Systems and MS in Information Systems

Combined Degree Programs with Dyson College

BS in Chemistry and MS in Computer ScienceBS in Chemistry and MS in Information Systems

5

Collaborative Masters Programs

MS in Accounting Information Systems (Lubin program supported by CSIS)

MBA in Information Systems (Lubin program with heavy support from CSIS)

MS in Nursing Informatics ( Lienhard program with heavy support from CSIS)

Undergraduate Minors

Computer ScienceInformation Assurance in the Criminal Justice System

(in collaboration with the Department of Criminal Justice)Information SystemsInternet TechnologyTechnology SystemsWeb Media

Certificate Programs (composed from undergraduate courses)

Applications ProgrammingComputer ProgrammingInformation Assurance in the Criminal Justice System

(in collaboration with the Department of Criminal Justice)Information SystemsInternet Technologies CertificatePersonal Computer Applications for the Workplace ProfessionalPersonal Computer and Multimedia ApplicationsWeb Media

Advanced Certificate Programs (composed from graduate courses)

Computer Communications and NetworksComputing for TeachersInformation SystemsInternet TechnologyNursing Informatics (in collaboration with the Lienhard School of Nursing)

Object-Oriented ProgrammingSoftware Design and DevelopmentTelecommunications and Information Assurance

6

3. Planning for Assessment -- Building on Our Solid History

A. The Three Underlying Reasons for Learning Outcomes Assessment

The activities, instruments, use of acquired information, and purposes of reports can seem muddled without the realization that learning outcomes assessment is performed for three distinctly different, although related, reasons: accountability, quality control, and "product" upgrade.

accountabilityStudents, parents, organizations granting scholarships, employers, accreditation agencies, and the University itself wants to know that students in CSIS are being well-served. If the mission of a program is to prepare graduates for careers in software development, students will enroll (and employers will hire and depend on them).

Accountability makes the program's "owner" (us) responsible for demonstrating that it produces the intended result. Assessment for accountability is tantamount to showing prospective stake-holders that the program is a reliable purchase. The data should remove all doubts or, better yet, convey confidence and enthusiasm.

For particular stake-holders, accountability extends to demonstrations of cost-effectiveness (i.e. economy; good use of resources), student retention, timely graduations, and external validations of program quality.

quality control Minding the educational process, while it is going on, is the concern here.

One intent is to catch and rectify problems affecting the development of students as they move through a program. A textbook illustration is the "rising junior" achievement test, termed a formative assessment. Time remains to remedy deficiencies detected in the attainment of individual students as well as in a cohort's core experience.

Another intent is to catch and rectify problems for the sake of the program's strength. The results from a pre-graduation achievement test, termed a summative assessment, will not help the exiting seniors but be valuable for improving the job we do with the advanced electives or for bolstering the capstone experience.

7

product upgrade Programs in CSIS function within an environment of change, both in the technology we teach as well as in the technology we use for teaching.

The former implies the need to monitor the skills and abilities sought by organizations when hiring our graduates and to keep the curriculum of each program in synchrony. This means making the necessary adjustments to course outlines and materials, exercises, projects, and examinations presented to students. Also, it means creating new electives or changing one or another of the program's requirements.

The latter impacts: the nature of some, most, or all face-to-face classes (e.g. greater utilization of video-

conferencing for sharing courses between campuses and adopting pedagogical techniques for maximizing the utility of laptop initiatives)

forms of assignments and course-related communications (e.g. Blackboard discussions, electronic student collaborations, extensive student-faculty interaction through email)

course structure (e.g. Web assisted courses and courses that are fully on-line)

Program improvement is evolutionary. Innovations are tried and assessed. What is good is retained, and what is not is discarded. Decisions are based on appropriate evidence.

Assessment reports tend to be oriented toward accountability, quality control, or product upgrade; and a specific assessment may be conducted with a particular report in mind. However, any one assessment investigation usually serves all three purposes. This is the reason that all the data collected for assessment, and all reports, are centrally archived within the School. For just one example, consider a study of performance reviews by internship supervisors. Its objective may have been to acquire feedback from those who supervise and train beginning professionals on the skills and abilities with which our students are equipped (i.e. accountability). However, the data would also point to areas of background (e.g. principles of data communications) and practical experience (e.g. Linux) that our students may be missing. This would be pertinent input relative to the on-going evaluation of the curriculum in multiple majors (i.e. product upgrade). Furthermore, if gaps were exposed, we could intercede before students enter the workforce as graduates (i.e. quality control).

8

B. The CSIS Assessment Plan is to Continue and Extend Past Practice

Pace University is embarking upon a new era of formalized assessment, and CSIS enters this era with considerable experience and corroborated success. Over the years the Computer Science Accreditation Board (CSAB), now succeeded by the Computing Accreditation Commission (CAC) of the Accreditation Board for Engineering and Technology (ABET), requires a process that includes a program's educational objectives, outcomes, assessment and evaluation. The CSAB/ABET concern for outcomes assessment culminated in 2000 with program accreditation standards that were entirely assessment-centric. That year, the BS in computer science earned re-accreditation for the period of six years with no concerns. The following year, the BS in information systems, examined by ABET applying corresponding standards, was similarly accredited with no concerns

Earlier, in the 1998 report by the Middle States Association on their visit to Pace, CSIS was singled out for special commendation on the comprehensiveness and effectiveness of its array of assessment activities.

Reasonably, the CSIS assessment plan is to continue its established practices. One of its practices, a meta-practice, is to broaden, deepen, and refine its assessments on a continuing basis.

Broadening refers to expanding the scope of assessment to introduce it to all programs..

Deepening refers to developing new instruments and/or procedures in order to get better data and to learn about aspects of programs not previously examined. For instance, the assessment depth of the baccalaureate programs in computer science will be enhanced with achievement tests, administered to graduating seniors, that supply quantitative confirmation of professional masteries. The assessment depth of the DPS was recently enlarged with an investigation into the quality of the emerging dissertations and with another investigation into how a three-year graduation might be facilitated for a greater proportion of students.

Refining refers to building upon experience to improve existing instruments and/or procedures.

9

C. Assessment to Date

The following offers a view of our history and current position.

Certain fundamental assessments span all programs and are conducted every year. For more than the past ten years, every course in CSIS, with no exception, concluded with a Course Opinion Survey eliciting student feedback on central aspects of instruction, from the clarity and fulfillment of course objectives to the availability and respectfulness accorded by the instructor.

For almost as long, graduates emerging from all CSIS associate, baccalaureate, and masters degree programs have been queried about the knowledge, skills, and abilities acquired through their coursework and about the learning environment. (DPS students are queried, in-depth, about curriculum and program conduct at the end of the first year and, again, at the end of the second year.)

BS alumni in computer science and information systems are queried about their careers (where they are working, job title, and responsibilities) to ascertain confirmation that they are succeeding professionally. Masters alumni in all areas are queried periodically (three years out, six years out, and nine years out).

The idea of asking industrial practitioners about the learning deemed to have applied value originated with faculty in Computer Science (in 1997) in response to the Curriculum Committee's quandary over the extent of coverage of certain traditional content (e.g. the depth of coverage of assembly language) in contrast to material that was more applicable but less foundational (e.g. networks). A questionnaire was sent to over sixty companies that hire Pace students, and members of the CSIS Advisory Board were interviewed. Curriculum adjustments were made as a result. Since that time the procedures and methods of inquiry have continued to change, but not the concept of querying those in the field. Also, Information Systems now conducts similar inquiries; so this is one of those assessments adopted for a second set of programs (those in IS) after proving useful to a first set of programs (those in CS). As things have evolved, curriculum inquiries either focus on the whole curriculum or on a particular issue. A general study of the IS curriculum came-up with the recommendation that collaborative projects are needed from which students learn about teams including how to build consensus, integrate, and lead. A single issue study was the investigation by Computer Science on whether to switch from C++ to Java.

The idea of using course portfolios as a direct view into the instructional content and activities within courses as well as to document attainment came from the CSAB in 1986. A course portfolio (including all sections) includes the course outline; tests and a sample of excellent, good, and papers; and similarly for projects, significant assignments, and Web referents. (In the future, portfolios will be Web based in part or in entirety) Thus it displays the accomplishment expected and the products showing fulfillment. Computer Science has been compiling these for courses in the BS in CS since 1985. Information Systems, in preparing for their first accreditation (Fall, 2001) now does likewise for courses in its BS program.

10

New and special assessment projects come-up continually. During the Fall semester of 2003, for instance, CSIS undertook a painstakingly detailed mapping of program curricula onto sets of standards from the National Security Association relative to obtaining certification under the Information Assurance Courseware Evaluation Program. This was followed by a rigorous self-study in application for designation as a Center of Academic Excellence in Information Assurance Education. During the Spring semester of 2004 and throughout the summer, an assessment was made of the quality of doctoral dissertations completed by students in the DPS. Also during the spring and summer, an extensive investigation was completed relative to strategies for increasing the proportion of students who graduate in three-years of matriculation (a timely graduation has been an important program objective).

D. High Priority Needs

Gaps in our assessment process are the formal tools for student-by-student formative and summative assessments in all programs. ("Formative assessments" are the tests of skills and abilities acquired through a major's core. "Summative assessments" are the tests of proficiencies with which students leave a program for professional practice.) These will convey the exact distribution of accomplishment levels among student aggregates that course portfolios may not. While a formative assessment or a summative assessment may sound like a single, paper-and-pencil (or online) test; it may actually have several parts that are administered at different times. For example, part could be an oral presentation rated on several criteria by two or more instructors and part could be an applied skills test. As stated previously, creating these achievement tests are projects that must begin immediately.

Another upcoming project is to examine the job placement experiences of new graduates with more detail than we have in the past. Past activities were restricted to tabulating data, as available from the Office of Cooperative Education and Career Services, on organizations and positions. CSIS needs to go further; we need to close the skills and abilities loop connecting industrial guidance, the curriculum, and job placements. We need to know the exact nature of the jobs for which our graduates are selected, what might strengthen their candidacies and performance, whether there are desirable jobs they tend not to be getting, and if so why.

11

4. CSIS Assessment (as per pages 10 and 11 of the Pace University Assessment Plan for Continuous Improvement , May 1, 2004)

Assessment in CSIS is supervised by Judith E. Sullivan, the CSIS Assessment/Research Analyst, working under the direction of Dean Susan M. Merritt and Dean Bernice Houle in consultation with Allen Stix. Responsibility for managing the assessment of particular programs and special services was shown above. These individuals, collectively, will be referred to as the assessment leadership team.

A. Preliminary Tasks to be Completed by Assessment Leaders

The following preliminary tasks and routine activities apply to each program and are to be handled by the leaders in conjunction with his or her faculty. [Please see the Calendar of Due Dates for Preliminary and Ongoing Tasks, on pages 19-20, for a chronological listing of the tasks in this and the following section (4B).]

1) Mission, Objectives, Outcomes [of all programs listed in section 2A] Deadline: October 15, 2004

Each program will require a mission statement, consistent with the CSIS mission statement, that will describe in high level terms what the program will accomplish.

Each program or service will require a set of educational objectives -- statements that, as a body, "describe the career and professional accomplishments that the program is preparing graduates to achieve" [Common ABET Terminology]. These statements must be sufficiently concrete as to be measurable.

Each program or service will require a set of educational outcomes directly related to the objectives -- "narrower statements that describe what students are expected to know and are able to do by the time of graduation" [Common ABET Terminology].

Leaders will need to review the existing mission statement for currency and appropriateness or they will need to draft one. Similarly for program objectives and outcomes. The appendix provides guidance with a discussion of mission statements, a typology for classifying objectives, and an illustration of how a set of measurable objectives and expected outcomes might look.

12

2) Standard Syllabi for all Courses in Baccalaureate Programs Deadline: November 30, 2004

Each course in the program will require the following:

a description suitable for the catalogprerequisites by topic and earlier coursesthe textbook (or two from which individual instructors may choose)a list of the topics covereda list of learning goals (e.g. the specific abilities expected of emerging students)the project(s) completed by students in each sectionthe theoretical contentthe problem analysisthe design of solutions

Leaders will need to review existing syllabi for currency and descriptive completeness and where necessary construct new ones. It is important to note that these define the learning and experiences that will be common from section to section. Instructors will base their course outlines on the standard syllabi. A program's set of syllabi, through the learning objectives, will enumerate its educational outcomes, indexing where each is acquired. The appendix shows a sample standard syllabus and gives the desired format.

3) Formative Assessments for Baccalaureate Programs Deadline: January 31, 2005

Formative assessment will begin in the Spring 2005 semester. Focusing on the educational outcomes derived through the major's core, the assessment may consist of multiple instruments and activities (e.g. demonstrations of skills). Results will indicate the levels with which educational outcomes are realized. Instruments and procedures are due by 1/31/2005.

4) Summative Assessments for Baccalaureate Programs Deadline: January 31, 2006

Summative assessment will begin in the Spring 2006 semester. As with the formative assessments, a summative assessment may consist of multiple instruments and activities. For instance, it could include a writing sample, evaluated with respect to a rubric, and a speech, evaluated by two or three faculty members. Instruments and procedures are due by 1/31/2006.

5) Standard Syllabi for all Courses in AIT Programs Deadline: November 30, 2004

13

6) Summative Assessments for AIT Programs Deadline: January 31, 2005

Summative assessment will begin in the Spring 2005 semester. Instruments and procedures are due by 1/31/2005.

7) Standard Syllabi for all Courses in Masters Programs Deadline: November 30, 2005

8) Summative Assessments for Masters Programs Deadline: January 31, 2006

Summative assessment will begin in the Spring 2006 semester. It may consist of one or more instruments, including evaluations of capstone projects with respect to a rubric. Instruments and procedures are due by 1/31/2006.

9) Assessment of Dissertations Emerging from the DPS Deadline: March 15, 2004

The quality of research completed and reported by students in the DPS program is an indicator of program achievement. The assessment of dissertations will begin in the Spring 2004 semester. [Note: Instruments and procedures were completed by 3/15/2004. The report on the Spring 2004 assessment was delivered to the DPS Executive Committee on June 10, 2004.]

10) Assessment Plan for CIS101 Deadline: November 30, 2004

CIS101, Introduction to Computing, has an annual throughput of approximately 2,300 students and involves around 30 instructors. The University constituencies served by CIS101 will require its assessment as part of their assessment plans (e.g. the task force on the University Core Curriculum, the Lubin School of Business, and the Lienhard School of Nursing). Likewise, its assessment will be of concern to Middle States Association of Colleges and Secondary Schools.

11) Assessment Plan for Learning Communities, Deadline: June 15, 2005Writing Enhanced Courses, and Other Service Offerings

12) Assessment Plans for Minors, Concentrations, Deadline: March 31, 2006Certificates, and Combined Degree Programs [of all programs listed in section 2B]

These programs are composed from courses in programs with assessment activities that will have been in place before 3/31/2006; hence they are receiving attention indirectly. Their assessment is deferred until last because they serve substantially fewer students than those for which assessment is begun earlier.

14

B. Routine Activities to be Completed by Assessment Leaders

1) Formative assessment of baccalaureate programs, conducted annually toward the end of the Spring semester, beginning in Spring 2005

2) Summative assessment of Applied Information Technology programs, conducted annually toward the end of the Spring semester, beginning in Spring 2005

Summative assessment of baccalaureate and masters programs conducted annually toward the end of the Spring semester, beginning in Spring 2006

3) Compilation of materials for Course Portfolios; materials are collected each Fall and each Spring semester beginning with the 2005-6 academic year (i.e. Fall 2005) [Materials from courses offered only during Summer I or Summer II, or at other special times, are collected as these courses are presented.]

4) Curriculum Review, conducted annually, beginning in 2005, following the Spring semester, using data from surveys supplied by the CSIS Assessment/Research Analyst -- program objectives, learning outcomes, and standard syllabi will be revisited for currency

5) Comprehensive Program Reviews: Review of formative assessment data and summative assessment data, as well as data supplied by the CSIS Assessment/Research Analyst, on the employment and/or graduate school matriculations of the most recent graduating class and the performance reviews of CoOp supervisors in order to identify opportunities for program improvement -- this will take each year following the end of the Spring semester, beginning in 2005

The expectation exists that plans will be formulated for implementing the most needed or most promising improvements.

6) Review of instruments and procedures for formative and summative assessment, conducted annually, following the Spring semester beginning in 2006

7) An annual report, due July 1, beginning in 2005, will be submitted to the Dean and to the CSIS Assessment/Research Analyst on the year's formative and summative assessment results, changes to the curriculum or standard syllabi, and changes to the formative and summative assessment instruments. Particularly important are the newly identified opportunities for program improvement, the improvements pursued, how they are being pursued, and, retrospectively, an evaluation of the success of earlier improvements.

15

C. Preliminary Tasks to be Completed by the Assessment/Research Analyst

The following activities will be conducted by the CSIS Assessment/Research Analyst:

Preliminary Tasks

1) Create a new survey for undergraduates, similar to the existing exit survey, to be given during the Spring semester confirming that students are informed on the structure of their program, that students have a fix on which courses they need to take and when, that the needed courses are available, and that students do not take CSIS courses out of sequence.

This survey should also confirm that in each CSIS course the standards and procedures for grading were transparent and fair, that the instructor was respectful, and that the instructor was available for consultation. It should also ask about other aspects of the learning climate.

D. Routine Activities to be Completed by the Assessment/Research Analyst

1) Oversee the administration of the new annual survey of undergraduates, the summarization of the results, and the reporting and archiving of the results.

2) Oversee the annual exit survey.

3) Oversee the course opinion surveys at the end of each semester, including summer I and summer II

4) Conduct an annual inquiry on the abilities and skills sought by employers in new graduates.

This may involve the CSIS Advisory Board, organizations that hire Pace graduates, organizations that supply internships, etc. The investigation might be designed in consultation with assessment leaders (i.e. department or program chairs and curriculum committees). Reports should go to the Dean and to assessment leaders.

5) Obtain the CoOp interns' performance reviews from Cooperative Education and Career Services, summarize the results, and send a report to the Dean and assessment leaders.

16

6) Obtain information on the job placement and/or pursuit of graduate school of the new graduates from each program. (Cooperative Education and Career Services should be help.) Summarize the results and send a report to the Dean and assessment leaders.

7) Conduct alumni surveys. Annually, one third of the total alumni body from each program will be surveyed on where they are employed, their title, and their day to day responsibilities.

8) Look after assessment throughout CSIS to make sure that activities are proceeding properly and on schedule, and that data is thoroughly analyzed, results appropriately examined, and improvements identified and implemented. When necessary, provide guidance or hands-on help. Watch for ways in which individual instruments and procedures as well as overall policies and plans might be made more effective or efficient and propose improvements to the Dean.

Good assessment, like good science, is methodical and rigorous. Also like good science, good assessment rests upon human perspicacity and creativity. CSIS depends upon the Assessment/Research Analyst for astute top-level planning and piloting.

9) Prepare the annual report; for delivery to OPARAB in August, beginning in 2005; on CSIS assessment activities (month of delivery subject to OPARAB's approval).

The report will center on the uses to which new findings were put. The probable format is a table with terse descriptions in the following columns:

ProgramObjective

Expected Outcome

How Outcome Was Measured

When Measured Improvement Identified

Improvement Implemented

The report will include augmenting discussion as needed for completeness and clarity.

Other items that may be in a report are evaluations of improvements from the past, focusing on their effectiveness, unanticipated benefits (if any), and unexpected dysfunctions (if any) as well as accounts of changes in CSIS assessment procedures and anything new (e.g. new efforts or new instruments).

17

10) Collect and record semesterly CSIS demographics. The following provide insights into factors that bear upon the learning and the efficiency with which CSIS delivers education:

total enrollment in each program as well as disaggregated by campus and, where pertinent, by year

the number of students beginning each program with a description of where they are from and of their scholastic background

the number of transfer students entering each program at an intermediate point with a description of where they are from and of their scholastic background

a complete list of classes (including tutorials and independent studies) offered by CSIS including, for each, its size, its classification (face-to-face, Web assisted, or on-line), its site, day or evening, and instructor

faculty publications

CSIS grants

18

Attention Assessment Leaders:Calendar of Due Dates for Preliminary and Ongoing Tasks

2004October 15 Mission, Objectives, Outcomes [of all programs listed in section 2A]

November 30 Standard Syllabi for all Courses in Baccalaureate Programs

November 30 Standard Syllabi for all Courses in AIT [Applied Information Technology] Programs

November 30 Assessment Plan for CIS101

2005January 31 Formative Assessment Instruments for Baccalaureate Programs

January 31 Summative Assessment Instruments for AIT Programs

Spring Semester

All Baccalaureate Programs: First Annual Administration of Formative Assessment tests

Spring Semester

All AIT Programs: First Annual Administration of Summative assessment tests

Following the Spring Semester

All Programs Listed in Section 2A: First Annual Curriculum Review -- program objectives, learning outcomes, and standard syllabi will be revisited for currency


All Programs Listed in Section 2A: First Annual Comprehensive Program Review to identify opportunities for program improvement

June 15 Assessment Plan for Learning Communities, Writing Enhanced Courses, and Other Service Offerings

July 1 First Annual Assessment Report to the Dean and the CSIS Assessment/Research Analyst

Fall Semester All Courses in Programs Listed in Section 2A: First Semesterly Collection of Materials for course portfolios

November 30 Standard Syllabi for all Courses in Masters Programs

19

2006January 31 Summative Assessment Instruments for Baccalaureate

Programs

January 31 Summative Assessment Instruments for Masters Programs

Spring Semester

All courses in programs listed in section 2A: Second Semesterly Collection of Materials for Course Portfolios

Spring Semester

All Baccalaureate Programs: Second Annual Administration of Formative Assessment tests

Spring Semester

All AIT Programs: Second Annual Administration of Summative assessment tests

Spring Semester

All Baccalaureate and Masters Programs: First Annual Administration of Summative Assessment tests


All Programs Listed in Section 2A: Second Annual Curriculum Review -- program objectives, learning outcomes, and standard syllabi will be revisited for currency


All Programs Listed in Section 2A: Second annual Comprehensive Program Review to identify opportunities for program improvement


All Programs Listed in Section 2A: First annual review of the instruments and procedures used for the formative and summative assessments

March 31 Assessment Plan for Minors, Concentrations, Certificates, and Combined Degree Programs [i.e. of all programs listed in section 2B]

July 1 Second Annual Assessment Report to the Dean and the CSIS Assessment/Research Analyst

Fall Semester All courses in programs listed in section 2A: Third Semesterly Collection of Materials for course portfolios

20

Appendix One

Index of Assessment Tools Used in CSIS

1. surveys to ensure progressiveness of the curriculum and professional pertinence (coverage, breadth, depth, and level of rigor)

2. coherence review of course syllabi to ensure coverage of each program's curriculum and a prerequisite structure that minimizes instructional redundancies and offers flexibility in schedule planning

3. section portfolios as a window into the challenges presented to students and the accomplishment being achieved

4. course opinion surveys, to learn about students' satisfaction with individual courses

5. deposits into the CSIS suggestion box from students

6. deposits into the CSIS suggestion box from staff and faculty

7. a "mid-way" achievement test, as a formative assessment, to ensure mastery of each major's core (may include demonstrations of skills)

8. a demonstration of expertise acquired through the program as a summative assessment (may include demonstrations of skills)

9. evaluations by students of the contribution of their internship to their professional development (kindly supplied by the Office of Career Services and Cooperative Education)

10. evaluations of students on internships by Co-Op supervisors (kindly supplied by the Office of Career Services and Cooperative Education)

11. surveys of students on how well-informed they are on the program's course requirements vis-à-vis the worksheet, what to take next, and what lies ahead; the availability of advising; the clarity of course objectives, requirements, and how grades are determined; and the supportiveness of the environment

12. a survey of program graduates of satisfaction with their education, the supportiveness of the CSIS learning climate, and, in general, the Pace experience

13. a compilation of job placements of program graduates (kindly supplied by the Office of Career Services and Cooperative Education) or continued schooling

14. regular surveys of alumni on long-term career advancement

21

Appendix Two

For Reference: A Compilation of Missions and Objectives

Pace University's Mission

from: Second Century Strategic Plan: 2003-2008 page 1

Pace University is committed to providing the best possible private education to a diverse and talented student body at each Pace Location. The University's continuous commitment to Opportunitas must be retained, strengthened and, when necessary, redefined in an uncertain world where many are excluded from the full benefits of higher education due to economic and social factors beyond their control. Pace's sustained emphasis on excellence will be a constant, which will guide decision-making regarding academic programs and other endeavors. In a world of increasing interdependence, Pace must strengthen its international efforts and continue its significant investment in technology. In addition, the University's commitment to self-evaluation and civic engagement must remain as major goals. All of this must be accomplished during a time of considerable economic change and in the context of strengthening Pace's own financial base.

from: Pace University Undergraduate Catalog: 2002-2004 page 8

Expression of the Pace University mission, as approved by the Board of Trustees in January 1998:

In keeping with its motto, Opportunitas, Pace University offers qualified students the opportunity to discover and fulfill their potential. At Pace, supportive and challenging programs prepare graduates for meaningful lives and successful careers in a rapidly changing world. The University's commitments combine respect for traditional academic values with readiness to meet the challenges of the future with energy and innovation.

22

Mission of the General Education Core Curriculum


The University Core Curriculum provides experiences that enrich students and enable them to explore the arts, history, literature, languages and cultures, philosophy, the social sciences, mathematics, computing, and the natural sciences. Knowledge of these areas is central to an educated person and is continually affirmed as essential to professional success. The Core endeavors to engage students in learning, which illuminates the human condition, encourages exploration of both the physical and moral realms, and emphasizes the importance of social and individual responsibility. The Core strives to support the student's personal and professional goals, while also producing responsible citizens. Students experience the humanities, the sciences and the social sciences in order to meet the challenges of a complex and changing world as well as to understand its cultural and intellectual heritage. They are helped to develop clear, critical and creative thinking, sound judgment, effective oral and written expression, and quantitative skills.

The courses are arranged to progress from intellectual skills, through a balanced range of disciplines, to enhanced study in the arts and sciences which may be dictated by the personal or professional development of the student. The Core reaffirms the essential power of the liberal arts to enrich, broaden and enhance the education of students so they are not only successful professionally but also enabled to live rich and satisfying lives. By exposing students to the wide range of ideas and experiences, which have characterized the history and development of modern life, the general education core enables individuals to ask significant questions about themselves and their world and empowers them to develop meaningful responses.

The University Core Curriculum seeks to balance:

** Western and Non-Western ideas and experience

** An understanding of diversity with an appreciation of the need for community

** Classical ideas and works with emerging voices

** Global views and issues with national perspectives and concerns

** Social responsibility with individual responsibility

** Technological literacy with human social interaction

23

Mission of the School of CS & IS


The School of Computer Science and Information Systems aspires to innovative leadership in preparing man and women for meaningful work, lifelong learning, and responsible participation in a new and dynamic information age. The School does this through a broad spectrum of educational programs on campuses in New York City and Westchester County and at other locations with community and corporate partners from the local and global community.

The School has a unique role: it provides professional education in the computing disciplines, supporting education for programs in other schools, general education for all students, and continuing education. Because change characterizes information technology, programs are built upon a strong foundation in the arts and sciences and emphasize competency in the theory and methodology of the computing disciplines. At the same time, programs are responsive to the rapid pace of technological development.

The School was founded in 1983 in creative response to the educational challenge and opportunity inherent in emerging disciplines, and is characterized by its core values:

** excellent teaching that is informed by scholarship, professional practice, and community service,

** the integration of theory and practice in teaching and scholarly activity,

** creative programs and partnerships with the local and global community,

** attentiveness to professional and social responsibility.

The School values diversity and welcomes qualified students of various experiences and origins, both regional and international. It provides excellent service to students both within and outside the classroom. It uses the power of technology to offer broad opportunity to student and to enable them to achieve excellence. Throughout its programs and services, the School of Computer Science and Information Systems consistently recognizes that information technologies are tools for the empowerment of people.

24

Objectives of CSIS Programs


Consistent with the Pace tradition, the School seeks to integrate theory and practice in its programs and research. The design, development, analysis, application and management of computers, and communication and information systems comprise the broad spectrum over which the School creates, interprets, criticizes, and applies knowledge with strict attention to academic standards. Change characterizes information technology; of particular importance, therefore, is the development of competency in the foundation and methodologies of the discipline, in order to enable ongoing learning and effective response to change.

The School is dedicated to the service of men and women of all ages of every race and culture through educational programs that develop skills, enhance individual and community effectiveness, extend knowledge, and enhance critical understanding of the culture. The educational process is undertaken with concern for the development of personal, professional, and social responsibility.

Mission of the Bachelor of Science in Computer Science


The B.S. program in computer science is a professional program that is accredited by the Computing Accreditation Commissi0on (CAC) of the Accreditation Board for Engineering and Technology (ABET). The curriculum is based upon algorithms and data structures, the principles of programming languages, computer architecture, data communications, and theoretical foundations. It includes advanced work in various areas including software engineering, operating systems, compilers, artificial intelligence, and graphics. Program requirements include the liberal arts core and other academic requirements that specifically apply to this Bachelor of Science degree.

The B.S. program provides excellent preparation for graduate study in computer science or for professional placement.

from the blue brochure created by Admissions (2002):The Bachelor of Science in Computer Science

This program focuses on the fundamentals of computing, including programming, data structures and algorithms, and computer architecture, as well as advanced study in operating systems, compilers, artificial intelligence, and graphics.

25

Mission of the Bachelor of Arts in Computer Science


The B.A. program in computer science shares the computer science core with the B.S. program and is structured in such a way that allows for more program diversity for the student who wishes to pursue a minor in business, Internet technology, or one of the arts and sciences.

from the blue brochure created by Admissions (2002):The Bachelor of Arts in Computer Science

Integrating computer science with a career in business or a liberal arts field, this program has the same core as the B.S. program; however, instead of taking as many advanced electives in computer science, these students take a concentration of courses in business, information systems, or the liberal arts.

Mission of the Bachelor of Science in Information Systems


The B.S. program in information systems is the first program nationally to be accredited by the Computing Accreditation Commission (CAC) of the Accreditation Board for Engineering and Technology (ABET). The program is designed to provide the student with current technical skills and knowledge of those information systems concepts that remain constant in the face of technological change, as well as detailed awareness of a cohesive body of knowledge to prepare students to function effectively as an IS professional in the IS environment. The continual appearance of new and increasingly powerful software tools for applications development, as well as the availability of low-cost hardware (personal computers),has created new organizational approaches to building computer information systems.

from the blue brochure created by Admissions (2002):The Bachelor of Science in Information Systems

This program focuses on the technical aspects of information systems and how they are organized. Courses focus on programming, systems analysis and design, business communications, and database management.

26

Mission of the Bachelor of Science in Technology Systems


As the personal computer and other technologies are integrated into the workplace, the technical knowledge required to select, manage, and maintain these systems is critical. The B.S. in technology systems combines theoretical and technical skills with practical applications to prepare the graduate with the skills necessary for the contemporary workplace.

from the blue brochure created by Admissions (2002):The Bachelor of Science in Technology Systems

A variety of skills is needed to develop, manage, and maintain the emerging technologies. Students in this program focus on all aspects of the personal computer, including evaluation of software packages, multimedia, telecommunications, end-user information systems, technical writing, and hardware maintenance and troubleshooting.

Mission of the B.S. in Professional Computer Studies


The B.S. in professional computer studies has been developed for those computer professionals with considerable on-the-job experience who would benefit from having a baccalaureate degree in computing and would most likely be interested in pursuing a masters degree in a computer-related field once the undergraduate requirements are completed.

from the blue brochure created by Admissions (2002):The Bachelor of Science in Professional Computer Studies

Developed for computer professionals with considerable on-the-job experience, this program would benefit students who are interested in earning a baccalaureate degree in computing, and would most likely be interested in pursuing a master's degree in a computer-related field.

27

Mission of the B.S. in Professional Technology Studies


The B.S. in professional technology studies is an online accelerated degree program designed to prepare students for successful professional careers in a global economy in the midst of rapid technological change. It incorporates a multidisciplinary concentration in Internet technologies for e-commerce with courses in the liberal arts and sciences. It is an innovative, online accelerated degree program for adults who have experience in the workplace and an A.S. (Associate in Science) or A.A. (Associate in Arts) degree or the equivalent (64 credits).

from the blue brochure created by Admissions (2002):The Bachelor of Science in Professional Technology Studies

Using a structured, accelerated format, adult students earn a bachelor's degree with a concentration in Internet Technologies for e-Commerce in two years. The degree is designed for working professionals who are interested in pursuing an online program.

28

Appendix Three

Writing Program Mission Statements, Objectives,and Expected Outcomes

~ A Typology for Objectives ~

~ An Example of a Program's Mission, Objectives, and Outcomes ~

Missions address two different sets of objectives: (i) the objectives of the program itself (the department, the school, or the University) as a working and developing entity, and (ii) the outcomes held for the students it serves. Mission statements would be clearer if these sets of objectives were explicitly distinguished and not intermingled.

Every program mission statement needs to be "expanded" (i.e. made concrete) through the elaboration of objectives. For each objective, one (or more) expected outcomes must be specified. Expected outcomes need to be "measurable" (empirically tractable) inasmuch as they specify the data to be gathered as evidence of one dimension of program effectiveness (think of each objective as a dimension). This same data may indicate where improvement is warranted. Valid quantitative data elicited through stable instruments enables a view of how well improvements are working as well as trends over time.

The following is a general typology of objectives for baccalaureate programs in CSIS:

Categories of Objectives for Students in CSIS Baccalaureate Programs:

i) Liberal Arts objectives (fulfilled by courses satisfying the University Core Curriculum)

ii) Soft skills beyond the Core (e.g. collaborative skills)

iii) Cultural and Ethical sensitivities beyond the Core

iv) Technical and professional abilities specific to the program (as well as theoretical learning)

v) Career placement and/or graduate school goals for graduates and alumni

vi) Extracurricular opportunities and distinctive advantages extended to students

All baccalaureate programs will probably be similar relative to the first three sets of objectives (and assessment activities). The fourth set of objectives will be program specific, derived from the program's curriculum. Perhaps this fourth set could be broken into core abilities and advanced abilities. For the fifth, the same assessment tools and procedures should apply across all baccalaureate programs.

29

To exemplify, below is a prospective mission statement and set of program objectives and expected outcomes for the B.S. in computer science. Notice how the objectives together with the outcomes convey a detailed, concrete view of what students will be learning. Notice also how the expected outcomes, in stating how masteries will be measured, furnish a description of the instruments that need to be developed. The "realization" statement could have been titled "realized through" or "realized by way of." It describes how the objective is to be accomplished, providing additional information about the objective.

The following is for illustration only. The expected outcomes associated with objectives 5, 6, and 7 are properly framed, although hypothetical. The expected outcomes associated with objectives 9, 10, and 11 are insufficiently descriptive. Acceptable expected outcomes are much like a list of topics (or possible questions) students are told to study for an exam.

Mission of the B.S. in Computer Science

The B.S. in computer science provides a college education steeped in the liberal arts with specialized emphasis in the theory and practice of software development. Students graduate with the soft and the technical abilities to begin careers as software professionals and remain effective as technology advances. Internships are advocated. CSIS maintains an environment that encourages individual accomplishment and the cohesiveness of the student body.

Objective 1: Students will receive a broad general education as specified by the Pace University Core Curriculum. This includes the development of written and oral presentation abilities (see Objective 2 also) as well as courses in literature, the fine arts, the social sciences, and foreign cultures and languages.

Realization: All students complete the University's Core Curriculum.

Expected Outcomes: These are enumerated by the Dyson College of Arts and Sciences and assessed by Dyson (possibly with the Academic Profile, the general education assessment published by the Educational Testing Service).

30

Objective 2: Students will develop oral and written communication skills.

Realization: All students will have made an oral presentation of an algorithm (in CS242, "Data Structures and Algorithms II"), an oral and written presentation on a socially related issue (in CS312, "Research Methods in Computers and Society"), written concise pieces of technical documentation (in many courses) as well as an extended piece of documentation (in CS389, "Software Engineering"), and written explanations of processes and concepts (in many courses).

Expected Outcomes: Part of the summative assessment will be a test of communications skills. All students will earn an acceptable score on an essay graded independently by two faculty members on an explicit set of criteria. All students will perform acceptably on an oral presentation that will be similarly graded. Also, course portfolios will contain samples of students' writing to exemplify the development of written skills.

Objective 3: Students will develop collaborative skills.

Realization: All students will work in small, self-directed, goal-directed groups. A group software development project is centerpiece to CS389, "Software Engineering." Expected Outcomes: As part of the summative assessment, students will be asked about aspects of their collaborative experience, focusing on problems that arose and how these were managed. Also, hypothetical situations will be presented for resolution. Acceptable responses are required.

Objective 4: Students will understand and be able to discuss issues of social salience and will be equipped to perform as ethical professionals.

Realization: The ACM and IEEE codes of ethics are examined and illustrated in CS312, "Research Methods in Computers and Society," which all students are required to complete. Social issues of a legal and ethical nature are discussed in all courses as they arise tangentially to content or through current events.

Expected Outcomes: As part of the summative assessment, students will satisfactorily elaborate on the differences between the law and ethics, answer questions about ethics, and write an analysis of a social issue.

31

Objective 5: Students will acquire an appropriate foundation in quantitative reasoning and skills.

Realization: All students will successfully complete two semesters of differential and integral calculus (MAT131 and MAT132), two courses in discrete mathematics (MAT137 and MAT256), and a course in probability and statistics (MAT234).

Expected Outcomes: Parts of both the formative and the summative assessments will require that students successfully apply formal understandings to account for aspects of digital computation, the properties and performance of data structures and algorithms, and for the solution of quantitative problems.

Competencies include the ability to use, to explain or to demonstrate such things as (these are only samples!):

why a bit string of length n can hold 2n distinct messages

why fractional values concisely represented in base 10, but which are not sums of integral powers of ½, cannot be expressed as bit strings of a finite length binary (and what this implies relative to tests of equality)

how to compute a numerical solution to a differential equation with a recursive expression derived from a difference equation

how to evaluate expressions in postfix notation, the ability to convert infix expressions to postfix, the ability to explain what makes postfix useful

that the height of a full m-ary tree with n nodes is on the order of logmn and the ability to describe what makes this useful

what is meant by exponential growth and by a "combinatoric explosion"

how pseudo-random generators work, why they cycle, and how to detect when cycling begins

why the sum of pairs of values from the standard uniform distribution is not uniformly distributed

how to transform a sequence of values from the standard uniform distribution into discrete and continuous uniform distributions with various ranges

why inferential statistics are needed to derive reliable results from a discrete system simulation (along with such principles as why sample size needs to increase with an increase in a variable's standard deviation)

how to perform proofs of program correctness

32

Objective 6: Students will develop proficiency in object-oriented programming techniques and strength in designing solutions to programming problems.

Realization: All students will complete two semesters of programming and problem-solving with Java (CS121, "Computer Programming I" and CS122, "Computer Programming II"), followed by additional courses with a heavy component of programming (CS241, "Data Structures and Algorithms I"; CS242, "Data Structures and Algorithms II"; and many others).

Expected Outcomes: Parts of both the formative and the summative assessments will be on programming skills. Students will satisfactorily demonstrate the use of derivation and polymorphism to achieve extensibility, the use of interfaces to implement callbacks (and listeners), and the use of facilities in java.util for sorting and for implementing a variety of abstract data types (e.g. a priority queue with a LinkedList). Students will also satisfactorily demonstrate familiarity with the idea of a pattern and an ability to implement Adapters and Iterators. Course portfolios will further document these abilities as well as problem-solving.

33

Objective 7: Students will develop an understanding of the foundational principles of computer science.

Realization: Through the courses in the CS core and through required advanced courses, all students will --

learn about the organization and architecture of computers and data communications networks

have experience with stacks, queues, trees, and graphs as abstract data types as well as their implementations and applied uses

be familiar with the concept of an algorithm, time and space complexity, and the general classification of algorithms

Expected Outcomes: A major portion of the formative assessment will require satisfactory performance on items pertaining to data structures and algorithms. Items will focus on the algorithms central to computer science, including --

the bubble, selection, insertion, heap, quick, merge, and radix sorts along with properties involving stability, natural versus unnatural behavior, time complexity, space complexity, and the physical versus the logical re-permutation of records

the binary search; inserting, searching, and deleting in binary search trees; B-trees; and hashing

traversing graphs, the topological sort, finding minimal spanning trees, computing reachability, finding strong components, Dijkstra's shortest path algorithm, the traveling salesperson's problem, and NP-completeness

Course portfolios will also document student mastery.

Objective 8: Students will develop an understanding of the deeper principles of computer science.

Realization: All students will complete CS361, "Programming Languages and Implementation," and CS371, "Operating Systems and Architecture I"

Expected Outcomes: All students will perform satisfactorily on items on the summative assessment pertaining to non-imperative programming languages, the structure of compilers, the structure of operating systems, and concurrent processing (including race conditions, deadlocks, and knowledge of the classical problems in concurrency). Course portfolios will also document student achievement.

34

Objective 9: Students will acquire an acquaintance with the conceptual foundation, current practices, and technology associated with data communications.

Realization: All students will complete CS388, "Data Communications."

Expected Outcomes: Pertinent items will appear on the summative assessment, and all students will perform satisfactorily. Course portfolios from CS388 will also document student achievement.

Objective 10: Students will acquire an understanding of the principles of software engineering and experience in applying contemporary best practices in software design, construction, and maintenance.

Realization: All students will complete CS389, "Software Engineering."

Expected Outcomes: Pertinent items will appear on the summative assessment, and all students will perform satisfactorily. Course portfolios from CS389 will also document student achievement.

Objective 11: All students will develop a familiarity with two specializations.

Realization: Students are required to choose two advanced electives, (e.g. CS383, "Computer Graphics"; CS385, "Artificial Intelligence"; CS387, "Database Design"; or one of the Special Topics offerings).

Expected Outcomes: Items on each advanced elective will be compartmentalized on the summative assessment. Students will select appropriate compartments for completion and perform satisfactorily. Course portfolios will also document achievement in the advanced electives.

35

Objective 12: Students will be acquainted with the needs for information assurance and familiarized with the measures to promote secure computing.

Realization: Security is treated as it applies in each course.

Beginning level classes explain security as a composite of a system's availability, integrity, and confidentiality. A password is put into the perspective of the typology of authenticating a user's identity based on knowledge, possessions, or personal attributes (biometrics). Best practices for safe computing on a personal computer are emphasized.

In CS122, "Computer Programming II," a Vigenere cipher is assigned as a programming project, supplemented with a general discussion of substitution and transposition ciphers.

In CS242, "Data Structures and Algorithms II," student presentations cover industrial strength secret key cryptosystems (e.g. triple DES) and asymmetric cryptosystems (e.g. the RSA public key cryptosystem) as well as Java's tools for implementing triple DES and RSA, message authentication codes (MACs), and message digests.

In CS383, "Computer Graphics," steganography is illustrated.

In CS387, "Database Design," the use of MACs for guaranteeing the authenticity and integrity of fields is presented.

In CS388, Data Communications, public key cryptosystems, certificates, and digital signatures are studied as are techniques for building firewalls.

Expected Outcomes: Parts of both the formative and the summative assessments will be on aspects of information assurance. Students will perform acceptably. Course portfolios will also document pertinent achievement.

Objective 13: All students will have the opportunity to preview the computing profession.

Realization: Industrial internships are available through the Office of Career Planning and Cooperative Education.

Expected Outcome: Over half the graduates will have experienced a career-related internship in a corporate or other organizational environment.

36

Objective 14: Graduates will be prepared for careers as computing professionals and/or graduate school.

Expected Outcome: Data will confirm that those in each graduating class will have either secured positions commensurate, with their education, from which professional advancement is expected or will have directly entered graduate school.

Objective 15: Alumni, over time, will build successful careers supported by ongoing learning and effective response to change.

Expected Outcomes: Surveys will confirm advancement in organizational hierarchies and participation in professional and civic activities at increasing levels of responsibility. Surveys will also confirm that alumni have acquired advanced training and education, professional licensures, read professional publications, and participate in professional activities.

Objective 16: To offer a supportive climate for learning both instrumentally and interpersonally.

Realization: A CSIS computer lab is available for use on a 24x7 basis (along with a microwave oven and congenial facilities for discussing programs, projects and holding group meetings). Programming consulting is available for free at scheduled times. A computer club promotes extra-curricular and co-curricular activities. CSIS hosts events at the beginning of each semester for students and faculty the meet and mix. Classes are small, so there is a high degree of student-faculty interaction.

Expected Outcomes: Various indicators on surveys will show student morale to be high; for instance, that CSIS is regarded as caring and nurturing, that faculty is regarded as respectful, available, and "on the students' side," and that students have no difficulty finding faculty members to write letters of reference. Retention in CS is high among students who have completed CS122, "Computer Programming II," and determined that computer science as a major suits their abilities and interests. (Students for whom computer science is not the right major tend to select another major in CSIS.)

37

Appendix Four

Template for Reporting upon Improvements Originating from Assessments

ProgramObjective

Expected Outcome

How Outcome Was Measured

When Measured Improvement Identified

Improvement Implemented

The report will include augmenting discussion as needed for completeness and clarity.

38

Appendix Five

Course Portfolios: Keith Barker's Views on Inclusions

Keith Barker, Ph.D., is Professor of Computer Science and Engineering, Associate Vice Provost, and Director of the Institute for Teaching and Learning at the University of Connecticut.

The course portfolio should be either in hardcopy or online, and it should provide the following:

Course name and number, number of credits

Course objectives, pedagogical approach, assessment methods (and how these relate to the program objectives, if appropriate)

Instructor's name, meeting times, textbook, syllabus

Assignments, projects, and exams as well as writing, projects, teamwork, ethics

Any feedback mechanism/examples to students that may be online

any substantive electronically posted communication, threaded discussion

course evaluations (measures of success may include the results of student surveys, formative or summative assessments, or the achievements of

students in concurrent or subsequent courses)

proposed and/or implemented changes as a result of formative, summative, or other assessments

For wholly online courses (or a complete degree program), the results of an electronic CAC survey to that group of students regarding their experiences in the program, comparative to the usual on-site class visit, should be made available to the visiting team. If a course is taught wholly online by a non-resident faculty member, then data about that faculty member must be included in the self study or provided in separate documents for credentialing purposes.

Location of URLs on a CAC-visit Website (or list of URLs) that would allow an evaluator to retrieve the data described above should be provided for data not supplied in the form of hardcopy.

It would be helpful to the visiting team if all assessment documentation, including the display materials, were available in the same location.

39

Appendix Six

A Rubric for Evaluating Prose Written by Students

The following is Pace University's Grading Standards for written work, by Linda Anstendig, Professor of English and Communications in Pleasantville, and Bette Kirschstein, Assistant Professor of Literature and Communications in Pleasantville. The rubric's description and a check-sheet are supplied.

Pace University Grading Standards for Written Work

The purpose of the following, from the English Department, is to bring objectivity to the determination of grades for pieces of critical writing. This rubric, or one like it, is needed for evaluating student writing on assessments.

A . An "A" paper presents interesting, insightful ideas. There is a clear focus (thesis, controlling idea) which is developed in an organized, concise, logical manner. Unified and coherent paragraphs include specific, relevant supporting evidence and examples. sentences are varied and well constructed. Word choice is fresh, precise, and vivid. There are virtually no errors in grammar, punctuation, spelling, or usage.

B. A "B" paper demonstrates a thoughtful, solid understanding of the subject. Although ideas are interesting, they tend to lack originality or insight. Focus is clear and content well-organized, but paragraphs may be slightly underdeveloped or need more support. Most sentences are varied and well constructed. Word choice is generally appropriate. Although there may be some minor errors in grammar, punctuation, spelling, or usage, none of these problems is glaring or highly distracting.

C. A "C" paper is in average paper, presenting ideas that may be obvious or unexceptional. Part of the essay may be unclear and information general or repetitious. The essay is somewhat developed and organized. Paragraph breaks may not always correspond to shift in topic. The sentence structure can be repetitive or awkward, and word choice imprecise or inappropriate. Errors in grammar, punctuation, spelling or usage may distract the reader, but do not prevent comprehension.

D. A "D" paper tends to lack insight and interesting ideas. Focus is often confusing or not easily identified. The essay is usually undeveloped and poorly organized. Paragraphs breaks can be arbitrary. Statements are unsupported, repetitive, or irrelevant. Sentence structure and word choice may be inaccurate, confusing, or awkward. There are many grammar, punctuation, spelling, and usage errors.

F. An "F" paper presents simplistic, inappropriate, or incoherent ideas, and lacks focus. It tends to be undeveloped and disorganized. Paragraphs are incoherent, and paragraph breaks often do not correspond to shifts in topic. Statements are unsupported, repetitive, or irrelevant. Sentence structure and word choice are inaccurate, confusing, or awkward. There are many grammar, punctuation, spelling, and usage errors that also prevent comprehension.

40

Evaluation Form for Written Work

A "check" means that your work corresponds to the description next to it.

A Excellent Work:_____Insightful ideas_____Clear thesis (main idea) which is developed in an

organized, concise, logical manner_____Unified and coherent paragraphs with relevant

supporting evidence and examples_____Sentences are varied and well constructed._____Word choice is fresh, precise, and vivid._____Almost no errors in grammar, punctuation, spelling,

or usage.

B Good Work:_____Interesting ideas. _____Clear focus_____Well-developed paragraphs_____Varied sentences._____Word choice is generally appropriate. _____Correct grammar, punctuation and spelling

C Adequate or average work:_____Ideas lack insight or are too obvious_____Focus is unclear or information is too general

or repetitious_____Poor organization of paragraphs and undeveloped idea_____Awkward sentence structure_____Imprecise word choice_____Grammar, punctuation and spelling errors

D Poor work:_____Ideas lack insight_____Confusing focus, unidentifiable thesis_____Poor organization of paragraphs, ideas are unsupported,

repetitive or irrelevant_____Awkward sentence structure_____Many grammar, punctuation and spelling errors

F An "F" paper presents simplistic, inappropriate, or incoherent ideas, and lacks focus. It tends to be undeveloped and disorganized. Paragraphs are incoherent, and paragraph breaks often do not correspond to shifts in topic. Statements are unsupported, repetitive, or irrelevant. Sentence structure and word choice are inaccurate, confusing, or awkward. There are many grammar, punctuation, spelling,

41

and usage errors that also prevent comprehension.

Grade:

Comments:

42

Appendix Seven

"Assessment Plan Procedures"(Pages 10-11 from the Pace University Assessment Plan for Continuous Improvement, May 1, 2004)

1. The Provost’s Office will be responsible for the general oversight of assessment. This oversight will include those Offices reporting directly to the President on behalf of the President (e.g. DoIT, Human Resources, etc.).

2. The Office of Planning, Assessment, Research and Academic Budgeting (OPARAB) will provide guidance and support for assessment activities across the University. It will provide funding assistance, when possible, for planning and implementing assessment and will be a repository of information and resources on assessment.

3. OPARAB will also serve as a repository for outcomes studies and reports.

4. In the submission of assessment findings and outcomes reports, every effort will be made to protect the privacy of students, faculty, and staff in the reporting and use of assessment data.

5. OPARAB will organize events for the dissemination of information, e.g., NSSE results, and at which Schools, Departments, Divisions, Units and other members of the University community will report on their assessment efforts.

6. The Provost’s Office will require that beginning on September 1, 2004 and continuing through December 31, 2004, all University units will submit an initial written assessment plan for their area. Plans within the schools and academic departments will focus on degree programs and on major departmental activities. Plans within academic support and administrative units will focus on student learning, service and university functions as appropriate. Plans will include:

a. The name or names of those responsible for assessment within each Unit, School, Department or Program.

b. A mission statement for each Unit, School, Department or Program.

c. General goals for each Unit, School, Department or Program.

43

d. Specific objectives and related measurement tools1 for the implementation of assessment that will be available for review on request. Objectives and measurements will focus on student learning outcomes, student feedback, and on various aspects of the Pace environment and Pace University operations as appropriate.

e. An appropriate timeframe for measuring outcomes, e.g., annually; on a 5-year cycle; at the beginning, middle, and end of a baccalaureate program; in the junior year; five years after graduation.

f. As assessment activities are implemented, Deans and Unit Heads will provide data, analysis, action plans, and the results of those plans related to assessment activities.

g. Assessment will include graduates’ experiences after completion of degree programs, i.e., career progress and reflections on what in their Pace experience was helpful and what needs to be improved.

7. The Provost’s Office will provide written feedback on each assessment plan and its implementation either directly or through other competent reviewers selected in collaboration with the Unit, School, Department or Program.

8. The Provost’s Office will initiate and support an assessment of the University-wide Assessment Plan and will act on its findings.

In setting forth this Assessment Plan, the Committee bears in mind the unique needs and goals of each academic and administrative unit and encourages those units to develop and implement assessment activities that best serve their mission and goals within the University’s Mission and Strategic Plan. The Committee is also sensitive to the dynamic nature of assessment planning, which must respond rapidly to changes in intended outcomes or objectives and to lessons learned from the actual use of various methods of assessment. As a consequence, the assessment plan should be seen as a dynamic document that will change with the needs of the University and its stakeholders.

______________________1 Measurements of learning outcomes could include pencil and paper tests, portfolios that present the products of learning, rubrics, case analyses, work products from problem-centered learning activities, nationally recognized examinations, presentations and other performances that are evaluated by faculty and other experts. Measurements of major activities that support learning and contribute to the Pace experience could include opinions surveys, narrative descriptions, results of focus groups, and post graduation employment data.

44

Documents

July 19, 2004 - Pace University in New York | PACE … · Web viewWriting Program Mission Statements, Objectives, and Expected Outcomes 29 (with an Example for a Hypothetical Bachelor