Collegiate Learning Assessment 2010-11

Collegiate Learning Assessment 2010-11Montclair State UniversityAssessing Learning Outcomes in General EducationThe assessment of learning outcomes related to general education is essential for continued accreditation by the Middle States Commission on Higher EducationStandard 12 of Middle States Characteristics of Excellence asks institutions to provide evidence that their students demonstrate college-level proficiency in general educationStandard 14 asks institutions to continuously assess learning outcomes, including general education outcomesThe University has agreed to assess learning outcomes in general education through its participation in the Voluntary System of Accountability (VSA)The General Education Committee reviewed available instruments, and determined that the Collegiate Learning Assessment (CLA) was most suitable for the University

The InstrumentThe Collegiate Learning Assessment (CLA) assesses Analytic Reasoning and Evaluation, Writing Effectiveness, Writing Mechanics, and Problem SolvingIt relies on constructed-response tasks rather than on multiple-choice itemsStudents are asked to complete one Performance Task, or a combination of one Make-an-Argument, and one Critique-an-Argument, promptPerformance Tasks include a document library that students use to answer several open-ended questions about a hypothetical, but realistic, situationMake-an-Argument prompts ask students to support or reject a position on some issue, while Critique-an-Argument prompts ask students to evaluate the validity of an argument made by someone else

CLA AdministrationFall 2010100 Freshmen TestedSampling Universe: First-time Freshmen with available SAT scores (N=2,101)Invitations to random sample of students with participation voluntaryIncentive: $75 Red HawkOnline test in a proctored setting in CADASpring 2011100 Seniors TestedSampling Universe: Native Seniors with available SAT scores (N=1,125)Invitations to random sample of students with participation voluntaryIncentive: $75 Red HawkOnline test in a proctored setting in CADARepresentativeness of Test-TakersFreshman and Senior test-takers were generally representative of the entire populationFreshman test-takers had higher SAT scores, higher HS GPA, and lower HS rank, but none of the differences were statistically significantSenior test-takers had lower SAT-M scores, higher SAT-CR and SAT-W scores, and higher college GPA, but only the GPA difference was statistically significantDifferences between test-takers and non-test-takers for both freshman and senior populations were statistically significant for race/ethnicity, but not genderDifferences between test-takers and non-test-takers, by College/School, were statistically significant for seniors, but not for freshmen

Means Comparisons

Freshman Representativeness

Senior Representativeness

CLA: Unadjusted PerformanceFreshman MeansTotal Score: 1073Performance Task: 1033Analytic Writing: 1113Make Argument: 1113Critique Argument: 1110SAT: 1026

Senior MeansTotal Score: 1177Performance Task: 1198Analytic Writing: 1155Make Argument: 1135Critique Argument: 1174SAT: 1013

CLA: Value-Added & EstimatesValue-Added ScoreTotal Score: 0.71Performance Task: 1.26Analytic Writing: -0.11Make Argument: -0.30Critique Argument: 0.14

ExpectedTotal Score: NearPerformance Task: AboveAnalytic Writing: NearMake Argument: NearCritique Argument: Near

CLA: Fr. Sub-Score ComparisonsMSUAllFreshmenFreshmenDifferencePerformance Task:Reasoning2.72.8-0.1Writing Effectiveness2.93.0-0.1Writing Mechanics3.03.1-0.1Problem Solving2.82.9-0.1Make an Argument:Reasoning3.53.20.3Writing Effectiveness3.63.20.4Writing Mechanics3.63.40.2Critique an Argument:Reasoning3.12.80.3Writing Effectiveness3.22.90.3Writing Mechanics3.63.40.2CLA: Sr. Sub-Score ComparisonsMSUAllSeniorsSeniorsDifferencePerformance Task:Reasoning3.63.40.2Writing Effectiveness3.83.50.3Writing Mechanics3.83.50.3Problem Solving3.63.40.2Make an Argument:Reasoning3.63.60.0Writing Effectiveness3.63.7-0.1Writing Mechanics3.83.80.0Critique an Argument:Reasoning3.33.30.0Writing Effectiveness3.53.40.1Writing Mechanics4.03.90.1Performing Beyond Expectations

Next Steps?Faculty analysis of resultsDevelopment of actionable itemsImplementation of proposed enhancementsFollow-up assessment of effects of changes