The Changing World of Assessment and Placement Craig Rutan, CAI Steering Committee Vice Chair Jennifer Coleman, CAI Statewide Program Director 2015 ASCCC Fall Plenary Session CAI Overarching Goal Develop a comprehensive, common assessment system that: aligns to state legislation reduces unnecessary remediation provides statewide efficiencies effectively supports faculty and staff to ensure accurate student placement, resulting in more successful student outcomes Fall Plenary Session November 5 - 7, 2015 Irvine, CA2 Key Objectives A test that covers curricular areas of Math English English as a second language (ESL) Adaptive at the Testlet level Multiple (Supplemental) Measures Assessment Preparation Accessibility as a primary consideration Fall Plenary Session November 5 - 7, 2015 Irvine, CA3 Assessment, Not Placement Placement is locally determined Raw data is portable No traditional cut scores Map of Student Competencies Validation steps Statewide Local Fall Plenary Session November 5 - 7, 2015 Irvine, CA 4 Current CAI Work Groups Math (includes above college-ready) English (includes Reading) ESL (includes Noncredit) Multiple Measures Professional Development Test Development Process Platform (User Interface, Reporting) CAI is ALWAYS looking for additional faculty. If you are interested in participating, please complete an Application for Statewide Service! Fall Plenary Session November 5 - 7, 2015 Irvine, CA5 Single Test for English Language Arts? When given a choice between taking an English assessment or an English as a Second Language (ESL) one, most students will choose to take the English test. Could there be a single English Language Arts test that directs students to English or ESL questions based upon their answers to certain questions? What types of questions would help destinguish English and ESL students? Fall Plenary Session November 5 - 7, 2015 Irvine, CA6 Writing Sample Resolution F14 included the following: Resolved, That the Academic Senate for California Community Colleges recommend that the Common Assessment Initiative include writing samples as a required component of the common assessment and that the writing samples are scored by human readers whose participation will inform assessment procedures that promote the growth of students across the composition sequence; and Resolved, That the Academic Senate for California Community Colleges urge the Common Assessment Initiative steering committee to ensure that English and ESL instructors with knowledge and experience as to how integrated assessment programs inform curriculum and pedagogy participate in the design and evaluation of writing samples to ensure that the assessment test is grounded in the latest research on language learning and assessment practices. Fall Plenary Session November 5 - 7, 2015 Irvine, CA7 Writing Sample (2) The writing sample will be a part of the CAI and will serve both English and ESL students The sample can be human scored, but the CAI system will also include a machine scoring algorithm. RFP for this machine scoring system will be out soon The AI must be able to learn from writing samples that have been hand scored Colleges will not be required to use the machine score The writing sample will be provided to colleges Should human scoring results move with the student? Fall Plenary Session November 5 - 7, 2015 Irvine, CA8 Multiple Measures Overlap with Multiple Measures Assessment Project (MMAP) Currently convening Pilot college overlap Non-cognitive measures Research to come What should be offered? Use of multiple measures for placement is required. No single placement tool (assessment test or single multiple measure) is sufficient. A multiple measure isnt multiple if you choose to not use the assessment test and then use a single alternative placement tool! Fall Plenary Session November 5 - 7, 2015 Irvine, CA9 CAI Pilot Colleges Bakersfield CollegeButte College Chaffey CollegeDeAnza College Delta CollegeDiablo Valley College Fresno City CollegeRio Hondo College Sacramento City CollegeSaddleback College Santa Monica CollegeWest Los Angeles College Fall Plenary Session November 5 - 7, 2015 Irvine, CA10 Pilot Phase Two components Test Data collection and Validation Technology/Platform Student information system interface Professional Development Iterative process Not a static solution Fall Plenary Session November 5 - 7, 2015 Irvine, CA11 Project Timeline Fall 2015 Pilot Phase Begins (item quality, NCVs) Spring 2016 Field testing and test validation Fall 2016 Release and Implementation Ongoing feedback and development Success! Fall Plenary Session November 5 - 7, 2015 Irvine, CA12 Launch and Implementation Pilot Colleges then Phased Release Sister colleges Compass users External factors for phased implementation Existing contracts College resources Ongoing support, continued validation Fall Plenary Session November 5 - 7, 2015 Irvine, CA13 Local Implementation The implementation of the Common Assessment at your campus will require the collaboration with All Faculty Administors IT Staff Admissions and Records Assessment Center Staff Research Fall Plenary Session November 5 - 7, 2015 Irvine, CA14 Faculty Role in Implementation Discipline Faculty need to look at the competency maps and determine how they align to existing curriculum The conversations should involve discipline faculty outside of math, English, and ESL because placement decisions affect the entire campus Couselors must be involved in discussion to have accurate information for students Your Academic Senate should take the lead in coordinating the implementation efforts! You should start this work now, if you havent done so already. Fall Plenary Session November 5 - 7, 2015 Irvine, CA15 Information for Instructional Faculty The new assessment test will produce a diagnostic report for each student. There could be class level reports that would give faculty information about the overall class skills before instruction begins. What type of curriculuar changes would be possible with this information? Fall Plenary Session November 5 - 7, 2015 Irvine, CA16 Consderations for Counselors Counselors need report access, but they also need to know how to interpret the results to advise students Counselors will be working with the students using the student reports What informaion should be shared with the student and what information should be kept out of the student report? Fall Plenary Session November 5 - 7, 2015 Irvine, CA17 Unanswered Questions There are several additional questions that the CAI Steeting Committee is looking at: What does it mean for the assessment to be portable? How long should the assessment results be good for? How often can students retake the assessment test? Should students be prevented from retaking the assessment test if they have already enrolled in a course in that area? If you feel strongly that the ASCCC needs a position on any of these, resolutions are due by 5 PM this afternoon! Fall Plenary Session November 5 - 7, 2015 Irvine, CA18 Fall Plenary Session November 5 - 7, 2015 Irvine, CA19 QUESTIONS?