CFAC MeetingJuly 24, 2014

Welcome!

• Breakfast

• Welcome/New Member Introductions

• Grant Update/Item Bank Update

• Schedule next CFAC meeting

• Project Needs, Current item writers

• Accessing items, planning for assessment development

• Survey: Clicker Use

• Break/Lunch

Agenda/Topics to Be Covered

• Schedule next CFAC meeting

Calendars:

• Clarification of Eduphoria Use

• 1. If we release items with incorrect standards, what implications will that have for districts?

• The items will be linked to the old standards. Therefore, any tests built now will have the old standards. It will depend on how a district’s assessment platform handles updating standards on item bank items and any items included on a test. Please consult your vendor regarding how tests and items will be updated when new standards are made available. 2. Will they be able to be re-exported later, with the correct standards, to replace the items exported now?

• Yes, but the updated items will have the same ID number so it will be the responsibility of the receiving assessment platform to do the replacement.

Eduphoria Update

• 2. Once the standards are updated, what modifications will happen for the items in the bank?

• eduphoria! will crosswalk the items to the new standards in the database directly. They will simply appear to be connected to the new standard in the application.

• Will they disappear and be replaced with a copy of the item and the new standard?

• The old item will not disappear, as the standard connection is only an element of meta data for each item. Updating that meta data will only change the standards association.

• Will the standards be deleted and automatically updated?

• While the old standards will still exist in the eduphoria! database along with the new standards, the meta data that associates a standard to an item will be replaced with the new standard connection.

Eduphoria Updates

• 3. What is the time frame from the time the new standards are released to the updates being ready at the item level?

• After eduphoria! receives the CPALMS preK-12 course files (July 25 estimate), they will review the files (2-3 days). Then eduphoria! will provide an estimated date of completion. The CTE & ESE standards are “on hold” until CPALMS gets what is needed from the DOE.To date, eduphoria! has only provided item exports of grant-covered courses to the IBTP/Equella State Platform with direct instruction from CFAC. Any QTI Item Bank distribution will be managed and released by CFAC on a course-by-course basis to participating CFAC districts.

Eduphoria Update

• 1. Access for Districts to upload IBTP items into their own platforms seems a bit inconsistent. Smaller districts have inquired but have not received any response from the state.

• How is the decision to release items to District/classroom tiers being conducted? We are concerned that items from state-assessed courses are being released entirely at the teacher level. This would impose a financial burden on Districts who were counting on using the IBTP for their interims.

• For Districts who are allowed to export items into their own platforms, how is the security of the tired access being upheld? The security agreement Districts must agree to is quite broad and does not seem to protect the tiered access established by the FDOE.

• 8. Can a consortium (NEFEC) have one security agreement instead of individual agreements for each District in the consortium

• 9. Can the IBTP handle export of assessments in addition to export of items?

• 4. Are there plans to roll out a paper-based testing option for year 1?

• 5. We have been told that the current load testing is based on 100,000 simultaneous clicks. Is this realistic?

• 6. What is the timeline for implementation going forward with the IBTP?

• 7. Are we on track (August) for ability for CFAC to share tests and items in the IBTP to participating CFAC districts....these districts would need appropriate permissions to access tests and items.

• What does the item analysis for progress monitoring look like? Does it identify strength and weaknesses by standard for each student? Is it exportable into Excel?

• Is the platform compatible with FOCUS?

Category Sub-category ExampleClassification N/A Content area, item type, course

name     Content Accessibility, Reasoning Clear wording, alignment with

standard, depth of knowledge, cognitive process dimension

     Directions Stimulus, Accessibility Definition of expectations, scoring

rubric     Style & Format

General, Accessibility Grammar/mechanics, readability, format, structure

     Options (Selected Response Only)

General, Clueing Single answer key, distractor plausibility, distractor homogeneity

Research Project: Rubric

Content Area

Number of Tests

Number of

Teachers

Number of

Districts

Number of

Courses

Number of

Items

ELAR 18 18 12 9 294

Math 8 7 3 6 232

Total 26 25 12 15 526

Sample by Content

Content Area

Number of

Tests

Minimum

Number of

Items

Maximum

Number of

Items Mean SDELAR 18 1 60 16.3 13.7

Math 8 5 65 29.0 19.7

Total 26 1 65 20.2 16.5

Number of Test Items by Content Area

Content Area

Number of

Items

Percentage of Items Evaluated

Selected Response Constructed Response MC TF MA SA ER PT GR

ELAR 294 63.6 7.8 4.8 20.4 3.4 0 0

Math 232 69.0 0.0 1.3 11.6 0 1.7 16.4

Total

526 66.0 4.4 3.2 16.5 1.9 0.8 7.2

Item Type by Content Area

Content Area

# of Item

sPercentage of Items Evaluated

RememberUnderstan

d Apply Analyze Evaluate Create

ELAR 294 46.3 37.8 3.7 9.9 0.3 2.0

Math 232 4.7 28.9 61.2 5.2 0 0

Total

526 27.9 33.8 29.1 7.8 0.2 1.1

Cognitive Processes by Content Area

Content Area

# of Items

Percentage of Items Evaluated

Low/Level1Moderate/

Level 2High/

Level 3ELA 294 67.7 25.5 6.8

Math 232 33.2 61.6 5.2

Total 526 52.5 41.4 6.1

Cognitive Complexity by Content Area

Item Type Content Area

Low Complexity/

Level 1 N (%)

Moderate Complexity/Level 2 N(%)

High Complexity/

Level 3 N (%)Multiple Choice

ELAR 124 (66.3%) 60 (32.1%) 3 (1.6%)Math 52 (32.5%) 102 (63.8%) 6 (3.8%)

True/False ELAR 23 (100%) 0 (0%) 0 (0%)Math - - -

Matching ELAR 14 (100%) 0 (0%) 0 (0%)Math 3 (100%) 0 (0%) 0 (0%)

Short Answer

ELAR 34 (56.7%) 13 (21.7%) 13 (21.7%)Math 10 (38.5%) 17 (63.0%) 0 (0%)

Essay/Extended Response

ELAR 4 (40%) 2 (20%) 4 (40%)

Math - - -Performance Task

ELAR - -Math - 4 (100%) -

Gridded Response

ELAR - - -Math 12 (31.6%) 20 (52.6%) 6 (15.8%)

DOK by Item Type & Content Area

Content Area

Aligned to Standard

Not Aligned to Standard

Standard Not Identified by Teacher

Standard Correctly Identified by Teacher

Standard Incorrectly Identified by Teacher

ELAR173 (58.8%)

121 (41.2%)

262 (89.1%) 10 (3.4%) 22 (7.5%)

Math218 (94.0%)

14 (6.0%) 53(22.8%)

165 (71.1%) 14 (6.0%)

Standards Alignment

  ELA Math TotalRubric Category Mean SD Mean SD Mean SDContenta .729 .445 .920 .272 .812 .391Directions/Stimulus/Stemb .814 .389 .903 .296 .852 .356

Style & Formatc .883 .321 .874 .332 .879 .326Options - Generald .831 .375 .831 .375 .848 .359

Options - Clueinge .956 .205 .971 .168 .962 .190

Overall .829 .376 .910 .286 .864 .343

Item Quality Results

• Brainstorm

Questions for FLDOE

• 1:00 FLDOE call

• Eduphoria Update

• CFAC Fiscal Agent Agreement

• ACCESS Assessment Update

• District Discussion/Sharing

Agenda: Afternoon Meeting

• What do you want to share?

• From SDOC: Electives, SharePoint

District Discussion/Sharing

2014-2015 Course Codes: MS Advanced Reading

6th grade M/J Research 1 1700000A1700000B1700000C

7th grade M/J Research 2 1700010A1700010B1700010C

8th grade M/J Research 3 1700020A1700020B1700020C

6th grade M/J Exploratory Wheel 3

0600020A & 0600020B

7th grade M/J Exploratory Wheel 4

0600030A & 0600030B

8th grade M/J Exploratory Wheel 5

0600040A & 0600040B

2014-2015 Course Codes: MS Intensive Math

2014-2015 Course Codes: MS Electives

Teacher Assistants or TA

M/J Exploratory Wheel 1

0600000T

Library Assistants

M/J Exploratory Wheel 1

0600000L

Semester Spanish

M/J Exploratory Wheel 1

0600000S

Semester Dance

M/J Exploratory Wheel 1

0600000D

Semester Theatre

M/J Exploratory Wheel 1

0600000P