North Carolina Computer Skills Tests CTE Summer Conference Koury Convention Center Greensboro July 27, 2005 Presenter: Randy Craven Information Technology

North CarolinaNorth CarolinaComputer Skills TestsComputer Skills TestsCTE Summer ConferenceCTE Summer ConferenceKoury Convention CenterKoury Convention CenterGreensboroGreensboroJuly 27, 2005July 27, 2005Presenter: Randy CravenPresenter: Randy CravenInformation Technology Manager,Information Technology Manager,Technical Outreach for Public SchoolsTechnical Outreach for Public SchoolsNC State UniversityNC State University

Session PurposeSession Purpose

This session will provide an overview of This session will provide an overview of the North Carolina Computer Skills the North Carolina Computer Skills Tests to be used during 2005-06 Tests to be used during 2005-06 including:including:

1992 & 1998 Curriculum Tests1992 & 1998 Curriculum Tests2004 Curriculum Online Test2004 Curriculum Online Test2004 Curriculum Alternate Assessment2004 Curriculum Alternate Assessment

CAUTIONCAUTION

This session will contain factual This session will contain factual information concerning an actively information concerning an actively developing process. Statements developing process. Statements made will be representations of made will be representations of current plans for the near future. current plans for the near future. End results may turn out to be End results may turn out to be different.different.

The RequirementThe Requirement

• In May 1991, the North Carolina State Board In May 1991, the North Carolina State Board of Education (SBE) adopted a policy which of Education (SBE) adopted a policy which required that all students, beginning with required that all students, beginning with the class of 2000, demonstrate computer the class of 2000, demonstrate computer proficiency in order to receive a high school proficiency in order to receive a high school diploma (Feature C of the Quality Assurance diploma (Feature C of the Quality Assurance Program). In October of 1995, the SBE Program). In October of 1995, the SBE modified the requirement which made it modified the requirement which made it effective beginning with the graduating effective beginning with the graduating class of 2001.class of 2001.

Curricula - TestsCurricula - Tests

• 1992 Curriculum Tests for Students Who 1992 Curriculum Tests for Students Who Entered Grade 8 From 1996-1997 through Entered Grade 8 From 1996-1997 through 1999-20001999-2000

• 1998 Curriculum Tests f1998 Curriculum Tests for Students Who Entered Grade 8 from 2000–2001 through 2004-2005

• 2004 Curriculum Tests for Students Who Entered Grade 8 from 2005-06 and beyond

One RequirementOne Requirement

• 1992 & 1998 Curricula1992 & 1998 Curricula– Passing two separate tests needed to Passing two separate tests needed to

fulfill computer proficiency requirement fulfill computer proficiency requirement for students entering 8for students entering 8thth grade during grade during school years 1996-97 to 2004-05school years 1996-97 to 2004-05•Multiple-ChoiceMultiple-Choice•PerformancePerformance

• 2004 Curriculum2004 Curriculum– Passing only one test requiredPassing only one test required

What - TestsWhat - Tests

• 1992 Curriculum Multiple Choice Scored Locally1992 Curriculum Multiple Choice Scored Locally• 1992 Curriculum Performance Scored Centrally1992 Curriculum Performance Scored Centrally• 1992 Portfolio Scored Locally1992 Portfolio Scored Locally• 1998 Curriculum Multiple-Choice Scored Locally1998 Curriculum Multiple-Choice Scored Locally• 1998 Curriculum Performance Scored Centrally1998 Curriculum Performance Scored Centrally• 1998 Portfolio Scored Locally1998 Portfolio Scored Locally• 2004 Curriculum Online Combination Scored 2004 Curriculum Online Combination Scored

CentrallyCentrally• 2004 Curriculum Alternate Assessment Field 2004 Curriculum Alternate Assessment Field

Test Scored CentrallyTest Scored Centrally• 2004 Curriculum Alternate Assessment Scored 2004 Curriculum Alternate Assessment Scored

LocallyLocally

When - ScheduleWhen - Schedule

• Now - 2005-06 School YearNow - 2005-06 School Year• 2004 Curriculum Alternate Field Test – ASAP at the 2004 Curriculum Alternate Field Test – ASAP at the

beginning of the school year for 9beginning of the school year for 9thth Graders Graders• 1992 & 1998 Curriculum Tests – Early October - Mid 1992 & 1998 Curriculum Tests – Early October - Mid

NovemberNovember• 2004 Curriculum Online Primary Window – Mid October – 2004 Curriculum Online Primary Window – Mid October –

Mid JanuaryMid January• 2004 Curriculum Alternate – Early November – Mid January2004 Curriculum Alternate – Early November – Mid January• 1992 & 1998 Curriculum Tests – Early February – Mid 1992 & 1998 Curriculum Tests – Early February – Mid

MarchMarch• 2004 Curriculum Standards Analysis – Mid January – Mid 2004 Curriculum Standards Analysis – Mid January – Mid

FebruaryFebruary• 2004 Curriculum Standards Set By SBE2004 Curriculum Standards Set By SBE• 2004 Curriculum Results Reported Early March2004 Curriculum Results Reported Early March• 2004 Curriculum Online & Alternate Secondary Window – 2004 Curriculum Online & Alternate Secondary Window –

Late March – Mid JuneLate March – Mid June

Grades AffectedGrades Affected

• Grade 8Grade 8– Online TestOnline Test– Alternate AssessmentAlternate Assessment

• Grade 9Grade 9– Alternate Assessment Field TestAlternate Assessment Field Test

• Grades 9 – 12 (for those not yet meeting the standard)Grades 9 – 12 (for those not yet meeting the standard)– 1998 Curriculum Tests 1998 Curriculum Tests – 1998 Curriculum Portfolio1998 Curriculum Portfolio

• Students who entered grade 8 between 1996-97 through Students who entered grade 8 between 1996-97 through 1999-2000 (seeking diploma – not yet meeting the 1999-2000 (seeking diploma – not yet meeting the standard)standard)– 1992 & 1998 Curriculum Tests1992 & 1998 Curriculum Tests– 1992 & 1998 Curriculum Portfolio1992 & 1998 Curriculum Portfolio

Administration Times*Administration Times*• 1992 curriculum1992 curriculum

– Multiple Choice – 105 minutesMultiple Choice – 105 minutes– Performance – 90 minutesPerformance – 90 minutes

• 1998 curriculum1998 curriculum– Multiple Choice – 110 minutesMultiple Choice – 110 minutes– Performance – 133 minutesPerformance – 133 minutes

• 2004 curriculum2004 curriculum– One Online Test Estimated 120 minutesOne Online Test Estimated 120 minutes– One Alternate – Time not known currentlyOne Alternate – Time not known currently

*Note: Administration times do not include distribution of *Note: Administration times do not include distribution of material, printing and organizing of student printouts material, printing and organizing of student printouts [performance], packaging, shipping, and other logistical [performance], packaging, shipping, and other logistical activitiesactivities

More About…More About…

• 1992 & 1998 Curricula Tests1992 & 1998 Curricula Tests

• 2004 Curriculum Online Test2004 Curriculum Online Test

• 2004 Curriculum Alternate Test2004 Curriculum Alternate Test

• NCDeskNCDesk

• NCRegistrationNCRegistration

• 2004 Curriculum Test Development Histor2004 Curriculum Test Development Historyy

• Other ItemsOther Items

Menu 1992 & 1998 Test Details1992 & 1998 Test Details

• Pencil-Paper-Computer-Printer processesPencil-Paper-Computer-Printer processes

• 2 Separate Tests with different standards 2 Separate Tests with different standards to be passedto be passed

• Multiple-Choice Scored LocallyMultiple-Choice Scored Locally

• Performance Scored CentrallyPerformance Scored Centrally

• Must be given once per year to students Must be given once per year to students who have not yet met the standardwho have not yet met the standard

Menu Old Paper TrailOld Paper Trail

• Printing of test materialsPrinting of test materials– Test bookletsTest booklets– Answer documentsAnswer documents– Header sheets and shipping listsHeader sheets and shipping lists

• Excess Ordering (~100% overage)Excess Ordering (~100% overage)

• Printing of student work for Printing of student work for performanceperformance

• Handling of materialsHandling of materials

Menu Test Files SupportTest Files Support

•Currently supporting 23 different Currently supporting 23 different software packages for software packages for performance testperformance test

•Distribution of test files to fieldDistribution of test files to field•Equity of administration Equity of administration

questionablequestionable•Online Test delivers required files Online Test delivers required files

to students as test is in progressto students as test is in progress

Menu Scoring PerformanceScoring Performance

• Currently performance test booklets Currently performance test booklets are hand scored by a scoring are hand scored by a scoring contractor at a central locationcontractor at a central location– TimeTime

•Fall administration – approximately 2 monthsFall administration – approximately 2 months•Spring administration – approximately 1 Spring administration – approximately 1

monthmonth– CostsCosts

•HighHigh•Scorers, staffing, spacing, suppliesScorers, staffing, spacing, supplies

Menu Scoring Performance (cont.)Scoring Performance (cont.)

• Scores are reliable and validScores are reliable and valid• Inter-rater reliability is high [93% +]Inter-rater reliability is high [93% +]• Reliability monitoring, qualified scorersReliability monitoring, qualified scorers

But…But…

• Potential for human error still existsPotential for human error still exists• Student work does not always provide Student work does not always provide

evidence that student used correct evidence that student used correct methods to accomplish taskmethods to accomplish task

Menu 1992 & 1998 Test Details1992 & 1998 Test Details

• End of SectionEnd of Section



• NCDeskNCDesk


• 2004 Curriculum Test Development History2004 Curriculum Test Development History


Menu

2004 Online Test Details2004 Online Test Details

• Tests the adopted 2004 curriculumTests the adopted 2004 curriculum

• Uses NCDesk as the client interfaceUses NCDesk as the client interface

• Delivered through the Internet from Delivered through the Internet from central testing servercentral testing server

Menu

Goals of the Online TestGoals of the Online Test

• Merge two tests into one…Merge two tests into one…• Reduce the administration, testing, scoring, and Reduce the administration, testing, scoring, and

logistical time required…logistical time required…• Provide a universal delivery to increase equity for a Provide a universal delivery to increase equity for a

“standardized” test…“standardized” test…• Eliminate costs of printing and reduce paper waste…Eliminate costs of printing and reduce paper waste…• Decrease handling so much paper…Decrease handling so much paper…• Maximize reliability and validity of scores…Maximize reliability and validity of scores…• Return scores more efficiently and timely…Return scores more efficiently and timely…• Keep pace with the changing face of technology, Keep pace with the changing face of technology,

testing, and scoring…testing, and scoring…

Menu

The Online TestThe Online Test

• 72 items total72 items total

• 4 sections4 sections

• 18 items per section18 items per section

• Items not delivered randomlyItems not delivered randomly

• Not divided into strand specific sectionsNot divided into strand specific sections

• Not divided into performance and Not divided into performance and multiple-choice sectionsmultiple-choice sections

Menu

The Online TestThe Online Test

• Different forms automatically Different forms automatically delivered to each studentdelivered to each student

• Performance vs. Multiple-ChoicePerformance vs. Multiple-Choice– 50% performance based50% performance based– 50% multiple-choice based50% multiple-choice based

• Embedded field test items for future Embedded field test items for future test developmenttest development

Menu

NCDesk ClientNCDesk Client

• NCDesk Integrated Java Applications SuiteNCDesk Integrated Java Applications Suite– Delivery of multiple-choice itemsDelivery of multiple-choice items– Delivery of performance items within testing Delivery of performance items within testing

environmentenvironment– Includes:Includes:

• editor/word processing applicationeditor/word processing application

• database applicationdatabase application

• spreadsheet applicationspreadsheet application

• e-mail composer applicatione-mail composer application

• window management applicationwindow management application

Menu

Online Test EnvironmentOnline Test Environment

• SecureSecure– EncryptedEncrypted– Save function disabled in NCDesk during testSave function disabled in NCDesk during test

• Self-containedSelf-contained– No “surfing” of Internet within NCDeskNo “surfing” of Internet within NCDesk– No “cut, copy, paste” functionality outside of testNo “cut, copy, paste” functionality outside of test

• Data WarehousingData Warehousing– Student responses stored on server when moving Student responses stored on server when moving

between sections and questionsbetween sections and questions– Allows for recovery of test and data if Allows for recovery of test and data if

workstations crash or other technical problems workstations crash or other technical problems are encounteredare encountered

Menu

Online Test ScreenOnline Test Screen

• Restore, flag control and position textRestore, flag control and position text

• Item stemItem stem

• Foils [multiple-choice] or application Foils [multiple-choice] or application [performance][performance]– Separate scroll bars for eachSeparate scroll bars for each

• Navigation buttonsNavigation buttons

• Pause buttonPause button

Menu

Online Test NavigationOnline Test Navigation

• Section number identified on each Section number identified on each item page [i.e., Section 1]item page [i.e., Section 1]

• Item number within section identified Item number within section identified on each item page [i.e., Question 2 on each item page [i.e., Question 2 of 18]of 18]

Menu

Online Test ControlsOnline Test Controls

• RestoreRestore– Clears item of changes and restores to original Clears item of changes and restores to original

formatformat

• FlagFlag– Identifies item on navigation bar at section end Identifies item on navigation bar at section end

with a red question mark as an indicator that with a red question mark as an indicator that student may need to re-visit prior to exiting student may need to re-visit prior to exiting sectionsection

– Student Student cancan still exit a section if items are still exit a section if items are flaggedflagged

Menu


• Students can move forward and Students can move forward and backward within a sectionbackward within a section

• Returning to section is Returning to section is notnot permitted permitted once section is completedonce section is completed

• Navigational buttons [PREV Navigational buttons [PREV (previous), NEXT](previous), NEXT]

• Linear movement backwards or Linear movement backwards or forwards within sectionforwards within section

Menu

Online Test FunctionsOnline Test Functions

• End SectionEnd Section– Links to section summary pageLinks to section summary page

• PausePause– Pauses the test at the immediate location Pauses the test at the immediate location

for recovery without exiting test for recovery without exiting test environmentenvironment

– 15 minute timeout – test administrator 15 minute timeout – test administrator will have to log student back into the test will have to log student back into the test if paused longer than 15 minutes.if paused longer than 15 minutes.

Menu


• Section SummarySection Summary

• Navigation bar at end of sectionNavigation bar at end of section

• Non-linear movement to any item in Non-linear movement to any item in sectionsection

• Displays which items have been flaggedDisplays which items have been flagged

• Displays which items have been answeredDisplays which items have been answered

• Moving to next section displays warningMoving to next section displays warning

Menu

Online Test Details EndOnline Test Details End

• End SectionEnd Section



• NCDeskNCDesk




Menu

Online Test Development Online Test Development HistoryHistory

• Feasibility Study/Trial Fall 2003Feasibility Study/Trial Fall 2003

• Feasibility Study/Trial Fall 2004Feasibility Study/Trial Fall 2004

• Spring 2005 Field TestSpring 2005 Field Test

Menu

Feasibility StudiesFeasibility Studies

• Conducted research into feasibility of delivering Conducted research into feasibility of delivering Internet-based test environmentInternet-based test environment

• Fall 2003Fall 2003• Fall 2004Fall 2004

• Conducted research into performance of local and Conducted research into performance of local and central technology during deliverycentral technology during delivery

• Conducted research into overall performance of Conducted research into overall performance of test environment and applicationstest environment and applications

• Conducted research into item performance within Conducted research into item performance within test environment and applicationstest environment and applications

• Received feedback and implement debugging, Received feedback and implement debugging, redevelopment, or new developmentredevelopment, or new development

Menu

Feasibility StudiesFeasibility Studies

• Fall 2003 Fall 2003 •Volunteer sitesVolunteer sites•Adults onlyAdults only•1,926 starts: 1,351 finishes1,926 starts: 1,351 finishes•62 LEAs represented, 193 schools62 LEAs represented, 193 schools

• Fall 2004Fall 2004•At minimum, 10 locally chosen students per At minimum, 10 locally chosen students per

school containing eighth-grade studentsschool containing eighth-grade students•5,620 starts: 4,783 finishes5,620 starts: 4,783 finishes

Menu

Field TestingField Testing

• Spring 2005Spring 2005• Continued research into overall performance Continued research into overall performance

of test environment and applicationsof test environment and applications• Conducted research into item performanceConducted research into item performance• Data used to construct operational form(s)Data used to construct operational form(s)• Receive feedback and implement debugging Receive feedback and implement debugging

or redevelopment where needed. or redevelopment where needed. • Note: implementation of change can only Note: implementation of change can only

occur where not affecting performance of occur where not affecting performance of itemitem

Menu

Spring 2005 Field TestSpring 2005 Field Test

• Sampled population of schools and Sampled population of schools and studentsstudents

• Window: April 11 – June 15Window: April 11 – June 15• 8,510 students chosen for sample8,510 students chosen for sample• 6,361 starts: 6,198 finishes6,361 starts: 6,198 finishes• Alternate Assessment also field testedAlternate Assessment also field tested

– Window: May 9 – June 15Window: May 9 – June 15– 2000 students chosen for sample2000 students chosen for sample

Menu

Online Test Development Online Test Development History EndHistory End





• NCDeskNCDesk



Menu

Computer Skills Computer Skills Alternate AssessmentAlternate Assessment

• Results of feasibility study and Federal Results of feasibility study and Federal mandates required development of an mandates required development of an alternate assessment instrument for alternate assessment instrument for two distinct populations:two distinct populations:

•Students with special needs who could not Students with special needs who could not access the online test using available access the online test using available accommodationsaccommodations

•Students who could not access the online test Students who could not access the online test as a result of technical/technology limitations as a result of technical/technology limitations [i.e., unable to meet minimum requirements [i.e., unable to meet minimum requirements for bandwidth, computer, etc.]for bandwidth, computer, etc.]

Menu

Computer SkillsComputer SkillsAlternate AssessmentAlternate Assessment

• Field tested Spring 2005Field tested Spring 2005– ~2000 students sampled~2000 students sampled– 1338 total documents processed1338 total documents processed– 1323 contained multiple choice data1323 contained multiple choice data– 765 contained performance data765 contained performance data

• Different delivery from online test, but Different delivery from online test, but equal rigor of standard [item difficulty equal rigor of standard [item difficulty level, thinking skills, etc.]level, thinking skills, etc.]

• One test consisting of two distinct sectionsOne test consisting of two distinct sections

Menu


• Multiple-Choice SectionMultiple-Choice Section– 36 items36 items– TraditionalTraditional

• Performance SectionPerformance Section– 27 total items27 total items

• 26 performance-based, administrator rated [yes or no] items26 performance-based, administrator rated [yes or no] items• 1 administrator rated [yes or no] item evaluating student proficiency 1 administrator rated [yes or no] item evaluating student proficiency

with computer over course of timewith computer over course of time– Computer-basedComputer-based– Individualized administrationIndividualized administration– Use supplied files and local applications [i.e., word Use supplied files and local applications [i.e., word

processing, database, etc.] to complete tasks required by processing, database, etc.] to complete tasks required by itemsitems

– Files provided in text format for conversion into local Files provided in text format for conversion into local applications [PDFs provided to serve as blueprints]applications [PDFs provided to serve as blueprints]

Menu


• Item performance, results, and Item performance, results, and feedback being analyzed at this timefeedback being analyzed at this time

• Highly likely to be field tested again Highly likely to be field tested again in early Fall 2005 to 9in early Fall 2005 to 9thth graders graders

• Test files will be provided for State Test files will be provided for State supported platforms/packages in supported platforms/packages in future administrationsfuture administrations

Menu

Computer SkillsComputer SkillsAlternate Assessment EndAlternate Assessment End




• NCDeskNCDesk




NCDeskNCDesk

• Center of the UniverseCenter of the Universe– TestTest

• Access test at Log in page [School code, User name, Password]Access test at Log in page [School code, User name, Password]

– Test SimulationTest Simulation• Practice activity to simulate real test environmentPractice activity to simulate real test environment

– Verify ConnectionVerify Connection• Runs test to verify if secure connection to test server is Runs test to verify if secure connection to test server is

establishedestablished

– DocumentationDocumentation• Links to website for information, updates, etcLinks to website for information, updates, etc

– ApplicationsApplications• Access to all applications integrated in test environment for use Access to all applications integrated in test environment for use

and familiarizationand familiarization

TechnicalTechnical

• NCDesk is a locally installed client Java applicationNCDesk is a locally installed client Java application– Client computers must have Java runtime installedClient computers must have Java runtime installed

• Quality Internet connection required for accessing test Quality Internet connection required for accessing test environmentenvironment– Internet connection not required for NCDesk applications when used for Internet connection not required for NCDesk applications when used for

learning and practicelearning and practice• NCDesk communicates with a central server for testing [not NCDesk communicates with a central server for testing [not

hosted locally]hosted locally]• Auto-update system check for current NCDesk versionAuto-update system check for current NCDesk version• Sufficient RAM recommendedSufficient RAM recommended• CPU of good clock speed and recent vintage recommendedCPU of good clock speed and recent vintage recommended• Minimum amount of drive space availableMinimum amount of drive space available• Sufficient amount of bandwidth required during testingSufficient amount of bandwidth required during testing• Best Resource for technical recommendationsBest Resource for technical recommendations

– http://ncdesk.ncsu.edu/ncdesk/technote.asphttp://ncdesk.ncsu.edu/ncdesk/technote.asp

Technical Notes - Proposed Client Computer RequirementsSpecial Note: Client computer systems running the minimum 128Mb RAM need to reduce the number of background applications running when trying to use NCDesk. Background applications consume memory resources that can become critically low when other applications are running. These types of applications include hidden applications, system inits (Macintosh) and system tray applications (Windows). The following proposed client computer requirements are posted with the assumption that currently active background applications are at a minimum.

SupportedPlatform Minimum Java Runtime

Oldest OS Version

Minimum ProcessorMinimum Free Hard Drive Space

Minimum RAM

No Microsoft Windows 95 NA NA NA NA NA

Yes Microsoft Windows 98 Java 2 Runtime Environment (JRE) Version 1.4.2_061 1st EditionPentium166 MHz

57 Mb 128 Mb

YesMicrosoft Windows Me

Java 2 Runtime Environment (JRE) Version 1.4.2_061 MePentium200 MHz

57 Mb 128 Mb

NoMicrosoft Windows NT

NA NA NA NA NA

YesMicrosoft Windows 2000 Pro

Java 2 Runtime Environment (JRE) Version 1.4.2_061 SP3Pentium233 MHz

57 Mb 256 Mb

YesMicrosoft Windows XP

Java 2 Runtime Environment (JRE) Verstion 1.4.2_061 SP1Pentium200 MHz

70 Mb 256 Mb

NoMacintosh"Classic"

NA NA NA NA NA

YesMacintoshOS X

Java 2 Runtime Environment Version 1.4.1 Update 1Jaguar OS X 10.2.x PowerPC x 26 Mb 128 Mb

YesMacintosh OS X

Java 2 Runtime Environment Version 1.4.2 Update 2 (based on 1.4.2_05 SDK) 2

Panther OS X 10.3.3

PowerPC x 26 Mb 128 Mb

YesMacintoshOS X

Java 2 Runtime EnvironmentVersion TBA

Tiger OS X 10.4

PowerPC x TBA TBA

NCRegistrationNCRegistration

A test registration system A test registration system (NCRegistration) is being developed to (NCRegistration) is being developed to work in conjunction with any of the work in conjunction with any of the online testing programs being online testing programs being developed/offered to public schools in developed/offered to public schools in North Carolina. School systems and North Carolina. School systems and schools will use the system to indicate schools will use the system to indicate students who are eligible for testing students who are eligible for testing within a testing window and schedule within a testing window and schedule test administrations within predefined test administrations within predefined testing sessions for groups of students. testing sessions for groups of students.

NCRegistration FunctionsNCRegistration Functions

• Administrative - User access rights functions. Users are Administrative - User access rights functions. Users are state level administrators, regional level administrators, state level administrators, regional level administrators, local district test coordinators, school test coordinators, test local district test coordinators, school test coordinators, test administrators, and possibly teachers. administrators, and possibly teachers.

• Bulk Registration - Function to allow bulk file uploads of Bulk Registration - Function to allow bulk file uploads of student records to register large groups of students to a student records to register large groups of students to a testing window. testing window.

• Single Registration - Function to allow registration of single Single Registration - Function to allow registration of single student to a testing window. student to a testing window.

• Test Session Scheduling - Indicating numbers of students at Test Session Scheduling - Indicating numbers of students at a school per test administrationa school per test administration

• Student Information Questions (SIQ) -- additional data Student Information Questions (SIQ) -- additional data collection process collection process

• Reports – Test ResultsReports – Test Results

Other ItemsOther Items

• Current ActivitiesCurrent Activities• Accessibility IssuesAccessibility Issues• HelpdeskHelpdesk• Mobile LabsMobile Labs• Future PlansFuture Plans• Necessities for SuccessNecessities for Success• Web Site ReferenceWeb Site Reference• Contact InformationContact Information

Current ActivitiesCurrent Activities

• Analysis of field test dataAnalysis of field test data– Item performance, results, feedback from fieldItem performance, results, feedback from field

• Development and implementation of scoring Development and implementation of scoring parameters for itemsparameters for items

• Analysis of technical issues arising during field Analysis of technical issues arising during field testingtesting– Ongoing debugging and development of technology Ongoing debugging and development of technology

and test environmentand test environment• Creation of operational form(s) based on analysis Creation of operational form(s) based on analysis

of field test dataof field test data• Development of new items [item writing] for Development of new items [item writing] for

embedding in the futureembedding in the future• Additions and testing of NCDesk & NCRegistrationAdditions and testing of NCDesk & NCRegistration

Accessibility IssuesAccessibility Issues

• Definite accessibility issues with online testing!Definite accessibility issues with online testing!• Standard accommodations are still availableStandard accommodations are still available• Choice of large or regular font size for NCDeskChoice of large or regular font size for NCDesk• Keyboard and mouse actions functionalKeyboard and mouse actions functional• Currently developing the ability to integrate Currently developing the ability to integrate

and support assistive technology [i.e., screen and support assistive technology [i.e., screen readers - JAWS]readers - JAWS]

• Exploring multiple options for accessibility Exploring multiple options for accessibility [zoom functions, etc.][zoom functions, etc.]

• Implementation of additional assistive Implementation of additional assistive technology likely an extended processtechnology likely an extended process

HelpdeskHelpdesk

• Activated for feasibility studies/trials, field testing, Activated for feasibility studies/trials, field testing, and will be available for operational administrationand will be available for operational administration

• Assistance provided prior to, during, and after Assistance provided prior to, during, and after testingtesting

• Addresses NCRegistration, NCDesk, Computer Addresses NCRegistration, NCDesk, Computer Skills Alternate Assessment, and any other issues Skills Alternate Assessment, and any other issues involved in delivery and implementation of online involved in delivery and implementation of online testtest

• http://cskills.ncsu.edu/ncdesk/helpdesk.asphttp://cskills.ncsu.edu/ncdesk/helpdesk.asp

• Email Email [email protected]@ncsu.edu to request assistance to request assistance

http://cskills.ncsu.edu/ncdesk/helpdesk.asp

mailto:[email protected]

Mobile LabsMobile Labs

•Available for schools/systems Available for schools/systems unable to test because of unable to test because of technical limitationstechnical limitations

•By request only [actual process By request only [actual process for requests still in development]for requests still in development]

•Availability issues and division of Availability issues and division of timetime

The Future The Future

• Online testing is the futureOnline testing is the future– Most states are in the process of either implementing or Most states are in the process of either implementing or

maintaining an online testing programmaintaining an online testing program– North Carolina is moving forward with online testing…North Carolina is moving forward with online testing…

this is only the beginningthis is only the beginning• Students are more positive about online testing than Students are more positive about online testing than

administrators/teachers/staffadministrators/teachers/staff– Trends suggest students more comfortable and engaged Trends suggest students more comfortable and engaged

with online testing…overwhelming support from themwith online testing…overwhelming support from them• Technology concerns are warranted, but…Technology concerns are warranted, but…

– Implementation of technology will become seamless over Implementation of technology will become seamless over time as traditional options for testing are exhaustedtime as traditional options for testing are exhausted

– Systems/schools have been successful in implementing Systems/schools have been successful in implementing this testthis test

Necessities for SuccessNecessities for Success

• Dissemination and sharing of informationDissemination and sharing of information– Local, State, National, InternationalLocal, State, National, International– Use resources available and act as a resourceUse resources available and act as a resource

• QuestionQuestion– Online testing is a new world so do not be afraid to Online testing is a new world so do not be afraid to

question things or offer your opinionquestion things or offer your opinion• SupportSupport

– There will be some growing pains, but never waver in There will be some growing pains, but never waver in your supportyour support

– Support at all levels, between all divisions and peoples is Support at all levels, between all divisions and peoples is absolutely requiredabsolutely required

• LearnLearn– Do not be complacent in your knowledge, always seek Do not be complacent in your knowledge, always seek

moremore

Questions???Questions???

Web Sites ReferenceWeb Sites Reference

• http://cskills.ncsu.eduhttp://cskills.ncsu.edu– Link to home page of the North Carolina Online Test of Computer SkillsLink to home page of the North Carolina Online Test of Computer Skills

• http://ncdesk.ncsu.eduhttp://ncdesk.ncsu.edu– Direct link to the home page for the NCDesk application suiteDirect link to the home page for the NCDesk application suite

• http://www.ncpublicschools.org/curriculum/computerskillshttp://www.ncpublicschools.org/curriculum/computerskills– Link to Computer/Technology Skills Standard Course of Study on the North Carolina Link to Computer/Technology Skills Standard Course of Study on the North Carolina

Department of Public Instruction websiteDepartment of Public Instruction website• http://community.learnnc.org/dpi/techhttp://community.learnnc.org/dpi/tech

– Link to Computer/Technology Skills page for Curriculum and School Reform on the North Link to Computer/Technology Skills page for Curriculum and School Reform on the North Carolina Department of Public Instruction websiteCarolina Department of Public Instruction website

• http://www.ncpublicschools.org/accountability/testing/computerskillshttp://www.ncpublicschools.org/accountability/testing/computerskills– Link to computer skills testing information on the North Carolina Department of Public Link to computer skills testing information on the North Carolina Department of Public

Instruction websiteInstruction website• http://tps.dpi.state.nc.us/http://tps.dpi.state.nc.us/

– Link to Technology Implementation & Planning Services page on the North Carolina Link to Technology Implementation & Planning Services page on the North Carolina Department of Public Instruction websiteDepartment of Public Instruction website

• http://www.ncpublicschools.org/techserviceshttp://www.ncpublicschools.org/techservices– Link to Technology Services page on the North Carolina Department of Public Instruction Link to Technology Services page on the North Carolina Department of Public Instruction

websitewebsite

http://cskills.ncsu.edu/nccs

http://ncdesk.ncsu.edu/

http://www.ncpublicschools.org/curriculum/computerskills

http://community.learnnc.org/dpi/tech

http://www.ncpublicschools.org/accountability/testing/computerskills

http://tps.dpi.state.nc.us/

http://www.ncpublicschools.org/techservices

Contact InformationContact Information

Scott RagsdaleScott RagsdaleProject Manager, North Carolina Computer Skills Project Manager, North Carolina Computer Skills [email protected][email protected]

Randy CravenRandy CravenTechnical ManagerTechnical [email protected][email protected]

Jim KroeningJim KroeningProgram Manager, Performance AssessmentsProgram Manager, Performance [email protected]@dpi.state.nc.us




Documents

North Carolina Computer Skills Tests CTE Summer Conference Koury Convention Center Greensboro July 27, 2005 Presenter: Randy Craven Information Technology