1

23rd Annual Conference

September 11-13, 2003

Westin Harbour Castle

Toronto, Canada

2

Technology, Service and Delivery

September 12th, 2003

1:30 – 3:00 p.m.

Dan Luk

CTO, Capstar

3

First Topic:“Point and Click – Using the

Internet to Enhance your Testing Program”

4

Why the Internet?

Convenient

Fast

24 X 7

Efficient

Cost-Effective

                                                            

5

Why Not the Internet?

Security

Control

Fairness

6

Web Usage and GrowthSource http://www.nielsen-netratings.com/new.jsp on 1/8/2003

7

When to use the Internet?

Pre-Test

Post-Test

8

Test Development

Web-based Test Development Tools

Collaborative Computing

Item Development and Review

Resolve Geographical Differences

Security

Authentication

Access Restrictions

Encryption

9

Communications

Types of Technology

Push Technology

e.g. e-mail

Pull Technology

e.g. Web Site

10

Push Technology

Main Advantages

Selective or Targeted

Customized Contents

Concise

Main Disadvantages

“Spam”

Frequent Address Changes

                                

11

Pull Technology Main Advantages

General Availability

Broad-based Information

Information Available On-Demand

Consistent Messaging

Interactive Contents

Main Disadvantages

Must be Easy to Navigate

Must Visit Frequently to View New Contents

Random Arrival

12

Types of Information

Bulletins

Application Procedures

Application Forms

Study Materials

Practice Exams

Etc.

13

Registration/Scheduling Self-Service Vs. Call Center

Accessibility

Skills

Online Help Vs. Help Desk

High Availability

Easy to Review Information or to Make Changes

Implementation Methods

Real-time

Back-end Processed

14

Registration/Scheduling (Continued)

Business Rule Enforcement

Upper Vs. Lower Case

Data Consistency Vs. Data Cleansing

Browser Compatibility

ADA Accommodation such as Section 508 Compliancy

E-commerce (Electronic Payment Required)

Identity Theft

15

Test Delivery

High Stake Vs. Low Stake

Anyone Can Test Vs. Positive ID

Proctored Vs. Self-proctored

Connection Performance

Timed Exams

Test Driver Features

Test Security

Browser Compatibility

                                                               

16

Data Management

On-Demand Vs. FTP

Dynamic Vs. PDF

Registration/Scheduling Records

Eligibility Processing

Test Reporting

Psychometric Analysis

17

Capstar Internet Usage

Exceeded Expectations

Anticipate Continued Growth

Continued Area of Focus

18

Second Topic:

“Innovative Uses of Technology In Testing”

19

When to use Technology?

Pre-Test

Post-Test

20

When to use Technology?

Lie!

Cheat!

Steal!

21

Lie

Whom Are You Really Testing?

22

Lie

Typical Preventive Measures

Policies and Procedures as Deterrent

Positive Identification Via Proctor

23

Traditional Identification Techniques Government Issued Pictured Ids

ATT

Tamper Resistance Paper and Printing

Smart Cards

Holographic Images

Microprints

Bar Code

Data Strip

24

Biometrics

‘Recently, the term "Biometrics" has also been used to refer to the emerging field of technology devoted to identification of individuals using biological traits, such as those based on retinal or iris scanning, fingerprints, or face recognition.’

http://stat.tamu.edu/Biometrics/

25

Biometrics (Continued)

Access to Testing Area

Authentication Process

Re-entry to Testing Area (Same Day Vs. Future)

Re-authentication Process (Same Day Vs. Future)

Initial Testing Vs. Repeat Testing

Same Person Vs. Different Person

26

Cheat

Typical Preventive Measures

Policies and Procedures as Deterrent

Rules for Taking Breaks

Business, Eligibility or Re-take Rules

Proctored Exams

Clean Test Rooms

27

Traditional Monitoring Techniques

Active Proctoring

Proctor Window

Video and Audio Surveillance

28

Other Monitoring Techniques

Remote Proctoring

Remote Video and Audio Surveillance

Back-end Monitoring

Extreme Low Score

Extreme High Score

Sudden Score Fluctuation

Time-to-respond

Item Performance

Form/Test Performance

DB Cross-checking

29

Other Monitoring Techniques

Biometrics

Candidate’s Photo

Access to Testing Area

Authentication Process

Re-entry to testing area

Re-Authentication Process

30

Steal Reasons for Prevention to Include:

Intellectual Property Protection

Exam Integrity

Public Safety

31

Steal

Typical Preventive Measures

Policies and Procedures

Business, Eligibility or Re-take Rules

Rules for Taking Breaks

Active Proctoring

Proctor Window

Video and Audio Surveillance

32

Other Techniques

Remote Proctoring

Remote Video and Audio Surveillance

Back-end Monitoring

Extreme Low Score

Extreme High Score

Sudden Score Fluctuation

Time-to-respond

Item Performance

Form/Test Performance

DB Cross-checking

33

Other Techniques (Continued) Initial Testing Vs. Repeat Testing

Same Person Vs. Different Person

New Items

New Forms

Continuous Testing Vs. Windowed Testing

Software Algorithms

Randomization

Computer Adaptive

Item Algorithms

                             

34

Trade-off

                             

Cost

Security

35

Other Technologies Used in Testing

Improved Paper-and-pencil Process

Dynamic Scanning and Scoring

Fax-based Scoring

Simulation-based Testing

Innovative Item Types

Computer-based Practical Testing

3-D Display

36

Other Technologies Used in Testing (Continued) Standards

SCORM

IMS QTI

Other Testing Devices

PDAs

Tablet PCs

IVR/Cell Phones

Alternate Devices

Wireless Technologies

                                                               

37

Q & A

38

Presentation Follow-up

Please pick up a handout from this presentation –AND/OR-

Please give me your business card to receive an e-mail of the presentation materials –OR-

Presentation materials will be posted on CLEAR’s website

39

Technology, Service and Delivery

September 12th, 2003

1:30 – 3:00 p.m.

Dan Luk

CTO, Capstar