Upload
aron-obrien
View
215
Download
0
Tags:
Embed Size (px)
Citation preview
T/TAC I
Final PresentationGeorge Mason UniversityGraduate School of EducationImmersion ProgramMay 11, 2004
T/TAC I & II
T/TAC I
ONLINE TRAINING
SCHOOL IMPROVEMENT
T/TAC I & II
T/TAC I
SITE LAUNCH:Special Education CouncilAPRIL 20, 2004
T/TAC I & II
T/TAC I
T/TAC I - Online Training
T/TAC II - School Improvement
T/TAC I & II -Overview of T/TAC Online Site
T/TAC I & II
T/TAC I
SEMESTER II
Tasks to Address from Last Semester
• Functionality
– www.Workshops
• Documentation
– Workflow Process
– Quick Start Guide
– E-Learning Content Guidelines
– Accessibility Guidelines
– Online Tutorial
T/TAC I Goal
To improve the overall site usability and usefulness by providing best practices along with support mechanisms and guidance in delivering good meaningful online instruction.
Project Plan
• System Usability
• Content Development
System Usability
• Determine if T/TAC staff can easily upload their training content to the T/TAC Online site and uncover any potential difficulties with the upload process.
• Determine the best practices for the inclusion of multi-media by establishing which content formats will best meet the needs of the staff and the site's audience.
• Assess the effectiveness of a pre-production workflow process designed to assist the T/TAC staff in the preparation of their content for uploading
Content Development
• T/TAC staff will feel confident about their ability to develop quality online content in any of a variety of formats
• Content posted to T/TAC Online Training site will be developed based on best practices for online learning
• Identify practices used to develop and delivery good online training.
Formative Evaluation
Using formative evaluation to achieve the team goal by seeking all information necessary to help improve the system and its content.
Formative Evaluation
Types of evaluations conducted were:– One-to-One
– Expert Reviews
– Usability Testing
T/TAC I
USABILITY EVALUATION
Usability evaluation
• Experts’ perspective– Found several issues – Changed “Courses” to “www. workshops”
• Users’ perspective– Usability tests
Usability Test
A structured interview focused on specific features in a user interface.
Usability Testing Goals
• Conduct a Kirkpatrick Level 3 (performance) evaluation on the site’s Online Training section
• Determine how to improve the site
• Gather data for constructing tutorials
• Validate tutorials
Usability Testing Procedure
• Select representative users T/TAC staff at GMU
Three without the tutorial Two with the tutorial
• Select representative tasksView, create, edit, publish for
webshops & workshops.• Develop a testing script
to ensure consistent testing• Observe users performing the tasks
Gathering Usability Results
• Users were asked to think out loud during the testing
• Observers noted thoughts & users’ errors while performing tasks
• Observers noted areas of user confusion that could be addressed in tutorials
• Observers noted areas of the user interface that could be improved with some redesign
• Observers tracked the time of completing the tasks
Usability Testing Results
• Found several design issues that when addressed improved user performance
• Users completed tasks faster with fewer errors after completing the tutorials than without the tutorials
• Users’ ratings of ease of use & time required to perform tasks were higher for those who had used the tutorials first
T/TAC I
TUTORIALS
Staff Education
• Earlier Performance Analysis indicated staff desire for tutorials
• Usability testing confirmed the need for tutorials
• Two aspects of staff education were considered– System training– Content creation and conversion
Training Deliverables
• The evaluation plan that was then developed called for the following training deliverables:
– Quick Start Guide – Preproduction Workflow– Creating Online Workshops that Work:
Introduction – Getting Started: A Framework for Designing
Online Training– Removing the Teacher without Losing the
Student– Implementing Multimedia Elements
System Tutorials
• We started with two tutorials– How to build a Webshop– How to build a www.Workshop
• Our goal: – Measure the effect of having a tutorial as
opposed to not having a tutorial as part of our formative evaluation process.
– Collect feedback on the tutorial itself
• The tutorial was only given to half of the usability test subjects.
Creating The Tutorials
• We used feedback from the initial usability tests as guidelines to create the initial tutorials.
• We concentrated on making the instruction clearly understandable
Creating The Tutorials
Shown here are some of the visual devices we used
• Easy to see and locate step numbers
• Arrows pointing to buttons mentioned in the text
Creating The Tutorials
Another visual device we used was subtle shading and a cutout window to help focus the user on the area discussed in the tutorial.
The Feedback
We gathered a number of good suggestions. The feedback included but was not limited to items like:
• Breaking original tutorials in smaller sections and combining the sections into a Workshop
• Modifications to screen language and instructions for clarity
• Additional Visuals for clarity
• Suggestions for sequencing various parts of the tutorial
• Items to be defined for the end user
Preproduction Workflow
• We have also prepared a preproduction workflow checklist for content authors
• It will be designed as a job aid for planning and construction of Webshops and Workshops
• It presents ideas and things to think about in the areas of:– Project planning– Visual asset preparation and organization– Narrative creation– Copyright issues
Quick Guide
• A quick guide can now be accessed when a Webshop or a www.Workshop
• The Quick guide provides basic, textual instructions for creating content
T/TAC I
CREATING ONLINE WORKSHOPSTHAT WORK
Development Process
Phase One: • Analysis of Existing Workshop
Materials• Recommendations for Training
Focus
Phase Two:• Incorporate Findings into • Webshops and www.Workshops
Phase One – Expert Review
1.Identify
Content Types
2.Identify
Goal Types
3.Associate Goals with Techniques
4.Identify & Apply
e-learning Principles
1. Identify Content Types
Fact
Specific and unique data or instanceYES
Concept
Category with multiple examplesYES
Process
Flow of events or activitiesYES
Procedure
Task with step-by-step actionsNO
Principle
Task performed by adapting guidelinesYES
2. Identify Workshop Goals
Inform
Communicate informationYES
Perform – Procedure
Build procedural skillsNO
Perform – Principles
Build principle-based skillsYES
3. Link Teaching Techniques with Goals
Receptive – Inform Goals
Lots of information
Limited practice opportunities
YES
Directive – Perform Procedure Goals
Require frequent responses
Provide immediate feedback
NO
Guided Discovery – Perform Principle Goals
Job-realistic problems
Supporting resources
YES
4. E-Learning Principles
Multimedia Contiguous
Coherent
Multiple Modes
Personalization
Practice
Examples
Phase One: Summary of Results
Good Presentation Techniques• Well-organized• Capture attention• Effective use of case studies
and examples
Training Focus on Web Environment• Ground in theory• Personalization• Multi-media
T/TAC I
PHASE TWO: Incorporate Findings into
Webshops and www.Workshops
Phase Two
Create www.Workshop
Conduct One-to-one Evaluation
Conduct Level One Evaluation
Create www.Workshop
Creating Online Workshops that Work
Webshops:• Getting Started: A Framework for
Designing Online Training• Removing the Teacher without Losing
the Student• Incorporating Multi-media Elements
Getting Started: A Framework for Design
This Webshop provides a structure you can use when developing online training. It discusses what is the same and what is different and how your previous experience will help you.
Removing the Teacher without Losing the Student
This Webshop introduces six techniques you can use to help keep students involved in a dialogue with the training when you are not there.
Incorporating Multimedia Elements Effectively
This Webshop discusses how you can use multimedia elements to reinforce information transfer based on the theory of human cognition
Conduct One-to-one Evaluation
One-to-one Evaluation– Teachers review raw content– Logs used for recording: page being
reviewed, comments or suggestions and changes incorporated by the designer of Webshop
– Revisions are made and content is transferred to T/TAC Online in the form of Webshops
Conduct Level One Evaluation
Level One Evaluation– Purpose: To collect reactionary data– Surveys emailed to T/TAC staff from all
regions– Feedback:
• T/TAC staff were pleased with the content of the www.Workshop
• T/TAC staff members feel more confident in their ability to create Online training
T/TAC I
RECOMMENDATIONS
Tracking Site Issues
• Web-based quality assurance system for
tracking user experience issues
• Issues filtered by page, category, date
opened, status
• Issues sorted by severity, page, category
• Screen shots included to assist with
communicating the issues
Types of Quality Assurance Issues
• Accessibility – Violations of Section 508 or WC3 WCAG
• Anomaly - Unexpected behavior or bug
• Data Entry – Using text fields, check boxes, radio buttons, etc.
• Information Architecture – Structure of information & labeling
• Management – Issues for the site manager, not programmer
• Navigation - Intuitive navigation menus and buttons
• Printing – Acquiring print outs in the most useful form
• Search - Easy options for filtering and sorting data
• User Guidance - Clarity of instructions, messages & online help
• Visual Design - Professional appearance and layout
Future Recommendations
• Determine where the Webshops and the www.Workshops with information for T/TAC staff will be located on the system.
• Determine how staff will access Webshops and www.Workshops with staff-specific information.
• Develop additional tutorial webshops on using T/TAC Online as users indicate their needs
• Review and address the issues in the Quality Assurance Tracking System
T/TAC I
SITE LAUNCH:Special Education Council
APRIL 20, 2004
T/TAC I & II
Site Launch Activities
Audience AnalysisScope, Sequence, Strategy
Review SiteConsistencyOrientation Video
SurveyBrochureNew Tab – SOL Lessons
T/TAC I
VA DEPTMENT OF EDUCATION SPECIAL EDUCATION COUNCIL PRESENTATION
T/TAC I & II
Introduction to T/TAC Online
• A community linking people and resources to help children and youth with disabilities
• An online extension of T/TAC Services– Usage tracked the same way
• Categories and subcategories• Same forms to collect data• Track by region and category
Accessing the Site
• www.ttaconline.org
• Map of Virginia– Region– Drop-down county lists– Out-of-state
• No login required to visit most of the site - only training is tracked by user
T/TAC I
Q&A
T/TAC I & II
T/TAC I
Thank You!
T/TAC I & II