Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
Virtual High School Program Evaluation
2008–09
September 2009
Virtual High School Program Evaluation
2008–09
September 2009
Submitted to
Liz Pape, Chief Executive Officer
Virtual High School
By
Jonathan Margolin (Principal Investigator)
Jenelle Akerstrom
Kelly Reese
20 N. Wacker Drive, Suite 1231
Chicago, IL 60606-2901
800-356-2735 312-288-7600
www.learningpt.org
Copyright © 2009 Learning Point Associates. All rights reserved. 3983_09/09
Contents
Virtual High School Program Evaluation 2008–09 ........................................................................ 1
The VHS Program....................................................................................................................... 1
Evaluation Methods .................................................................................................................... 3
VHS Growth in Membership, Enrollment, and Offerings .............................................................. 6
Growth in Student Enrollement .................................................................................................. 6
Growth in Membership ............................................................................................................... 7
Growth in Capacity: Course Offerings and Seat Utilization ...................................................... 8
Course Quality .............................................................................................................................. 10
Passing Rates ............................................................................................................................ 10
AP Testing Outcomes ............................................................................................................... 11
Satisfaction With Course Quality ............................................................................................. 12
Ratings of Course Quality ......................................................................................................... 16
Student Learning Outcomes ...................................................................................................... 22
Summary of Course Quality ..................................................................................................... 24
Professional Development Quality ............................................................................................... 25
Professional Development Course Quality ............................................................................... 25
Mentoring Quality ..................................................................................................................... 27
Site Coordinator Orientation ..................................................................................................... 28
Benefits and Feasibility of Participation ....................................................................................... 31
Overall Level of Satisfaction With VHS .................................................................................. 31
Benefits of Participation ........................................................................................................... 32
Feasibility of Participation ........................................................................................................ 36
Barriers to Participation ............................................................................................................ 37
Summary of Benefits and Feasibility of Participation .............................................................. 38
VHS Support and Communication ............................................................................................... 40
Satisfaction With VHS Administrative Procedures .................................................................. 40
VHS Communications .............................................................................................................. 43
VHS Support Services .............................................................................................................. 44
Summary of Findings .................................................................................................................... 49
Recommendations ......................................................................................................................... 53
References ..................................................................................................................................... 55
Appendix A. Summaries of Open-Ended Responses ................................................................... 56
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—1
Virtual High School Program Evaluation
2008–09
The Virtual High School Global Consortium (VHS) is a nonprofit collaborative of high schools
that offers full-semester, yearlong, and summer school courses online to high school students.
These courses can supplement school offerings, providing the opportunity for students to take
classes that are not offered in their school or for which the student demand exceeds the number
of openings.
The mission of VHS is ―to be the leader in online education by working collaboratively with
high schools to offer the highest quality courses for students and teachers‖ (VHS, n.d.a.). During
the 2008–09 school year, VHS offered 171 unique online courses to 10,762 students in 515
member schools.1
In support of the VHS mission, this evaluation report presents quality benchmark indicators of
course instructional quality, professional development quality, and program and services quality.
In regard to these benchmarks, the annual program evaluation aligns with the following research
questions to determine if these benchmarks are being met:
1. Is VHS feasible for schools?
2. Does VHS offer participation benefits to schools, teachers and students?
3. Are schools, teachers, and students satisfied with the services provided by VHS?
4. Is the rigor and quality of VHS courses and professional development sufficiently high?
5. Has VHS expanded teaching and learning opportunities for schools, teachers and
students?
6. How well does VHS communicate, and how responsive is it to its members?
In addition to the focus on instruction, professional development, and service quality, the
program evaluation also tracks the growth of VHS operations across several indicators. To
provide context to the VHS quality benchmark indicators, the following section describes the
operation of the VHS program.
The VHS Program
At the heart of the program is the VHS exchange model, in which schools trade the time of their
teachers for student access to VHS courses. When a teacher from a school instructs a single
course in the VHS network, the school receives a certain number of slots (or ―seats‖) in any of
the courses that the VHS network offers. Therefore, this model allows a school to provide
numerous courses to its students in exchange for assigning a single teacher to teach a single VHS
course. The online program includes courses, professional development for online teachers, and
support services for schools and teachers.
1 The enrollment figure reflects the total enrollment at the end of the no-penalty drop period. The total enrollment at
the end of the add period but prior to the end of the drop period was 11,902. The former enrollment figure is used for
evaluation purposes because it represents the same group of students for which the passing rate is calculated.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—2
VHS Courses
VHS offers courses across the major curriculum areas and at four different levels: middle school,
regular high school, honors/pre-Advanced Placement, and Advanced Placement. A VHS
curriculum coordinator oversees course development to assure adherence to the following VHS
course standards (Virtual High School, n.d.b.), which in turn reflect the North American Council
for Online Learning (NACOL) Standards of Quality for Online Learning (NACOL, n.d.):
Instructor Led—The VHS online pedagogy standards call for clear and consistent
teacher presence as part of the online course delivery.
Student Centered—The VHS design and delivery standards call for an instructional
style in which instructors serve as facilitators and educational coaches, designing
challenging activities and providing effective guidance and quality feedback to students.
Collaborative—All VHS courses contain small-group activities and team projects on
which students must collaborate to foster an online community of learners.
Asynchronously Scheduled—To accommodate student schedules and provide optimal
flexibility across time zones, all VHS courses are scheduled (students must complete
established activities within a given time frame), but courses are scheduled
asynchronously so that students can access and work on their courses 24 hours per day,
seven days per week.
Foster 21st Century Learning Skills—Online courses should fully utilize the medium
to develop effective online communication and interpersonal skills, collaboration and
team-building skills, and inventive thinking skills, such as creativity, problem solving,
and critical thinking.
The evaluation examined customer satisfaction with course quality in accordance with these
standards, with a focus on the topics of course instruction, course content, and student
interactions.
VHS Professional Development
VHS provides its teachers with professional development and mentoring to promote effective
online facilitation. All new VHS teachers participate in NetCourse Instructional Methodologies
(NIM), a 10-week course offered via the Internet. NIM was designed to prepare face-to-face
classroom teachers to become online course instructors. It introduces the pedagogy,
methodology, and moderation techniques, along with technical skills, that teachers need to
effectively teach an online course (Virtual High School, n.d.). VHS also offers the Teachers
Learning Conference (TLC), a 22-week online course that prepares face-to-face classroom
teachers to become online course developers and instructors with VHS (three teachers completed
TLC this past year compared with 72 completers of NIM). The content of the NIM and TLC
courses reflect best practices for online pedagogy (e.g., National Education Association, n.d.).
During each teacher‘s first semester, they are assigned a faculty advisor (typically a person on
the VHS staff who serves as the curriculum coordinator). The role of this advisor is to oversee
the quality of facilitation and to provide feedback and support as necessary. Teachers are
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—3
evaluated at the end of their first semester to determine whether they meet or exceed VHS
standards for course facilitation. Those who do not meet these standards are retained in a high
level of supervision for another semester.
VHS Support Services
VHS member schools have several responsibilities related to managing their participation. These
include recruiting students, providing guidance to students about what courses to take, and
providing time and equipment for students and teachers to work on their VHS courses. Each
school designates a local site coordinator to assist with the management of online learning. The
site coordinator‘s responsibilities include the following: registering students into VHS courses;
managing course materials for students; monitoring student grades; serving as intermediaries
between students and teachers; and reporting student grades to the local school‘s registrar. VHS
supports schools by offering the following services:
Site coordinator orientation (SCO), a four-week online graduate-level professional
development course for site coordinators to learn VHS policies and procedures and the
skills they need to manage online learning at their school. Completion of SCO is a
prerequisite to serving as a site coordinator.
Technical support (available 24 hours) for site coordinators, teachers, and students,
offered through the Contact Center.
Administrative support for site coordinators for creating and accessing student
accounts, filing and obtaining grade reports, and registering students
Mentoring for teachers who have specific questions or issues with student management,
site coordinator communication, and school support. This support is provided by faculty
advisors and other VHS staff.
PROGRESS (PROfessional GRowth ESSentials), a Web-based ongoing professional
development system VHS inaugurated in 2007–08 to help teachers deepen their online
teaching skills as well as provide just-in-time training and support to answer teacher
questions on technical and course-related issues.
SC Resource Area (Site Coordinator Resource Area), a blog that serves as a centralized
resource area for site coordinators to help them better support their students and
administer their vhs program.
Evaluation Methods
The indicators of program quality are drawn from three sources: (1) VHS programmatic data,
encompassing VHS records of school membership in VHS, course registration and grades, and
teacher evaluations; (2) annual surveys of superintendents, principals, teachers, and site
coordinators, and end-of-semester student surveys; and (3) participation and passing rates for
students in Advanced Placement (AP) courses, as reported by VHS member schools. Specific
measures based on these data sources are the following (additional detail regarding each source
is provided in the body of the report):
Indicators of VHS growth in membership, enrollment, and offerings
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—4
Growth in student enrollment
Growth in VHS school membership and retention of member schools
Growth in course offerings, course sections, and seat utilization
Indicators of course and instructional quality
Course passing rates (overall and for credit recovery classes)
AP testing outcomes: number of students taking and passing AP exams
Customer ratings of course quality, overall and in relation to student learning,
instructional practices, course content, and student participation
Student learning outcomes
Professional development quality
Customer ratings of effectiveness of VHS professional development NetCourses
VHS mentoring quality
Customer ratings of effectiveness of site coordinator orientation courses
Benefits and feasibility of participation
Overall level of satisfaction with VHS
Benefits of VHS participation
Feasibility of VHS participation
Barriers to VHS participation
VHS support and communication
Customer ratings of VHS administrative procedures
Customer ratings of VHS communication
Customer ratings of VHS support services
VHS Extant Programmatic Data
VHS compiled all indicators based on extant programmatic data, covering the most recently
concluded program year and extending back as far as data were available for each indicator. In
addition to this summary, VHS provided the evaluator with the raw data files upon which the
indicators were calculated. Evaluation staff examined data files for irregularities (e.g., missing
data, out-of-range values, and inconsistent identifiers) and verified the process by which VHS
calculated each indicator. In cases where the evaluation staff arrived at final figures that differed
from those reported in the VHS summary, the discrepancies were resolved by discussion.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—5
Customer Surveys
VHS developed its own customer surveys covering the topic areas of educational quality,
professional development quality, and service quality. VHS administered these customer surveys
each semester to students and each year to teachers, principals, site coordinators, and
superintendents. All surveys were administered online, although VHS sent paper copies of
surveys to principals and superintendents who did not complete their online survey. The response
rates for the 2008–09 rates are reported in Table 1. Teachers had the highest response rate at 60
percent, and superintendents had the lowest response rate (16 percent).
Table 1. Survey Response Rates by Respondent Category (2008–09)
Respondent Sent (N) Received (N) Response
Rate
Teachers 341 206 60.4%
Site coordinators 744 348 46.8%
Principals 394 108 27.4%
Superintendents 294 47 16.0%
Students 9,997 2,188 21.8%
Note: Student response rates were based on a combination of surveys
administered in the fall and spring semesters. Number sent to students
is based on number of students who had not withdrawn at time of survey.
Survey Analysis Procedures. Surveys were analyzed descriptively and presented by displaying
frequency of response for each item. The student survey responses from the fall and spring
semesters were combined and analyzed together. It is important to note that numerous items
asked for opinions about VHS services and other aspects of the program with which respondents
may not have been familiar (e.g., the Contact Center or the PROGRESS website). Those
respondents who reported that they were not aware of these services were removed from the
denominator when calculating the response frequencies.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—6
VHS Growth in Membership, Enrollment, and Offerings
The rate of growth of the VHS program is presented first, because it is the most fundamental of
all measures of program quality. In particular, the growth of the program is a reflection on the
satisfaction of customers with the program and its feasibility for schools, addressing evaluation
questions, ―Are schools, teachers, and students satisfied with the services provided by VHS?‖
and ―Is VHS feasible for schools?‖ VHS growth is measured in terms of changes in number of
member schools, student enrollment, and course offerings.
Growth in Student Enrollment
During the 2008–09 school year, enrollment in VHS courses continued to increase compared
with previous years, as displayed in Figure 1.
There were 10,762 students enrolled at the end of the no-penalty drop period in 2008–09, a
15 percent increase from the previous year.2 The rate of growth is slightly more than the
previous year, when enrollment grew by 12 percent from the 2006–07 to the 2007–08 school
years.
The total enrollment is approximately double what it was four years ago (2004–05).
Figure 1. Total Student Enrollment and Percentage Change, Across Years
8118
6,864
9,186
10,762
5,342
28%
15%18%
12%
16%
0
2000
4000
6000
8000
10000
12000
2004-05 2005-06 2006-07 2007-08 2008-09
Nu
mb
er
of
Stu
den
ts
0%
20%
40%
60%
80%
100%
Perc
en
tag
e C
han
ge
Enrollment Change in Enrollment
2 There were 11,902 students enrolled at the end of the add period, as mentioned in the previous footnote.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—7
Growth in Membership
Growth in membership is tracked with the number of member schools and membership retention
rates. In 2008–09, there were 515 schools affiliated with VHS, a 7 percent increase from 2007–
08.3 As highlighted in Figure 2, the rate of growth in school membership has decreased
somewhat relative to the previous two years.
Figure 2. Number of Member Schools and Percentage Change, Across Years
371423
480515
235
18%
58%
14% 13%7%
0
200
400
600
2004-05 2005-06 2006-07 2007-08 2008-09
Nu
mb
er
of
Sch
oo
ls
0%
20%
40%
60%
80%
100%
Perc
en
tag
e C
han
ge
Total Schools Percentage Change
The slower rate of growth in 2008–09 is further illustrated by Figure 3, which displays the
number of new schools and retained schools. VHS added 84 new schools in 2008–09, the fewest
new schools since 2004-05. At the same time, however, the VHS program in 2008–09 retained
about 90 percent of schools that had participated during the previous year. This the highest
membership retention rate in four years.
3 The number of schools reflects a particular point in time, namely, the beginning of the school year.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—8
Figure 3. Number of New and Retained Member Schools and Percentage of School
Retained, Across Years
179 189
318 34443156
182
105136
84
90%86%
90%
80%
81%
0
100
200
300
400
500
600
2004-05 2005-06 2006-07 2007-08 2008-09
Nu
mb
er
of
Sch
oo
ls
0%
20%
40%
60%
80%
100%
Perc
en
tag
e R
eta
ined
Retained Schools New Schools Retention rate
Growth in Capacity: Course Offerings and Seat Utilization
In 2008–09, VHS offered 170 unique courses, a 14 percent increase from the previous year. The
number of course sections increased to 352, a 20 percent increase from the previous year. Figure
4 displays the growth in courses and sections. In light of the increase in the number of course
sections, it is useful to examine the proportion of all seats that were filled by students. As
reflected in Figure 5, the rate of seat utilization has fluctuated somewhat over the past five years,
somewhat reflecting the interplay between enrollment and the number of available seats.
Figure 4. Number of VHS Courses and Course Sections, Across Years
131 132150 149
170184
216241
293
352
0
50
100
150
200
250
300
350
400
2004-05 2005-06 2006-07 2007-08 2008-09
Nu
mb
er
of
Co
urs
es o
r S
ecti
on
s
Courses
Sections
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—9
Figure 5. Seat Utilization and Percent Change in Enrollment and Sections, Across Years
76%83% 85%
79% 78%
16%
28%
12%15%
29%
14%20% 19%18%
16%
0%
20%
40%
60%
80%
100%
2004-05 2005-06 2006-07 2007-08 2008-09
Perc
en
tag
e C
han
ge
Seat Utilization
Increase in Enrollment
Increase in Seats
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—10
Course Quality
VHS course quality is evaluated in terms of passing rates (overall and for credit recovery in
particular), AP testing results for students in AP courses, and customer evaluations of the
adherence to course standards and for overall satisfaction with courses and student learning.
Together, these indicators address the evaluation question, ―Is the rigor and quality of VHS
courses and professional development sufficiently high?‖
It must be acknowledged that the passing rates in VHS courses are not direct measures of course
quality, given that an excellent course and a mediocre course may have identical passing rates.
However, changes in the overall passing rates may call attention to fluctuations in course quality
that may need further examination through other methods. For this reason, they are included
here.
Passing Rates
The passing rate for a given VHS course is the proportion of students who achieve at least a ‗D‘
grade (60 percent or greater). Students who withdrew from the course after the no-penalty drop
period or with a grade of ―Incomplete‖ were counted toward the denominator of the passing rate.
The overall course passing rates have remained high over the past five years and stood at about
79 percent for the 2008–09 school year. This is consistent with the passing rate from the previous
two years (see Figure 6).
Figure 6. Overall VHS Course Passing Rate, Across Years
81%77% 78% 78%
79%
0%
20%
40%
60%
80%
100%
2004-05 2005-06 2006-07 2007-08 2008-09
Passin
g R
ate
Note: The passing rate is the ration of students earning a D or higher to the
total number of students enrolled after the no-penalty drop deadline.
Students taking VHS summer courses may elect to take the course for credit recovery. In the
summer of 2008, 189 summer school students elected the credit recovery option; 128 of these
students earned credit, for a success rate of 68 percent. The historical trend in the credit recovery
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—11
success rate is depicted in Figure 7, which indicates that this rate continues to fluctuate. More
students (189) participated in credit recovery courses in 2008–09 than in any previous year.
Figure 7. Credit Recovery Success Rate, Across Years
39%
78%
62%
76%68%
0%
20%
40%
60%
80%
100%
2004-05 2005-06 2006-07 2007-08 2008-09
Passin
g R
ate
N = 119 N = 108 N = 82 N = 193N = 128
AP Testing Outcomes
In 2008–09, 949 students enrolled in AP courses through VHS, a large increase (43 percent) over
the previous year. Out of these 949 students, VHS successfully tracked test outcomes of 740 (78
percent), 739 of whom took the AP exam. Thus, nearly every AP student took the AP exam, an
increase from the 89 percent who took the exam the previous year. The proportion of students
passing the exam increased to 62 percent from 58 percent the previous year. However, the
passing rate has remained fairly consistent over the past four years (see Figure 8).
Figure 8. Total Enrollment in AP Courses and AP Exam Passing Rate, Across Years
299
499
958
664613
70%62%61% 61% 58%
0
100
200
300
400
500
600
700
800
900
1000
2004-05 2005-06 2006-07 2007-08 2008-09
AP
En
roll
men
t
0%
20%
40%
60%
80%
100%
Pa
ss
ing
Ra
te
Taking AP %Passing
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—12
Satisfaction With Course Quality
This section reports the level of satisfaction with course quality and student learning as reported
by superintendents, principals, site coordinators, VHS teachers, and VHS students on customer
surveys.
Satisfaction With VHS Courses Among School Administrators
Superintendents, principals, and site coordinators rated their overall level of satisfaction with
VHS courses. The response options for these items were very satisfied, moderately satisfied,
somewhat satisfied, slightly or not satisfied, and don’t know/no opinion. As shown in Figure 9,
the vast majority of respondents was very satisfied or moderately satisfied with VHS courses. In
addition, over half (53 percent) of principals, almost two-thirds (63 percent) of site coordinators,
and over three-quarters (76 percent) of superintendents were very satisfied.
Figure 9. Rating of Satisfaction With VHS Courses by Role
53%63%
76%
38%33%
19%7%
0%
20%
40%
60%
80%
100%
Principal Site Coordinator Superintendent
Perc
en
tag
e
Very Satisfied Moderately SatisfiedSomewhat Satisfied Slightly or Not Satisfied
Note: Principal N = 90; Site coordinator N = 295; Superintendent N = 42.
The N for each excludes respondents who selected Don’t know/no opinion.
These ratings of course quality are consistent with the high levels of satisfaction reported in past
years.
All respondents were asked to comment on their ratings; 55 site coordinators, five principals,4
and four5 superintendents did so. The main themes among these comments are as follows:
General Satisfaction. Sixteen of 55 site coordinators (29 percent) conveyed that they were
generally satisfied with their VHS courses. One representative comment was, ―All teachers were
extremely competent and communicative, and most students had a tremendous experience.‖ One
principal mentioned that VHS has a good variety of classes. A superintendent noted that running
4 There were a total of 7 principal responses, but two of the responses were not applicable.
5 There were a total of 5 superintendent responses, but one of the responses was not applicable.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—13
VHS is easy because of a high-quality site coordinator, and another gave a positive review of the
site coordinator training.
Course Difficulty. Seven of 55 site coordinators (13 percent) suggested that their students‘ VHS
course needed to be adjusted to incorporate more difficulty. One site coordinator elaborated:
―[There was an] issue with overall rigor in several courses and student complaints about their
classmates not completing partner assignments and group tasks.‖ One principal commented
about course difficulty, saying that the courses were too difficult.
Variability in Teaching Quality. Seven of 55 respondents (13 percent) noted high variability in
teaching quality. One site coordinator commented, ―[There is] great variety in teaching
competency and degree of attention to students. Some were not as well-organized or thorough as
the students needed, and some seemed to hurry through challenging subjects.‖
Alignment Between VHS AP Courses and AP Exams. Three site coordinators (5 percent)
voiced a need for better alignment between VHS AP courses and the material covered on the AP
test. For example, one site coordinator commented on a particular course, ―The AP computer
science class did not seem to meet expectations in regards to the AP test.‖
Individual comments from site coordinators, principals, and superintendents touched on the
following topics: addressing the needs of middle school students, improving grade book
organization, improving course descriptions, and fostering student motivation.6
Satisfaction With VHS Courses Among Students and Teachers
VHS students and teachers rated their overall level of satisfaction with the quality of the VHS
course they took or taught (respectively). The frequency of these satisfaction ratings are
summarized in Figure 10. Eighty-two percent of students and 86 percent of teachers were
moderately or very satisfied with the quality of their VHS course.
6 Sixteen of 55 site coordinators expressed a need for greater responsiveness from some VHS teachers; these
comments will be summarized in the chapter on VHS Service Quality.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—14
Figure 10. Student (N=2,020) and Teacher (N=192) Satisfaction With Course Quality
44% 48%
42% 34%
10% 12%
0%
20%
40%
60%
80%
100%
Teachers Students
Perc
en
tag
e
Very Satisfied Moderately SatisfiedSomewhat Satisfied Slightly or Not Satisfied
Note: The students and teachers who selected ―Don‘t know/No opinion‖ are excluded from the
denominator.
The combined proportion of respondents who were very or moderately satisfied is
summarized across years in Figure 11. This chart indicates that there is an upward trend for
course satisfaction among students but a downward trend among teachers. As a result, these
two groups are now similar in their level of overall course satisfaction.
Figure 11. Student and Teacher Satisfaction With Course Quality, Over Time
77%
86%91%
83%
75%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
2006-2007 2007-2008 2008-2009Perc
en
tag
e V
ery
or
Mo
dera
tely
Sati
sfi
ed
Teachers Students
Note: Students were not asked the course quality item in the 2006–07 survey.
Reasons for Student Satisfaction. Students were invited to describe in open-ended fashion what
they ―liked best‖ about their course. Since the responses to this items were numerous (N=1,687),
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—15
a 10 percent sample of responses was analyzed (n=169).7 The five major themes among these
responses are listed below in order of prevalence, along with a representative quotation. For a
more extensive summary of these comments, see Appendix A.
Students developed independent learning skills (43 percent).
―[I valued] the independence we are given to complete our assignments on time
without a teacher physically reminding us every day. The flexible schedule is also
awesome as a senior to help prepare us for college.‖
Students appreciated the quality, content, and structure of their courses (34 percent).
―VHS courses let me learn, the way I want and need to, and not just memorize data
for standardized tests.‖
Students valued the opportunity to interact with new people (17 percent).
―You get to meet many different people from all different backgrounds which
provides a very interesting learning atmosphere.‖
Students liked the expanded access to courses (14 percent).
―The online courses that are available were not part of my in-school curriculum so I
decide[d] to expand my learning and prepare for college.‖
Students felt they had quality instructors (8 percent).
―My instructor gave us problems that are related to real things in life and other
problems that made us think.‖
Students were asked to describe what could help improve the quality of their VHS courses; 267
students responded to this item. Students identified opportunities for improvement in the
following six areas, listed in order of prevalence (see Appendix A for a more extensive
summary):
Improve teacher communication and involvement (33 percent).
―Please tell teachers that it is very important to be involved in class discussion. In my
fall VHS course, my teacher was just as active in our class discussions as every other
student, but for my spring course, my teacher never posted in a general discussion. I
feel that myself and my classmates would have benefitted more if she had commented
on our discussions. Then we would also know what we were doing wrong if we got
points off for assignments.‖
Provide more variation and creativity in course activities (17 percent).
―I think that it would help for students to do a variety of different types of
assignments. Almost every week our assignments were a discussion, a comment on
someone else's post and a current events journal. It got to be monotonous and seemed
like busy work at times.‖
Provide clarity on assignments and expectations of the course (15 percent).
―Having assignment expectations clearly stated and grading policies and expectations
previously established would be very helpful.‖
7 This random sample was obtained through random number generation, and 60 additional random responses were
reviewed to support the validity of the sample. Within these additional 60 responses, no additional topics were
mentioned.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—16
Facilitate better student collaboration (11 percent).
―Group work was very difficult to do because classmates couldn't always log on and
[it] often affected your ability to work on the project.‖
Improve technology used in the course (9 percent).
―Possibly integrate video lectures, and in other ways make it more like an actual
classroom.‖
Escalate rigor and difficulty of courses (8 percent).
―Having the students be forced to be more involved and having more interesting and
difficult work.‖
Reasons for Teacher Satisfaction. With respect to course quality, teachers were also asked in
what ways their VHS course could be improved for the next year. A total of 147 teachers
responded to this item. Teachers mentioned six major themes, each listed below in order of
prevalence, with a representative quotation. For a more complete summary of teacher responses
to this item, see Appendix A.
The content or structure of the course should be revised (44 percent).
―Different discussion prompts and more in-depth lessons on essay types.‖
The course could integrate or improve the use of wikis and blogs (29 percent).
―[I need to] utilize more creative applications—blog and Wiki need to be used better.
I want students to take on more ownership of their learning.‖
VHS could refine important technical details (15 percent).
―There are a couple technical glitches I just haven‘t been able to work out—like a few
images are broken.‖
Teachers could better facilitate collaboration among students (15 percent).
―Kids lose interest in team activities around the middle of the course. It is hard to
have teams when only half the members participate regularly. I am thinking of
possible solutions.‖
Courses could add more audio and video components (10 percent).
―I plan on adding audio and video for tutorials.‖
Courses could add more current events in class work and discussion (5 percent).
―I should update the materials with all the changes that have happened in the
economy and business over the course of the last year.‖
Ratings of Course Quality
In alignment with VHS standards for course quality, teachers and students rated several
statements about course instruction, course content, and student interactions. These ratings are
presented in three respective sections.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—17
Ratings of Course Instruction
Effective instruction is a central component of the VHS vision for online learning. The student
and teacher surveys asked respondents to rate statements describing different aspects of course
instruction, within sub-categories of course facilitation, course management and feedback on
assignments.
Teacher Ratings. When surveyed about their instructional practice in the areas of course
facilitation, class management, and their provision of feedback, teachers‘ levels of agreement
varied by survey item (see Figure 12).
Teachers most strongly agreed with statements about whether they provided clear and
consistent course management (e.g., providing guidelines, answering questions within 24
hours).
Teachers also typically strongly agreed with statements that they provided constructive
and timely feedback on student work.
Teachers typically agreed or strongly agreed with statements about course facilitation
(e.g., moderating discussions, facilitating group activities). Fewer teachers strongly
agreed with this group of instruction-related items than with items about course
management and the provision of feedback.
Figure 12. Levels of Teacher Agreement With Statements About Their Instructional
Practice
26% 22%
53%64% 69%
54%48%
62%58%
44%35% 28%
41%40%
10%19%
10%
0%
20%
40%
60%
80%
100%
I Promoted
Discussion
I Facilitated
Group
Activities
I Provided
Guidelines
I Clearly
Explained
Standards
I Responded
in 24 Hours
I Provided
Constructive
Feedback
I Provided
Timely
Feedback
Facilitation Management Feedback
Perc
en
tag
e o
f T
each
ers
Strongly Agree Agree Disagree Strongly Disagree
Note: N for these items ranged from 161 to 184.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—18
Most of these trends are consistent with the data from the 2007–08 report; however, there are
three items for which the proportion selecting agree or strongly agree declined by five to 10 raw
percentage points. These three items, and the raw decrease in agreement, were as follows:
Promoting discussions (five percent decrease)
Facilitating group activities (seven percent decrease)
Providing timely feedback (ten percent decrease)
Student Ratings. Students typically agreed or strongly agreed with statements about instruction
(see Figure 13).
Students rated statements about course management highly with just under half strongly
agreeing with statements that their teachers provided guidelines and clearly explained
standards.
About a third of students strongly agreed with statements that their teacher provided
timely and constructive feedback.
Within the facilitation category, students tended to rate the item on their teacher
promoting of discussion higher than the facilitation of group activities.
Figure 13. Levels of Student Agreement With Statements About Instructional Practice
44%
27%
47% 46%38%
32%39%
45%
49%
46% 42%
41%46%
43%
8%19%
6%9%
14% 16% 13%
7% 6%
0%
20%
40%
60%
80%
100%
Promoted
Discussion
Facilitated
Group
Activities
Provided
Guidelines
Clearly
Explained
Standards
Responded
in 24 Hours
Provided
Timely
Feedback
Provided
Constructive
Feedback
Facilitation Management Feedback
Perc
en
tag
e o
f S
tud
en
ts
Strongly Agree Agree Disagree Strongly Disagree
Note: The N‘s ranged from 1,986 to 2,070.
As with previous years, a lower proportion of students strongly agreed with these items than
teachers. Two contrasts between these groups are noteworthy:
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—19
A higher proportion of students than teachers strongly agreed that teachers promoted
discussion (44 percent of students versus 26 percent of teachers).
A lower proportion of students strongly agreed with items related to timeliness (provided
timely feedback and responded in 24 hours).
A higher proportion of teachers (7 percentage points higher than last year) agreed that
teachers provided constructive feedback.
Ratings of Course Content
This section reports the ratings of students and teachers regarding course content. Students and
teachers were asked whether the amount of work in the course was about right, too much, or too
little. As seen in Figure 14, most teachers (93 percent) and students (81 percent) felt that the
workload was about right, though 16 percent of students indicate that there was too much work.
This finding does not differ substantially with results from previous years.
Figure 14. Student (N=2,105) and Teacher (N=193) Ratings of Statements About Amount
of Work Assigned
Teachers
93%
4%3%
About Right Too Much Too Little
Students
4%
16%
80%
About Right Too Much Too Little
Students and teachers rated their level of agreement with several statements about the course
content. These included three items related to the appropriateness of topics and assignments, and
three items related to whether the content fostered student engagement.
Teacher Ratings. Teachers generally expressed a positive opinion about course content. As
shown in Figure 15, almost all teachers (at least 91 percent across items) surveyed agreed or
strongly agreed with items regarding course content. One half to three quarters of respondents
strongly agreed with these items.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—20
Figure 15. Level of Teachers Agreement With Statements About Course Content
75% 68%60%
50%66% 65%
24%29% 38%
41%
33% 30%
8%
0%
20%
40%
60%
80%
100%
Importance
of Topics
Assignments
Practical
Depth of
Content
Content Tied
to Student
Interests
Materials
Relevant
Real-World
Application
Course Content Student Engagement With Materials
Perc
en
tag
e o
f T
each
ers
Strongly Agree Agree Disagree Strongly Disagree
Note: N for these items ranged from 174 to 186.
Student Ratings. At least 90 percent of students agreed or strongly agreed with items about
course content (see Figure 16). However, students differed from teachers in the intensity of their
agreement—they tended to agree less strongly than teachers. For example, 75 percent of teachers
strongly agreed that the course covered important topics, compared with 49 percent of students.
Figure 16. Level of Student Agreement With Statements About Course Content
49%41% 41% 44% 44% 46%
44%54% 50% 46% 49% 45%
8% 7% 7%
0%
20%
40%
60%
80%
100%
Importance
of Topics
Assignments
Practical
Depth of
Content
Content Tied
to Student
Interests
Materials
Relevant
Real-World
Application
Course Content Student Engagement With Materials
Perc
en
tag
e o
f S
tud
en
ts
Strongly Agree Agree Disagree Strongly Disagree
Note: N for these items ranged from 951 to 2,035.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—21
Teacher and student ratings of these course content items did not differ importantly from
previous years.
Ratings of Student Interactions
The third and final area of course quality rated by students and teachers is the extent of student
interactions with each other and the instructor as part of VHS courses. Students and teachers
rated their agreement with statements about student discussions and group work, with two
statements corresponding to each topic. The survey items differed somewhat in accordance with
the perspective of the two groups of respondents.
For statements about how frequently students used discussion boards to discuss course
content, the quality of student discussion in the course, and discussions among students
helping them to better understand the course, 85 percent or more of the teachers indicated
that they agreed or strongly agreed with these items, with between a quarter and half of
respondents strongly agreeing (see Figure 17).
Items about group work were rated lower. For the statements about the importance of
group work, and whether group projects help students understand course concepts, one-
fifth of the teachers strongly agreed with these statements, and about three quarters
agreed or strongly agreed.
Finally, the lowest rated item was regarding the effectiveness of group projects. Only 10
percent of teachers strongly agreed that students worked effectively on group projects, 49
percent agreed, 35 percent disagreed, and 6 percent strongly disagreed.
Figure 17. Level of Teacher Agreement With Statements About Student Participation
46%26%
45%
20% 20%10%
40%59%
51%
53% 51%
49%
12% 13%25% 25%
35%
6%
0%
20%
40%
60%
80%
100%
Frequently
Used
Discussion
Boards
Student
Discussion
High Quality
Student
Discussion
Understand
Course
Concepts
Group
Projects
Important
Group
Projects
Understand
Course
Concepts
Group
Projects
Effective
Student Discussion Group Work
Perc
en
tag
e o
f T
each
ers
Strongly Agree Agree Disagree Strongly Disagree
Note: N for these items ranged from 157 to 186.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—22
Students, for the most part, rated the items about the importance, quality and frequency of
student participation lower than teachers.
Less than 30 percent of respondents strongly agreed with any of the items regarding
student the quality of discussion and the importance of working in groups. In particular,
items about group work were rated low, however, 15 percent of students strongly agreed
with the item about the effectiveness of group projects, as compared with 10 percent of
teachers (see Figure 18).
For items about student how frequently students used discussion boards to discuss course
content, the quality of student discussion in the course, and discussions among students
helping them to better understand the course, more than three quarters of students agreed
or strongly agreed with survey items, and about a half to two thirds of students agreed or
strongly agreed with items regarding group work.
Figure 18. Level of Student Agreement With Statements About Student Participation
29% 28% 29%17% 16% 15%
21%
47% 55% 48%
45%40% 47%
48%
19%14%
18%
26%31%
27%23%
6% 6%11% 13% 11% 8%
0%
20%
40%
60%
80%
100%
Frequently
Used
Discussion
Boards
Student
Discussion
High
Quality
Student
Discussion
Understand
Course
Concepts
Group
Projects
Important
Group
Projects
Understand
Course
Concepts
Group
Projects
Effective
Feeling of a
Classroom
Community
Student Discussion Group Work
Perc
en
tag
e o
f S
tud
en
ts
Strongly Agree Agree Disagree Strongly Disagree
Note: N for these items ranged from 863 to 2,058.
Student Learning Outcomes
Teachers and students rated their impressions of the quality of student learning in their course.
VHS teachers responded to items regarding whether their students learned a lot in their course,
gained an understanding of key concepts, were highly engaged in the class, and were motivated
to do their best (see Figure 19).
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—23
Almost all teachers agreed or strongly agreed that students ―learned a lot in this course‖
and gained a ―deeper understanding of important concepts,‖ with teachers evenly divided
in their level of agreement (nearly half selecting agree and strongly agree, respectively).
Most teachers agreed or strongly agreed that students were highly engaged and
motivated, although teachers typically selected agree for these items.
Figure 19. Teachers’ Agreement With Statements About Student Learning (N=191)
47% 49%32% 27%
49% 47%
57%60%
8% 9%
0%
20%
40%
60%
80%
100%
Overall Student
Learning
Understanding
Key Concepts
Student
Engagement
Student
Motivation
Perc
en
tag
e o
f T
each
ers
Strongly Agree Agree Disagree Strongly Disagree
Students rated the same items as the teachers, although phrased in the first person. Overall,
students typically agreed with statements about their overall learning, engagement, and
motivation, but to a lesser extent than did teachers.
Nearly 90 percent of students agreed or strongly agreed that they learned a lot in the
course and gained understanding of important concepts.
More than three quarters of students reported being highly engaged and motivated by
their course (see Figure 20).
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—24
Figure 20. Students’ Agreement With Statements About Student Learning
41% 39%31% 30%
47% 50%
46% 46%
9% 8%17% 18%
6% 6%
0%
20%
40%
60%
80%
100%
Overall Student
Learning
Understanding
Key Concepts
Student
Engagement
Student
Motivation
Perc
en
tag
e o
f S
tud
en
ts
Strongly Agree Agree Disagree Strongly Disagree
Note: N for these items ranged from 2,016 to 2,024.
The ratings of these statements has changed somewhat over time, with agreement decreasing
slightly among teachers and increasing slightly among students.
Summary of Course Quality
This section reported findings relevant to course quality. Course passing rates have remained
steady overall, but the credit recovery success rate continues to fluctuate. The number of students
enrolled in AP courses increased 43 percent from last year. The proportion of students passing
their AP tests has remained steady over time and now stands at 62 percent. Approximately 66
percent of students completed their credit recovery courses, and the number taking these courses
rose to 193.
Satisfaction with VHS courses among site coordinators and administrators was high. Moreover,
most teachers and students agreed that these courses are educationally valuable. Most teachers
perceived high levels of student engagement and motivation in their classes, and the large
majority of students themselves agreed that they were highly engaged and motivated to do their
best. Teachers and students provided feedback on the extent to which their course reflected VHS
standards for course instruction, content, and student interactions. Course instruction was rated
highly by both types of respondents, except in regards to facilitation of group work. Similarly,
students and teachers rated the quality and level of student interactions highly—when
considering student discussions but not when considering group work. Finally, both types of
respondents rated course content highly and agreed that the amount of work was appropriate.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—25
Professional Development Quality
Several indicators of the quality of the professional development offered by VHS are explored in
the following chapter. These include the passing rate for the NetCourse Instructional
Methodologies (NIM) and Teachers Learning Conference (TLC) courses, ratings of effectiveness
of NIM, teacher suggestions for improvement in new teacher preparation, the proportion of
teachers who successfully completed their first year of teaching without being retained in
mentoring, ratings of effectiveness of mentoring, ratings of effectiveness of site coordinator
training, and site coordinator suggestions for improvement. Overall, these indicators are used in
this section to address the evaluation question: ―How well does VHS communicate and how
responsive is it to its members?‖
Professional Development Course Quality
The graduation rate for the NIM and TLC professional development courses has remained fairly
steady for the past four years, with 90 to 92 percent of participants graduating since 2005–06. In
2008–09, the number of teachers participating in VHS NetCourses (NIM and TLC, combined)
decreased somewhat from the previous year: 83 teachers enrolled in the classes in 2008–09, a 27
percent decrease from enrollment of 114 in 2007–08 (see Figure 21).
Figure 21. Total Participants in Professional Development Courses (Graduates and
Nongraduates), and Percentage Graduating, Across Years (N=83 for 2008–09)
177 7 11 7
70 83 84
103
7680%
92%90%
92%92%
0
20
40
60
80
100
120
2004-05 2005-06 2006-07 2007-08 2008-09
NIM
Part
icip
an
ts
70%
80%
90%
100%
Perc
en
t G
rad
uati
ng
Non-Graduates Graduates Percentage
Customer Ratings of Professional Development Quality. All teachers were asked to rate the
effectiveness of their professional development NetCourse (NIM or TLC) in preparing them for
a variety of online teaching tasks. These findings are summarized in Figure 22.
The typical rating for all of these items was very effective, with at least three quarters or
more of teachers ratings the course at least moderately effective.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—26
Teachers rated the professional development most effective for preparing them to do the
following: Meet VHS course delivery standards, foster online discussion with students,
implement a VHS course of their own, and use technology in teaching.
The lowest ratings of effectiveness, on a relative basis concern fostering group activities,
building online collaboration, and using the online grading system (OGS). Although
about a quarter still rated these components of professional development as very or
moderately effective, a lower percentage (less than half) reported it as being very
effective.
Figure 22. Ratings of Effectiveness of Professional Development Courses
57% 57%46% 44%
58%
43%
61%
31% 31%
32% 38%
31%
30%
30%
7% 8%17% 13%
9%
15%
13%
0%
20%
40%
60%
80%
100%
Implement
VHS course
Use
technology in
teaching
Foster group
activities
Build online
collaboration
Foster
discussion
Use the
OGS
Meet VHS
standards
Perc
en
tag
e o
f T
each
ers
Very Effective Moderately EffectiveSomewhat Effective Slightly or Not At All Effective
Note: N for these items ranged from 171 to 190.
Compared with last year, ratings of very effective are slightly higher all items concerning the
effectiveness of the VHS professional development except for two: Use the VHS Online Grading
System, in which there was a slight decrease, and meet VHS course delivery standards, in which
the response remained consistent.
All teachers were asked ―Please tell us how VHS should improve its preparation of new online
teachers.‖ There were 90 valid responses to this item, encompassing the following suggestions:
Provide more tutorials that feature hands-on practice, mentioned by 59 of 90
respondents (66 percent). One representative comment recommended, ―Continued hands-
on training—making teachers do everything their students will do.‖ Within these
responses, 30 comments (51 percent) directly addressed the use of the grade book; one
representative comment stated: ―I would have benefited from an online workshop or
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—27
collaborative regarding the grade book. It took both semesters for me to become
comfortable using it.‖
Facilitate on-going mentoring and sharing of best practices, mentioned by 28 of 90
respondents (31 percent). Several respondents suggested that they would benefit from
seeing examples of other teachers‘ effective lessons, as expressed in the following
comment: ―Allow us guest access to other courses/use actual examples from other VHS
courses, so that we have a better feel of the expectations and see how others address the
same issues.‖
Address scheduling conflicts between schools and VHS, mentioned by four
respondents (4 percent). These comments indicated that school vacations present
problems because VHS does not provide for students to take this time off from their VHS
course as well, and this is often not communicated effectively to students.
Mentoring Quality
All VHS teachers in their first semester of teaching are assigned a faculty advisor, who is a VHS
staff member, to assist them with the new experience of online teaching. VHS faculty advisors
submit regular evaluations of their mentees to the VHS Education Programs Coordinator, a
process called Level 1 supervision. Based on these evaluations, the faculty advisor and VHS
Education Programs Coordinator decide whether teachers need to continue to be supervised
during the subsequent semester. Figure 23 illustrates the proportion of teachers who exited Level
1 supervision after their first semester, across years.
During the 2008–09 school year, 88 percent of teachers successfully completed their first
semester without needing to be retained in mentoring, which dropped slightly from its highest
success rate in the past five years of 92 percent in 2008–09.
Figure 23. Percent of Teachers Exiting Level 1 Supervision After Their First Semester,
Across Years
88%
70%
80%
88%
92%
50%
60%
70%
80%
90%
100%
2004-05 2005-06 2006-07 2007-08 2008-09
Perc
en
tag
e o
f T
each
ers
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—28
Teacher Ratings of Mentoring Quality
Respondents to the teacher survey who indicated they had taught one to two semesters of a VHS
course (and who therefore had recently worked with a faculty advisor) were asked to rate the
effectiveness of their advisor.
In general, these ratings were very positive, as displayed in Figure 24. For each aspect of the
faculty advisor‘s role, teachers typically described their faculty advisor as very helpful. At least
94 percent rated the faculty advisor at least moderately helpful for each item. Ratings were
uniformly high for giving encouragement, helping with troubleshooting, and helping with course
or student management. Although still very positive, teachers were somewhat less positive
regarding the helpfulness of course content suggestions. These ratings are consistently more
positive than those from the previous year.
Figure 24. Ratings of Faculty Advisor Helpfulness
76%83% 85% 87%
16%11% 13% 7%
0%
20%
40%
60%
80%
100%
Course content
suggestions
Help with
course/student
management issues
Help with technical
issues/troubleshooting
Encouragement when I
needed/wanted it
Perc
en
tag
e o
f T
each
ers
Very Helpful Moderately HelpfulSomewhat Helpful Slightly or Not At All Helpful
Note: N for these items ranged from 88 to 97.
Site Coordinator Orientation
The VHS Site Coordinator Orientation (SCO) is a mandatory four-week online training course
for new site coordinators to learn VHS policies and procedures and the skills they need to
manage online learning at their school. Site coordinators were asked how effective the
orientation was in preparing them to fulfill several key components of their role. As shown in
Figure 25, site coordinators typically reported that the VHS SCO was very effective in preparing
them to accomplish different site management tasks.
Site coordinators typically rated the SCO as very effective in preparing them for all
aspects of their role. The SCO was rated most highly in regards to two tasks technology
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—29
related tasks (use technology well and use the VHS system); over two thirds rated the
SCO as very effective for preparing for these tasks.
Site coordinators perceived the SCO as relatively less effective in preparing them to
recruit students, with nearly one quarter rating it as somewhat or slightly/not at all
effective. Although, even for this item, 78 percent rated it very effective (41 percent) or
moderately effective (37 percent).
Figure 25. Site Coordinator Ratings of Effectiveness of Site Coordinator Orientation in
Preparing for Various Tasks
68%75%
41%
61%52%
24%20%
37%
30%
33%
7%
17%
7%11%
6%
0%
20%
40%
60%
80%
100%
Use technology
well
Use the VHS
system (such as
online grading
system,
registration, and
catalog)
Recruit students
for VHS classes
Work more
effectively as a
VHS coordinator
Understand the
VHS program
Perc
en
tag
e o
f S
ite C
oo
rdin
ato
rs
Very Effective Moderately Effective
Somewhat Effective Slightly or Not At All Effective
Note: Ns for these items range from 303 to 319. Three to 8 percent of respondents selected Not
applicable/Don't know; these were excluded from the denominator.
Site coordinators were asked, ―Are there any aspects of being a site coordinator that should
receive more emphasis within site coordinator training?‖ The major themes and representative
quotations from the 75 valid responses are as follows (see Appendix A for full summary):
Include more hands-on practice time (25 percent).
―The registration process was not really clear until I experienced it.‖
Provide more training on how to market the program and recruit students (25 percent).
―Determining what types of students will be successful‖ should be more of a focus.‖
Provide more training on effective communication with VHS teachers, students, and
school personnel (19 percent).
―How to effectively deal with VHS teachers‖ was an area that should receive more
emphasis within training.
Make explicit the expectation of ongoing monitoring of students (13 percent).
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—30
―[Emphasize] the importance of monitoring the students closely, particularly in the
beginning of the course. The first few weeks can make or break a VHS experience for
a first time student.‖
Tailor training to the unique needs of each school (12 percent).
―It should be stressed that if your school year is different from the VHS school year,
this impacts quite a lot on the VHS student experience and for SCs as well.‖
Site coordinators also recommended providing more training on working with media kits
(6 percent), making additional on-going training available (5 percent), giving site
coordinators recommendations on time management (5 percent), and providing more
information on course content and quality measures (4 percent).
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—31
Benefits and Feasibility of Participation
This section presents findings on the benefits of VHS participation for schools, teachers, and
students, as well as the feasibility of this participation. The findings address the evaluation
questions of, ―Is VHS feasible for schools?‖ and ―Does VHS offer participation benefits to
schools, teachers, and students?‖
Overall Level of Satisfaction With VHS
The overall level of customer satisfaction is related to both of these questions. VHS customers
reported high levels of satisfaction with their school‘s or district‘s overall experience with VHS
during 2008–09, as displayed in Figure 26. At least 90 percent of respondents reported being
very satisfied or moderately satisfied with their school experience with VHS. Over half (58 to 71
percent) of all respondents stated that they were very satisfied.
Figure 26. Overall Level of Satisfaction With VHS
58%71% 70%
34%22% 24%
7% 7%
0%
20%
40%
60%
80%
100%
Principal Site Coordinator Superintendent
Perc
en
tag
e
Very Satisfied Moderately SatisfiedSomewhat Satisfied Slightly or Not Satisfied
Note: Principal N=91; site coordinator N=314; superintendent N=46.
Forty-nine of the site coordinators, 10 principals, and five superintendents commented on their
ratings. The main themes among these comments are summarized as follows:8
General satisfaction. Eighteen site coordinators (37 percent) provided general positive
comments about the Virtual High School. For example, one site coordinator remarked,
―Everyone at VHS is always more than helpful and professional. You are a pleasure to
work with.‖
Student participation. Twelve site coordinators (25 percent) raised concerns about
students dropping VHS courses or not being screened properly for online learning. One
8 The remainder of comments were suggestions about improving VHS operations, as summarized in a separate
chapter of the report.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—32
coordinator mentioned that ―the students who actively participated were successful, I still
have students enrolling for the wrong reason and holding expectations that VHS is an
easy option, which it isn't.‖
School constraints. Six site coordinators (12 percent) mentioned school-level constraints
to their involvement in VHS, such as a lack of time provided by the school to perform
site coordinator duties.
Teachers rated their overall satisfaction with their experience as being a VHS instructor. Almost
all (93 percent) of teachers stated that they were either very satisfied (64 percent) or satisfied (29
percent) with their experience acting as a VHS instructor. Twenty-eight teachers commented on
their ratings. Their comments may be categorized as follows:
Lack of student readiness/high drop-out rate was noted by 12 teachers (43 percent).
One teacher remarked, ―There were half a dozen students who dropped or failed my AP
course throughout the year because of lack of participation on their part. Schools may not
be screening students well before enrolling them.‖
Courses were time consuming or presented scheduling problems was noted by nine
teachers (32 percent). Many of these comments referred to teachers‘ personal issues.
Technology problems/slowness of the learning management system was mentioned
by seven teachers (25 percent). As one teacher explained, ―[Blackboard] is slow and
requires a lot of clicks. It can take up to 3 hours a day to maintain the VHS course.‖
Need to improve or update course content was noted by five teachers (18 percent). A
new teacher said, ―This was my first on line class and so it was definitely a learning
experience. I also spent a lot of time modifying and updating the course.‖ For more on
course content, see the Course Quality section of this report.
Students also rated their overall satisfaction with the VHS program. The majority of students (88
percent were either very satisfied (57 percent) or moderately satisfied (31 percent).
Benefits of Participation
Part of the VHS mission is to ―expand students‘ educational opportunities and 21st century
skills, and to offer professional development to teachers to expand the scope and depth of their
instructional skills‖ (VHS, n.d.a.). To speak to the success of this part of the mission, this section
summarizes customer opinions about the benefits schools, teachers, and students reported they
gained as a result of VHS participation.
Benefits to Schools
In the 2008–09 VHS customer surveys, superintendents, principals, and site coordinators each
were asked to select from a list of benefits the ones that were applicable for their schools. These
three groups of customers were asked, ―In which of the following ways has VHS expanded
learning opportunities for your high school?‖ The rates of endorsement of five types of expanded
opportunities are presented in Figure 27. The main findings are as follows:
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—33
The two most highly endorsed benefits were access to courses not offered at the school
(95 percent or more endorsing) and access to more elective or highly specialized courses
(72 percent or more endorsing).
A lower proportion of respondents (28 to 35 percent) agreed that VHS expanded access
to academically rigorous courses.
The remaining two benefits, related to expanding access to highly qualified teachers and
expanding enrollment capacity (for courses currently offered at the school), were
relatively infrequently selected (15 to 19 percent).
Figure 27. Proportion of Respondents Endorsing School Benefits of VHS Participation
19%
15%
32%
72%
15%
17%
28%
74%
16%
19%
35%
77%
95%
96%
0% 20% 40% 60% 80% 100%
Greater enrollment capacity for classes already
offered at your school
Access to courses not offered at your school
Access to more courses with highly qualified
teachers
Access to more academically rigorous courses
Access to more elective or highly specialized
courses
Principal Superintendent Site Coordinator
Note: Principal N=95; site coordinator N=325; superintendent N=46.
The following changes from previous years are noteworthy:
There was an increase in the percentage of principals endorsing access to elective or
highly specialized courses (from 67 percent in 2007–08 to 72 percent this year).
There was an increase in the percentage of superintendents endorsing access to courses
not offered at your schools (from 94 percent in 2007–08 to 100 percent this year).
There was a decrease in the percentage of principals endorsing access to more
academically rigorous courses (from 41 percent in 2007–08 to 32 percent this year).
There was a decrease in the percentage of all three respondent types endorsing access to
more courses with highly qualified teachers (in particular, the percentage of site
coordinators endorsing this item went from 25 percent in 2007–08 to 19 percent this
year).
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—34
Benefits to Students
All three types of respondents were asked, ―In which of the following ways have your students
benefited from participating in VHS courses?‖ The rates of endorsement of six types of
expanded opportunities are presented in Figure 28.
The two most frequently endorsed student benefits, selected by at least 80 percent of all
respondents, were [opportunity to] pursue personal interests and develop independent
learning skills.
The item about exposure to online learning environment was endorsed highly by
superintendents (91 percent) and principals (86 percent), but not as highly endorsed by
site coordinators (64 percent).
One half to 62 percent of respondents endorsed develop technology skills.
Less than half of respondents endorsed interacted with students from other parts of the
country and developed information literacy skills).
There was a decrease in the percentage of site coordinators endorsing Students interacted
with students from other parts of the country (from 67 percent last year to 27 percent this
year).
Figure 28. Proportion of Respondents Endorsing Student Benefits of VHS Participation
47%
57%
87%
87%
86%
62%
48%
50%
86%
80%
91%
57%
48%
62%
89%
86%
64%
27%
0% 20% 40% 60% 80% 100%
Students develop information literacy skills
Students develop technology skills
Students develop independent learning skills
Students pursue personal interests
Students get exposure to online learning
environment
Students interacted with students from other
parts of the country
Principal Superintendent Site Coordinator
Note: Principal N=91; site coordinator N=322; superintendent N=44.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—35
Benefits to Teachers
Superintendents and principals were asked whether their teachers received particular benefits.
Figure 29 summarizes the endorsement of items presented to administrators.
Half to three quarters of administrators endorsed the statements that VHS teacher has
gained new technology skills, VHS teacher has gained new instructional skills, and VHS
teacher can teach a new course.
Relatively few administrators endorsed the statement that VHS teacher teaches fewer
course outside his/her area(s) of certification.
There was an increase in percentage of principals and superintendents endorsing the
statement that VHS teacher can teach a new course (from 56 percent last year to 61
percent this year for principals, and from 60 percent last year to 77 percent this year for
superintendents).
There was a decrease in percentage of principals and superintendents endorsing the
statement that VHS teacher teaches fewer course outside his/her area(s) of certification
(from 11 percent last year to 5 percent this year for principals, and from 10 percent last
year to 3 percent this year for superintendents).
Figure 29. Proportion of Respondents Endorsing Teacher-Related
Benefits of VHS Participation
61%
59%
67%
5%
77%
65%
52%
3%
0% 20% 40% 60% 80% 100%
VHS teacher can teach a new course
VHS teacher has gained new technology
skills
VHS teacher has gained new instructional
skills
VHS teacher teaches fewer courses outside
his/her area(s) of certification
Principal Superintendent
Note: Principal N=87; superintendent N=41. For this group of items, 23 principals
and 10 superintendents were removed from the denominator because they responded
that they did not currently have a VHS teacher.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—36
Feasibility of Participation
This section summarizes the three types of customers‘ perceptions of the feasibility of
participation with regard to the following aspects: teachers‘ perceptions of the time commitment
and whether it was manageable; students‘ perceptions about time and technology resources;
principals‘ and superintendents‘ perceptions of barriers to participation; superintendents‘
perceptions of community support; and ideas for improvement of services and operations of
VHS.
Time Commitment for Teaching
For VHS to operate successfully, teachers need designated time to prepare for and to teach their
course. Teachers were asked to rate how the time commitment compares with a face-to-face
course and to indicate (in yes/no fashion) whether this time commitment was acceptable and
whether their school provided sufficient time for them to work on their course. These survey
findings were as follows:
As described in Figure 30, most instructors (68 percent) reported that teaching a VHS
course is more or a lot more time consuming than a face-to-face course.
Most (88 percent) teachers indicated that the amount of time needed to teach a VHS
course was acceptable to them. This is an increase from 73 percent in 2006–07 and 86
percent in 2007–08.
About three-quarters (77 percent) of teachers reported that their school provided enough
time for them to work on their VHS course.
There was a decrease in the percentage of teachers reporting teaching a VHS course is a
lot more time consuming than a traditional course (from 32 percent in 2007–08 to 23
percent in the current year).
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—37
Figure 30. Teachers’ Ratings of Time Spent Teaching VHS Course Relative to Face-to-Face
Course (N=203)
23%
27%
45%
4% 1%
A lot more More About the same Less A lot less
Student Resources
As indicated by the following survey findings, students appear to have the technological
resources and time during the school day to participate in their VHS course:
Ninety-seven percent of students indicated that they have a suitable computer available at
their school to use when they need it. This is up from 93 percent last year.
Ninety-three percent of students reported that they have class time during the school day
for their VHS course. This is up from 89 percent last year.
Barriers to Participation
As shown in Figure 31, principals and site coordinators reported few barriers to participation in
VHS; in fact, 51 percent of principals and 37 percent of the site coordinators reported that there
were no barriers.
The most frequently selected barriers were the following:
Low enrollment due to lack of student interest (selected by 21 percent of site coordinators
and 22 percent of principals)
The membership fee (selected by 22 percent of principals).
In addition, 20 percent of site coordinators and 10 percent of principals selected ―Other.‖ These
included: Lack of funds; system issues (―Union concerns‖ and ―Insufficient support from
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—38
superintendent's staff‖); student and parent concerns (amount of time, amount of busywork, and
students being unaware of what VHS courses are like in advance); and course-specific issues
(―For some reason this year, we did not fill our seats‖ and ―course offerings in some area are
small‖).
Figure 31. Principal and Site Coordinator Reported Barriers to Participation in VHS
22%
22%
11%
6%
9%
11%
4%
1%
51%
10%
21%
19%
7%
18%
13%
5%
37%
21%
0% 20% 40% 60% 80% 100%
The membership fee*
Low enrollment/Lack of student interest
Not enough seats available to meet student demand
Diff iculty providing computer or Internet access at school
Student w ho lack computer or Internet access at home
Diff iculty providing staff time for VHS duties
Insufficient number of teachers w illing to participate
Insufficient support from school board of the community*
There are no signif icant barriers
Other
Principal Site Coordinator
Note: Site coordinator N=320; principal N=90.
*Site coordinators were not presented with this item on the survey.
One aspect of feasibility is community support. Ninety-eight percent all of the superintendents
(N=44) agreed that participation in VHS was perceived as ―a positive district offering by school
board members, parents, and community members.‖
Summary of Benefits and Feasibility of Participation
In summary, participation in VHS appears both beneficial and feasible. Most customers
perceived that VHS participation had expanded access to courses not offered at their schools. A
majority of school staff agreed that VHS provided a variety of benefits to students, in particular
the exposure to online learning, opportunity to pursue personal interests, and the development of
independent learning skills.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—39
Teachers and students typically reported having the time and equipment to work on their courses,
although a sizable minority of teachers reported that they did not receive sufficient time to work
on their online course. Principals and site coordinators generally reported few barriers to
participation in VHS, and most superintendents reported that VHS participation was viewed
positively by their constituents.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—40
VHS Support and Communication
VHS offers several types of support to teachers and schools for management of courses and sites.
This section describes the satisfaction of customers (including superintendents, principals, VHS
teachers, and site coordinators) with VHS administrative procedures, communications, and
support services in relation to the evaluation questions, ―Are schools, teachers, and students
satisfied with the services provided by VHS?‖ and ―How well does VHS communicate and how
responsive is it to its members?‖
Satisfaction With VHS Administrative Procedures
Site coordinators, principals, and teachers rated their satisfaction with a variety of VHS
administrative procedures, including creating student accounts, registering students for VHS
classes, dropping students from classes, requesting service through the contact center, and
obtaining grade reports. Each respondent rated the administrative services relevant to their
particular role.
Satisfaction With Administrative Procedures: Site Coordinators
As shown in Figure 32, almost all site coordinators were very satisfied with creating student
accounts, registering students for VHS classes, dropping students, and requesting service through
the contact centers. About half of site coordinator respondents were very satisfied with obtaining
grade reports (55 percent) and course availability (49 percent).
Figure 32. Site Coordinator Satisfaction With VHS Administrative Procedures
89% 85%
49% 55%
82% 83%
9% 12%
34%34%
14% 14%14%8%
0%
20%
40%
60%
80%
100%
Creating
student
accounts
Registering
students for
VHS
classes
Course
availability
Obtaining
grade
reports
Dropping
students
Requesting
service
through the
contact
center
Perc
en
tag
e o
f S
ite C
oo
rdin
ato
rs
Very Satisfied Moderately Satisfied
Somewhat Satisfied Slightly or Not Satisfied
Note: N for these items ranged from 278 to 317.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—41
The site coordinator survey responses indicate that the level of satisfaction is softest in two areas:
course availability and grades. In response to the question, ―In what ways should VHS improve
any of the above operations?‖ the vast majority of the 117 responses pertained to the same two
areas: the grading system and course availability. The following are the main findings related to
the following categories (see Appendix A for more detail):
Offer more sections of the most popular courses, noted by 38 site coordinators (32
percent). For example, one site coordinator stated, ―Classes fill too quickly...add sections
when waitlist is deep enough.‖
Improve teacher promptness in posting grades, expressed by 21 site coordinators (18
percent). A typical comment was, ―Some teachers did not have assignments graded long
after the work was posted by the students. My students complained about not having
current grades frequently.‖
Expand options and improve clarity of grade reports, noted by 13 site coordinators
(11 percent) suggested expanding options for grade reports and improving their clarity. A
common suggestion was ―to be able to obtain [all] student grade reports in one location,
rather than logging in as each student individually.‖
Improve capacity to monitor student participation, as noted by 12 site coordinators
(10 percent), such as by ―mak[ing] it easier for site coordinators to sign into the student's
courses.‖ Others requested greater detail on grade reports.
Increase the frequency of grade reports, as suggested by ten site coordinators (9
percent).
Clarify the meaning of Current Grade Average (CGA). Seven site coordinators (6
percent) expressed confusion over the meaning of CGA.
The remaining suggestions focused on improving procedures for course registration, add/drop,
and exchange of media kits. These comments are summarized as follows:
Streamline the interface and process for registering and dropping students, as
suggested by 10 site coordinators (9 percent). For example, one site coordinator stated,
―The registration process to find and select a course requires way too many clicks and
steps to complete.‖ Two site coordinators noted that students who drop a class should be
removed promptly from all rolls.
Ensure timely return of media kits. Four site coordinators suggested that the use of the
Media Tracker (a program for site coordinators that tracks student use of media kits)
should be mandatory or at least highly encouraged.
Satisfaction With Administrative Procedures: Principals
Principals typically have less direct contact with VHS administrative procedures than teachers or
site coordinators. In general, they were satisfied with the broad categories of student registration,
course availability, obtaining student grades, and technical support (see Figure 33). Specifically,
at least half of principals were very satisfied, and almost all principals were very or moderately
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—42
satisfied. Their ratings resemble those of site coordinators in respect to the procedures with
which they are most satisfied (student registration) and least satisfied (course availability).
Figure 33. Principal Satisfaction With VHS Administrative Procedures
71%
51% 56% 64%
25%
35%31%
26%
14% 9% 9%
0%
20%
40%
60%
80%
100%
Student
registration
Course availability Obtaining student
grades
Technical support
Perc
en
tag
e o
f P
rin
cp
als
Very Satisfied Moderately SatisfiedSomewhat Satisfied Slightly or Not Satisfied
Note: N for these items ranged from 87 to 95.
Satisfaction With Administrative Procedures: Teachers
Most teachers were very or moderately satisfied with viewing student enrollments, filing student
grade reports, and requesting service through the Contact Center. Only 65 percent of teachers
however were very or moderately satisfied with dropping students (see Figure 34).
Figure 34. Teacher Satisfaction With VHS Administrative Procedures
69%
33%
59%
83%
19%
32%
25%
13%7%
20%
8%16% 9%
0%
20%
40%
60%
80%
100%
Viewing student
enrollments
Dropping students Filing student
grade reports
Requesting
service through
the Contact
Center
Perc
en
tag
e o
f T
each
ers
Very Satisfied Moderately Satisfied
Somewhat Satisfied Slightly or Not Satisfied
Note: N for these items ranged from 87 to 95.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—43
Teachers who were not satisfied with the administrative procedures were asked to describe how
VHS could improve them. Fifty-five teachers responded. As with the previous year, the
comments most frequently focused on procedures for dropping students and entering grades.
These are summarized as follows:
Inform teachers when students drop and remove them promptly from grading
system, as noted by 33 teachers (60 percent). One teacher suggested that ―notifications of
additions/deletions should come in the form of an email from the registrar not just
‗popping up‘ in the roster.‖ Other teachers criticized the lack of response from site
coordinators to inquiries about students who drop.
Improve the grading system to make it more user-friendly, as noted by 20 teachers
(36 percent). Their suggestions included the following: (1) the ability to import grades
straight from the grade book and (2) filling out reports for multiple students in a single
window.
Improve communication with schools. Six teachers (11 percent) referred to difficulties
in communicating with site coordinators at schools. One respondent did not know which
site coordinator to contact when there were more than one, and another suggested that
VHS follow up with schools where multiple students seem to be falling behind.
VHS Communications
VHS uses a variety of forms of communication to keep superintendents, principals, teachers, and
site coordinators apprised of program news. These include a website (for visitors and for
member-only access), a newsletter, annual report, weekly announcements, monthly account
management email updates, and direct contact via e-mail or phone. Different modes of
communication are targeted toward different types of roles. Each type of respondent was asked
to rate how beneficial each form of communication was that they received.
The form of communication respondents found most useful varied by their position. A summary
of the ratings of these forms of communication is as follows:
The member-only VHS website and weekly announcements were most beneficial for
teachers and site coordinators. Eighty percent or more of teachers and site coordinators
rated the VHS website and weekly announcements as very or moderately beneficial.
The VHS website (visitor access) was rated as very or moderately beneficial by three-
quarters of principals (77 percent), less than half of teachers (39 percent), and just over
half of site coordinators (54 percent). Twenty-seven percent of teachers and 18 percent of
site coordinators are unaware of this communication.
The monthly account management e-mail updates sent to superintendents and
principals were rated as moderately beneficial, with half (50 percent of principals and 47
percent of superintendents) rating them as very or moderately beneficial. About a quarter
of each of these groups are unaware of this communication.
Teachers typically rated faculty Web announcements as very beneficial, with 74 percent
rating them as very or moderately beneficial.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—44
Principals typically rated direct contact via e-mail or phone as very beneficial, with 58
percent rating this mode of communication as very or moderately beneficial.
Finally, the VHS newsletter was rated as very to moderately beneficial by about half of
principals (48 percent), teachers (54 percent), site coordinators (49 percent), and
superintendents (53 percent).
Teacher and Site Coordinator Satisfaction with Communications
Teachers and site coordinators were asked to describe, in open-ended fashion, the ways that
should VHS improve its communication. Forty-six teachers and 57 site coordinators responded.
The majority of both groups of respondents indicated that the communication was good as it was.
Reflecting the ratings described above, several respondents specifically mentioned the weekly
announcements as being the most effective mode of communication. A minority of respondents
(seven teachers and six site coordinators) criticized the weekly announcements for being
repetitive or not relevant to their role. For example, one site coordinator stated, ―I began to tune
out announcements because they were not relevant most of the time.‖
Teachers offered few suggestions for improving communications. The most frequent suggestion
was for VHS to provide ongoing support from faculty advisors, as described in more detail in the
next section. However, as reported in the previous section (page 43), several teachers requested
more prompt notification when students drop from their courses.
Site coordinators suggested the following improvements in communication:
Encourage communication with teachers. Seven site coordinators (12 percent) asked
VHS to encourage teachers to communicate with site coordinators about issues with
students. One site coordinator noted, ―Continue to encourage teachers to maintain
effective and prompt communication with students and site coordinator. An ounce of
prevention is worth a pound of cure.‖
Provide timely reminders. Five site coordinators (nine percent) noted that they were
lacking key information to help them do their job. They requested that VHS send
reminders about ordering AP exams, enrolling students promptly in popular courses, and
finding different support services for site coordinators.
Clarify grading system. Several site coordinators reiterated their concerns about the
clarity of the grading system, as discussed in the previous section of this chapter.
VHS Support Services
Providing technical support to site coordinators, teachers, and students is an important
component of VHS. There are two main types of support services for site coordinators, teachers,
and students outlined in this section: the VHS contact center and technical support, and Web-
based resources. The following section will describe ratings and comments about the support
services that VHS provides to these groups.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—45
VHS Contact Center and Technical Support
Site coordinators described the frequency with which they used the contact center, the reasons
for their use, and site coordinators and students rated their satisfaction with the contact center.
The contact center was frequently used, with about two thirds of the site coordinators (69
percent) reported having contacted a support representative from the contact center over
the past year.
Site coordinators were asked to indicate the different issues for which they sought
assistance from the contact center. The prevalence of these issues is as follows, in order:
student access (54 percent), grading (27 percent), student passwords (18 percent), and
course lessons (16 percent).
As shown in Figure 35, Of the students who received supports from the VHS Contact
Center (N=565), just under half (44 percent) were very satisfied, 37 percent were
moderately satisfied, 14 percent were somewhat satisfied, and 5 percent were slightly or
not satisfied.
Of those who sought support from the contact centers (N = 247), almost all site
coordinators (88 percent) were very satisfied.
Figure 35. Site Coordinator and Student Satisfaction With the VHS Contact Center
44%
88%
37%
10%14%
0%
20%
40%
60%
80%
100%
Students Site Coordinators
Perc
en
tag
e
Very satisfied Moderately satisfied
Somewhat satisfied Slightly or Not Satisfied
Note: For students, N = 565, and for site coordinators, N = 247
Of the 107 site coordinators who commented on their satisfaction rating, almost all (96 percent)
praised the prompt and helpful assistance from the contact center. One respondent explained, ―I
have always been pleased with the quick responses and how helpful your staff is. They not only
answer my question, but give me additional helpful information to make life easier.‖
Student Satisfaction With Support Services. Students were also asked about other academic
support services that would have been helpful. The proportion indicating that each service would
have been helpful is as follows: real-time discussions with an instructor (44 percent), 24-hour
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—46
access to a homework helpline (34 percent), online tutoring (24 percent), 24-hour access to a
technical support representative (21 percent), and other (5 percent).
Of the 107 students who selected ―Other,‖ 60 made comments about VHS courses or services
(the remaining comments simply indicated that none of the other options would have been
useful). The major themes among the 60 comments on courses or services are as follows:
Real-time chat with fellow students, mention by 30 students (50 percent of comments).
Several of these comments pointed out the important of live chat for group projects. One
student requested the ability ―to do a live discussion with the class on a group activity,
like a chat room for discussing [the project].‖
More immediate access to the teacher, suggested by 10 students (17 percent). Some
students wanted to know the specific times their teacher would be online, so they could
get immediate answers to their questions. Other students simply wanted more feedback
and participation from their teacher.
Access to a subject-matter expert at their school, suggested by five students.
The remaining 15 comments comprised isolated suggestions, most of which did not pertain to
academic support services.
Teacher Ratings of Helpfulness of Technical Support. Teachers were asked how helpful VHS
staff members were in providing technical support. Almost all (81 percent to 87 percent) of
teachers rated VHS staff members at being very helpful in assisting them in the following areas:
Using the Blackboard platform, handling problems with students, site coordinators or schools,
and other technical assistance.
Teachers were asked to offer suggestions about how to improve the helpfulness of VHS‘s
technical support. Seventy-seven teachers responded to this open-ended question. The most
typical response, stated by seventy percent of respondents (54 teachers), was that VHS support
was effective and prompt, and that no improvement is necessary. A typical comment of this sort
was, ―The quality of service is outstanding and immediate.‖ A small number of teachers offered
the following suggestions for improvement:
Provide feedback on instruction. Three teachers requested ongoing feedback or
additional communication with their VHS advisor (six teachers made a similar comment
in describing how to improve administrative procedures). A typical comment requested
―a check in with our original VHS support person once per semester, at least, and maybe
at the end with feedback? I know I can do better but I‘m not sure how.‖
Provide access to technical support on demand. Five teachers suggested improvements
to technical support. Typically, these requested more immediate access to support, such
as via the telephone.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—47
Online Support Resources
VHS provides support to site coordinators and teachers through online resources targeted to each
role. Site coordinators and teachers were asked to describe show often they used the SC resource
area, and the PROGRESS (PROfessional Growth ESSentials) support site, respectively.
One percent of site coordinators visit the resource area daily, six percent visit weekly, 16
percent visit monthly, 41 percent visit one or twice per semester, and 36 percent visit
never or almost never.
Less one percent of teachers visit PROGRESS daily, three percent visit weekly, 14
percent visit monthly, 36 percent visit one or twice per semester, and 47 percent visit
never or almost never.
As seen in Figure 36, of the site coordinators who visited the resource area, 43 percent
found it to be very helpful, 35 percent found it to be moderately helpful, 17 percent found
it to be somewhat helpful, and 4 percent found it to be slightly or not at all helpful. The
VHS Site Coordinators were asked to explain their ratings of helpfulness of the SC
Resource Area. Twenty site coordinators made relevant comments.
Of the teachers who visited PROGRESS, under a quarter (21 percent) found it to be very
helpful, 29 percent found it to be moderately helpful, 34 found it to be somewhat helpful,
and 15 percent found it to be slightly or not at all helpful to answer questions about day-
to-day course operations.
Figure 36. Site Coordinator and Teacher Satisfaction With the VHS Web-Based Resources
43%
21%
35%
29%
17%
34%
15%
0%
20%
40%
60%
80%
100%
SC Resource Area (Site
Coordinators)
PROGRESS (Teachers)
Perc
en
tag
e
Very helpful Moderately helpful
Somewhat helpful Slightly or not helpful
Note: For teachers, N=99, and for site coordinators, N=231. Respondents who never accessed these
resources were excluded from the denominator.
Thirty site coordinators made comments about their ratings on the helpfulness of the resources
area. Eight respondents used the site and found it to be helpful. They used it to obtain handouts,
help with recruitment, and access the VHS handbook. Seven explained that they did not use it;
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—48
instead, some of these site coordinators follow-up with it directly with the necessary party—
either the VHS Contact Center or the teachers. Five site coordinators found the area too large
and/or complicated to use, or found the information and forms on the site to be outdated.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—49
Summary of Findings
This report may be summarized according to the guiding questions for the program evaluation.
As context for the answers to the research questions, the growth of VHS over time, and a
summary of VHS growth in membership, enrollment, and offerings also is presented here.
What is the growth of VHS over time?
VHS has experienced growth in many areas, including student enrollment, number of member
schools, and number of courses and course sections offered, but there was a decline in number of
new schools, and percentage increase has slowed in many areas.
Total student enrollment is increasing as is the rate of growth in student enrollment. In
the 2008–09 school year, there were 10,762 students enrolled in VHS courses, an
increase of 15 percent from 2007–08.
Corresponding to the increase in student enrollment, VHS increased the number of course
sections by 20 percent. VHS now offers 352 sections of 170 courses.
There were 515 member schools in VHS in 2008–09, a 7 percent increase. For the third
year in a row, however, the rate of growth in member schools has slowed. In the current
year, there were 84 new schools compared with 136 the previous year.
The rate of retention of member schools has increased to 90 percent for 2008–09, up from
81 percent the previous year.
Is VHS feasible for schools?
VHS participation appears to be highly feasible for schools as indicated by extant program data
and survey data.
Seat utilization has remained stable over time, with the utilization rate currently at 78
percent.
According to survey responses, most teachers are provided with time during the school
day to attend to their VHS course, which is a slight increase from the previous year. A
sizable minority (23 percent), however, do not agree that their school provided sufficient
time to work on their VHS course.
Teachers typically noted that preparing for and teaching a VHS course is more or a lot
more time consuming than instructing face-to-face courses. However, there was a
moderate decrease from 2007–08 in teachers reporting that it is a lot more time
consuming.
Most students report that they have the resources (time, space, equipment) within their
schools to work on their VHS courses.
According to superintendents, district participation in VHS was nearly always perceived
positively by school board, parents, and community members.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—50
About 37 percent of site coordinators and 51 percent of principals reported that there
were no barriers to participation. The most frequently mentioned barriers were low
enrollment or lack of student interest (principals and site coordinators) and the
membership fee (principals), which is consistent with the previous year.
Does VHS offer participation benefits to schools, teachers and students?
This question is combined with the related question of ―Has VHS expanded teaching and
learning opportunities for schools, teachers and students?‖ VHS customers expressed broad
levels of satisfaction with VHS and reported several benefits of participation for schools,
teachers, and students:
Administrators were nearly unanimous in confirming that VHS expanded access to
courses not offered at their school. Moreover, their endorsement of this benefit increased
from the previous year (2007–08) to the present year (2008–09).
Administrators frequently indicated that VHS expanded access to elective classes and, to
a lesser extent, to rigorous courses. The proportion indicating expanded access to elective
classes increased from last year, whereas the proportion noting expanded access to
rigorous courses decreased from last year.
A majority of site coordinators, principals, and superintendents endorsed a variety of
student benefits of VHS participations. The three most commonly endorsed benefits were
the opportunity to develop independent learning skills; the opportunity to pursue personal
interests; and exposure to the online learning environment. However, there was a large
decrease in site coordinators perceiving that students benefited from VHS by interacting
with students from other parts of the country.
A majority of administrators perceived that teaching a VHS class allowed teachers to
teach a new course, and expanded their technology and instructional skills.
Administrators reporting that teachers teaching a new course was a benefit increased, and
that teachers teacher fewer courses outside certification areas was a benefit decreased.
Are schools, teachers and students satisfied with the services provided by
VHS?
Customers expressed their satisfaction with VHS administrative procedures, technical support,
and online resources. Site coordinators, teachers, and principals were broadly satisfied with
administrative procedures, but their satisfaction differed by role and by procedure.
Site coordinators and principals typically were highly satisfied with procedures for
creating student accounts, registering and dropping students, and requesting technical
support.
For site coordinators, there was the most room for improvement in their satisfaction
regarding course availability and grade reports. Several site coordinators expressed
concern about the frequency and clarity of grade reports.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—51
Most teachers were highly satisfied with procedures for viewing student enrollments and
requesting service through the contact center.
For teachers, there was the most room for improvement in procedures for dropping
students and filing grade reports.
Satisfaction with technical support varied considerably according to role:
Most site coordinators were very satisfied with the contact center; likewise, most teachers
were very satisfied with VHS support in relation to a number of different technical
difficulties.
Conversely, students were only moderately to very satisfied with the contact center, with
one fifth of indicating that they were somewhat to slightly satisfied.
When asked what additional support services would have been helpful, students
frequently cited real-time discussions with instructors and other students and 24-hour
access to homework help.
Teachers and site coordinators rated the helpfulness of the Web-based resources PROGRESS
and the SC Resource Area, respectively.
Almost two thirds of the site coordinators visited the SC Resource Area, and of those
who visited it, three quarters found it to be very or somewhat helpful.
Just over half of the teachers reported that they had accessed the PROGRESS system. Of
these teachers, about half rated it as very or somewhat helpful, which is a decline from
the previous year.
Is the rigor and quality of VHS courses and professional development
sufficiently high?
Course Passing Rates. Extant data on passing rates and credit recovery rates provide a
perspective on course quality.
The overall course passing was 79 percent for the 2008–09 school year, which was
consistent with the previous two years.
The number of students taking summer school courses for credit recovery rose to 193 (up
from 82 the previous year). Sixty-six percent of these students passed their classes, a rate
that continues to fluctuate from year to year.
AP Test Outcomes. Success on AP tests provides an indicator of the quality of AP courses
offered by VHS.
There was a large increase (43 percent over the previous year) in the number of students
taking AP courses with 949 students enrolled in 2008–09.
The proportion of students passing the exam (out of those taking the exam) rose slightly
to 62 percent from 58 percent the previous two years.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—52
Satisfaction With Courses. Principals, superintendents, and site coordinators expressed high
satisfaction with overall VHS course quality in the 2008–09 academic year. Students and
teachers reported high satisfaction with courses taken or taught, respectively. Teachers and
students were asked to rate the extent to which courses reflected standards for quality online
learning in terms of instructional practice, course content, and student interactions.
The instructional practice of teachers was rated highly by both teachers and students, and
in particular, the management of classes with the largest room for improvement was the
facilitation of group activities.
Most students and teachers felt that the amount of work was appropriate for the course.
Student participation was rated relatively lower than all other aspects of course quality,
with student use of groups and student collaboration as the lowest rated items.
Regarding course content and student learning, teachers typically expressed high
satisfaction and students expressed moderate to high satisfaction.
Professional Development Quality. Customer survey and extant program data provided
indicators of professional development quality. Even thought the graduation rate has remained
stable, the number of professional development participants is at its lowest point seen since the
surveys were first tracked in 2004-05.
Teachers typically rated the NIM or TLC courses as somewhat effective or very effective
for preparing them to accomplish a range of different online instructional tasks. These
ratings are slightly higher than in the previous year in all areas except for two
The effectiveness of professional development is reflected in the small percentage of
teachers who were retained in the faculty advisoring program during 2008–09.
Teachers typically rate their faculty advisors as very helpful in regards to the different
aspects of the advisor‘s role. These ratings are slightly higher than the previous year.
In open-ended responses to questions about communication and service quality, some
teachers requested periodic communication with the curriculum advisor.
How well does VHS communicate, and how responsive is it to its members?
Customers rated the benefit of various modes of communication with VHS. Different forms of
communication were helpful for different groups.
Most teachers and site coordinators rated the VHS website and weekly announcements as
very or moderately beneficial. A minority viewed the weekly announcements as
repetitive or not relevant to their role.
For principals, the visitor-access VHS website and direct contact via phone or e-mail
were the most beneficial forms of communication.
Site coordinators suggested that VHS could improve communication by providing timely
reminders about certain tasks and opportunities, and by encouraging teachers to
communicate more effectively about student issues.
Several teachers requested better communication about students who drop.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—53
Recommendations
The following are recommendations for improvement based on the findings of this report:
Provide more hands-on training for teachers and site coordinators. Based on
comments and ratings from of site coordinators and teachers, the NIM and Site
Coordinator Orientation should provide more tutorials, workshops, or interactive
components to allow teachers and site coordinators to become more familiar and
comfortable with the basic tasks and tools needed for their duties.
Facilitate communication about student performance. According to both teachers and
site coordinators, there is a breakdown in communication about students. Respondents of
each type mentioned the lack of mutual communication as a problem. The following are
some specific suggestions:
Provide explicit guidance (e.g., in NIM and SCO or refresher training) about when
and how to communicate about students.
Clarify for teachers which site coordinator is the primary contact at schools with
multiple site coordinators.
Reevaluate their online infrastructure to determine if there is a way to more easily
facilitate this communication, such as chat rooms or direct links from student names
to site coordinator contact information for teachers, and vice versa for a more
streamlined communication process.
Improve site coordinator capacity to monitor student performance. Site coordinators
gave extensive comments about how to improve the quality of, and access to, information
about student participation and performance in classes. These suggestions, described in
Appendix A (―Site Coordinator Comments About Service Quality‖) include the
following: increasing the amount of information on grade reports, providing a tool that
indicates the level of student activity in courses, increasing the frequency of grade
reports, and providing this information in one place for all students.
Facilitate collaboration. Teachers and site coordinators have mentioned their feelings of
isolation and a desire for new ideas. In particular, some site coordinators find student
recruitment challenging, and some teachers mentioned wanting to work with other
teachers to continue improving their online courses.
Check in with teachers in their second or third semesters on a regular basis to see if
they are encountering any difficulties or to give feedback.
Provide incentives for teachers and site coordinators to post questions to their
electronic discussion forums.
Create opportunities for meaningful collaboration among teachers within a discipline.
Improve online infrastructure. Satisfaction with the online grade book among teachers
and site coordinators was low. Although it is beyond the purview of this report to suggest
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—54
specific improvements, VHS is encouraged to review the concerns expressed by these
two groups in detail (see Appendix A).
Address concerns about course registration. Some site coordinators suggested that the
course registration procedures were not equitable, because schools on the east coast had a
head start at enrolling students in high-demand courses. One way to address this issue
would be to open registration at a later time in the day. A less tractable concern with
registration relates to course availability. Although there is no clear-cut recommendation,
VHS should be aware that it is a highly prevalent concern among site coordinators and
principals.
Recommendations for evaluation topics and procedures.
Investigate why so few teachers know about PROGRESS and why satisfaction with
this resource is relatively low.
Promote completion of the surveys to obtain higher response rates among teachers
and site coordinators, possibly by offering incentives and by alerting them farther in
advance about the expectation that they participate.
Eliminate redundant items on surveys. The analysis of survey items indicate that
several open-ended responses could be eliminated.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—55
References
North American Council for Online Learning (n.d.). National standards of quality for online
courses. Retrieved September 1, 2009, from http://inacol.org/research/nationalstandards/.
Virtual High School. (n.d.a) VHS mission and beliefs. Retrieved September 24, 2009, from
http://www.govhs.org/Pages/WhyVHS-Home
Virtual High School. (n.d.b). Why virtual high school. Retrieved September 24, 2009, from
http://www.govhs.org/pages/whyvhs-mission/
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—56
Appendix A
Summaries of Open-Ended Responses
Teacher Comments on Course Quality
With respect to course quality, teachers responded to the question, ―In what ways, if any, should
your VHS course be improved for next year?‖ (N=147)
Revise Course Content or Structure. Sixty-four out of 147 teachers (44 percent) who answered
the question stated that their course should be improved by revising the content or structure of
the class. Many teachers indicated that they would be slightly revising next year‘s curriculum
based on the reaction of students to this year‘s course. Most respondents spoke of small changes
rather than large ones. A generally representative comment simply listed items to be revised,
such as: ―Different discussion prompts and more in-depth lessons on essay types.‖
Integrate or Improve the Use of Wikis and Blogs. Twenty nine of 147 teachers (20 percent)
mentioned wanting to integrate or improve the use of Wikis, blogs, and other technological
applications in their classrooms. Some of these teachers indicated that the interactive aspect of
the Wiki application was key, and some also stated that the blog was helpful for teaching
students to provide useful feedback. Many of these respondents suggested that these applications
were not currently being used to their full potential. One person expressed this by saying, ―[I
need to] utilize more creative applications—blog and Wiki need to be used better. I want
students to take on more ownership of their learning.‖
Refine Important Technical Details. Fifteen teachers (10 percent) talked about needing to
refine strictly technical aspects of their course. These comments covered topics such as restoring
dead links and upgrading various software. One representative comment was ―There are a couple
technical glitches I just haven't been able to work out—like why a few images are broken.‖
Facilitate Better Collaboration Among Students. Fifteen teachers (10 percent) talked about
finding ways to facilitate more collaboration among students. Most comments mentioned group
work as a way to accomplish this, although responders also talked about challenges that group
work presents. One representative comment was, ―Kids lose interest in team activities around the
middle of the course. It is hard to have teams when only half the members participate regularly. I
am thinking of possible solutions to this.‖
Add Audio And Video Components. Fourteen respondents (10 percent) mentioned wanting to
add audio and/or video components to their courses. A typical comment stated a need for ―more
videos and interactive activities online.‖
Add More Current Events in Class Work and Discussion. Seven teachers (5 percent)
mentioned adding more current events in their class work and discussions. A few of the
comments mentioned updating reading lists based on current events. One teacher commented: ―I
should update the material with all the changes that have happened in the economy and business
over the course of the last year.‖
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—57
Student Comments on Course Quality
Students’ Favorite Aspects of VHS Courses
Students were asked to describe ―What did you like best about taking a VHS course?‖ in an
open-ended manner. Because response to this question was particularly great (N=1,687), a 10
percent sample of responses was analyzed (n=169).9 The major themes among these responses
were as follows:
Development of Independent Learning Skills. Forty-three percent or 73 of 169 students
indicated that the development of independent learning skills was one of the things they liked
best about their VHS course. Students articulated that this was an area in which they needed to
improve to be ready for the college environment and that VHS helped them to do so. One aspect
of VHS courses that simulated the college environment was the flexible schedule, and students
cited a gain in time-management skills and responsibility as a result of taking the courses. When
responding to the question of what he or she liked best, one student stated: ―The independence
we are given to complete our assignments on time without a teacher physically reminding us
every day. The flexible schedule is also awesome as a senior to help prepare us for college.‖
Another said: ―Learning how to be independent with my work. That is some thing that I will take
and use in the future.‖
Quality, Content, and Structure of Courses. Thirty-four percent of student responses analyzed
(57 out of 169) referred to the quality, content, or structure of their courses. Several students
found their course challenging but not too difficult. Many students expressed being very engaged
with the content of their courses because it was something in which they were specifically
interested. Other comments highlighted structural aspects that were appealing, such as the way
the course was organized being conducive to learning. Overall, these comments indicated that
students found the courses to be interesting and relevant. One student emphasized this by saying:
―VHS courses let me learn, the way I want and need to, and not just memorize data for
standardized tests.‖
Expanded Access to Courses. Fourteen percent or 23 out of 169 respondents mentioned that
they had gained access to a larger variety of courses through VHS. Many students indicated that
VHS offered courses that were not available at their school, and that they benefited from this
greater variety of courses from which to choose. One student said, ―The online courses that are
available were not part of my in-school curriculum so I decide[d] to expand my learning and
prepare for college.‖
Opportunity to Interact With New People. Seventeen percent of students (28 out of 169)
mentioned that they appreciated getting to meet and interact with new people from other states
and countries through their VHS classes. Many students indicated that the different experiences
and points of view that these diverse classmates brought with them enriched the learning
9 This random sample was obtained through random number generation, and 60 additional random responses were
reviewed to support the validity of the sample. Within these additional 60 responses, no additional topics were
mentioned.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—58
environment for everyone. One student stated: ―You get to meet many different people from all
different backgrounds which provides a very interesting learning atmosphere.‖
Quality Instructors. Eight percent of respondents (13 out of 169) talked about having very
positive experiences with their VHS instructor. One student said, ―My instructor gave us
problems that are related to real things in life and other problems that made us think.‖ These
students also seemed to feel that their instructor put forth a lot of effort on their behalf: ―My
teacher was great about explaining things I didn't understand and teaching me instead of just
giving me the week‘s layout and hoping I understand it.‖
Student Opinions About How to Improve VHS Courses
As follow-up to the question, ―How satisfied are you with the quality of the VHS course you
took this semester?‖ students were invited to comment on the following prompt: ―If you were not
satisfied, how can we improve the quality of your VHS courses?‖ Students expressed
opportunity for improvement in the following areas (N=267):
Improve Teacher Communication and Involvement. Thirty-three percent, or 87 of 267
responding students, expressed that VHS could improve course quality through working to
improve teacher communication and involvement in courses. These students felt that their
teacher was not able to answer their questions in a timely manner and that this lack of timeliness
reduced what they were able to learn. One student elaborated: ―The instructor could have given
feedback and grades in a much more timely manner. I felt that I was getting comments way too
late to incorporate them in my products.‖ Other students commented about a need for greater
teacher involvement in general, including more teacher preparation for the course and more
teacher engagement in online conversations. One student expressed a need for VHS teachers to
be more engaged in posting on discussion boards, rather than letting the conversation take place
only between the students: ―Please tell teachers that it is very important to be involved in class
discussion. In my fall VHS course, my teacher was just as active in our class discussions as
every other student, but for my spring course, my teacher never posted in a general discussion. I
feel that myself and my classmates would have benefitted more if she had commented on our
discussions. Then we would also know what we were doing wrong if we got points off for
assignments.‖
Provide More Variation and Creativity in Course Activities. Seventeen percent of students
(45 out of 267) expressed an opportunity for improvement in the activities of their course. Many
respondents felt that the course design would benefit from the introduction of more creative or
varied tasks: ―I think that it would help for students to do a variety of different types of
assignments. Almost every week our assignments were a discussion, a comment on someone
else's post and a current events journal. It got to be monotonous and seemed like busy work at
times.‖ Several students expressed that teachers should also consider lessening the work load,
specifically the amount of ―busy work‖ that was assigned.
Provide Clarity on Assignments and Expectations of the Course. Fifteen percent of students,
or 40 out of 267 respondents, spoke of a need for more clarity on assignments and expectations
of the course. Comments indicated more clarity was needed from the teacher when describing
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—59
directions for completing assignments and that more clarity was also needed on the initial course
description. Some students felt directions for assignments were often too brief and ambiguous
and that grading procedures were not always clear. One student commented: ―My teacher […]
was rarely detailed in explaining the kinds of things she wanted in an assignment and how she
was grading it.‖
Escalate Rigor and Difficulty of Courses. Eight percent of students (21 out of 267) expressed a
need for more rigor and difficulty in the course that they took. Specifically, several of these
students mentioned that the content and expectations of their course were not appropriate to their
grade level, and that this may be a product of students from multiple grade levels being taught
within one course. One general comment suggested that ―having the students be forced to be
more involved and having more interesting and difficult work‖ would improve course quality.
Also, students noted that when the subject matter was too easy for them, they became bored and
disengaged.
Improve Technology Used in the Course. Nine percent of student respondents (25 out of 267)
suggested that there was an opportunity for improvement in technological aspects of their course.
Some felt that navigation of their course site was cumbersome and a few requested software that
enabled real-time editing between participants. Three respondents also requested video in order
to facilitate more interaction between students and create a more classroom-like setting:
―Possibly integrate video lectures and in other ways make it more like an actual classroom.‖
Facilitate Better Student Collaboration. Eleven percent of students (29 out of 267) mentioned
that collaboration with other students was sometimes problematic, and that course quality could
be improved through the facilitation of more effective student interaction. Five of these students
suggested doing away with group projects in which the grade of one student was dependent on
the effort and timeliness of other students. One student explained: ―Group work was very
difficult to do because classmates couldn‘t always log on and often affected your ability to work
on the project.‖
Site Coordinator Comments on Orientation Quality
Site coordinators were asked, ―Are there any aspects of being a site coordinator that should
receive more emphasis within site coordinator training?‖ There were 75 valid responses to this
item, encompassing the following suggestions:
Include more hands-on practice time. Twenty-six site coordinators (35 percent)
expressed that they needed more step-by-step instruction during the training, including
hands-on practice with working through tasks. They felt the best way to learn how to
effectively complete tasks was by doing them. One site coordinator commented, ―The
registration process was not really clear until I experienced it.‖ Therefore, they felt that
the training would be more effective if it included practice time.
Provide more training on student recruitment. Nineteen of 75 site coordinators (25
percent) expressed the need for more training on how to effectively market the program
to students and then recruit them. This included ways to screen students to determine if
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—60
they were in fact a good fit for the program, as well as how to effectively keep current
VHS students. One site coordinator expressed this by answering that ―determining what
types of students will be successful‖ should be more of a focus in the training.
Provide more training on effective communication. Fourteen respondents (19 percent)
of the 75 indicated that they would like more training on effective communication and
collaboration tools. This included interactions with teachers, students, and school
personnel such as guidance counselors. A representative comment indicated that ―how to
effectively deal with VHS teachers‖ was an area that should receive more emphasis
within training.
Make expectations of ongoing monitoring explicit. Ten respondents (13 percent) stated
they felt more emphasis should be put on what is expected of site coordinators in terms of
ongoing monitoring of students. They also suggested that direction was needed on how to
effectively achieve this ongoing monitoring. Several comments stated a need for
recommendations on when specifically a site coordinator should intervene to correct a
student‘s trajectory. One respondent suggested that early concentration of monitoring
should be emphasized: ―The importance of monitoring the students closely, particularly
in the beginning of the course. The first few weeks can ‗make or break‘ a VHS
experience for a first time student.‖
Tailor training to the unique needs of each school. Nine responses (12 percent) talked
about the need to adapt VHS practices to the reality of their specific school. Because each
school is unique in its schedule, expectations, and atmosphere, they expressed the need
for training that is tailored to the specific issues that they will encounter when conducting
their tasks within their unique school setting. For example, the following comment
touched on a problem arising from variations in school schedules: ―I think that it should
be stressed that if your school year is different from the VHS school year, this impacts
quite a lot on the VHS student experience and for SCs as well, for example trying to get
up-to-date grades for report cards when they aren't available, etc.‖
Provide more training on working with media kits. Eight site coordinators (6 percent)
mentioned that they felt they needed more instruction on how to manage working with
the media kits. Specifically, this included how to send and receive them, especially for
international students. Tracking the packages was mentioned as an important detail that
was not stressed enough in training.
Make additional on-going training available. Four respondents (5 percent) said that
they would appreciate more refresher or ongoing training. They felt this was crucial for
keeping up to speed on new technologies and VHS practices.
Give site coordinators recommendations on time management. Four respondents (5
percent) expressed needing more initial direction on how to best manage their site
coordinator responsibilities. To do this, they felt they needed more up-front information
about the expected time commitment of the site coordinator position.
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—61
Provide more information on course content and quality measures. Three site
coordinators (4 percent) talked about needing to know more about the course content and
its quality measures in order to better assist their students. By knowing what their
students are experiencing and what is expected of them, these site coordinators felt that
they could anticipate needs and head off potential problems for students.
Site Coordinator Comments About Service Quality
The site coordinator survey responses indicate that the level of satisfaction is softest in two areas:
course availability and grades. In response to the question, ―In what ways should VHS improve
any of the above operations?‖ the vast majority of the 117 responses pertained to the same two
areas. Over half of all comments dealt with the grading system, and one third related to course
availability. The following are the main findings related to these two categories:
Improve course availability. The most prevalent comment of this sort was a suggestion
that VHS offer more sections of popular courses. For example, one site coordinator
stated, ―classes fill too quickly...add sections when waitlist is deep enough.‖ Some site
coordinators offered suggestions for improving waitlist procedures (e.g., communicating
the likelihood of getting into a course, allowing a student to register for another course
while on the wait list), and others commented on issues of fairness regarding the waitlist
(e.g., that student-only schools and those in later time zones having a disadvantage in
registering for popular courses).
Improve teacher promptness in posting grades. The most prevalent comment about
grades, expressed by 21 of 117 site coordinators, was that teachers should stick to the
schedule for posting grades or for grading assignment. A typical comment was, ―Some
teachers did not have assignments graded long after the work was posted by the students.
My students complained about not having current grades frequently.‖
Improve grade reports. The next most prevalent suggestion about grades, expressed by
13 of 117 site coordinators, was to expand options for grade reports and to improve the
clarity of these reports. Several site coordinators wanted ―to be able to obtain [all] student
grade reports in one location, rather than logging in as each student individually.‖ A
variation on this theme was the request that grade reports include every assignment, not
just a final grade.
Improve capacity to monitor students. Twelve site coordinators made suggestions
pertaining to their capacity to monitor student participation. Some site coordinators
suggested better information about student activity, such as by ―mak[ing] it easier for site
coordinators to sign into the student's courses‖ or by providing an ―activity monitor‖ for
each student. From a different perspective, some requested greater detail on the grade
reports and in the student grade center. As one site coordinator explained, ―I have
students that have earned 30 of 70 points with no explanation given.‖
Increase the frequency of grade reports, as suggested by 10 site coordinators. One site
coordinator elaborated on the difficulties with the current system, saying, ―The week‘s
time lapse in receiving CGA‘s can be problematic when students are having problems.‖
Learning Point Associates Virtual High School Program Evaluation Report: 2008–09—62
Clarify the meaning of Current Grade Average (CGA). Seven site coordinators
expressed confusion over the meaning of CGA, with some noting that the teachers
themselves appeared to be confused about how to calculate it.
The remaining suggestions focused on improving procedures for course registration, add/drop,
and exchange of media kits. These comments are summarized as follows:
Streamline the interface and process for registering and dropping students, as
suggested by 10 site coordinators. Each comment described a particular aspect of the
process that ought to be improved in convenience and efficiency. For example, one site
coordinator stated, ―The registration process to find and select a course requires way too
many clicks and steps to complete.‖ Another site coordinator suggested that VHS
consider upgrading the entire site coordinator interface to improve consistency and
functionality. Two site coordinators noted that students who drop a class should be
removed promptly from all rolls.
Ensure timely return of media kits. Four site coordinators suggested that the use of the
Media Tracker should be mandatory or at least highly encouraged, or that tardy returns
should be accompanied by a fine.