Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
BVTE 6450 Project 2 Page
Photo by M. Mabry 11/21/2007
Team 1
Jennifer Bingham
Robert Bourgeois
Marsha Mabry
Linda Mamuscia
BVTE 6450 Project 2: Product Use Testing
Dr. Lilla Holsey
December 5, 2007
Classroom Management Software
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 1 of 31
Table of Contents
1) Project Definition
2) Evaluation Process
3) Evaluation Tools
i) Client‟s Needs Analysis (a) Pre-Assessment Client Survey
(b) Pre-Assessment Observation Guide
(c) Pre-Assessment Interview Questions ii) Technical Team Analysis
(a) Selected Products Review (b) Scoring Sheet
(c) Product Features Checklist iii) Product Demonstration
(a) Selected Products Introduction and Overview (b) Product Comparison
(c) Client Scoring Sheet iv) Pilot
(a) Pilot Survey (b) Pilot Observation
(c) Pilot Interview Questions v) Solution
(a) Final Recommendation Submitted
(b) Full Product Roll Out (c) Product Training
(d) Post Installation Survey (e) Post Installation Observation
(f) Post Installation Interview
4) Summary of Results i) Client‟s Needs Analysis
ii) Technical Team Analysis iii) Product Demonstration
iv) Pilot
v) Solution
5) Appendices A. McNamara‟s Checklist for Program Evaluation Planning
B. Pre-Assessment Client Survey C. Product Specification Table
D. Technical Team Product Scoring Sheet Form E. Client Product Scoring Sheet Form
F. Completed Technical Team Product Scoring Sheets Summary
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 2 of 31
Project Definition
A common issue for those responsible for computer labs, whether in
academic or business environments, is to control:
1) computer use policy violations,
2) improper organization resource utilization, 3) missed IT opportunities,
4) unwanted accumulation of viruses and spyware, 5) and excessive bandwidth consumption.
Although most IT departments have Internet filtering software and virus
scan software in place, users are often able to bypass the filtering devices
using proxy web sites and less secure browsers such as Mozilla Firefox. This
is a growing issue in open computer labs, but also in the classroom or
corporate training facility. It is too easy for users to proxy themselves over
to alternate sites instead of paying attention to the intended instructional
material.
This lack of control costs the organization valuable resources. Our
team has been charged with the task of identifying a solution to this
problem. To address these concerns, a number of companies have
developed network management software. The program is installed on each
computer in the lab or classroom and controlled from the facilitator‟s
workstation. Some typical features of this type of software include the ability
to lock computers, broadcast web sites or programs, capture screenshots,
facilitate internal chat sessions and exchange files with user workstations.
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 3 of 31
The objective for this technical team is to:
1) Identify the needs of the client organization
2) Isolate site-specific technical or administrative requirements 3) Review possible solutions
4) Test recommended program(s) 5) Implement the solution
Some of the items to be addressed include software functionality and ease of
use, pre- and post-installation issues, financial and human resource
constraints and other product specifications.
Evaluation Process
The prescribed steps that the Team will follow during the study are:
1) Find out what people want/need with a pre-assessment survey. 2) From the survey responses, identify issues and areas that require
further clarification and perform one-on-one interviews with the users (the questions can be more in depth and detailed.)
3) Additional insight provided by observation of the client. 4) Identify any constraints on the choice of products according to
organization standards or brand loyalties (executive or administrative edicts.)
5) Determine a „short list‟ of products to evaluate based on the client survey and interview.
6) Test out the products. 7) Measure how well they meet the client‟s requirements.
8) From a technical aspect determine if they are reliable with the user‟s
operating system (OS) and platform. 9) Determine system requirements and whether the software will require
hardware upgrades. 10) Identify aspects like ease of use and implementation or learning curve
factor. 11) Demonstrate the evaluated products for the client.
12) Allow the client to rate the products using a prescribed scoring sheet. 13) Review the client‟s opinions along with the technical team‟s
assessment. 14) Rank the products and select the software to examine.
15) Organize for a pilot study with a select group of users. 16) Evaluate the results of the pilot study.
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 4 of 31
17) Finalize the product choice and make a recommendation for the
organization. 18) Repeat steps 11-15 as needed.
The evaluation plan will employ elements of McNamara‟s Checklist for
Program Evaluation Planning. This checklist is an excellent tool for outlining
the overall focus for product use testing. As outlined in the checklist in
Appendix A, the primary goal of our study is to find suitable classroom
management software packages for clients. Resources available for the
project include demo versions of the software, a Technical Team to conduct
preliminary testing based on survey and interview responses from the clients
and clients to conduct pilot testing. Careful consideration of the client‟s
needs is essential in yielding an acceptable solution.
Evaluation Tools
Client‟s Needs Analysis is the starting point for the technical team.
Prior to the team‟s involvement, the management/administration of the
organization has been made aware of a problem with the existing classroom
or training room computer management methods. For the team, the process
begins with identifying the needs of the client.
One of the first steps in collecting data is the Pre-Assessment Client
Survey. This tool is designed to get an overview of the ways that the client
wants to be able to control the use of the computers in the facility. Specific
short answer questions regarding current methods and problems, as well as
desired capabilities are posed. The delivery and collection of the survey data
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 5 of 31
is determined by the client‟s environment. Web based or email transfer is
quick, economical and makes the data easy to record and process.
The general categories of questions being asked are:
1) Prior experience with computer management software.
2) Description of the environment. 3) List of tasks/capabilities desired.
Please see Appendix B to view the complete user survey.
Observation is a good way of gathering additional information for the
needs analysis. It is important that the observer is discreet and does nothing
to alter the client‟s actions during the observation. The purpose of a
nonbiased individual observing the client is to record the way that they
currently do the task that we are looking to replace. If the person being
observed changes their normal activity during the observation, the data
collected is invalid.
The subjects selected to be observed should be in an area that is
currently doing business as usual. If the clients being observed are in the
process of working on special projects or activities out of the norm, the
results of the observation are not valid for the needs analysis. The objective
of the observation is to provide a picture of the way the client group
currently does business.
The observer focuses on activities related to those tasks we are
looking to replace with the computer management software. Data to be
recorded include:
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 6 of 31
1) Steps involved in the task.
2) Amount of time required for the task. 3) Resources used to accomplish the task.
4) Problems occurred during the task. 5) Other personnel involved in the task.
6) System requirements for the task. 7) Other processes involved in the task.
8) Quality and quantity of end results. 9) Downtime or wasted resources.
10) Redundant tasks.
Similar requirements for the survey and interview apply to the
observation phase. These three tools for collecting data from the client are
vital to the evaluation process. As assessment tools, they are used to
measure activities within the client group in order to identify an appropriate
solution to their problem. The information obtained to gain a total picture of
the client‟s environment allows the team to select a specific solution to their
needs. This step can be repeated during the pilot phase of the evaluation
process that will be introduced later in this document. This concludes the
initial data-gathering phase from the client group.
Follow-up interviews are often necessary to clarify data submitted
through the survey. From the initial results received, a list of interview topics
and questions is required to ensure that the information collected is
understood and valid. These one-on-one exchanges can be online or via
phone or most effectively, in person. As the survey data is recorded, any
responses that are unclear or appear to be questionable are pulled for
follow-up interviewing.
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 7 of 31
From the surveys submitted in this case, approximately five percent
required a more in-depth interview. Interview questions are determined by
the responses collected through the client survey tool. Once the replies are
entered into the database, any questionable responses should be noted. A
report can be generated identifying the clients whose survey requires
clarification. If the summary indicates an unexpected or confusing
cumulative response, a select group of clients can also be interviewed to
clarify this information. Certain clients may even request an opportunity to
discuss their survey responses with the team.
Here is a list of possible questions to be covered during the interview:
1) What is your understanding of how a classroom management program can help you with your job?
2) What kinds of issues are you concerned about with using a classroom management program?
3) Have you ever had difficulty learning a new software program? 4) What kind of training do you prefer in learning a new software
program? 5) What computer classroom/facility management tools do you
currently use? 6) What problems do you have with your current management tools?
7) What questions do you have about computer management tools?
8) Do you foresee any problems with students/learners as you transition to the new tool?
9) Are you comfortable with the team selecting a computer management program for you?
10) Do you have confidence in the team‟s ability to select and install a new program for you?
For clarification we need to be sure to parrot-back what we hear from
the client. It is imperative to listen carefully, accurately record the responses
and refrain from leading the client‟s response in any way. The interviewer
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 8 of 31
can begin by reading back the client‟s survey entries and ask if they have
anything that they want to change or add. The purpose is to ensure that the
data collected is a valid reflection of the interviewee‟s feelings. A needs
analysis is only as good as the data collection process. Like any good
assessment tool, the survey and interview must be:
1) Accurate
2) Appropriate 3) Comprehensive
4) Clear 5) Confidential
6) Properly administered
7) Purposeful 8) Relevant
9) Reliable 10) Secure
11) Suitable 12) Unbiased
13) Understandable 14) Valid
Once these items were clarified, the client data could be processed and
a list of product/program specifications was created. Please see Appendix C
to view the Specification Table.
The User Specification Table can be matched against various vendor
specification lists to find those products/programs that meet the client‟s
requirements. Vendor websites will likely provide their product‟s features
and specifications.
Before selecting software products to evaluate further, the team
considered input from executive/administrative stipulations. Many
organizations will partner with specific vendors in order to secure price and
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 9 of 31
service enhancements. This often requires them to confine certain
acquisitions to a single vendor.
In our case, we were free of any brand loyalty constraints. From the
client‟s needs assessment analysis, four computer classroom management
programs were identified for further evaluation: LanSchool, NetSupport,
SynchronEyes and Vision6. The following section details the Technical
Team‟s reviews of these products.
LanSchool
LanSchool is a very basic classroom management software package.
Installation is effortless for both teacher and student computers. The
installation wizard finishes in less than 45 seconds with no errors. The user
interface is very intuitive and takes little time to become acquainted with.
As illustrated by the icons in the screenshot below, the key features of
LanSchool are show teacher screen, show student screen, run a program,
control computer remotely, view all computers (as shown), send a message,
blank screens, limit web surfing, limit applications, prevent printing,
shutdown/logoff computers, and set LanSchool options. Using Pentium IV
computers running Windows XP Professional, the features work flawlessly.
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 10 of 31
LanSchool is recommended for facilitators who merely need a core set
of features. At $599 per classroom (1 - 4 classrooms), with no annual
maintenance fee, it is a great product for schools on budget. The modest
system requirements are also a plus: Windows 98, 2000, XP, or Vista;
Pentium 166 MHz processor, 48MB – 256MB of memory depending on
operating system of PCs, TCP/IP network protocol.
NetSupport
NetSupport is one of the most comprehensive computer classroom
management software programs available. It is applicable in academic and
business environments, both large and small. The vendor is a British
company, NetSupport Software Ltd., selling the program for an approximate
US dollar equivalent $500 for 10 users, $3600 for 100 users, and upwards to
$40,000 for a 2500 User License. The fact that the program works for 10 as
well as 2500 indicates that it is designed to meet the needs of a variety of
environments. Basic features include powering computers on or off, remote
log off or log on, blank out screens, lock mouse and keyboard, and create
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 11 of 31
individual facilitators‟ profiles. The program can reach users beyond the
confined classroom subnet, locating them in other parts of the campus. It
connects to computers over wireless connections, optimizing performance to
match access point speeds, working effectively with laptop and tablet
computers, and publishing a class allowing students to join on demand.
There is a designated „Tutor‟ that is the PC taking over the „Student‟
PC that is being viewed. Another component is a Synchronized Multimedia
Player for WAV, MOV, AVI, MPG, etc. files. The program provides for
effortless test design, including text, picture, audio, and video questions,
with the ability to set grading levels, track question and student progress in
real time, and create a resource library to be shared with other educators.
During instruction not only can the teacher select computers to
broadcast their screen to, but also students can be instantly surveyed to
provide feedback on their understanding of the material. The responses can
be viewed individually or summarized for the entire class, and establish
groups dynamically based on the results. Selected students can be elevated
to Group Leader status and given rights to act as a tutor.
Besides monitoring student internet use, including background sites or
applications open on each PC, a feature called Internet SafeSearch prevents
inappropriate content being returned within search results. The teacher can
also synchronize a website with the browser on the student PC.
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 12 of 31
NetSupport School allows the Tutor to manage other devices including
USB storage, CDR/DVD, and audio on classroom computers. The extensive
printer management control provides for preventing, pausing, and deleting
print jobs. Printer usage can be limited by number of pages or only if
authorized.
System requirements include 65MB free disk space. NetSupport
School runs on Win 95/8, Win ME, Win 2000, 2003, Win XP & Windows Vista,
IPX/SPX, NetBEUI and TCP/IP. CPU speed required is 300MHz or faster with
at least 32 MB of RAM.
This is a unique software product providing tools to create and manage
examination, testing and training environments for classes comprising
networked PCs. It‟s very easy to use, and utilizes wizard based help for
working with many of its features.
SychronEyes
SynchronEyes software is an extremely valuable classroom
management tool. It gives teachers a better sense of security and control
within computer labs and makes students more responsible in their
computer use. As a result, both the teacher and the students are more
successful. With SynchronEyes software, teachers can manage their
classroom, communicate with their students and assist individuals, all
without leaving their computer.
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 13 of 31
Using SynchronEyes software, teachers can monitor and control their
students‟ computers using the Thumbnails view, which displays real-time
images of students‟ desktops (see screenshot below). In addition to viewing
the thumbnails, teachers can capture snapshots of a student‟s screen, take
control of a student‟s desktop, place a temporary lock on computers,
remotely shut down computers, block Internet access, block the use of
specific applications and the software allows the teacher to broadcast his/her
desktop to students. SynchronEyes also allows for the integration of other
SMART Products. Teachers have the option to use their SMART Interactive
Whiteboards to control, monitor, and write over student computer screens.
Unfortunately, there are several excessive system requirements which
may be prohibitive for some schools. System requirements for the Teacher
PC include: Pentium III processor at 500 MHz or equivalent, 128 MB of RAM,
Windows 98 SE, 2000, Me, XP, Windows Vista or Windows Server 2003, 50
MB of free hard disk space, 800 x 600 or higher resolution display with full
color, DirectX 7.0a, and Microsoft Internet Explorer 5.0 Internet browser or
higher. System requirements for the Student PC include: Pentium II
processor at 300 MHz or equivalent, 64 MB of RAM, Windows 98 SE, 2000,
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 14 of 31
Me, XP, Windows Vista or Windows Server 2003, 15 MB of free hard disk
space, 640 x 480 or higher resolution, Microsoft Internet Explorer 5.0
Internet browser or higher, DirectX 7.0a, and 10 Mbps TCP/IP network
interface adapter.
Cost of the SynchronEyes classroom software varies depending upon
how many computers are in the school‟s lab(s). The chart below shows
product pricing. SynchronEyes software is fairly inexpensive based upon the
features it provides.
SynchronEyes 7.0 Product Price # of Computers Price
1-4 $1,198.00
5-9 $1,000.00 10-24 $839.00
25-49 $599.00 Site License $4,999.00
Vision
Vision allows classroom teachers to easily manage their computer
lab(s). It is designed with the help of teachers, specifically with teachers in
mind. Installation is quick and error-free. The administrative screen is
intuitive with large buttons for the most commonly used tasks. Key features
of this package include: screen sharing, file sharing, remote-control of lab
computers, blank screens and classroom chat sessions. GenevaLogic offers
a number of plug-ins for additional functionality, allowing for individualized
software that meets a variety of classroom needs. These plug-ins include
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 15 of 31
functionality for: Internet blocking, application control and an on-screen
pointer. The entire package can be purchased at once or one can purchase
individual pieces as needed, see pricing below. Other applications offered by
GenevaLogic that teachers might find useful include: Print-Limit, Protect-On2
and Plan-It, which can also be bundled into the classroom management
packages.
In terms of operation, Vision6 runs in the background on student
computers and offers a teacher-desktop called the Dashboard which places a
drop-down toolbar on the teacher computer (see screenshot below.)
System requirements include: wired network with no restrictions on
multicast/broadcast traffic, teacher computer running Windows 2000 or XP
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 16 of 31
Professional, student computers running Windows 98, Me, 2000 or XP
Professional, Intel Pentium III or equivalent, 500Mhz or higher, minimum of
64MB of RAM (128MB or higher recommended), VGA compatibility and 30
MB of free hard disk space.
Unfortunately, the cost of Vision may be prohibitive for some school
systems. Cost of the Vision Class Kit, which includes Vision, Surf-Lock2,
App-Control and Pointer costs $999 with an installation limit of two labs per
school. Individual pieces can be purchased for much less at $35 per vision
installation, $3 per Surf-Lock2 or AppControl installation (10 item minimum
for each of these options.) The system is designed for use as a total solution
and purchase by package over individual components is recommended.
School systems can purchase the software for their entire district at $3999
for districts with less than 1000 students and $4499 for districts with more
than 1000 students. For an additional $500, GenevaLogic offers a plus
version including Print-Limit, which many schools may find useful.
Product Scoring
The technical team also completed a uniform scoring sheet for each of
the prospective solution products. This allowed the team to rate the
programs according to specific criteria that have been identified as being
pertinent to the client‟s needs. In general the criteria provide a snap shot of
the products‟ installation and use from a technical aspect. The results will be
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 17 of 31
discussed in the summary section of the report. Please see Appendix D to
view the Technical Team Product Scoring Sheet Form.
Once the team was familiar with the four products, a client
demonstration was organized. Based on the survey and interview results, as
well as management requests, a select group of five percent of the client
group participated in the presentation. The participants were provided a
Client Score Sheet to rate the four program products. Each was rated on the
following categories:
1) Ease of Use 2) Features
3) Level of Control 4) Classroom Integration
5) Help Tools
Please see Appendix E to view the Client Product Score Sheet.
The client group‟s input was recorded and summarized. The results of
the clients scoring process have been included in the Summary of Results
section of this report. In some cases there is a hands down winner, but in
others, the decision requires reconsideration of the Technical Team‟s
assessment. The client may be swayed by the similar feel to another
commonly used product by the group, a single feature that could cancel out
all other requirements or any number of surprising factors. It is up to the
Technical Team to provide accurate and objective information, but the
decision ultimately rests with the client.
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 18 of 31
Based on the results of the client‟s and team‟s scoring, the program
selected to be tested in a real life application was NetSupport. From this
point the team was responsible for initiating a pilot study using the product
under typical conditions by a select client group. Factors to be considered
when choosing a group to test the product are a department/area:
1) Currently doing ongoing business as usual
2) Representative of the entire client group 3) Significant enough in size and range of operations to employ the
products features and tools 4) Not currently participating or recently participated in a pilot study
As the pilot group proceeds with evaluation, the team is in contact
with them to monitor the test run. If problems arise with the use of the
program, the technical support group logs the problems encountered. This is
a good opportunity to discover platform, compatibility, resource and/or user
issues. Utilization of a Client Pilot Survey, Interview, and Observation,
similar to the Pre-Assessment tools is suggested. With slight modifications to
content these tools will offer a comprehensive gauge of the product‟s
effectiveness as a solution to the client‟s needs. The results of this Pilot for
NetSupport are discussed in the Summary of Results section of the report.
Following a successful Pilot, the team can now make its final
recommendation. If this receives executive/administrative approval, the
Evaluation Team can commence with the plan for the full product roll out.
This includes product introduction, launch and training. It is critical to
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 19 of 31
continue to monitor the product through this final phase with a client Post
Installation Survey, Observation and Interview.
Summary of Results
The organization was faced with a problem dealing with control of their
computer classroom‟s resources. This is a growing concern for many
organizations. The team was engaged to recommend a solution to the
problem.
The team proceeded by laying out an evaluation process. The plan
included identification of the client‟s needs through surveys, interviews, and
observation. Each of these provided insight and clarification of the criteria to
be used in the evaluation.
Results of the Pre-Assessment survey indicated that the client was
looking for a product that would offer the following features:
1) Broadcast of teacher screen 2) View student screens
3) Application control 4) Block Internet access
The Pre-Assessment follow up interview questions yielded information
about the client‟s concern with interruption of existing operations. They also
expressed lack of understanding of how the product could be used to help
them do their job. The interview clarified the specific features that the client
wanted to be able to use and tasks they hoped to be able to perform.
The initial client observation was helpful in getting an inside view of
the client‟s daily work process. This unbiased perspective allowed the team
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 20 of 31
to understand how the selected solution needs to function for the client.
What the team observed was that the client:
1) Is concerned about cost of the product because of the budget of
his/her school system. 2) Desires a program that requires little to no training with self-
explanatory installation. 3) Seeks a user interface that is well designed and easy to use so that
the software compliments the teaching process in his/her classroom. 4) Wants the software to meet his/her individual needs.
Once the requirements and any selection limitations were understood,
the team targeted four computer classroom management software programs
for further review. Based on the features and specifications identified by the
needs analysis, the team closely assessed LanSupport, NetSupport,
SynchronEyes, and Vision. A rating system was employed to determine how
closely each matched the client‟s needs. To see the results of the team‟s
rating of the products, please view the collection of Completed Technical
Team product Scoring Sheets in Appendix F.
The team presented the four products to a select client group. Similar
features and tasks were demonstrated during the presentation and the
clients were asked to complete a Client Scoring Sheet for viewing a selection
of ratings for each program. The ratings by the client and team were then
merged to make a final product selection.
The program of choice was NetSupport. Before it was installed
organization wide, a sample group was used for a pilot. The results of the
test run indicated that NetSupport indeed was the solution that the client
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 21 of 31
wanted. The pilot group in our case reported the following concerns after
using the program:
1) Minor hardware incompatibility in one of the labs.
2) Occasional poor performance due to increased traffic on the network.
3) Intermittent transfer of files on two occasions (some PCs received files, but others did not.).
4) Ability of some students to close the software in the notification area.
Ultimately the program‟s benefits to the organization far outweighed
its problems and its price. The client group determined that this was an ideal
solution to their computer classroom/ training facility management needs.
The final recommendation of NetSupport School was submitted to the
executives and approved.
Continuing with the project process, the team negotiates with the
vendor for the product purchase and service level agreement. The team
participates in the full product roll out including acquisition, installation,
training, and follow up. Involvement of the client‟s training and IT
departments is critical for the team to begin to turn over ownership of the
project to the client. The installation and training process occurs in various
areas of the organization over a period of time, according to resource
availability and other ongoing IT projects.
Before the team is done, a repeat data collection phase must be
completed. A final requirement to the process includes a post installation
survey, observation, and interview with select client groups.
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 22 of 31
Appendix A
McNamara‟s Checklist for Program [Product] Evaluation Planning http://www.managementhelp.org/evaluatn/chklist.htm
Purpose of Evaluation? What do you want to be able to decide as a result of the evaluation? For example:
____ Understand, verify or increase impact of products or services on customers/clients (e.g. outcomes evaluation)
____ Improve delivery mechanisms to be more efficient and less costly (e.g. process evaluation)
____ Verify that we're doing what we think we're doing (e.g. process evaluation) ____ Clarify program goals, processes and outcomes for management planning
____ Public relations ____ Program comparisons, e.g., to decide which should be retained
____ Fully examine and describe effective programs for duplication elsewhere
____ Other reason(s) (Primary reason: keep students on task; Other reasons: reduce spyware and viruses, unnecessary bandwidth use, and policy violation)
Audience(s) for the Evaluation?
Who are the audiences for the information from the evaluation, for example: ____ Clients/customers
____ Funders/Investors ____ Board members
____ Management ____ Staff/employees (CTE teachers and/or lab coordinators)
____ Other(s)
What Kinds of Information Are Needed? What kinds of information are needed to make the decision you need to make and/or
enlighten your intended audiences, for example, information to understand:
____ The process of the product or service delivery (its inputs, activities and outputs) ____ The customers/clients who experience the product or service
____ Strengths and weaknesses of the product or service ____ Benefits to customers/clients (outcomes)
____ How the product or service failed and why, etc. ____ Other type(s) of information? (System requirements of monitoring software)
Type of Evaluation?
Based on the purpose of the evaluation and the kinds of information needed, what types of evaluation are being planned?
____ Goal-based? ____ Process-based?
____ Outcomes-based? ____ Other(s)?
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 23 of 31
Where Should Information Be Collected From?
____ Staff/employees (Technical Team) ____ Clients/customers (Selected CTE teachers and/or lab coordinators)
____ Program documentation ____ Funders/Investors
____ Other(s)
How Can Information Be Collected in Reasonable and Realistic Fashion?
____ questionnaires [pre-assessment surveys, interviews, scoring sheets] ____ interviews
____ documentation (Review information on software companies‟ sites) ____ observing clients/customers
____ observing staff/employees
____ conducting focus groups among _________________________ ____ other(s) (Install software in selected labs and perform testing)
When is the Information Needed?
______________________________________________________________________
What Resources Are Available to Collect the Information? ______________________________________________________________________
Before April 1, 2008
Demo versions of software, Team testing of software, Client surveys and pilot testing of software
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 24 of 31
Appendix B Classroom/Lab Monitoring Software Pre-Assessment Client Survey
Directions: Please answer each question below as it pertains to your classroom.
1. Do you currently use a classroom management software program and if so, which one? (If no, skip to question #4.)
2. What are its most beneficial features?
3. What are the problems, if any that you have encountered using this software?
4. How many workstations do you need to manage simultaneously?
5. Do you need to control other peripherals besides the client‟s computer and if so, which ones? (If no, skip to question #6.)
6. Do you use the client‟s computer to deliver instruction and if so, what type of instruction
do you deliver? (If no, skip to question #7.)
7. Do you need to manage workstations in more than one physical locale and if so, where?
(If no, skip to question #8.)
8. Do you need to be able to completely control the client‟s/student‟s computer for the following? (A description of these features can be found at the end of this document.)
Check all that apply.
Lock Computer
Broadcast Teacher Screen Broadcast Student Screen View Class Screens
Remote Control Chat
Instant Messenger Sound Quiz Students
Push Files Create Workgroups
Hardware Control
Print Job Control Remote Start-up/Shutdown Limit Webpage Access
Block Internet Access Restore Desktop Settings
Restore PC Configuration Application Control Interactive Whiteboard
Multiple Configuration
9. Is there a task that you need to do that hasn‟t been mentioned above?
Thank you for completing this survey. Your feedback is greatly appreciated!
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 25 of 31
Appendix C
Classroom Management Product Features Specification Table
Features
Lan
Sch
oo
l
NetS
up
po
rt
Syn
ch
ro
Eyes
Vis
ion
Lock Computer
Broadcast Teacher Screen
Broadcast Student Screen
View Class Screens
Remote Control
Chat
Instant Messenger
Sound
Quiz Students
Push Files
Create Workgroups
Hardware Control *
Print Job Control **
Remote Start-up/Shutdown
Limit Webpage Access **
Block Internet Access **
Restore Desktop Settings **
Restore PC Configuration **
Application Control **
Interactive Whiteboard **
Multiple Configuration
* Printer Only ** Requires Plug-in
Lock Computer – Locks all input from user
Broadcast Teacher Screen – Share information from instructor screen with all workstations in the lab
Broadcast Student Screen – Share information from single student screen with all
workstations in the lab
View Class Screens – Monitor workstations in simple, single desktop view
Remote Control – Take control of an individual workstation
Chat – Chatroom capability
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 26 of 31
Instant Messenger – Allow sharing of private messages from teacher workstation to single student workstation
Sound – Allow voice communication via headsets between teacher and workstations
Quiz Students – Share quizzes, tests and polls through the environment and allow
results to be shared
Push Files – Share documents from teacher to workstations
Create Workgroups – Allow teacher to create groups of workstations to work together on projects/assignments and share information
Hardware Control – A hardware component can be used to control select functions of the
software via button pressing
Print Job Control – Environment allows for control over print jobs such as canceling or
limiting the number of pages printed
Remote Start-up/Shutdown – Allows teacher to control booting and shutting down of computer from their workstation
Limit Webpage Access – Allow students access to a single or select group of webpages
but nothing else
Block Internet Access – Block all Internet access in the lab or single workstations if desired
Restore Desktop Settings – Allows restoration of desktop settings if changed by students
Restore PC Configuration – Allows teacher workstation to reconfigure all PC settings from their workstation
Application Control – Environment allows for control over which applications can be
opened and used during login session
Interactive Whiteboard – Allows teacher to use a virtual interactive whiteboard to share information on workstation screens
Multiple Configuration – Environment allows for a variety of “classrooms” to be set up for
management. For example: one‟s own class can be one classroom, all computer labs in the school can be a second classroom, two teacher‟s classrooms managed separately as
a third classroom.
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 27 of 31
Appendix D Technical Team Product Scoring Sheet Form
Name of classroom management software:
Please rate your experiences with the above named software package. If a question does not apply to you, please choose N/A.
Scale: 1 = Strongly Agree 2 = Agree 3 = No Opinion 4 = Disagree 5 = Strongly Disagree
1. The installation guide and user guide were clearly written and easy to follow.
1 2 3 4 5 N/A
2. If hardware and software updates were required, they were able to be completed in a
timely manner.
1 2 3 4 5 N/A
3. Installation of the software was error-free.
1 2 3 4 5 N/A
4. The installation process (install wizard) completed in a timely manner.
1 2 3 4 5 N/A
5. The user interface was intuitively designed and easy to learn.
1 2 3 4 5 N/A
6. The most common features sought (e.g., lock screens, broadcast screens, thumbnail
views of screens) are evident in this product.
1 2 3 4 5 N/A
7. During testing of the software, the software‟s features worked properly and no errors
were received.
1 2 3 4 5 N/A
8. If you had to contact technical support, were they professional and able to solve the
problem?
1 2 3 4 5 N/A
9. This software is a good value for the money.
1 2 3 4 5 N/A
Comments:
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 28 of 31
Appendix E Client Product Scoring Sheet Form
Client Name: Software Package:
Please rate the software package above using the scale provided. Please select only one answer per
question. Scale: 1 = Strongly Agree 2 = Agree 3 = No Opinion 4 = Disagree 5 = Strongly Disagree
1. Teacher desktop was intuitive and easy to operate
1 2 3 4 5 N/A
2. Software contained the requested features (e.g., lock screens, block Internet access, observation of student screens, etc.)
1 2 3 4 5 N/A
3. Software allowed for supervision of student computers as desired
1 2 3 4 5 N/A
4. Sharing of files and instruction on student computers was relatively easy and problem free
1 2 3 4 5 N/A
5. Blocking Internet access was easy and intuitive
1 2 3 4 5 N/A
6. Remote management of lab computers was simple and worked as desired
1 2 3 4 5 N/A
7. Software performed as expected and desired
1 2 3 4 5 N/A
8. Use of software compliments the teaching process in your classroom
1 2 3 4 5 N/A
9. Help screen is useful when learning to use software
1 2 3 4 5 N/A
10. Our instruction of system operation assisted you in learning the software
1 2 3 4 5 N/A
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 29 of 31
11. What features were you most impressed by? Please place a check next to the features
you used during your examination of the product:
Lock Computer Hardware Control
Broadcast Teacher Screen Print Job Control
Broadcast Student Screen Remote Start-up/Shutdown
View Class Screens Limit Webpage Access
Remote Control Block Internet Access
Chat Restore Desktop Settings
Instant Messenger Restore PC Configuration
Sound Application Control
Quiz Students Interactive Whiteboard
Push Files Multiple Configuration
Create Workgroups
12. Which features were missing that you would like in your software package? Please place
a check next to the features you desire:
Lock Computer Hardware Control
Broadcast Teacher Screen Print Job Control
Broadcast Student Screen Remote Start-up/Shutdown
View Class Screens Limit Webpage Access
Remote Control Block Internet Access
Chat Restore Desktop Settings
Instant Messenger Restore PC Configuration
Sound Application Control
Quiz Students Interactive Whiteboard
Push Files Multiple Configuration
Create Workgroups
Comments:
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 30 of 31
Appendix F Completed Technical Team Product Scoring Sheets Summary
Please rate your experiences with the software package. If a question does not apply to you,
please choose N/A.
Scale: 1 = Strongly Agree 2 = Agree 3 = No Opinion 4 = Disagree 5 = Strongly Disagree
Technical Team Scoring Sheet Questions
Lan
Sch
ool
NetS
up
port
Syn
ch
roEyes
Vis
ion
1. The installation guide and user guide were clearly written and
easy to follow. 1 1 2 1
2. If hardware and software updates were required, they were
able to be completed in a timely manner. N/A 1 N/A N/A
3. Installation of the software was error-free. 1 1 1 1
4. The installation process (install wizard) completed in a timely
manner. 1 2 1 1
5. The user interface was intuitively designed and easy to
learn. 1 1 2 1
6. The most common features sought (e.g., lock screens,
broadcast screens, thumbnail views of screens) are evident in this product.
1 1 1 1
7. During testing of the software, the software‟s features worked properly and no errors were received.
1 1 1 2
8. If you had to contact technical support, were they professional and able to solve the problem?
N/A 1 N/A N/A
9. This software is a good value for the money. 2 1 2 2
COMMENTS
LanSchool NetSupport SynchronEyes Vision
This software sports basic
features and is extremely
easy to set up and use. The
installation wizard
completes very quickly, so
LanSchool could be up and
running in a 25 seat lab in
less than an hour (provided
no software or hardware
upgrades are necessary).
None were needed on the
Pentium IV, XP Pro test
PCs. It was not really
necessary to read the
installation guide or user
guide.
Of the computer
classroom management
software programs
currently available, it is
the most comprehensive
and extensive product.
The cost is perhaps the
only factor that would
prohibit certain
organizations from
acquiring this program.
Ultimately, the benefits of
NetSupport School far
exceed its price for those
that can afford it.
SynchronEyes offers competitive
features and is easy to install and
use. If you have trouble installing
or using the software, SMART
offers a variety of training
resources and services to help you
get the most from your SMART
product including free materials
or instructor-led sessions.
SynchronEyes is more expensive
than other classroom management
software but it keeps students
engaged and focused, and helps
teachers better manage class time
so you get what you pay for!
Vision allows the user to
control the computers in
an intuitive manner and
is very easy to use.
Everything worked as
anticipated. Plugins
work flawlessly with the
core program. It is a
very comprehensive
package with additional
features available.
However, the cost is
prohibitive for a number
of organizations.
BVTE 6450 Project 2 Team 1 (Bingham, Bourgeois, Mabry, Mamuscia) Page 31 of 31
Works Cited
“Checklist for Program Evaluation Planning.” Free Management Library. 1997. 19
November 2007. <http://www.managementhelp.org/evaluatn/chklist.htm>
"Classroom Management Software Tools from GenevaLogic for North America." GenevaLogic: Active Teaching Systems. 2007. GenevaLogic. 25 Nov 2007.
<http://www.genevalogic.com/index.php?id=us>
“LanSchool Classroom Management Software.” LanSchool. 2005. 19 November 2007. <http://lanschool.com/>
“NetSupport School – The Classroom Management Software.” NSS NetSupport School.
2007. 19 November 2007. <http://www.netsupportschool.com/>
“SMART – SynchronEyes Classroom Management Software.” SMART Technologies. 2007.
21 November 2007. <http://www2.smarttech.com/st/en-US/Products/SynchronEyes+Classroom+Management+Software/>