Questions of Analysis Can Be Answered Through a Survey

7/28/2019 Questions of Analysis Can Be Answered Through a Survey

1/4

questions of analysis can be answered through a survey. for example, a survey could be used to

describe the composition of the groups that attend crime prevention or weight reduction classes as

well as to analyze the relationship between descriptive data of sex and weight reduction success.

surveys are usually performed for summative (impact) evaluation. did the program accomplish what

it was proposed to do? was the program perceivedas successful by clients? By personel? if the

program was considered successful, what parts were most helpful? Least helpful? what should be

changed? Left unchanged? The questions asked by survey are determined by the initial list of

questions about program evaluation.

like the case study method, the answers on surveys come from the perceptions, values, and belief

systems of the respondents. the response given to questions of program usefulness bythe nurse who

planned and implemented the program may be very different from the answers of the participants.

awareness of perseption bias can direct evaluation efforts to consider the perceptions of all persons

( providers, clients, and management) involved in program implementation.

surveys that are used to measure program evaluation must be concerned with the reliability and

validity of the information collected. reliability deals with the repeatability, or reproducibility of the

data (ie, if the same questions were asked of the same people 1 week later, would the same

responses be recorded?). validity is the correctness of the information. if questions are written to

evaluate knowledge and the answers of the respondents reflect behaviors, then the questions are

not valid because they do not measure what they claim to measure.

USEFULNESS TO EVALUATION

surveys can be very valuable to answer questions of relevance, or the need for proposed or existing

programs, especially if the perceptions of clients, providers and management are solicited. in like

fashion, progress can be measured. people critiquing surveys as an evaluation strategy may be

concerned with the subjectivity of the survey indeed, individual perception affects every response to

every question. However, most decisions are based on subjective judgements, not objective reality.

the important concern is to understand whose subjective impression is being used as a basis for

judgment; it is imperative for community health nurses to ensure that clients perceptions are

represented alongside those of health care providers and management.

cost efficiency, effectiveness and outcome are difficult to measure by using a survey. although a

survey can measure the perceived efficiency of the program or ideas on alternative ways of

operating to make the program more cost efficient, these perceptions are formed only in the

context of the existing program. there is no other comparison program against which recorded

perceptions can be measured. A survey can provide information on the characteristics of program

activities that are perceived by the respondents to have caused changes in their health status, but


2/4

these impressions are reported in the absence of any comparasion group. A comparasion group is

especially important with regard to effectiveness and impact because it is impossible to tell if an

alternative program (or no program at all) might have been more or less effective in accomplishing

the same objectives.

EXPERIMENTAL DESIGN

Completed correctly, an experimental study can provide an answer to the crucial questions : Did the

progrm make a difference? Are health behaviors, knowledge, an attitudes changed as a result of the

program activities? Is the community healthier because of the programs offered by the Rosemont

Health Promotion Council? however, the problem with experimental studies in program evaluation

is that they require selective implementation, meaning that people who participate are selected

through a process such as random assignment to a control group and an experimental group. for

many ethical, political, and community health reasons, selective implementation is difficult tocomplete and is sometimes impossible. despite these problems, the experimental study remains the

best method to evaluate summative effects (outcomes) of a program and the only way to produce

quantified information on whether the program made a difference.

The following designs are the most feasible and appropriate to health care settings. Apply the

research process the research process to each design.

PRETEST-POSTTEST ONE-GROUP DESIGN

The pretest-posttest design applied to one group is illustrated in Table 13-3. two observations are

made, the first at Time 1 and the scond at Time 2. The observation can be the prevalence of a health

state (eg, the percentage of adults in Rosemont who exercise regularly, the teenage pregnancy rate,

cases of child abuse and so on), knowledge scores or other important health facts in the community.

between Time 1 and Time 2, an experiment is introduced. The experiment may be a planned

program aimed at a target group, such as teen sexuality classes or with a community-wide focus, like

a crime-prevention program. The evaluation of the program is measured by considering the

difference between the health state at Time 1 and the health after the program at Time 2.

If the experiment in Table 13-3 was teen sexuality classes for 10th-grade girls at Hampton Highschool, Time 1 was a teen pregnancy rate of 5 per 100 and Time 2 (1 year later) was a teen

pregnancy rate of 3 per 100 among the girls taking classes, then would you agree that the teen

sexuality program was responsible for the decrease in teenage pregnancies? ( Are there other

factors that could account for the decrease in the teen pregnancy rate? Perhaps family-planning

programs have been focused on teenagers, or maybe local churches and social service agencies have

sponsored teen sexuality programs. Teen access and use of contraceptive methods may have

changed.) None of these factors can be eliminated as unassociated with the decrease in the teen

pregnancy rate. To eliminate othe possible explanations for program effectiveness, a control group

must be added.


3/4

PRETEST-POSTTEST TWO GROUP DESIGN

A pretest-posttest with a control group design is illustrated in table 13-4. The design has both an

experimental group and a control group. At Time 1, an observation is made of both the experimental

and control groups.

Table 13-3 .....PRETEST-POSTTEST ONE GROUP-DESIGN

TIME 1 TIME 2

Experimental group Observation 1 Experimental Observation 2

Table 13-4 .....PRETEST-POSTTEST TWO GROUP-DESIGN

TIME 1 TIME 2

Experimental group

Control group

Observation 1

Observation 1

Experimental Observation 2

Observation 2

Between Time 1 and Time 2, an experiment is introduced with the experimental group. At time 2,

second observations are made on both the experimental and control groups. Program evaluation is

the difference between Observation 1 and 2 for the experimental group when compared to the

comparison group ( which has been selected to be as similar as possible to the experimental group).

Will the pretest-posttest with a control group design eliminate the effect of outside factors that

occured simultaneously withh the experiment and that might account for the change between

Observation 1 and Observation 2, the very problem that plagued the pretest-postest one-group

design? The answer is yes, if the experimental and control groups are similar.

To explain, lets return to Rosemont and the idea of a teen sexuality class for 10th-grade students atHampton High School. If a group of 10th-grade students, similar in social, economic and geographic

characteristics, were randomly selected and then randomly assigned to the experimental or control

group then it could be assumed that any other factors that influenced the experimental group would

also affect the control group . However, frequently, the decision is made that all students must be

given the same program, thereby eliminating a comparison group. at the Rosemont Health

Promotion Council, when the information was received that all 10th graders must be given a teen

sexuality program that had been proposed by the school nurse as response to an increasing number

of teen pregnancies, the suggestion was made that perhaps another high school could be used as a

control group. How would you respond to that suggestion? Perhaps another high school class 10th

graders could be used, if thestudents were similar in social, economin and geographic characteristics

to the students at Hampton High School (an unlikely situation).

Another possibility mentioned by the Rosemont Health Promotion Council was to offer the program

in one school year to one half of the Hampton High 10th graders (using the other half as a control)

and then in the following year to offer the program to the remaining students. This method would

ensure that all students would be given the program but would also allow for an experimental

pretest-posttest design for evaluation.

A third method that was suggested to ensure an experimental design was to gove the control group

sexuality education plus assertiveness training. The assertiveness training would differentiate the


4/4

groups and allow an experimental design. All the suggestions were discussed with school officials

and it was decided to offer a training

Documents

Questions of Analysis Can Be Answered Through a Survey