Adventure Based Training and Its Adventure Based Training and Its Impact on the Team Cohesion and Impact on the Team Cohesion and
Psychological Skills Development of Psychological Skills Development of Elite Netball PlayersElite Netball Players
Where are we headed with Where are we headed with this presentation?this presentation?
A report on my doctoral thesis 2002!– Rationale– A pictorial of “The Journey To The Nationals”– Methodology: Strong Isomorphism– Findings of Mixed Mode Investigation– Implications for practice & our Industry– Questions
What Can This Presentation What Can This Presentation Offer Our Industry?Offer Our Industry?
Feedback from our clients. We must ensure we are taking them in the direction they need and want to go?
The need for strong Isomorphism in our programs. (Ensuring the program is tailored for the clients needs and mirrors their reality)
A learning tool for new facilitators
The Connection Between Sport The Connection Between Sport Psychology and AdventurePsychology and Adventure
My own journey!
The Connection Between Sport The Connection Between Sport Psychology and AdventurePsychology and Adventure
Overcoming self doubt and fears
Blocking out distractions and focussing on task
Mental skills learnt during during my outdoor adventures Mental skills learnt during during my outdoor adventures transferred to my sport and life!transferred to my sport and life!
The question I often pondered was could I take others into the outdoors and teach them
mental skills to help their sporting endeavours?
Motivation for the StudyMotivation for the StudyInitial concerns for psychology skills training
““It was boring, it put us to sleep”It was boring, it put us to sleep” ““We just sat there and listened it was just We just sat there and listened it was just
like school”like school”Sport psychology’s lack of support for my
ideas ““Why take athletes away from their Why take athletes away from their
normal training venue?”normal training venue?”Dissatisfaction with previous adventure
based training interventions from sport coaches.
1. Athletes who received an adventure-based training program intervention, would show increased team cohesion when compared to a control group.
2. The duration of the intervention will see longitudinal improvements in all four sub-scales of team cohesion, when compared to a control group.
Research Questions?Research Questions?Quantitative HypothesesQuantitative Hypotheses
Qualitative Outcome QuestionsQualitative Outcome Questions From an athletes’ or coaches’ perspective, what were the major
outcomes of the adventure-based training program; and how did they impact most upon the team in the following areas:
– What new skills or knowledge about themselves or other teammates did individuals take away with them from the adventure-based training camp?
– How did the team or individuals within the team change as a result of their adventure experience? What new skills were developed that helped the team?
– Was there any direct evidence that psychological skills learnt during the adventure training camp were directly transferable to netball training or competition?
Qualitative Process QuestionsQualitative Process Questions From an athletes’ or coaches’ perspective, what
processes during the adventure-based training weekend had the most impact on the team?
– What elements of the training intervention had the most impact on athletes? Why was this significant for these athletes?
– How did this camp differ (if at all), from previous adventure-based training camps, which the athletes had been on in the past?
– How did the outdoor bush environment impact on the program? Was it an advantage or a disadvantage traveling away from their usual training venues?
NSW Netball: Needs AnalysisNSW Netball: Needs Analysis
A bunch of individually talented athletes with the ability to win the nationals but failed the previous year.
Cliques and negative behaviour toward each other on and off the court, was impacting on the team.
No mental toughness during the pressure of competition.
Lack of mental skills to handle this pressure
In order to address these issues the intervention was structured to ensure……
STRONG ISOMORPHISM STRONG ISOMORPHISM ((Priest & Gass, 1997)Priest & Gass, 1997)
The adventure experience The adventure experience had to had to mirror mirror thethe clients needs! clients needs!
Assessed, identified & ranked client goals [coach meetings] Created a metaphoric experience that possessed a strong isomorphic
relationship to playing as team at the national championships. Provided opportunities to explore how resolution during the adventure
would correspond to life at netball. Strengthened the isomorphic framework by using client language that
mirrored their reality as netball players. Constant review of activity selection to ensure clients were motivated.
(If it does not move group towards your program goals then don’t do it). Conduct the experience with revisions. Look out for that teachable
moment; the program is not static. Debrief focussed on how behaviours learnt would help or hinder netball
training or performance.
STEPS STEPS THAT GUIDED THE INTERVENTIONTHAT GUIDED THE INTERVENTION ISOMORPHISM ISOMORPHISM ((Priest & Gass, 1997Priest & Gass, 1997))
The Intervention The Intervention
Sequencing of activities: sequencing of activities can improve team cohesion of groups! (Bisson, 1997)
– Group formation activities (Forming) Ice breakers. Categories, Have you ever (Ronke, 1984)
– Group challenge activities (Storming) Games requiring the team to function together
– Group support activities (Norming) Climbing, abseiling, giant swing
– Group achievement activities (Performing) Bush walking, team campout, group swim, caving
Journey to the NationalsJourney to the NationalsFrom Storming to PerformingFrom Storming to Performing
Friday NightFriday Night
Psychological skills
development
Goal setting
Awareness of ideal state
Self-Monitoring
Self-Regulation
AN ATHLETE’S PERSPECTIVE AN ATHLETE’S PERSPECTIVE OF THE INTERVENTIONOF THE INTERVENTION
Hi! I am the leg of one of the under 17 NSW netball players. Let me share with you part of the journey that I had to endure through the
bush, cliffs, rivers and caves of Kangaroo Valley!
It taught me a lot.
We were getting wet in the first 5 minutes, but it seemed easy. Everyone was saying it
would be a piece of cake
First up was the mother of all bush
walks that lasted over 12 hours
Things soon changed!
All us legs had to begin to work
together!
We began to have to
think as a team
I am glad we are all helping each
other through the difficult sections!
NOT!
What is wrong with this picture? Are we working like a team of netballers? Or, are we individuals who are in the same netball team that don’t care
about anyone else?
Over a break we talked about how
we could overcome the challenges the
river posed!
The ideas were similar to what
we needed to do on the netball
court
All us legs were beginning to step in time!
A Focus On Personal Mental SkillsCentre, focus on yourself, don’t think about the things out of your control.
Bush WalkBush Walk
Metaphor of a journey to nationals.
“Through the physical & psychological challenges of a netball nationals campaign a team must stick together. During this section of the walk today you must work together all the way to the top. Don’t leave the slower people behind”.
Setting up a double bind!Setting up a double bind!
They’re too slow, lets just go off ahead!
Debrief: We had two choices today. To work as a team or to carry on our path of self destruction
where we play one out and forget about our team-mates. What kind of behaviour do you
want your team to have at the national championships?
Oh no, not more scratches! This hurts. We legs
were really complaining
now.
Remember team rule #1, No Complaining!
Just stay focused on the task. Don’t waste energies on
distractions that will take us away from our goals
Once at the top it gave us an opportunity however to sit and reflect on who we
were and where we were headed
By the time we got to the top of the
mountain we were aching and tired!
“Why did we have to come all the way up here to sort this out?”
We began the walk like a team of
individuals, but we started to pull
together near the top
A group in conflict with clique behaviour damaging
performance!A question for the audience to see if you
are still awake!
Used in this session were:Mini Solo Reflection Time
Questions were given to ponder; Strengths & weaknesses of team?
Sharing circle used to share ideasFacilitated problem solving session
After hiking to the top of this mountain, how would you utilise this location &
situation to address some of the negative team issues that were distracting this team
from their goals?
The power of the wilderness to heal problems!
This session was the beginning of us
coming together as a team
Here I am, you can see the big cut on my leg!
After a big day out we still were not off the mountain, instead of complaining we
finally worked together
No cabins, no tents Only a big tarp to sleep under. We
thought we would be sleeping in cabins! Showers from a billy. “We stank so bad!” Then we had to cook & clean. By the time we got to bed it was
midnight. This was all about learning to deal with
the unexpected situations that may occur in sport and being ready to deal with them!
Upon Return To Camp Upon Return To Camp The Pressure ContinuedThe Pressure Continued
5.30 AM The Next Morning As 5.30 AM The Next Morning As Twilight Filtered Through The Twilight Filtered Through The Morning MistMorning Mist
5.30am! Well that is the best alarm clock I’ve ever had.. NOT!
I don’t work at this time
of the day
The bagpipe alarm clockThe bagpipe alarm clock sounded sounded
This is crazy
Block out the negatives. If you’re
tired at Nationals you still have to play no matter how you are
feeling.
Okay team lets swim across, we can do
this together
Meditation
Visualisation
Learning to prepare mind and body for
competition
After the swim we had to sit for an hour preparing our minds for competition!
We then headed out for a day of
caving!
Its dark like when we
play at night
I feel so nervous like before a big
gameI feel like I do when I have to
sink a goal under pressure. My heart is racing
For me, I have come to the realisation that I will not be able to get out of this hole [cave] on my own, we will
have to do it as a team
It reminds of the night before a
big game, I feel really anxious
Can anyone please tell me what the hell going caving has
to do with playing netball?
I’m not sure but I think it goes somewhere
“You are kidding me aren’t you, there is no way we can fit through
that space”
You let go and I will kill you
Don’t worry I will support you
Ahhhhhhhhhhhhhh!
Its okay we have got you!The coaching staff were in position to
catch & support each player through the laundry shute!
I can’t believe I am doing
this
So in our earlier exploration to find our way down here you all
felt it was impossible to get through the squeeze. This was
with lights on……
No way, it is impossible to get out of here with out lights on! What? We have to hand our headlamps
in????
Rather than exhausting our energies on what is not possible, what would it look like if we all concentrated our energies on how we can work as a
team to get out of here?
What you just described to me then, sounds exactly like the skills a team would need to win a national netball championship. How
about we give it a try?
The whole weekend built a
bond that was the foundation from which we won
the national championships.
After this weekend nothing in our lives seemed impossible!
We are behind and we’re in a hole. Remember how we worked together to get out of that cave. We can
do the same here. “1, 2, 3, out of this hole”.
10 Seconds
to go
My body has
frozen
My heart is racing I feel like
when I was caving
Centre, breathe, relax,
visualise
Focus on the things
I can control
The goal goes in to
tie the match
The Isomorphic Metaphor!The Isomorphic Metaphor!The connection between netball & adventureThe connection between netball & adventure
We are a goal
behind
I can hear the crowd screaming
We have to either tie or win to go into the grand
final
Controlling the stress of caving Controlling the stress of caving transferred transferred to to netballnetball
Breathe deep, exhale, feel your body relax, focus on you!
These same skills are exactly what you need to do to maintain control and
concentration during a netball game
It does not matter if we are on the bench we are still
part of the team
TeamworkTeamwork
“Never doubt that a small group of thoughtful, committed people can change the world. Indeed, it is the only thing that ever has” Margaret Mead.
NSW NETBALL: AUSTRALIAN CHAMPIONS 1999NSW NETBALL: AUSTRALIAN CHAMPIONS 1999
I hope you like the guided tour I
gave you!
Any Questions Related To Any Questions Related To What We Have Just Seen?What We Have Just Seen?
MethodologyMethodology
Recommendations & processes that guided this Recommendations & processes that guided this research projectresearch project
Recommendations From Recommendations From Existing ResearchExisting Research
Neill (1988) lack of description from past studies– More info on instruction and facilitation. [Will be available
from Amazon.com in a paper back book by the end of the year ]
Ewert (1982) what’s happening inside the black box? It is no longer a mystery! Why & how did change occur?
Martens (1987) positivists stifling sport psychology research through the removal of the person from the process of knowing.
Dale (1996) existential phenomenology and sport psychology. Emphasizing the experience of the athletes.
MethodologyMethodologyOverview of quasi-experimental mixed Overview of quasi-experimental mixed
mode research designmode research designTreatment Group n= 23Treatment Group n= 23Control Group n= 11Control Group n= 11
Data Collection
0
25
Time 1(Week prior))
Time 2 (Endof W'end)
Time 3 (6weeks Post)
Time 4(Interviews)
PostNationals
Data Collection Times
Time LineIntervention
InstrumentationInstrumentation
The Group Environment QuestionnairThe Group Environment Questionnaire Was e Was Administered At 4 Time MeasuresAdministered At 4 Time Measures
(Carron, Brawley, Widmeyer, 1985)
Measures Group Cohesion on 4 sub scales– Individual attraction to group on task– Individual attraction to group Social– Group integration on task– Group integration social
Quantitative Survey ResultsQuantitative Survey Results
(A): Analysis of variance tests were conducted across all three groups for each time-period.
(B): Effect size testing examined changes within each group for times 1-2, 2-3, 3-4. TIME
4321
Means
Sco
res
36
35
34
33
32
31
30
GROUP
Control Group
U/19 Team
U/17 Team
A
B
TIME
4321
Me
an
s S
core
s36
35
34
33
32
31
30
GROUP
Control Group
U/19 Team
U/17 Team
Repeated measures analysis showing interaction between groups and time on the ATG-T sub-scale.
Individual Attraction To Group On TaskIndividual Attraction To Group On Task ATG-T Subscale
Lines considered not to be parallel thus interaction occurred
Individual team member’s feelings about their personal involvement with the group task, productivity, goals and objectives
Effect Size Change ATG-T SubscaleEffect Size Change ATG-T Subscale
Comparison of effect size change for three groups across time for ATG-T subscale.
Effect Size Description
Below .1 No detectable change
.1 to .2 Small Change
.2 to .3 Small-Med Change
.3 to .4 Med Change (Ave OE)
.4 to .5 Med Large Change
.5 to .6 Large Change
Above .6 Particularly large change
Individual Attraction to Group SocialIndividual Attraction to Group Social ATG-S SubscaleATG-S Subscale
TIME
4321
Me
an
Sco
res
43
42
41
40
39
38
37
36
GROUP
Control Group
U/19 Team
U/17 Team
Repeated measures analysis showing interaction between groups and time on the ATG-S sub-scale.
Lines considered not to be parallel thus interaction occurred
Individual team member’s feelings about their personal involvement, acceptance and social interaction within the group.
Effect Size Change ATG-S SubscaleEffect Size Change ATG-S Subscale
Comparison of effect size change for three groups across time for ATG-S subscale.
Effect Size Description
Below .1 No detectable change
.1 to .2 Small Change
.2 to .3 Small-Med Change
.3 to .4 Med Change (Ave OE)
.4 to .5 Med Large Change
.5 to .6 Large Change
Above .6 Particularly large change
Group Integration on TaskGroup Integration on Task GI-T SubscaleGI-T Subscale
TIME
4321
Me
an
Sco
res
46
44
42
40
38
36
GROUP
Control Group
U/19 Team
U/17 Team
Repeated measures analysis showing interaction between groups and time on the GI-T sub-scale.
Lines considered not to be parallel thus interaction occurred
Individual team member’s feelings about the similarity, closeness, and bonding within the team as whole around the groups task.
Effect Size Change GI-T SubscaleEffect Size Change GI-T Subscale
Comparison of effect size change for three groups across time for GI-T subscale.
Effect Size Description
Below .1 No detectable change
.1 to .2 Small Change
.2 to .3 Small-Med Change
.3 to .4 Med Change (Ave OE)
.4 to .5 Med Large Change
.5 to .6 Large Change
Above .6 Particularly large change
Group Integration SocialGroup Integration Social GI-S SubscaleGI-S Subscale
TIME
4321
Me
an
Sco
res
28
27
26
25
24
23
22
21
GROUP
Control Group
U/19 Team
U/17 Team
Repeated measures analysis showing interaction between groups and time on the GI-S sub-scale
Lines considered to be parallel thus no interaction
Individual team member’s feelings about the similarity, closeness, and bonding within the team as a whole around the group as a social unit
Qualitative ResultsQualitative ResultsExistential PhenomenologyExistential Phenomenology What was the athlete’s story? Dale (1996) recommended What was the athlete’s story? Dale (1996) recommended
hearing from those involved in the research as to whether hearing from those involved in the research as to whether the intervention was beneficial to them as individuals or as the intervention was beneficial to them as individuals or as a team. a team. – Was the learning transferable to Netball?Was the learning transferable to Netball?– Did it help in netball competition?Did it help in netball competition?
The participants told their story!The participants told their story!
I believe the athletes story has a very strong I believe the athletes story has a very strong message for our industry and how we conduct message for our industry and how we conduct programs!programs!
Qu
alit
ativ
e O
utc
omes
Qu
alit
ativ
e O
utc
omes
Group-Cohesion
Improved on court performance
Change outside of netball
Pro
cess
Facto
rs T
hat
Pro
cess
Facto
rs T
hat
Help
ed
Ach
ieve
H
elp
ed
Ach
ieve
Ou
tcom
es
Ou
tcom
es
Unfreezing
Moving
Refreezing
Lewin’s Lewin’s Change Change TheoryTheory
4 Years Later4 Years Later
Many of the athletes that participated in the intervention are now playing in the National Netball League with the Sydney Sandpipers. The Under 19 Coach in now the coach of this team.
Talking to her last week, she was telling me how the adventure experience from this intervention is still having a powerful impact on the athletes.
Maybe an area for follow up research!?
Implications for practiceImplications for practice Ensure programming addresses the needs of the client. Include clients in the
planning phase. Facilitate the program at a level appropriate for desired outcomes.
Sequence activities to suit stages of group development (Bisson, 1997). Activities need strong isomorphism!Activities need strong isomorphism!
The correct style of facilitation and metaphor development for the clients needs can greatly enhance the experience. Greater emphasis on training facilitators in our own programs and the tertiary sector can help ensure quality programming.
A “Change Agent” that can teach, inspire, motivate, and lead a group to a new level of personal performance is critical in the adventure process with athletic teams [and all clients].
Sport coaches and sport psychologists should consider utilising experiential teaching methods in their sports.
?Questions and Comments??Questions and Comments?
I can do it!
[email protected] more Information contact:
THE ENDTHE END
[Under 19 Coach]: “At one point in the cave, I was asked to spot the girls as they came out that vertical hole. At that time all my instincts were telling me to get out of the cave, I felt like I could not breathe and I was losing control, and panicking. However, I focused, and realised that to learn to breathe, centre, relax, and gain control for the better of the team was more important.”
Putting the Team Before
Self
[Under 19 player] “I felt we were pushed both physically and mentally to the edge of what I could take when caving. I just wanted to get out of that cave and end it, but you couldn’t do a thing, you were stuck. Instead of freaking out you had to stay in control to help those around you so we all could get out. You had to trust yourself to get through it.”
Developing team-work in the face of extreme stress and anxiety.
[Under 19 player] “Our talk on the mountain made me feel that people would listen to me. I could share my feelings honestly, this helped me trust everyone. Up unto this point I felt like I was not part of the team, I did not connect with many in the team and felt like no one was interested in my feelings or what I had to offer. It was good to see we finally found a way to talk about the problems in the team.”
Improved Relationships
Decreased Dysfunction
[Extract from phenomenological interview with Under 19 player] “The game was so close and we were behind with several minutes to go. During a time out we had talked about how we had to really lift if we were going to narrow the other teams lead. The rest of the team just hustled, they chased down everything, and turned this into attack feeding me the ball. We were one goal down with only about ten seconds to go, and the other team had the ball down the other end of the court attacking. Marion however intercepted a pass and threw this giant bomb right at me, I caught it with only a few seconds left on the clock. My heart was just pounding; it felt like it was trying to jump out of my chest. My muscles felt heavy, like I could not control them, and I was all light headed. I had a flash back of lying in that squeeze section of the cave talking to you, the feelings I was having were the same, however this time I immediately knew what I had to do to gain control. I briefly closed my eyes, took a breath, and told myself to relax. I felt this unbelievable sense of calm. I saw myself using good shooting technique and the ball going into the net. I opened my eyes, and took the shot; it went in. We tied the match. We knew this was enough to get us into the final the next day.”
Transfer from the Adventure Transfer from the Adventure Environment to Netball was Environment to Netball was Clearly Evident in the DataClearly Evident in the Data
Improved Social Relations = Improved Social Relations = Platform for Improved Platform for Improved PerformancePerformance
[Under 17 coach] “To get away from the netball environment allowed the team to get to know each other in another dimension as people and not just netballers.”
Under 17 player] “Waking up at sunrise to the bagpipes and swimming across the lake, made us feel like we were all doing these hard things together, no one complained, we just got in and did it for each other.”
[Under 19 player] “During the caving, the coaches were crying and scared, they went through what we did. Instead of a “them and us” feeling between the coaches and players, we all felt as one, working toward the same goal.”
[Under 17 player] “Once we got out of the caves, we hugged her (the coach). After this we could talk to her, it made her more approachable and easier to talk to. We seemed to really trust her decisions and coaching much more.”
Team Cohesion: CavingTeam Cohesion: Caving to the Court to the Court
[Under 19 Team Captain] “I did not go into the cave with the girls because of my ankle, but I was there when they came out, it was amazing to see the emotion and relief when they made it out. Whatever occurred in that cave it helped us win the nationals without a doubt. The team came up with this call “out of this hole”, for me, I thought it was a bit silly to begin with as I could not relate to what it was supposed to mean. The others told me about it, and how they thought they were going to die, but because they all stuck together, they got out of the cave. Whenever we were in trouble we would use this to our advantage, and everyone would just put the extra in to get us over the obstacle. I can still vividly feel the power and enthusiasm of the others, it made a difference to the way we played.”
Transfer Of Skills Can & Does Occur
Changes Outside NetballChanges Outside Netball
[Under 17 coach] “During caving, I wanted to turn the light on when we were trying to get out of the cave with lights out. I was so scared. I felt I had no control whatsoever over my body, my thinking, I could not get it together. I got more out of this training than just things to do with netball. I was so emotional for days after this weekend. I would just keep breaking down in tears. I am someone who has always been able to do anything I tried, needing no help from others. Outside netball, I always try to take too much on. To get out of the cave I had to let go of the control I usually have, and rely on others to help me. This has taught me to ask for help, and that asking for help is okay.”
[Under 19 player] “Not only did the weekend teach us how to be good netballers, it taught us how to be the best at life.”
The Importance Of Facilitating To The Importance Of Facilitating To Client NeedsClient Needs
[Under 19 player] “The camp last year was just like fun games. However, this years camp we were just thrust into the thick of it. There were no easy options, like when we had to climb to the top of the mountain, we then had to remain together to get down again, otherwise we would not have got out. Last year, you were not put into a situation where you had to push yourself.”
[Under 19 player] “The first camp we went on, the leaders used all the buzz words about being a team, then they had us do low or high rope activities, like all stand on a log together and we were supposed to be a team. We didn’t have to push each other. This years camp was individual and group focused and we were all pushed hard, and we had to work together to get through it.”
[Under 19 Coach] “The adventure camp last year was at a superficial level, and I felt this is where our team was as well. This year was REAL, and it allowed us to get down to the deeper issues that I knew were affecting the team, and holding back performance on the court. I knew if we could solve the problems we would begin to see the talent, the girls really had.”
Wilderness As A Healing PlaceWilderness As A Healing Place
[Under 19 player] “The outdoors provided an environment where everyone was equal. The coaches, new and old players were all the same; you could say what you wanted to anyone without feeling threatened.”
[Under 19 player] “The outdoor environment provided an atmosphere to air grievances, which we had not been able to do back in Sydney.”
[Under 17 player] “Being out in the bush was new, and provided a safe place for us to communicate.”
[Under 19 player] “Why did we have to come all the way up here to sort this out? Couldn’t this of happened in Sydney?”
ReferencesReferencesBisson, C. (1997). The effects of varying the sequence of categories of adventure activities on the development of group cohesion. Unpublished doctoral dissertation. University of Northern Colorado. Greeley.
Priest, S. & Gass, M. (1997) Effective leadership in adventure programming. Human Kinetics. Champaign. Ill. USA (Back)
Rohnke, K. (1995). Quicksilver. Kendall Hunt Publishing. Rohnke, K. (1989). Cowstails and Cobras II. Kendall Hunt Publishing. Rohnke, K. (1984). Silver Bullets. Project Adventure Inc. Publishers.
Back to Friday Night
GEQ Sub Scale ExplanationGEQ Sub Scale Explanation Individual Attraction to Group – Task: [ATG-T] Individual
team member’s feelings about their personal involvement with the group task, productivity, goals and objectives. Eg: “I do not like the style of play of this team.”
Individual Attraction to Group – Social: [ATG-S] Individual team member’s feelings about their personal involvement, acceptance and social interaction within the group. EG: “some of my best friends are on this team.”
Group Integration – TaskGroup Integration – Task: [GI-T] Individual team member’s feelings about the similarity, closeness, and bonding within the team as whole around the groups task. EG: “Our team is united in trying to reach its goals for performance.”
Group Integration – SocialGroup Integration – Social: [GI-S] Individual team member’s feelings about the similarity, closeness, and bonding within the team as a whole around the group as a social unit. EG: “Members of our team do not stick to together outside of practices and games.”
Quantitative Data AnalysisQuantitative Data Analysis
Cohen (1994), significance testing “has not only failed to support the advance of psychology as a science but also has seriously impeded it”
Significance testing can lead to false conclusions, ie) type I or type II errors
– Type I errors falsely conclude an effect when there is none. Controlled by .05 error rate
– Type II errors falsely conclude no effect when there is an effect. Controlled by increasing sample size or increasing critical alpha to .10 or greater.
Traditional Significance TestingTraditional Significance Testing (Neill in press)(Neill in press)
Traditional Significance TestingTraditional Significance Testing 35% of outdoor ed studies may have reported false
conclusions through type II errors (Hattie, Marsh, Neill, & Richards, 1997)
Significance testing was designed to give binary answers, ie) either yes or no
Significance testing only tells whether an effect has occurred, but does not tell whether one program was more or less effective than another
Outdoor ed research could benefit more from understanding how much change occurred. (Neill, in press)
Effect SizeEffect Size
Effect size measures how much change (or difference) there is between two sets of scores
• A negative ES score indicates a reduced score
• A Score of 0 indicates no change
• A positive ES score indicates an increase in score
% Effect Size %% Effect Size %
Effect Size Description
Below .1 No detectable change
.1 to .2 Small Change
.2 to .3 Small-Med Change
.3 to .4 Med Change (Ave OE)
.4 to .5 Med Large Change
.5 to .6 Large Change
Above .6 Particularly large change
Effect Size DefinitionEffect Size Definition
The major new statistical tool, which is recommended for adoption by outdoor education researchers and evaluators, is called an “effect size” (ES). The ES measures “how much” change or difference there is between two sets of scores.
In mathematical terms, an ES is the average difference between two sets of scores in standard deviation units. An ES measures, for example, the standardized change in raw scores between the beginning and end of an outdoor education program.
Neill, (1999).