ICT in Education Usability and Educational Review on Body PingPong and GameBlocks Corne Kruger and Antonet Bekker

ICT in EducationICT in Education

Usability and Educational Review on Body PingPong and GameBlocks

Corne Kruger and Antonet Bekker

AgendaAgenda

Why was this study done? What are usability and educational evaluations? How were the studies done? Results for Body PingPong:

– Usability– Educational

Results for GameBlocks– Usability– Educational

Conclusion

Why was this study done?Why was this study done?

Why the models were createdWhy the models were created

To stimulate interest amongst children for ICT and Science

To encourage children to be more physically active

Teach children that there are different ways to interact with a computer (not only mouse, keyboard and monitor)

Why TDS was involvedWhy TDS was involved

To study if these models were:

1. usable for the children

and

2. if it had any educational value.

What are usability and What are usability and educational evaluations?educational evaluations?

Usability evaluationUsability evaluation

Definition of usability

“ISO 9241-11:ISO 9241-11: the extent to which a product can

be used by specified users to achieve

specified goals with effectiveness, efficiency

and satisfaction in a specified context of

use.”

Intuitiveness !!

Usability evaluationUsability evaluation

Context of use for the product is determined

Participants are recruited

Environment and tasks to represent real-life scenario

Testing session is recorded and analysed

Ideal usability testing Users and tasks

determined. Environment unsure

No control over participants

No real-life scenarios to use as benchmark

Testing session is recorded and analysed

Meraka usability testing

Educational evaluationEducational evaluation

New skills and knowledge to learn Learn by playing and exploring Pre-defined educational goals How the children will be exposed – the

process Could the child achieve the skills and

learning

How were the studies done?How were the studies done?

Usability and EducationalUsability and EducationalModels were presented at 2 Science Fairs

(Pretoria and Grahamstown)Children (at random) attend these sessionsBriefing session and demoChildren play on the modelPlay was video recorded and notes made by

testers. Interviews and questions to childrenAnalysis Reporting

Results for Body PingPongResults for Body PingPong

UsabilityUsability

Overall design was not intuitive – children could not figure out what the game was about

1. Effectiveness

UsabilityUsability

Hand-held controls:– Design: orientation confusing and uncomfortable

to handle– Functionality:

• Poor learnability, not intuitive• Positions of the 5 sensors did not correspond with

movement of paddle• Space between sensors• Move magnet in opposite than desired direction• No consistency in paddle movement (glide vs. jump)

1. Effectiveness

A B C D E

UsabilityUsability

5 min to get basic understanding – mainly because of coaching, not intuitiveness

Errors still made up to last group (e.g. move paddle in wrong direction)

No of errors reduced with every round – learnt from mistakes made in previous rounds and coaching

2. Efficiency

UsabilityUsability

Positive:– Excitement from players and audience– New challenging experience

Negative:– No “game-over” status– No consistency between look of design

and how game reacts

3. Satisfaction

UsabilityUsability

Frame more sturdyController:

– Reduce length– One continuous sensor area– “Attach” magnet to plate– “L” and “R” symbols to:

• Know in which direction to move magnet• Know which side of plate faces forward

Recommendations

L R

EducationalEducational

1. Educational outcome to achieveThe child must be able to : manipulate the movement of the paddle

with the controller use the magnet to choose five different

positions of the paddle manoeuvre the ball to hit the target Looking at the floor while playing Body

PingPong instead of looking vertically at a PC-screen


2. Process Minimum explanation Game will be active - children can hear the

sounds, reaction of the paddles


2. Process (continue) Moving the magnet over the sensors - realizing that

paddle reacts and that he can manipulate the paddle

Moving the magnet over the plate to move the paddle into 5 various positions

Controlling the ball with the controller and manage to hit the target in the middle of the screen - colour of the target changes , score increase

Playing the game on the floor and looking down for the whole duration of the game


3.1 Educational outcome (Manipulate the paddle)

Struggled to discover how to manipulate the movement of the paddle

On request – could move the paddle into a specific position

Able to watch the ball and then decide which direction the paddle must move to

Some children chose wrong direction


3.2 Educational outcome (Choose five different positions )

Difference between sensors unclear Middle sensor – place paddle in middle of

screen Two buttons on the left – move paddle to left Two buttons on the right - move paddle to

right One boy said - difference is that paddle

moves at different speeds


3.3 Educational outcome

(manoeuvre the ball to hit target) Took a while for the children to figure out

the function of the target Realised the colour change of the target



(Playing the game on the floor) No difficulty to look at the floor Some struggled to hold the plate One girl held the plate like a guitar


4. Conclusion

The children could successfully achieve : The handling of the plate together with the magnet Manage to hit the target in the middle of the screen Looking down to the game – don’t know about the

comfort

The children could not achieve: Positioning the paddle in five different positions


5. Recommendations One person must be able to play on his own Two persons to compete - positioned opposite each other Four persons playing as two teams Four persons playing as four individuals No closure of a game - time-out or certain amount of balls Initial idea – use arms and elbows - in line with main objective

to get the children more physical The ball must be a round shape Sides look the same e.g. a frame around the screen and on

each side the scores are displayed so that it is legible from any direction.

Results for GameBlocksResults for GameBlocks

UsabilityUsability

Overall design was not intuitive – children could not figure out what the game was about.

Symbols were not intuitive Could not understand correlation between

direction of symbol and direction robot moves Robot did not always act as expected

1. Effectiveness

UsabilityUsability

First child: 15 min to learn game and complete the task.

Most time was spent on learning and making errors

Second and third group took 6 minutes to complete task – due to learning from first child.

A lot of coaching from facilitator

2. Efficiency

UsabilityUsability

Positive:– Enjoyed the challenge – Enjoyed controlling a robot

Negative:– Could not hear what robot said– Cubes too big– Robot must move in direction of arrow, not relative

to current position– Robot movement not visible enough

3. Satisfaction

UsabilityUsability

More descriptive and less abstract

Recommendations - Symbols

Forward Back Right LeftForwardForwardForward BackBackBack RightRightRight LeftLeftLeft

UsabilityUsability

Show bottom, top, front and side of cube clearlyOlder children: smaller cubes

Recommendations - Cubes

ForwardForwardForward


1. Educational outcome to achieve Main idea - teach the learners the basics of programming

– identify the instructions– write down (pseudo code)– translate into programming language– execute the steps (results)

The child must be able to : – put valid instructions in a specific sequence– translate the written instructions into the symbols on the

cubes– place the cubes in the correct position on the correct tray– understand how the cubes communicate with the robot


2. Process Selecting instructions from the wall - put it in

sequential order Choosing a cube that simulates the correct

command Positioning of the cubes in the correct position on

the correct tray Asking questions, challenge them - need to

discover that the cubes communicate with the robot via infrared



(put instructions in specific sequence)

Without exception the children were able to order the instructions


3.2 Educational outcome (translate instructions into the symbols on

the cubes ) Without any information - children were not sure what to do First boy - fiddling with the grid and wires. Didn’t even look at

the cubes Instructions were confusing – positioning Instructor had to explain in detail to the children how the

positioning works By the end of the session - children were able to interpret the

positioning correctly By end of the session – were able to translate the written

instructions into the symbols on the cubes


3.3 Educational outcome (place the cubes in correct position, correct tray)

First child did not realize specific sequence for the cubes

He randomly wanted to place the cubes on the tray. After explanation - children were able to correctly

place the cubes in correct position on the trays.


3.3 Educational outcome (communication between cubes and robot)

Children mentioned different ways of communication: magnetic communication , antennas , receivers , radio waves , lasers , electricity and impulse waves

Could understand that it is wireless With guidance understood that it is infrared


4. Conclusion

The children could successfully achieve : ordering of the instructions sequentially. interpretation of the symbols on the cubes and the relation to

the written commands

The children could partially achieve: Correct positioning of the cubes and to put it on the trays in

the correct sequence With guidance and explanations they were able to identify the

communication between the cubes and the robot


5. Recommendations Jackie must talk more clearly. Jackie must not be able to time out automatically. The instructions on the cubes must be more clear

and understandable. The sensors inside the cubes were loose. Need to

fasten it in a better way or even maybe redesign it.

ConclusionConclusion

No frame of reference or past experiences:– Negative: uncertainty and prolonged learning process – Positive: excitement for players and audience

The facilitators played too big a role in assisting the participants to learn the games

The learning therefore did not take place as a result of self confident exploration or the intuitiveness of the designs.

The children had to spend too much time listing to instructions and learning, rather than playing and having fun

The lessons learnt during this first cycle of development were of great value, and should be implemented in future development cycles.

Documents

ICT in Education Usability and Educational Review on Body PingPong and GameBlocks Corne Kruger and Antonet Bekker