Pex4Fun: A Web-Based Environment for Educational Gaming via Automated Test Generation

Tao XieNorth Carolina State University

Nikolai Tillmann Jonathan de HalleuxMicrosoft Research

Judith Bishop Microsoft Research

secret

http://pex4fun.com

Testing Tool Educational Gaming

Dynamic Symbolic Execution (Pex)

secret

Pex for Fun: Interactive Gaming for Teaching and Learning

Support

http://research.microsoft.com/pex/ http://pex4fun.com

http://research.microsoft.com/pex/

Dynamic Symbolic Execution (DSE) aka. Concolic Testing [Godefroid et al. 05][Sen et al. 05][Tillmann et al. 08]

Instrument code to explore feasible paths

Background

http://research.microsoft.com/pex/

Coding Duels

1,378,742 clicked 'Ask Pex!'

Example User Feedback

“It really got me *excited*. The part that got me most is about spreading interest in teaching CS: I do think that it’s REALLY great for teaching | learning!”

“I used to love the first person shooters and the satisfaction of blowing away a whole team of Noobies playing Rainbow Six, but this is far more fun.”

“I’m afraid I’ll have to constrain myself to spend just an hour or so a day on this really exciting stuff, as I’m really stuffed with work.”

Released since 2010

X

Coding Duels

Pex computes “semantic diff” in cloudsecret reference implementation vs.code written in browserYou win when Pex finds no differences

secret

Behind the Scene of Pex for Fun

Secret Implementation

class Secret { public static int Puzzle(int x) { if (x <= 0) return 1; return x * Puzzle(x-1); }}

Player Implementation

class Player { public static int Puzzle(int x) { return x; }}

class Test {public static void Driver(int x) { if (Secret.Puzzle(x) != Player.Puzzle(x)) throw new Exception(“Mismatch”); }}

behaviorSecret Impl == Player Impl

8

Coding DuelsFun and Engaging

Iterative gameplayAdaptivePersonalizedNo cheatingClear winning criterion

Teaching and Learning

Coding Duel Competition @ICSE 2011

http://pexforfun.com/icse2011

Outlook: redesign as game

Usage Scenarios of Pex4Fun

• Massive Open Online Courses (MOOC)

• Course assignments (students/professionals)

• Student/professional competitions

• Assessment of testing, programming, problem solving skills for job applicants

Summary: Testing Tool Educational Gaming

DSE/Pex

secret

Pex for Fun:

Interactive Gaming for

Teaching and Learning

Support

http://research.microsoft.com/pex/ http://pex4fun.com

Coding Duels for Course Assignments

@Grad Software Engineering Course

http://pexforfun.com/gradsofteng

Observed Benefits• Automatic Grading• Real-time Feedback (for Both Students and Teachers)• Fun Learning Experiences

Skills Being Trained

InductionProblem solving/debuggingProgram understanding/programmingTestingSpecification writing