Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
Crowdsourced Conformance
Testing via Remote
Sandboxing
Joe Gibbs Politz
http://www.batchgeo.com
“Remote Sandboxing?”
1600 times?
grade.rktStudents' interpreters
Students' interpreters grade.rkt
#lang racket/base
(require 'interp.rkt')
(command-line“--test” test-dir(run-tests interp test-dir))
#lang racket/base
(require 'interp.rkt')
(command-line“--test” test-dir(run-tests interp test-dir) “--report”(sign (run-tests interp our-tests) “sooper-seekret”))
#lang racket/base
(define interp (dynamic-require “interp.rkt” 'interp))
(command-line“--test” test-dir(run-tests interp test-dir) “--report”(sign (run-tests interp our-tests) “sooper-seekret”))
000101010001010111000111010111
000101010001010111000111010111
000101010001010111000111010111
000101010001010111000111010111
Students' interpreters grade.bin
Remote Sandboxing
“Crowdsouced Conformance Testing”?
correct-interp
for-update-after-test
plus-coerces-strings
arity-before-evaluate
.
.
.
• tests/• loop-tests/
• for-test1.psl• for-test2.psl
• operator-tests/• subtract-test1.psl• subtract-test2.psl• add-test1.psl
• function-tests/• apply1.psl• apply2.psl• closure1.psl• closure2.psl
discard-if-effects
✓✓✓
ParselTongue Testing Assignment
> 10,000 tests from 250 people
That's as many tests as JavaScript'sconformance suite.
Crowdsourced Conformance Testing
Lessons
● Racket's dynamic code support is awesome, and Racket's build tools for distribution just work
● Value proposition for teaching PL online – get crowdsourced tools and tests