Embed Size (px)
Citation preview
NYC Open Data, March 18, 2015
Chaitanya Ekanadham
Representing
learning
experiences
personalize learning experiences
for students around the world
PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC.
Knewton’s mission
content inventory
student goals
interaction data
recommendations
analytics
content insights
provide high quality adaptive
education to everyone in the world
PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2014 KNEWTON, INC.
Knewton’s mission
leverage past learning experiences
to improve future learning
experiences
PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC.
data science team’s mission
when a learner is presented some content and
acquires knowledge as a result
PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2014 KNEWTON, INC.
what’s a “learning experience”?
learner
s
content
alice
sharo
n
... ...
video lecture on limits
derivatives definition
integrals quiz
limits exercise
related rates word problem
bob
pastfuture
leverage past learning
experiences to improve future
learning experiences
PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC.
past learning
experiences
modelslearner
representation
content
representation
models
past learning
experiences
learner
representation
proficiency
learning speed
time investment
preferred mode
content
representation
similarities
difficulty
length
effectiveness
leverage past learning experiences
to improve future learning
experiences
PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC.
modelslearner
representation
content
representation
hypothetical
future learning
experience
score
path
dependent
effects
difficult to
quantify
do not have
final grades
leverage past learning experiences to
improve future learning experiences
PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC.
unique challenges
content
representation
PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC.
video lecture on
limits
limits problem set
derivatives
definition
common derivatives
differentiation
exercises
integrals as limits of
Riemann sums
fundamental
theorem of calculus
summation notation
reference
not scalable!
PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC.
video lecture on
limits
limits problem set
derivatives
definition
common derivatives
differentiation
exercises
integrals as limits of
Riemann sums
fundamental
theorem of calculus
summation notation
reference
PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC.
video lecture on limits
limits problem set
derivatives definition
common derivatives
differentiation exercises
integrals as limits of
Riemann sums
fundamental theorem of
calculus
summation notation
reference
instruction
assessment
both
PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC.
measuring
learner ability
modelslearner
representation
content
representation
hypothetical
future
learning
experience
score
PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC.
learner
proficiency
item
difficulty
item
discrimination
PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC.
learner
s
assessment
content
alice
sharon
...
...
limits exercise
integrals quiz
limits challenge problem
related rates word problem
bob
differentiation problem
PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC.
learner
s
assessment
content
alice
sharon
...
...
limits exercise
differentiation problem
integrals quiz
limits challenge problem
related rates word problem
bob
PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC.
learner
s
alice
sharon
...
bob
assessment
contentlimits exercise
differentiation problem
integrals quiz
limits challenge problem
related rates word problem
...
PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC.
learner
s
alice
sharon
...
bob
assessment
contentlimits exercise
differentiation problem
integrals quiz
limits challenge problem
related rates word problem
...
PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC.
? ?
likelihood
prior
posterior
? ?
PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC.
likelihood
prior
?
?
?
?
?
?? ?
?
t=3? ? t=2
posterior
? ? t=1
PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC.
differentiation
related rates
word problem
3D shape volume
PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC.
assessing multiple concepts
compensatory likelihood: knowing 1 concept is good enough
differentiation
related rates
word problem
3D shape volume
PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC.
assessing multiple concepts
non-compensatory likelihood: have to know both concepts
differentiation
related rates
word problem
3D shape volume
learner
timing patterns
PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC.
time
timesession break
response times
histogram of log-taus
with highlighting!
48K students
9.6K modules
1M interactions
histogram of log-taus
with highlighting!
48K students
9.6K modules
1M interactions
histogram of log-taus
with highlighting!
session
breaks
response times
?
48K students
9.6K modules
1M interactions
log time
pro
bab
ility response times
session breaks
normalized response time
no
rma
lized
qu
ittin
g r
ate
normalized response time
no
rma
lized
qu
ittin
g r
ate
high
engagement
normalized response time
no
rma
lized
qu
ittin
g r
ate
low
engagement
high
engagement
normalized response time
no
rma
lized
qu
ittin
g r
ate
non-sticky low
engagement
high
engagement
normalized response time
no
rma
lized
qu
ittin
g r
ate
non-stickyboredom?
low
engagement
high
engagement
normalized response time
no
rma
lized
qu
ittin
g r
ate
non-stickyboredom?
low
engagement
high
engagement
slow
normalized response time
no
rma
lized
qu
ittin
g r
ate
non-stickyboredom?
low
engagement
high
engagement
slowfrustration?
deep thought?
PROPRIETARY & CONFIDENTIAL - NOT FOR REDISTRIBUTION © 2015 KNEWTON, INC.
exciting challenges
● offline learning
● learner affinity for pedagogical strategies
● automating content graphing
● student agency
● controlled experiments
● “adaptivity-ready” content design
chaitanya ekanadham
managing data scientist
Knewton, Inc.
Twitter: @knewton
thank you.
appendix
