The New Identity of Adaptive Math

The New Identity of Adaptive Math

October 17, 2016

Nigel Green• VP User Experience, DreamBox Learning• Architect and Developer of the DreamBox

“Intelligent Adaptive Engine” (2006 - present)• Former teacher (K - U)

The New Identity of Adaptive Math“When Clay Christensen, Curtiss Johnson, and I published Disrupting Class: How Disruptive Innovation Will Change How the World Learns in 2008, we didn’t use the phrase “adaptive learning” once in the book. Just eight years later, it’s nearly impossible to imagine writing a book about educational technology and neglecting the term”

– Michael B. Horn, Introduction to Decoding Adaptive


2006 2008 2009 2016

DreamBoxFounded

DreamBox K-2Launched

Disrupting ClassPublished

“When Clay Christensen, Curtiss Johnson, and I published Disrupting Class: How Disruptive Innovation Will Change How the World Learns in 2008, we didn’t use the phrase “adaptive learning” once in the book. Just eight years later, it’s nearly impossible to imagine writing a book about educational technology and neglecting the term”



2006 2008 2009 2016

DreamBoxFounded

DreamBox K-2Launched

Disrupting ClassPublished

“When Clay Christensen, Curtiss Johnson, and I published Disrupting Class: How Disruptive Innovation Will Change How the World Learns in 2008, we didn’t use the phrase “adaptive learning” once in the book. Just eight years later, it’s nearly impossible to imagine writing a book about educational technology and neglecting the term”


DreamBox K-8DreamBox K-5

2014

6

• Defining Adaptive• Content, Assessment, Sequence• Micro, Macro• Useful but insufficient?• Pedagogy should drive platform

• How Adaptive Learning Technologies Work

• Student actions and behaviors that inform an adaptive engine• Adaptivity in the moment and over time• How, when, and why evidence of learning is used

• Why Adaptive? Integrating Adaptive Learning Models into Blended Learning

• The Future(s) of Adaptive Learning

7

Defining Adaptive

Defining Adaptive• Pearson/EdSurge Whitepaper

• Adaptive Assessment• Adaptive Content• Adaptive Sequencing

• Tech & Learning Buyers Guide• Micro/Macro• Pedagogy

From: Decoding Adaptive

Defining Adaptive – Pearson/EdSurge“…we define digital adaptive learning tools as education technologies that can respond to a student’s interactions in real-time by automatically providing the student with individual support” (Decoding Adaptive p.17)

• Adaptive Content• Responds based upon the student’s unique response with targeted feedback, hints, additional

learning resources and scaffolded support.

• Adaptive Assessment• Changes the questions a student sees, based upon their response to the previous question.

• Adaptive Sequencing• Changes the sequence of what a student learns next by continuously collecting real-time data on

performance to automatically change a student’s learning experience.

10

Adaptive Tools• Collect specific information about

individuals• Track how they answer questions• Responds to each student• Uses data to change the learning

experience to better suit individual’s needs

Non-Adaptive Tools• Simply mark an answer as correct or

incorrect• Provide one learning path regardless

of the response• Don’t collect data in real-time• Don’t provide support in real-time• Collect data through one singular

assessment and then prescribe a path of learning

Defining Adaptive – Pearson/EdSurge

Sequence

Content

Assessment

Responds to:• Correct / Incorrect Answers• Mistakes madeResponds with:• Corrective feedback, hints, additional resources, scoffolded

support

Defining Adaptive Content

Responds to:• Correct / Incorrect Answers• Mistakes madeResponds with:• Corrective feedback, hints, additional resources, scoffolded

supportWhat’s Usually Missing:• Detailed analysis of how students respond• Assessment and Analysis of strategies used by student• Ability to dynamically adapt the content and challenges

presented within the current content as a student demonstratesunderstanding and learning progression

Defining Adaptive Content

Responds to:• Answers GivenResponds with:• Changes in difficulty of subsequent questionsCommonly used for:• Practice (pulling from a pool of questions)• Benchmark Assessment (usually more formal stand-alone)

Defining Adaptive Assessment

Responds to:• Answers GivenResponds with:• Changes in difficulty of subsequent questionsCommonly used for:• Practice (pulling from a pool of questions)• Benchmark Assessment (usually more formal

stand-alone)What’s Missing:• Assessment by itself doesn’t cause learning• The level and granularity of assessment is only

as good as the response analysis the content enables

Defining Adaptive Assessment

Responds to:• Data collected in real-time from students.Responds with:• Changes what skills are addressed or what content a

student sees next.

Defining Adaptive Sequencing

Responds to:• Data collected in real-time from students.Responds with:• Changes what skills are addressed or what content a

student sees next.What’s Missing:• “Big Data” but what data? And what quality? Clicks? Click Rates?

Multiple Choice Answers? Answers vs. Strategies?• Should utilize coherent Learning Pathways.• “Ideally should “Know” and “Understand” Content• No matter how good your algorithms and predictive analytics, Adaptive

Sequencing can only be as good as the data that was collected.

Defining Adaptive Sequencing

Defining Adaptive: Useful but Insufficient?• For many products the three categories are a good

place to start.• But are they merely a more structured and more

rigorous version of “if you have some of these features you are an adaptive learning product” approach?

• Misses the bigger perspective that a truly/Intelligently Adaptive Learning product is more than a collection of separate overlapping adaptive features.

Defining Adaptive: Pedagogy Should Drive PlatformWe know too much about human learning to embrace adaptive platforms that ignore pedagogy.• Optimizing for “behavioral profiles” does not necessarily

result in optimized learnings.• Example: Is it harder and more complex to make a great movie, or to

recommend one from a limited set?• Too many still model Teacher “delivers” content and students “receive” it.• Lots of data, but without opportunities to engage in authentic, independent thinking,

and ways to show that, what data is collected is more behavioral than demonstrations of understanding and cognitive development.

Defining Adaptive - Tech & Learning Buyers Guide• Micro-Adaptive Within Lesson

• Fluency: Lessons support students’ speed and accuracy to understand that there’s more than one way to solve a problem

• Feedback: Specific real-time scaffolding based on students’ actions and strategies that build understanding and fluency

• Formative: Assessment is seamlessly embedded and data to support instruction is provided in real time

• Macro-Adaptive Between Lessons• Choice: Students drive their own learning, and an exact next lesson is not predetermined by the

program• Path: Non-linear learning pathways support conceptual development across grade levels and

topics• Challenge: Problems in lessons increase in difficulty as the student progresses

Defining Adaptive – What About DreamBox?Developed Adaptive Learning in 2006, before there was “Adaptive”.• Originally created to serve those with specific learning disabilities, and those

struggling with Math.• Able to respond and adapt after the first interaction because lessons are

designed from the start to be highly adaptive learning experiences, not based upon textbook instruction.

• Required Continuous Assessment & Continuous Adaptation in the moment – a different approach to that still often used today.

• No “question banks”. Every problem generated dynamically for that student at that moment.

• Not aimed at the middle of the bell curve, but able to seamlessly adapt across entire range fromintervention to “advanced and gifted”.

“Over Time” / “Macro”

Utilizes coherent Learning Pathways based upon the skills and understandings assessed by each piece of content.

Informed and optimized by data mining and cohort analysis.

Defining Adaptive – DreamBox Learning (2006)

Content

Assessment

“In The Moment” / “Micro”

Rigor Lives Here. Complexity Lives Here. Driven byPedagogy. Content reveals what you are Assessing.

Adaptation &Micro-Sequence

Formative

Summative

Sequence

Interactions

Student Browser or iPad

Cloud

Analysis

1. Personalizes learning by adapting both in the moment and over time in a variety of ways

2. Designed according to evidence-based principles of learning science to support cognitive development

3. Ensures students are active learners and thinkers instead of passive receivers of information

4. Uses continuous, embedded assessment both formatively and summatively to collect longitudinal evidence of student proficiency

5. Real-time feedback is specific to a student’s independent and unique thinking, problem solving strategies, and answers.

6. Empowers self-directed, student-driven learning paths thatsupport conceptual development across grade level and topics.

Defining Adaptive – DreamBox Learning

23

How Adaptive Learning Technologies Work

How Adaptive Learning Technologies Work• Determining the appropriate (potentially, set of) content• Presenting the content, then observing, assessing and adapting• Determining the appropriate next (potentially set of) content.Example: DreamBox Learning

Lesson

In the Moment Adaptation Over Time Adaptation

How Adaptive Learning Technologies Work - Assessment

Lessons can, and usually do, assess one or more skills, pieces of knowledge or conceptual understandingEach interaction assesses one or more of these, with a score for each. Each score is based upon, and can be affected by, at least:

Difficulty of the current questionThe quantity and type of the mistakesUse of Hints and any other Assistance provided by the lesson prior to the interaction being assessedThe total number of mistakes made in this problem, previous problems and the lessonHow the student’s response time compares to those of similar students answering similar questions (Think/Prep/Act/Review)Any adjustments based upon the specific strategy used by the student.

Scores can also be influenced on where the response times fall within establishedthresholds and by the strategies used.

// Number-specificOFF_BY_ONEOFF_BY_TWOOFF_BY_THREEOFF_BY_FOUROFF_BY_FIVEOFF_BY_NINEOFF_BY_TEN OFF_BY_ELEVENOFF_BY_TWENTYOFF_BY_ONE_HUNDREDOFF_BY_A_MULTIPLEINCORRECT_VALUE

// Number-specific diagnosticDIGIT_REVERSALDIGIT_REVERSAL_TWO_FIVEDIGIT_REVERSAL_THREE_EIGHTDIGIT_REVERSAL_SIX_NINE

// BuildingINCORRECT_PLACEMENTINCORRECT_PLACEMENT_MULTIPLEINCORRECT_PLACEMENT_NON_ALTERNATEINCORRECT_COLORINCORRECT_COLOR_MULTIPLEONE_TOO_MANY_INTERACTIONSMANY_TOO_MANY_INTERACTIONS

// StrategyINCORRECT_STRATEGY

// TimingRESPONSE_TIME_OVER_THRESHOLD_ONERESPONSE_TIME_OVER_THRESHOLD_TWORESPONSE_TIME_EXCEEDS_FATAL

// SelectionINCORRECT_SELECTION

// Place ValuePV_TOTAL_IN_TENSPV_TOTAL_IN_HUNDREDSPV_TOTAL_IN_ALLPV_REVERSAL_COLUMNSPV_MULTIPLIED_OUTPV_NOT_OPTIMALPV_OFF_BY_PLACE_VALUEPV_REVERSAL_DIGITSPV_EXTRA_ZEROPV_DROPPED_ZEROPV_IGNORED_COLUMNPV_ADD_UP

// ArraysARRAY_SUM_OF_FACTORS // MultiplicationMULTIPLICATION_ONE_GROUP_TOO_MANYMULTIPLICATION_ONE_GROUP_TOO_FEWMULTIPLICATION_INCORRECT_FACTORMULTIPLICATION_INCORRECT_NUMBER_OF_GROUPSMULTIPLICATION_REVERSED_FACTORS

How Adaptive Learning Technologies Work – AssessmentExamples (From DreamBox K-2, 2009):

26

27

GHOSTINGCHUNKING

RESET_BUTTONAUDIO_BUTTONALLIGATOR_BUTTON

STATIC_FLASHCARDFLASHCARD_PEEK_FIRSTFLASHCARD_PEEK_NEXT

GENERALAUDIO_HINTVISUAL_HINTSHOW_STRATEGYSHOW_EXAMPLEREMOVE_COUNTERREMOVE_DISTRACTORSTART_OVER

SHOW_ANSWER

HIGHLIGHT_TOOLHIGHLIGHT_SELECTIONHIGHLIGHT_LOCATIONHIGHLIGHT_COUNTERS

How Adaptive Learning Technologies Work – AssistanceExamples (From DreamBox K-2, 2009):

How Adaptive Learning Technologies Work• Determining the appropriate (potentially, set of) content• Presenting the content, then observing, assessing and adapting• Determining the appropriate next (potentially, set of) content.

Lesson

In the Moment Adaptation Over Time Adaptation

BuildNumbergramDigitline1to10

BuildOptimalNumbergram1to10

Requirements:• Identify (quick image) numbers from 1 to 10 in sets

Assessments:• Form a set of 1 to 10

objects in Numbergrams• Identify a set of 1 to 10

objects in Numbergrams

Requirements:• Form a set of 1 to 10 objects in Numbergrams• Identify a set of 1 to 10 objects in

Numbergrams

Assessments:• Form a set of 1 to 5 objects in Numbergrams

using groups of 1, 2, 3, 5, 10• Form a set of 6 to 8 objects in Numbergrams

using groups of 1, 2, 3, 5, 10• Form a set of 9 to 10 objects in Numbergrams

using groups of 1, 2, 3, 5, 10

How Adaptive Learning Technologies Work – SequencingExamples (From DreamBox K-2, 2009):

30

Why Adaptive?Integrating Adaptive Learning Models into Blended Learning

Why Adaptive? Integrating Adaptive Learning Models into Blended Learning

Don’t Start by Telling

“Providing students with opportunities to first grapple with specific information relevant to a topic has been shown to create a ‘time for telling’ that

enables them to learn much more from an organizing lecture.”

How People Learn, p. 58

Intelligent Adaptive Learning Programs are designed as Learning Experiences, not as Instruction.Students:• Are provided with scenarios appropriate to their

skillset and level of understanding.• Can explore, make mistakes, create hypotheses.• Be provided with just the right amount of

support as they need it.• Discover for themselves fundamental

mathematical concepts.


• “An understanding is a learner realization about the power of an idea.”

• “Understandings cannot be given; they have to be engineered so that learners see for themselves the power of an idea for making sense of things.”

p. 113, Schooling by Design, Wiggins & McTighe, ©2007

Learning PrinciplesWhy Adaptive? Integrating Adaptive Learning Models into Blended Learning

Learning Math“If I cover it clearly, they

will ‘get it.’”“Presentation of an explanation, no matter

how brilliantly worded, will not connect ideas unless students have had ample

opportunities to wrestle with examples.”

From Best Practices, 3rd Ed., by Zemelman, Daniels, and Hyde, ©2005 From Understanding by Design, Wiggins & McTighe,

©2005


Adaptive Learning Programs let students have their own realizations about ideas.

Student Engages within

a Context

Student Transfer

s & Predicts

Student Receives Feedback

Engine Adapts & Differentiate

s

Student Independently Transfers


36

Adaptive Learning is...• Instructive, Engaging Learning Experiences• Engineered for Realizations• At the Initial Conception of Ideas• With Dynamic Content• That Stimulates Thought• Captures How Students Are Thinking• Accounts for Mistakes• Responds Formatively• And Reports Summatively• To Complement Classrooms• And Enable Independent Transfer


Source: Clayton Christensen Institute for Disruptive Innovation

37

The Future of Adaptive Learning?

The Future of Adaptive Learning?Everyone - Student, Educator, Administrator, Guardian – is a Learner.Each plays an important role.• Actionable information and instruction make every educator more effective.• Highly personalized and adaptive instruction enables every learner to be

successful.• Adaptive Learning tools become an essential and integral part of education by

truly understanding and supporting the needs of every learner.• Personal connections and impacts are maximized in the few one-on-one

minutes available with each student.

Content

Assessment Adaptation &Micro-Sequence

SequenceAnalysis

The Future of Adaptive Learning?Where is the educator?

• Educators included in, and benefitting from, the Adaptive Learning experience.• Adaptive Learning Engines, informed by student progress and educator input

drive sequencing for both educators and their students.• Adaptive Professional Development.• Adaptive Reporting.

The Future of Adaptive Learning?

Content

Assessment Adaptation &Micro-Sequence

SequenceAnalysis

Sequence

Analysis

PDContent

AdaptiveReporting

Assessment

Questions?

DreamBox Learning® K–8 MathAvailable in English & Spanish

DreamBox Lessons & Virtual ManipulativesIntelligently adapt & individualize to:• Students’ own intuitive strategies• Kinds of mistakes• Efficiency of strategy• Scaffolding needed• Response time

AssignFocus™

Differentiated assignments for every student through your Insight Dashboard

To accelerate learning, offer remediation, and adjust classroom instruction

iNACOL is right around the corner… Catch us at @DreamBox_Learn

Learn more and see how it works:www.DreamBox.com/request-a-demo

Efficacy: Independent Validation from CEPR at Harvard University

We value your feedback!Let us know how we’re doing:

www.surveymonkey.com/r/GC6ZCM7

Education

The New Identity of Adaptive Math