Building Learning Ecosystems with

the Experience API

Mike HruskaPresident/CEO

Problem Solutionsmike@problemsolutions.net

@mikehruska

The LandscapeAdaptive Learning:

“Adaptive learning systems will make education much more tailored to each individual student. These systems use artificial intelligence to customize lessons that match the individual student’s progress.”1

Experiential Learning: “Experiential learning to build capabilities is one of the most important elements of a successful company transformation.” 2

70/20/10 Rule: “Redefining the blend to bring learning closer to the workplace—and provide appropriate “scaffolding” for the learner’s needs—is still a struggle for most organizations. 3

http://www.mckinsey.com/insights/mgi/in_the_news/empowering_teachers_and_trainers_through_technologyhttp://www.mckinsey.com/insights/operations/experiential_learning_whats_missing_in_most_change_programshttp://insights.ccl.org/wp-content/uploads/2015/04/BlendedLearningLeadership.pdf

Current State

• Many companies required learning is beyond capabilities of current platforms

• The LMS is not “satisfying” needs• Learning is beyond formal (experiential and

informal)• Companies face cost and time challenges to

develop human capital and manage talent• Data from systems could drive learning

experiences, but data isn’t typically connected

the BIG QUESTIONS…

How can we leverage performance data to save time and money training personnel?

How can we increase training effectiveness by using data

collected along the continuum of training?

What are learning ecosystems?

“Learning ecosystems provide a combination of technologies and

support resources to help individuals learn within an environment“

Source: David Kelly: http://twist.elearningguild.net/2013/11/what-is-a-learning-ecosystem/

What are learning ecosystems?

6

Personalized and Brokered Content•Just-in-Time•Just-for-You•RIGHT TIME EXPERIENCES

Access FromAny Device

Learn From:•Recommendations•Intelligent Tutors•Mentors and peers via social networks•Self-Discovery

Learn Using:•Simulations•LMS•Web•Games•Virtual Worlds•Intelligent content

Ecosystem Elements

Actors Resources Events

Signals Sensors

Flows Patterns

AFFORDANCES

• Evaluate incoming student competencies to inform planned instruction

• Use data analytics to discover trends over time

• Discover which methods of training have had the most impact

• Make sound recommendations for the next training events based on performance data.

• Report performance data to the larger organization.

• Discover which training interventions caused the greatest transfer to the operational environment.

• Dynamically optimize training events in real-time based on performance.

• Assess capability and proficiency levels of individuals, teams, and groups

• Reveal competency gaps and their sources:

- Trainee skills - System faults

Incoming Data Current Data Outgoing Data

Use Cases

Historical proficiency / Performance over time

Live performance view

Macro adaptation (learning path)

Micro adaptation (performance based)

Trends analysis

RECOMMENDATIONS

• Is this person capable to do X?• What do they need to be ready to do X?• Are my people capable/ready to do X? If

not, how can we prepare them?• Who are the best people at X ? And why

are they so awesome?• What does someone need to do more

of? Less of?

Key Questions

Method of uniformly defining and describing experience and context to assess learning and performance

Ability to adapt training across a variety of environments, systems, and modalities

Observe, assess, evaluate, or assert performance by systems or observers

What is Interoperable Performance Assessment (IPA)?

Our [Research] ApproachBuild lightweight and useful tools that:

– Uniformly capture performance data AND context

– Enable learning ecosystem functionality• Allow the data to become immediately visible• Enable adaptive systems to respond to the data

– Allow multiple lines of research efforts to build upon work

– Inform early adopters and lower implementation risk

Army Research Laboratory (ARL) Supported Programs

Soldier Centered Army Learning Environment• Developed by Army Research Lab (ARL) and Army Research Institute

(ARI)• Data-driven architecture and web service-based means to allow the

integration with new technologies• Supports training and education across multiple hardware platforms

Generalized Intelligent Framework for Tutoring• Developed by Learning in Intelligent Tutoring Environments (LITE)

Laboratory• Supports Army’s vision of more efficient and effective learning • Computer-based tutoring framework to evaluate adaptive tutoring

concepts, models, authoring capabilities, and instructional strategies• Provides a generic tutoring capability, including remediation strategies

based on learner performance, to integrated learning environments

Human Performance Measurement Language

(HPML) Constructs• Experience • Position• Platform• Training Environment• Training

Characteristics• Measure• Assessment

• Project/Mission• Task• Subtasks• Competency• Objective• Standard• Knowledge/Skills

Reference: Stacy, W., Ayers, J., Freeman, J., & Haimson, C. (2006). Representing Human Performance with Human Performance Measurement Language. Washington, DC. Aptima, Inc.

Elements of an xAPI Statement• Actor• Verb• Object• Context• Results• Extensions

xAPI and HPML

HPML Constructs

Data via web

services

Import Tools

Games

Mobile

A/R

Virtual Worlds

Example Ecosystem

Multiple Simulators

AdaptationPersonalization

Tailoring

AnalyticsVisualization

EfficacyEfficiency

Effectiveness

• Encoding/data collection library• Reduces complexity to support xAPI• Implements encoding best practices• Describes rich performance context• Dynamic Link Library (.dll)• Individual and group performance

encoding support • www.pipelinexapi.com

TOOLS

19

Current Research

• Efficacy research • Using Pipeline with

gunnery and crew simulators

• Encoding individual data/team

using xAPI• Macro-adaptation• Show effectiveness and

ROI basis for adaptation

Effectiveness Research

• Recently completed human subject research using xAPI to adapt individual to drive team outcomes

• In the team training, participants in the Adaptive condition took nearly 40% less time to and nearly 60% fewer scenarios to achieve excellent qualification scores.

• Results will be published in Q4 2015 in research journal – I/ITSEC

Lessons LearnedCompetencies• Profiles should be built around competencies.• Systems should use competencies as an anchor for tracking

learner performanceGranularity• A balance exists between capturing large amounts of data and

capturing meaningful data that is useful to other systems. Standards and Interoperability• Though data may be captured by a given system, it may or may

not be relevant to other systems. • Most systems that adapt to the learner do so in a black box

fashion using proprietary models of the learner, domain, and data. • Highly adaptive systems are typically complex and designed as

isolated systems that do not communicate or interoperate with other digital learning systems.

Mike Hruska President/CEO Problem Solutions

mike@problemsolutions.net @mikehruska

Questions or want more info?

www.problemsolutions.net

APPENDIX SLIDES

Our work..

Support technical activities in and around learning technologies

Span applied research to product development

Build startups and early stage technologies

Design and build products and solutions

Design and deploy new technologies

Things We’ve DoneBuilt the Experience API with Gov’t and Industry over last 5+ years

Built more open source tools for xAPI than any other gov’t program

Brandon Hall Gold Award Winning xAPI Product on xAPI

Thought Leader Articles in Learning Solutions Magazine (Feature in June 2015) on Learning Ecosystems, xAPI and Design

CoAuthor of “Learning on Demand – ADL and the Future of e-Learning”

CoAuthor of US Dept of Education “Ed Tech Startup Guide” (April 2015)

CoAuthor of Increasing Access through Mobile Learning (2014)

Industry ArticlesLearning Solutions MagazineLearning Ecosystems, Analytics, and xAPI-http://www.learningsolutionsmag.com/articles/1766/elearning-authoring-taking-the-next-step-with-xapi -http://www.learningsolutionsmag.com/articles/1761/amplifying-the-experience-api-xapi-camp-at-devlearn-2015 -http://www.learningsolutionsmag.com/articles/1745/are-you-an-isd-a-business-process-engineer-or-both -http://www.learningsolutionsmag.com/articles/1693/learning-ecosystems-and-the-experience-api-xapi-camp-recap

-http://www.learningsolutionsmag.com/articles/1722/xapi-and-analytics-measuring-your-way-to-success --http://www.learningsolutionsmag.com/articles/1523/ten-steps-to-plan--communicate-your-xapi-design-to-a-web-developer

Training Magazine What is the xAPI?-http://www.trainingmag.com/content/what-experience-api

Elearning News - xAPI-http://elmezine.epubxp.com/t/112009/33

ASTD T&D Magazine - SCORM Evolution-http://www.astd.org/Publications/Magazines/TD/TD-Archive/2013/04/A-SCORM-Evolution

Future Learning in the DoD-http://www.kmimediagroup.com/military-training-technology/440-articles-mtt/unlimited-access-learning

Published Research on xAPIHruska, M., Medford, A., Murphy, J. (2015). Learning Ecosystems Using the Generalized Intelligent Framework for Tutoring (GIFT) and the Experience API.  17th International Conference on Artificial Intelligence in Education (AIED 2015). Madrid, Spain. June 2015.

Amburn, C., Goodwin, G., Michael, H., Murphy, J. (2015). Developing Interoperable Data for Training Effectiveness Assessment in Army Marksmanship Training. MODSIM World 2015.

Goodwin, G., Hruska, M., Murphy, J. (2015). Developing Persistent, Interoperable Learner Models in GIFT. GIFT Sym3. Orando, FL, June 2015

Hruska, M., Long, R., Amburn, C. (2014). Human Performance Interoperability via xAPI: Current Military Outreach Efforts.  Simulation. Fall Simulation Interoperability Workshop, 14F-SIW-035, Orlando, FL, September, 2014.

Hruska, M., Long, R., Amburn, C., Kilcullen, T., Poeppelman, T. (2014). Experience API and Team Evaluation: Evolving Interoperable Performance Assessment. The Interservice/Industry Training, Simulation & Education Conference (I/ITSEC). Volume: 2014.

Poeppelman, T., Hruska, Long, R., Amburn, C. (2014). Interoperable Performance Assessment for Individuals and Teams Using Experience API.  2nd Annual Generalized Intelligent Framework for Tutoring (GIFT) Users Symposium (GIFTSym). Pittsburgh, PA, June, 2014.

Hruska, M., Poeppelman, T. R., Dewey, M., Paonessa, G., Paonessa, M., Nucci, C., Ayers, J. (2013). Interoperable  Performance Tracking to Support Tailored Learning (Final Report). U.S. Army RDECOM Army Research Laboratory (ARL) – Simulation Training Technology Center (STTC).

Poeppelman, T., Ayers, J., Hruska, Long, R., Amburn, C., Bink, M. (2013). Interoperable Performance Assessment using the Experience API. The Interservice/Industry Training, Simulation & Education Conference (I/ITSEC). Volume: 2013.

{ "id": "9a9c6f5f-5bb7-4ed1-b993-96e9cee3cd2f", "actor": { "objectType": "Agent", "name": "PVT Jacob Daniels", "mbox": "mailto:name@us.army.mil" }, "verb": { "display": { "en-US": "experienced" }, "id": "http://adlnet.gov/verbs/experienced" }, "object": { "objectType": "Activity", "id": "activityId:uri/10th_CAB_AATX", "definition": { "name": { "en-US": "10th CAB AATX" }, "description": { "en-US": "On-Station - Area of Operations - Expert:15 degree safe firing range each way,Average:18 degree safe firing range each way,Novice:20 degree safe firing range each way - This measure assesses the way in which pilots approach the target when they get into the on-station area" } } }, "result": { "extensions": { "resultExtensions:uri/extensions/trainingobjective": "Meet all Time on Target objectives Meet all Time on Target objectives" }, "score": null, "success": false, "response": "Expert" }, "context": { "extensions": { "contextExtensions:uri/AVCATT/role": "FAC", "contextExtensions:uri/AVCATT/visual_scene_display_resolution": "20/10 visuals", "contextExtensions:uri/AVCATT/cockpit_content": "Content and layout exactly replicates aircraft", "contextExtensions:uri/AVCATT/visual_field_of_view": "180 horizontal FOV", "contextExtensions:uri/AVCATT/g_cueing": "No G cues", "contextExtensions:uri/AVCATT/axis_disturbance_motion_cues": "No disturbance motion cues" }, "contextActivities": { "parent": [], "grouping": [ { "id": "contextActivities:uri/01_2_5179_perform_aerial_deliberate_attack_operations", "definition": { "name": { "en-US": "01-2-5179 Perform Aerial Deliberate Attack Operations" }, "description": { "en-US": "Destroy enemy or force them to withdraw" } } }, { "id": "contextActivities:uri/support+troops+in+contact+(tic)", "definition": { "name": { "en-US": "Support Troops in Contact (TIC)" }, "description": { "en-US": "Friendly ground forces receiving effective fire" } } }, { "id": "contextActivities:uri/battlefield_communications", "definition": { "name": { "en-US": "Battlefield Communications" }, "description": { "en-US": "Right information, to the right person, at the right time" } } } ] }, "platform": "AH-64 - Apache" }, "timestamp": "04/17/2014 16:19:27", "authority": { "objectType": "Agent", "name": null, "mbox": null } }