May 2009 1Distribution authorized to U.S. Government Agencies only

Symmetric Multimodal Interactive Intelligent

Development Environments

•Dramatic reduction in time and cost of constructing software designs and related engineering artifacts

•Production of higher quality artifacts through interactive design.

•Metrics to be refined during seedling.

Multi-modal User Interfaces can now integrate sketch, gesture,& natural language to capture design, rationale more naturally. Mixed-Initiative User Interfaces enable symmetric collaboration between human and computer-based designers.Adaptive, Domain-Specific Model-Based Software Engineering tools enable flexible collaboration at a human level of abstraction.

Symmetric multimodal interaction brings intelligence to model-based development

Multi-modal Mixed-initiative User Interface• Supports sketching, gestures, natural

language.• System can query human for clarification,

suggestion, etc.• System can take initiative in design and

implementation, freeing the human from detailed specification of lower-level details

Adaptive Domain Specific Model-Based Software Engineering tools

• Support evolution of domain “vocabulary” • Support diagramming using symbology

specific to a domain.ASSUMPTIONS AND LIMITATIONS:

• Anticipated productivity gains can be demonstrated on well-defined domains.

• Current natural language technology is sufficient.

• Current Integrated Development Environments (IDEs) are developer-centric, detail-intensive, and error-prone

• Modality is essentially paper-based and human developer-centric

• IDEs support limited consistency checking and debugging

• Demonstrate program feasibility and develop program structure

•Prototype combination of symmetric multimodal HCI with model-based code generation technology

•Determine feasibility of program leading to mixed-initiative high-productivity environments

•Identify technology gaps and refine DARPA-hard program challenges.

•Demonstrate on a DoD-relevant domain

•Survey and position with respect to previous research

QUAN

TITA

TIVE

IMPA

CTEN

D-OF

-SEE

DLIN

G GO

AL

STAT

US Q

UONE

W IN

SIGH

TS

Code

Compil

e

Test

System Complexity

Cos

t (Ti

me)

Savings

With MMIIDE

Current IDEs

Does this process need access to the

database?

Let’s create a data interface component

May 2009

Review – Purpose of this Seedling

The penta and SOW identify the area of “Intelligent Development Environments” and make conjectures about:

– Current SOA in IDEs and what is possible (in terms of mixed-initiative design and multimodal interfaces that make this possible)

– The supposed technical challenges in doing that

Seedling seeks to test those conjectures by marrying advances in areas that may be convergent in this space: MIDOS and PAMS.

Investigate those conjectures more completely, and arrive at a focused set of technical challenges that form the basis of a new program

We should guard against getting too fascinated with a demo and keep our eye on this goal

Bottom Line: The purpose of the seedling is to create a new program initiative brief

2Distribution authorized to U.S. Government Agencies only

May 2009

My Goal for Today

Get sharper focus on our original conjectures:– The nature of the technical challenges, and which focused challenges will form the

basis for a program?– How far can we reach in terms of operational capability in the scope of a program?– What is the potential operational payoff?– What options for demonstrating the concept and measuring success?

Determine whether we are getting any closer to filling in the blanks of a new initiative briefing that will get across to the Director and others on a tech review panel

– A “Scientific American” statement of the technical goals of the program – A clear connection to compelling operational need– A convincing case for how this can be achieved

Exchange ideas, resolve questions, set goal for next IPR

Discuss potential workshop

3Distribution authorized to U.S. Government Agencies only

May 2009

The Heilmeyers…

1. What is the problem, why is it hard?

2. How is it solved today?

3. What is the new technical idea; why can we succeed now?

4. What is the impact if successful?

5. How will the program be organized?

6. How will intermediate results be generated?

7. How will you measure progress?

8. What will it cost?

4Distribution authorized to U.S. Government Agencies only

May 2009

“DARPA-Hard” Questions /Technical Insights?

grounding complex abstract diagrams in sufficient meaning to support the design task;

Determining how complex specifications can be segmented in ways that are functionally meaningful for reasoning and dialog;

determining what kind of human-computer interactions are needed at any given moment and mediating that interaction effectively;

determining the HCI principles that lead to productivity enhancements not attainable with model-based software engineering tools alone (including the advanced versions produced in the current DMT-SWP program).

Is there a body of kn about MBSE applicable across domains?– Dev of DSMLs suggest any common body of language, lexicon,…– How wide a variety of specification forms to consider/// tabular, text,

graphicalGraphical models vs. behavioral specifications? How do we get the benefits of graphical structural models for behavioral specs?

5Distribution authorized to U.S. Government Agencies only

Lowering the barrier to rationale capture as a means of addressing the questions on the left

May 2009

Can we make computers a design partner in MBSE task?

What does it take to do this?

What are the challenges preventing that from being realized?