Upload
paul-henderson
View
222
Download
0
Tags:
Embed Size (px)
Citation preview
Intelligent Systems
Software Assurance Symposium 2004
Bojan Cukic & Yan Liu, Robyn Lutz & Stacy Nelson, Chris Rouff,
Johann Schumann, Margaret Smith
July 22, 2004
“What”
• Intelligent Systems research will create “new generations of robust, fault-tolerant software for intelligent, cooperative space systems that operate largely autonomously from ground control” --NASA list of key technology areas for H & RT Advanced Space Technology, 6/04
• New technologies for V&V of Intelligent Systems
“What” (cont.)
• Technologies demonstrated at this year’s presentations:– Neural Networks– AI Planners – Support Vector Data Description algorithms– Bayesian-based safety envelopes– Autonomous contingency identification and
recovery technology– Model Checking– Hybrid formal methods
Information Systems Presentations
ScheduledPresentations
By TRL Real World Application
Verification andValidation of AdaptiveSystems
BojanCukic
5 IFCS, F-15
Bayesian Verificationand Validation toolsfor Adaptive Systems
JohannSchumann
7 F-15
Formal Approaches toSwarm Technologies
Chris Rouff 2+ ANTS
Information Systems PresentationsContingencySoftware inAutonomousSystems
Robyn Lutzand StacyNelson
2+ ARP
ModelChecking ofArtificialIntelligenceBasedPlanners
MargaretSmith
7 DS-4/ST-4JPL S/C
LyapunovStabilityAnalysis andOn-LineMonitoring
Bojan Cukic 5+ IFCS JPL’s FaultProtection engineEarth Orbiting
SSatellite
Intelligent Systems: Why ?
• Long lived missions
• Lower operations costs
• Swarms & constellations of satellites/spacecraft
• Currently used in other domains:– automotive– health– waste water management
• Intelligent Systems are here to stay!
Intelligent Systems: Why not
• Is the technology:– Scalable for usage?– Being oversold?– Just a piece of a larger puzzle?
• V&V of Intelligent Systems requires a new knowledge set: math, tools, control theory, and highly skilled software engineers.
• V&V is scrambling to catch up to new technologies for Intelligent Systems
Directions?
• Do we know yet how to design intelligent systems for verifiability? (or meaningless to lump them?)
• Is the IV&V process different for intelligent systems?
• Are we ready to demonstrate scalability on real systems?
• Should we be developing V&V standards for intelligent systems? Tied to criticality levels?
• How do we start establishing benchmarks for intelligent systems?
Verification and Validation of Adaptive Systems by Bojan Cukic
• Investigate the role of modern AI techniques (Support Vector Machines) in failure detection and identification.– Failure Detection
• Designing a fast (real-time) SVDD algorithm to detect failure conditions
– Failure Identification• Failures are identified by studying the correlation
between certain longitudinal and lateral dynamics parameters
– Validate the technology in extensive simulations
Bayesian Verification and Validation tools for Adaptive
Systems by Johann Schumann
Problems with traditional V&V methods applied to Adaptive Systems:
•Fault avoidance design testing applies to base case only
–Unanticipated failures?
– Unmodeled failures?
•Fault removal cannot test all possible configurations in advance
•Fault tolerant design does not consider all possible problems
–
Bayesian Verification and Validation tools for Adaptive
Systems by Johann Schumann
Methods for improvement:
•Improve performance estimation of the neural network (Bayesian approach)
•Use Envelope tool to answer:
– How large is the current safe envelope?
– How far is the operational point from the edge?
Formal Approaches to Swarm Technologies by Chris Rouff
• Survey formal approaches for agent-based, multi-agent and swarm-based systems for appropriate swarm-based methods• Apply most promising approaches to parts of ANTS• Evaluate methods for needed properties• Model and outline swarm-based formal method• Develop formal method for swarm-based systems• Do formal specification of ANTS using new method• Prototype support tools
Formal Approaches to Swarm Technologies An ANTS Overview - by Chris Rouff
Earth
Lagrange Point Habitat
1. Assembly & release
2. Self propelled transit
6. A messenger carriesfindings to Earthwhen needed.
4. Swarm (Fly by) Operations
5. Repeat steps 3 and 4.IR Worker
MAG Worker
X-Ray Worker Messenger
Asteroid belt
3. Long-Range Operations
Asteroid(s)
Workers
Messengers
Rulers
Workers
Workers
M. L. Rilee, EIT, S. A. Curtis, NASA/GSFC, 2001.
ANTS: Mission Concept 2020
Contingency Software in Autonomous Systems by Robyn Lutz
& Stacy Nelson
The Goal - Mitigate failures via software contingencies resulting in safer, more reliable autonomous vehicles in space and in FAA national airspace
How?
• Adding intelligent diagnostic capabilities by supporting incremental autonomy
• Responding to anomalous situations currently beyond the scope of the nominal fault protection
• Contingency planning using the SAFE (Software Adjusts Failed Equipment) method
Model Checking of Artificial Intelligence Based Planners
by Margaret Smith
• Goal: Using model checking, and specifically the SPIN model checker, retire a significant class of risks associated with the use of Artificial Intelligence (AI) Planners on Missions – Must provide tangible testing results to a mission using AI technology.– Should be possible to leverage the technique and tools throughout NASA.
• FY04 Activities:– Identify and select candidate risks– Develop and demonstrate technique for testing AI Planners/artifacts
on:• A toy problem (imaging/downlinking) – demonstrate tangible results with
an abstracted clock/timeline• A real problem (DS4/ST4 Champollion Mission) – demonstrate, using
DS4 AI input models, that Spin can determine if an AI input model permits the AI planner to select ‘bad plans’.
Lyapunov Stability Analysis and On-Line Monitoring by Bojan Cukic
The Problem:
•Issues with Adaptive Systems: uncertainty/newness
•Need Understanding of self stabilization analysis techniques suitable for adaptive system verification
•Need to investigate effective means to determine the stability and convergence properties of the learner in real-time
The Approach:
•Online Monitoring
•Confidence Evaluation
Lyapunov Stability Analysis and On-Line Monitoring by Bojan Cukic
Relevance to NASA:
• Artificial Neural Networks are increasingly important in flight control and navigation
• Autonomy and adaptability are important features in many NASA projects
• The theory is applicable to future agent-based applications