VERIFYING IMPLEMENTATION PROTOTYPE
Independent Test Capability Team - Bill Stanton- Jarrod Petersavage- Justin Morris- Steven Seeger- Mike Wise
1
VERIFYING IMPLEMENTATION
2
GOAL
IVV 09-1: Verify System Behavior
…Verify actual system behavior in the implemented system against expected (or designed) behavior…
Engineering Services Initiative #10:
Provide a capability to dynamically assess mission’s software against expected software behaviors.
STEPS1. Understand the Problem a. Vision b. Concept of Operations2. Practical Example: Case
Study a. Proof of Concept b. Role of the SRM c. Effort, Lessons Learned3. Capture System Behaviors a. Models b. GUI Mockups c. Design Documentation 4. Acquire Supporting Tools
and Develop Framework5. Maintain Capability and
Supports Applicable Projects
OBJECTIVES CASE STUDY APPROACH
ROLE OF THE PBRA & SRM
CASE STUDY RESULTS
LESSONS LEARNED
FUTURE WORK3
CASE STUDY APPROACHNavigator Software on GPM Project4
WHAT DID WE DO?Chose a Project GPMExamined PBRA Results
o GPM PBRA Profile: March 5, 2009o GPM PBRA-Lite: May 28, 2009
Chose a small example Navigator Softwareo Requirements:
o Code is Availableo Some Documentation Availableo Supporting Tool(s) Available o Modeling Artifacts Available
Successfully Run Code Executed using EDGE IDE with SimTest Simulator
o Develop and Execute Test Cases using Sequences from GPM SRM (Working) 5
ROLE OF THE PBRA AND SRM
6
GPM PBRA PROFILEMARCH 5, 2009, MACAULAY, DUNKERLEY
System CapabilitiesJ1: Launch and Achieve Initial OrbitJ2: Checkout SpacecraftJ3: Fly in Required OrbitsJ4: Obtain Science DataJ5: Maintain Health and Safety of SpacecraftJ6: Process Science DataJ7: Decommission Spacecraft
J4 J1
J5
J6
J3J7 J2
Impact
7
ROLE OF THE PBRA & SRMIDENTIFY TARGET BEHAVIOR(S)
ROLE OF THE PBRA & SRMIDENTIFY TARGET BEHAVIOR(S)
“Maintain Health and Safety of Spacecraft Activity Diagram”
8
H H
None
HH
H
Maintain Propulsion SystemH
ROLE OF THE PBRA & SRMIDENTIFY TARGET BEHAVIOR(S)
9
H
H
H
H
H
M
M
M
M
L
Maintain Propulsion System Activity Diagram
Determine Position & Delta-V
M
ROLE OF THE PBRA & SRMIDENTIFY TARGET BEHAVIOR(S)
Determine Position and Delta-V Behavior is implemented in the GPS Navigator The SC determines its position using GPS
position information.
10
RESULTS & LESSONS LEARNED
11
CASE STUDY RESULTS
Identified a capability using PBRA & SRM to drive Verification Implementation Activities (Case Study)
Executed Navigator Flight Software using a trial version of a COTS toolset in 2 months Duration includes obtaining
all required tools and artifacts and configuring the environment
Develop serial interface to provide conduit for testing
Utilize Elaborated Sequence Diagrams (to be developed by SRMV PL) to drive test cases
Accomplishments Future Work
12
GPM Electrical Block Diagram
13
Navigator Software Simulation Navigator software runs on a FreeScale
ColdFire 5307. On GPM, the Navigator is part of GN&C and
connects to C&DH via RS-422. GPM has two Navigator units, but we only
need to test one. Initial trial run used EDGE Development suite
from Mentor Graphics. Compiled code as x86 without Nucleus OS code. Nucleus runtime provided by simtest.
14
Navigator Commands All Navigator commands and responses are
transmitted over RS-422. ITC team determined that Navigator would be
a good place to start because sending and receiving serial data is not difficult.
Navigator commands over serial include read and write memory, patience, and ephemeris data. Easy proof of concept with simple write and read-
back operation. Can expand simulation further with more complex
commands. CCSDS message format.
15
Issues with Hardware Simulation Navigator has an RF board that receives GPS
signals. Difficult to simulate.
Developer usescomplex hardwareand softwaresimulation solution.
16
Software Simulation End Goal Run binaries provided by vendors on an
instruction set simulator. No need to compile Navigator software for x86. No chance of results varying by build process.
More hardware interaction. Simulation Real hardware
Headless operation with all simulations driven by test scripts.
17
LESSONS LEARNED Working with Trial Versions of Tools often Proves Difficult
Vendor Support 30-day to 60-day Trial Window Limited Tool Capabilities
Importance of Communications between Product Lines Modeling Artifacts help drive Verification Implementation
Activities Importance of SRMV and Verification Task Scheduling
Initial setup time will vary depending on test environment and requirements
Project Leads & Product Lines need to identify Verification Implementation targets early in lifecycle to allow time for tool acquisition, development time, and training
Leveraging of Developer’s Capabilities may prove Beneficial Parallel Activities 18
PARALLEL ACTIVITIESOTHER ITEMS BEING WORKED BY THE ITC TEAMNOT SCOPE OF PRESENTATION – INFORMATION SHARING
SoftSim All-digital system simulation with flight-like
interfaces Juno and GRAIL missions
VxWorks Utilized by almost all Science missions ITC is obtaining trial version, inquiring about
licensing, and training License required to support SoftSim testing
19
ANY QUESTIONS?
20
BACKUP SLIDES21
PROVIDES ADDITIONAL CAPABILITY“STATIC VERSUS DYNAMIC ANALYSIS”
Static Analysis Dynamic Analysis• Finds weaknesses in exact location• Allows quicker turnaround for fixes• Finds errors earlier in lifecycle• Automated Tools• Relatively fast• Can scan all of code
•Assess Mission Software against Expected Software Behaviors•Finds run-time vulnerabilities•Provide increased flexibility of what to look for•Identifies vulnerabilities that may have been false negatives in static analyses•Validation of Static Analysis Findings
22