Using Systems Engineering Processes and Methods to Make Procurements Easier and to Fully Meet Public Transportation Agency Expectations

© 2014 Noblis, Inc. Noblis proprietary and confidential.

Using Systems Engineering Processes and Methods to Make Procurements Easier and to Fully Meet Public Transportation Agency Expectations

Blake Christie and Ann DiephausApril 3, 2014

22© 2014 Noblis, Inc. Noblis proprietary and confidential.

The Transportation Challenge

■ More Vehicles■ More Drivers■ More Trips

But –

■ Fewer Resources■ Space Limits on New Road Construction■ Frequent Local Opposition to Mass Transit Solutions

Means –

We have congestion, long commutes, dangerous roadways, and …

http://www.freefoto.com/preview.jsp?id=31-43-12


So What’s The Solution?

■ More money would help, but that’s not the total answer

■ Technology can help – Intelligent Transportation Systems

■ Intelligent Transportation Systems (ITS) include:• Better information for the surface transportation system manager (e.g., sensors,

cameras, weather stations)• Better information for the surface transportation system user (e.g., 5-1-1, travel

time maps on the Internet)• Technology that reduces the time spent in travel-related activities, through the

use of automation (e.g., toll collection)


Technology Solutions Bring Their Own Challenges

■ New technology is generally proprietary – and expensive■ Competing technologies don’t start out interoperating, complicating the use

of similar items in different locations (e.g., Smart Tag vs. E-Pass)■ Users don’t necessarily have a “vision” for how to integrate all of the

necessary systems• In transportation, engineers are primarily Civil Engineers, experienced in

construction, not systems• Information systems involve software, which has its own known set of challenges• In the early days of ITS (and to a lesser degree today), there was the “NIH”

syndrome


U.S. Department of Transportation Role

■ Fostered research and development of:• “Intelligent Highways”• Intelligent Transportation Systems (successor to “intelligent highways”)

■ Sponsored the development of a National ITS Architecture• Conceptual description of the expected systems and the infrastructure that ties

them together• Framework for states and other jurisdictions building intelligent transportation

systems• Adding connected vehicles architecture• Provides the “Big Picture”

■ Recognized the need for standards to support the development of intelligent transportation systems and sponsored their development


Issues With ITS Standards Development

■ Early standards weren’t successful• Failed to capture the full scope of the areas they were intended to address• Were limited by the lack of systems experience by the people developing them• Were ignored (because of limitations) by the agencies fielding ITS systems• Were hard to read and ambiguous

■ Revisions to standards weren’t successful• Groups working on the standards lacked sufficient systems expertise• Users were inadequately involved in modifications, largely because of a lack of

systems expertise – but the standards were intended for them!!


Systems Engineering – To the Rescue!

■ Proposal made to use a systems engineering process for ITS standards development

■ Why?• Provide a context for the standard – concept of how it would be used in actual

operation of a system• Develop clear-cut requirements, based on user needs, for the interfaces and

devices requiring standardization• Trace the requirements back to user needs, to show users how the standard

evolved• Design standard solutions that addressed requirements• Trace standard solutions back to requirements• Create mechanism for testing products that claimed conformance to the standard


Development of the Systems Engineering Profile for ITS Standards

■ Identified the appropriate “ilities”• Usability• Readability• Maintainability• Interoperability• Flexibility

■ Mapped the interface standards development process to system life-cycle stages• Concept of Operations• Requirements• Needs-to-Requirements Matrix• Design• Requirements Traceability Matrix• Verification and Validation

9© 2014 Noblis, Inc. Noblis proprietary and confidential. 9

The “V” Diagram (Development Life-Cycle)

Conceptof Operations

High LevelRequirements

DetailedRequirements

High LevelDesign

DetailedDesign

Implementation

Operations &Maintenance

SystemAcceptance

SubsystemVerification

Integration &Test

Time

Relates to

Relates to

Relates to

Configuration Control Begins With the First

Product and Continues Throughout the Life of

the System

Development Translates Design Into Product


Systems Engineering and Quality

■ Quality Concerns Exist:• Within each stage• At the transitions between stages

■ Within a stage, solutions must validly reflect the intent and content of the product(s) of the previous stage

■ At the transitions, the stakeholders must verify the products that result from a stage• Each product must be complete• Each product must be correct




High LevelDesign

DetailedDesign

Implementation


SystemAcceptance


Integration &Test

We want to improve the quality of the standards


What Systems Engineering Provides to ITS Standards Development

■ Creates a quality standard■ Ensures that the standard meets user needs■ Verifies that the standard is complete ■ Validates that the standard is correct■ Supports deployments■ Reduces cost of interface development


Profile for a ConOps for a Standard

■ Define User Needs• Description of what the users want to do• Highest level of “requirement” in the system

■ Establish Operational Policies and Constraints• What policies govern the operation of the system• What constraints does the system have to accommodate

■ Delineate Modes of Operation• Normal mode (rush hour, scheduled events, etc.)• Other modes (accidents, sever weather, tec.)

■ Provide Operational Scenarios (optional) – used to give examples of how the user (or system) may operate with the capability desired

■ Tell a story■ Introduced use of Guides:

• IEEE Std 1362-1998 (ConOps) • INCOSE SYSTEMS ENGINEERING HANDBOOK, version 3.1, section

3.3.2• IEEE Std 1028-2008 (for technical reviews and walkthroughs)


Introduced the Characteristics of Good Functional Requirements

■ Necessary■ Concise (minimal, understandable)■ Attainable (achievable or feasible)■ Complete (standalone)■ Consistent■ Unambiguous■ Verifiable

Guided by IEEE 1233-1998 & INCOSE SYSTEMS ENGINEERING HANDBOOK version 3.1, Appendix I

Needs

Requirements

Based

Needs to Requirements Matrix


Introduced the Profile of Design Concepts

■ Based on the requirements■ Directly traceable to one or more requirements■ Consistent with the requirements■ Follow rule of one (a single) definition

(How many times should you define an object?)

Requirements

Designs

Driven

Requirements Traceability Matrix (RTM)


Introduced Basics of Verification and Validation

Verification and Validation are related concepts

■ Verification is “building the product right” -- ensures that all functions implemented in the product have been implemented correctly

■ Validation is “building the right product” – ensures that the desired functions have been implemented in the delivered product

V&V Methodologies


Introduced What V&V is at Each Stage




High LevelDesign

DetailedDesign

Implementation


SystemAcceptance


Integration &Test

V&V for ConOps asks, is this a complete set of needs and is each need correctly

described? V&V for requirements asks, do the requirements address all the

needs (completeness) and do respective requirements fulfill the

need (correctness)?

V&V for the design asks, are the requirement(s) all addressed

(completeness) and does the design fulfill each requirement

(correctness)?


Introduced the Needs-to-Requirements Matrix

Shows which requirements fulfill specific user needs Every Need Must Be Addressed By At Least One Requirement Every Requirement Must Relate to At Least One Need Any Need That is Not Addressed By At Least One Requirement may be

Irrelevant Every Requirement That Does Not Address At Least One Need is

Irrelevant


What is a Needs-to-Requirements Matrix used for?

■ Provides tool for completeness and correctness (V&V)■ Helps guide procurements■ Establishes high level picture (how it fits)■ Maps (references) to details■ Sets up basis for design


What Does the Needs-to-Requirements matrix Look like in ITS Standards

User Need ID

User Need FR ID Functional

Requirement Conformance Support Additional Specifications

2.5.2.3.3 Define a Message VMS:M Yes / NA

3.5.1.2.3.1 Determine Maximum Number of Pages

M Yes

3.5.1.2.3.2 Determine Maximum Message Length

M Yes

3.5.1.2.3.3 Determine Supported Color Schemes

M Yes

3.5.2.3.2.3Configure Default Flash-On and Flash-Off Times

O Yes / No

The DMS shall support all flash on times from ____ tenths of a second (0..255) to ____ tenths of a second (0..255) in ____ tenths of a second increments. The DMS shall support all flash off times from ____ tenths of a second (0..255) to ____ tenths of a second (0..255) in ____ tenths of a second increments.


How is the Needs-to-Requirements Matrix Used to make Procurements Easier?

The system owner selects the operational concepts they need The system owner addressed the optional requirements as needed The filled in needs-to-requirements matrix becomes the requirements

selected for the device interface When included “as specified in the standard” in contract language, off-the-

shelf implementations are achievable Basis for all interface testing

“Overall, VTTI feels that the project has been a resounding success.The specification process meets the goals of creating a more user-friendly environment for an agency to develop procurements.”

Ashwin AmannaVirginia Tech Transportation Institute

Deployment and Testing of an Updated Dynamic Message Sign (DMS) Standard, FINAL REPORT, April 24, 2007


What Does the Needs-to-Requirements matrix Look like in ITS Standards

User Need ID

User Need FR ID Functional

Requirement Conformance Support Additional Specifications

2.5.2.3.3 Define a Message VMS:M Yes / NA

3.5.1.2.3.1 Determine Maximum Number of Pages

M Yes

3.5.1.2.3.2 Determine Maximum Message Length

M Yes

3.5.1.2.3.3 Determine Supported Color Schemes

M Yes

3.5.2.3.2.3Configure Default Flash-On and Flash-Off Times

O Yes / No

The DMS shall support all flash on times from ____ tenths of a second (0..255) to ____ tenths of a second (0..255) in ____ tenths of a second increments. The DMS shall support all flash off times from ____ tenths of a second (0..255) to ____ tenths of a second (0..255) in ____ tenths of a second increments.


Introduced the Requirements Traceability Matrix (RTM)

■ Tells developers how specifically to meet a given requirement■ Traces from requirements to design (dialog and associated objects)

• Every Requirement Must Be Addressed By At Least One “Thing” (Dialog, Object, Element)

• Every “Thing” Must Relate Back to At Least One Requirement• Any Requirement That is Not Addressed By At Least One “Thing” is Unfulfilled

(Unsatisfied)• Any “Thing” That Does Not Address At Least One Requirement is Unnecessary


What is a RTM Used For?

■ Provides tool for completeness and correctness checks (V&V) of the design■ Specifies the “how” for development■ Maps (or references) requirements to design


What does the RTM Look like in ITS Standards?

FR ID FunctionalRequirement

Dialog ID

Object ID

Object

3.5.2.3.2.3 Configure Default Flash-On and Flash-Off Times

G.3

5.5.3 defaultFlashOn

5.5.5 defaultFlashOff

2525© 2014 Noblis, Inc. Noblis proprietary and confidential. Slide-25

Example Dialog

Center Signal Controller

GET (activeVolumeOccupancyDetectors)

RESPONSE (activeVolumeOccupancyDetectors)

GET (volumeOccupancyTable)

RESPONSE (volumeOccupancyTable)

Determine number of rows

in table

GET (volumeOccupancySequence)RESPONSE

(volumeOccupancySequence)Determine if a new table is available

Collect volume & occupancy

data

Read Volume/Occupancy Data Dialogue Sequence

Obj

ects


How is the RTM used to make procurements and testing easier?

The RTM specifies how the interface is to precisely behave Think of it as an Interface Control Document (ICD) When included “as specified in the standard” in contract language, off-the-

shelf implementations are achievable Basis for all interface testing

27© 2014 Noblis, Inc. Noblis proprietary and confidential. 27

Introduction of Advanced Verification Concepts




High LevelDesign

DetailedDesign

Implementation


SystemAcceptance


Integration &Test

Time

Relates to

Relates to

Relates to

Advanced Planning of Verification (Inspection,

Analysis, Demonstration, Test) Methods Introduced to work on the right

side of the “Vee”


Introduced standardized test procedures

Problem: Planning for verification was limited, resulting in inconsistent testing for conformance

Solution: Provides a requirements to test case/test procedure matrix Provides IEEE std 829-1998 guided test case/test procedures Standardized test procedures verify conformance Implementations use the test procedures; results are consistent

“… The testing team was able to quickly identify problems and assign responsibility. This process fostered an amicable environment between the agency and the vendor which produced very fast resolution to problems.”

Ashwin AmannaVirginia Tech Transportation Institute

Deployment and Testing of an Updated Dynamic Message Sign (DMS) Standard, FINAL REPORT, April 24, 2007


Requirements to Test Case/Procedure Martix

By planning the testing out, local agencies were assured that the system was verified.

Requirements to Test Case & Test Procedure Matrix

Requirement Test CaseID Title ID Title

3.5.3.1.1.3 Execute Climate-Control Equipment Testing C.3.5.3

C.3.5.4

Climate-Control Equipment Test - No ErrorsClimate-Control Equipment Test - Errors


Standardized test procedures

Test Case & Test Procedure Consistent Formats

Step Test Procedure Results Additional References

1 CONFIGURE: Determine the event class to utilize for this test (e.g., per the test plan). RECORD this information as: »Class_Index

16 GET the following object(s): »eventConfigStatus.Event_Index

Pass / Fail(Section 3.4.2.2)

Section H.3.1.2Step d

17 VERIFY that the RESPONSE VALUE for eventConfigStatus.Event_Index is equal to ‘log’ (3). NOTE--Valid enumerated values for eventConfigMode are defined in NTCIP 1103, Section A.7.5.1.9 (Event Log Configuration Status Parameter).

Pass/Fail (Section 3.4.2.2)

Includes test descriptions, identification of variables, test case results, etc.


Summary

Systems engineering ■ Formalizes ITS Standard development in three stages (ConOps,

requirements, design)■ Ensures quality in deployable standards that support interoperability and

interchangeability■ Provides for Verification & Validation of the standard at each stage■ Combines systems engineering expertise with existing domain expertise, as

best practices recommend

The new ITS Standards, developed using systems engineering concepts make it easier to procure and test


Questions?


Additional References

Applying Systems Engineering Principles to the Development of Transportation Communication Standards - See more at: http://www.standards.its.dot.gov/LearnAboutStandards/SystemsEngineering#sthash.dgcraB3G.dpuf

The systems engineering process is used during the development of ITS standards - See more at: http://www.standards.its.dot.gov/LearnAboutStandards/SystemsEngineering#sthash.yj2xUrat.dpuf

The ITS Standards Program is managed and funded by the United States Department of Transportation (USDOT), Intelligent Transportation Systems Joint Program Office (ITS JPO), Steve Sill is the Program Manager and can be contacted at 202-366-1603 or [email protected]

USDOT ITS Standards Website http://www.standards.its.dot.gov/ For further information, contact:

• Blake Christie at [email protected] and 202-488-5711• Ann Diephaus at [email protected] and 202-488-5718

http://www.standards.its.dot.gov/LearnAboutStandards/SystemsEngineering#sthash.dgcraB3G.dpuf



http://www.standards.its.dot.gov/LearnAboutStandards/SystemsEngineering#sthash.yj2xUrat.dpuf



mailto:[email protected]

http://www.standards.its.dot.gov/

http://www.standards.its.dot.gov/



Documents

Using Systems Engineering Processes and Methods to Make Procurements Easier and to Fully Meet Public Transportation Agency Expectations