Writing a Validator for TMATS

Bryan Kelly83rd FWS/D04

Tyndall AFB, FL

Internet Sites

• www.bkelly.ws/irig_106• www.bkelly.ws/irig

Purpose

• Common validator needed• Proprietary vendors– May not be responsive– Do not advise users of defects

• Secure systems have software restrictions• Limited software budgets• Open Source effective solution

Audience

• Programmers– Intermediate Level– C++ knowledge presumed– Working knowledge of TMATS

• Users– Simple Interface, Simple Results– So Far

Open Source

• Eliminates malware• Capabilities and defects available to all• Easier to introduce into secure areas• Can be customized• End result is usually better code

Platform

• Windows 7• Visual Studio• C++• Core code is in classes independent from GUI• Easy to port, hopefully

Programmer’s Advisory#pragma message statements put this information in the Output dialog during build

1>***************************************1> 1>This project will not build until you understand how to use Additional Include Directories1>You may reference a Code Project article found here.1>http://www.codeproject.com/Tips/588022/Using-Additional-Include-Directories1> 1>This project is discussed in my Bulletin Board found here:1>http://www.bkelly.ws/irig_106/1>And in my web pages found here:1>http://www.bkelly.ws/irig1> 1>***************************************

Validate in Three Phases

1. Read the definitionsa) Definitions are easily editableb) Application loads them into an STD::MAPc) The map makes them randomly accessible

2. Read the Dataa) Stored into STD::MAP

3. Validationa) Iterate through datab) Compare each item with appropriate definition

Definitions

• The definitions are directly from the standard.• TMATS data is validated with the definitions.• Application Requirements– Easily editable– Easily readable by application– Easy to parse

• Storage– CSV (Comma Separated Value) file– Created via Excel

Definitions

• Group: This is from group G• Separators: \ - : ;• Identifier: DSI and DST• Enumerators: n, ( x, m, d, s, r, p )• Payload: text and RF\TAP\etc

Payload Categories

• Exact List– RHCP\LHCP\LIN

• Type– text– decimal– Integer– binary

• Case is critical

Upper & Lower Case• Some Payload are Mixed

– P-d\SYNC3 (example)• Payload can contain a positive integer• Payload can be exact: NS (Not specified), or,

• Some Complex– NRZ-L\NRZ-M\NRZ-S\BIO-L\BIO-M\BIO-S\RNRZ-L\OTHER– ? How to convey this to the validator

• Use Case– Upper case denotes exact text– Lower case denotes type of field– Definition payload: NS\dec

STD::MAP

• Standard Template Library– Available to all compliant C++ compilers– Easy to use

• Stores all the definitions• Provides random access• Requires a key• Selecting Key is critical

Constructing the Key

• Key =– Group– Identifier

• Difficulties– Identifier not consistent, it moves– Duplicate Identifiers– Major problem

Duplicates

• G\DSI\N:<decimal>; key = GDSI• G\DSI-n:text; key = GDSI• V-x\VN:text; Identifier moved• T-x\AP:text; key = TAP• T-x\AP\POC1:text; key = TAP• T-x\AP\POC2:text;• T-x\AP\POC3:text;

Expand Identifier Definition

• Identifier consists of each field that is preceded by a backslash, and is before the colon.

• All identifier fields are concatenated to build the key.

Key Construction

• First character is Group• Iterate– Find backslash– Concatenate backslash– Concatenate next token– Repeat to semicolon

• TSCON• T\SCO\N

Goal of Definition Phase

• Read the definitions• Parse into strings• Combine strings into a record• Construct a key• Store the record in map using key

Helper Files and Classes

C_Get_CString_Tokens.h, .cppC_Get_Cstring_Tokens_And_Separators.h, .cppC_Helper_Class.h, .cppC_Log_Writer.h, .cppGnu_Notice.h

Worker Classes

C_Tmats_Definitions.h, .cppreads defintion file, contains def. map

C_Tmats_Data Reads the TMATS data, contains data map

C_TMATS_Basic_Grammar_Checker Checks the grammar. More work to do.

Definition Classconst unsigned int TOKEN_DEFINITION_GROUP = 0;const unsigned int TOKEN_DEFINITION_02 = TOKEN_DEFINITION_GROUP + 1;…const unsigned int NUMBER_OF_DEFINITION_TOKENS = TOKEN_DEFINITION_24_FORMAT + 1;typedef struct{ CString token[ NUMBER_OF_DEFINITION_TOKENS ]; unsigned int max_payload_length;} td_tmats_definition;

Definition Map

std::map < CString, td_tmats_definition > m_tmats_definition_map;std::map < CString, td_tmats_definition > ::iterator m_tmats_definition_map_iterator;

CString declares the format of the map key.td_tmats_definition declares the type of data stored in the map. In this case, an array of CString, one per token of the definitions.

This map is the heart of the definition class.Everything serves one of two purposes:1. Acquiring and formatting the data to put records in the map.2. Fetching selected records from the map for validation.

Definition Review

• Definition File• Definition parsing• Building a key• Storing the data

Data Preview

• Similarities to Definitions– Parse into tokens, then into a record– Build key for retrieval– Build key for definition– Save record

• Differences– Parser must save separators

Data Parsing

• No control over the format– Unlike the definitions

• Definitions use CSV, commas are separators, discarded

• Data separators: \ - : ;• All fields in data are significant, no discards• Parser is different.

Get_Cstring_Tokens_And_Separators

• Data parser is– C_Get_CString_Tokens_And_Separators

• Uses a simple FSM to parse the data• Constructs a definition key• Data key is simple counting integer– Read from data map in same order as in file

TMATS_Basic_Grammar_Checker while( mp_C_Tmats_Data->Get_Next_Record( &one_data_record,

&data_key ) ){ definition_key = one_data_record.token[ TOKEN_DATA_26_DEFINITION_KEY ];

definition_status = mp_C_Tmats_Definitions->Get_One_Definition( definition_key, &one_definition_record );

data_status = Compare_Definition_And_Data( &one_definition_record, &one_data_record );}

FSM States enum ENUM_STATE { GROUP, // Always begins here DETERMINE_NEXT_STATE, // Separators, Sets next state COUNTER, // one of the enumerators: -n, -x, etc IDENTIFIER, PAYLOAD, SEMICOLON, EMPTY, // after the colon all tokens must be empty EXIT };

Payload Validation

• Non-payload fields easy to validate• Payload varies widely• Parse the definition of the payload– Dedicated parser

• Examine payload

Payload Types

• text• int• dec• ANAL\CASS\HDDR\PARA\SSR\MD\N\OTHR• INC\DEC\int differentiated from INC\DEC\INT• NO\dec

End of TMATS Marker

• There is none defined• How to differentiate between the end of

TMATS data and a bad TMATS record– One bad record? Two? Three? Possibly consumes

good data from Chapter 10 section.• Suggest a specific identifier for end of data• Create group to mark end. • E:END;

Questions?

• www.bkelly.ws/irig_106– Bulletin board– Questions, discussions

• www.bkelly.ws/irig– Paper, presentation– Visual Studio solution– Definition file