Binary Instance Loading

©2011 by Pathfinder Solutions

Binary Instance Loading

Version 1.0

July 14, 2011

PathMATE Technical Notes

Pathfinder Solutions

www.pathfindermdd.com

888-662-7284

Technical Note: Binary Instance Loading

Table Of Contents

1. Introduction.......................................................................................... 1

2. Feature Details...................................................................................... 1

User Selection ........................................................................................ 1

Data Format........................................................................................... 1

3. Marking Rules ....................................................................................... 3

4. Creating and Using Binary Instance Data.............................................. 3

File Creation........................................................................................... 4

Binary Instance Data Transmission ............................................................ 4

5. Feature Implementation Impact Areas ................................................. 4

System Router........................................................................................ 4

Domain Loader ....................................................................................... 4

Class CreateFromBinaryData Method.......................................................... 5

Linking Methods...................................................................................... 5

File Reader............................................................................................. 5

Interprocess Message Routing................................................................... 5

pfdsocket.hpp......................................................................................... 5

6. ISSUES, QUESTIONS AND OPEN DISCUSSION....................................... 5

SW Domain ............................................................................................ 5


Technical Note: Binary Instance Loading 1

1. Introduction

This Technical Note outlines binary instance data loading capabilities. Creation of instances for PIM domain classes is a key capability for PI-MDD systems, and different systems – and even different domains within a system – require different solutions to meet the overall functional and performance requirements. The binary instance data can be sent to the application as a socket-based stream, or loaded via a process-local file.

Binary instance loading provides the most concise data format and best performance for loading and inter-processor transport of large instance populations.

2. Feature Details

User Selection

The binary instance loading feature is enabled by marking classes and associations with the BinaryInstanceData property. Setting this to “TRUE” will generate appropriate binary instance data packet routing code at the system and domain levels, and instance creation and linking methods at the PIM class level.

Data Format

Records

Binary instance data is organized into a record format. These records may be sent via tcp/ip socket message and are recognizable to the standard PfdProcess interprocess message receiver, which routes them to the system dispatcher.

General Record Format

type domain # contents EOR marker value 1 to 4 <domain #> <varies> 0XEEEEEEEE

# bytes 1 2 varies 4

A binary instance data record has a byte for the type (1 to 4), a domain number, the contents, and then 4 bytes used as an inter-record marker. While the code producing or consuming a record intrinsically knows its contents, a marker is provided as a second check to aid in detecting corrupted or incomplete data.

Record Stream

Records are formed into a Stream that contain the Class Instance Records and Link Records for a single domain. A Record Stream starts with a Population Record, and ends with a Population Complete Record.

Records are conveyed via files or inter-process messages. A Record Stream can be grouped into one or more Record Blocks, where a Record Block is an individual file or



message containing one or more whole records. Streams and Blocks are subject to these constraints:

The first record of a Stream must be a Population Record

The last record of a Stream must be a Population Complete Record

All records in a Stream must be for the same domain

A Block must contain complete records only.

Population Record

value 1 <domain #> <count> 0XEEEEEEEE # bytes 1 2 4 4

This record indicates the total number of class instances that are to be loaded for this specified domain from this Stream (via the full set of records, regardless of blocking). This allows the domain’s binary instance loader to allocate an appropriately sized instance id-to-pointer table.

The instance id-to-pointer table is an array of pointers used to retain the information needed to rapidly process Link Records. A simple array of PfdObject* (the base class for all modeled classes), it has one entry for each instance in the Stream. It is created when a Population Record is processed, and deallocated when a Population Complete Record is processed.

Class Instance Record

value 2 <domain #> <class #> <instance id #> <attribute values> 0XEEEEEEEE# bytes 1 2 2 4 varies 4

This record contains the attribute values for the specified class instance. The address of the newly created instance is stored in the transient instance id-to-pointer table for subsequent Link Records. This table is indexed by instance_id, therefore instance ids must:

Be unique for each instance – of any class – in this Stream

Start at 0

Have no gaps in the value range

The attribute values are the serialized forms for each attribute (but without the type identifiers since they are known), to be retrieved with the get_<type>_from_buffer serialization utility functions.

Link Record

value 3 <domain #> <assoc #> <part. 1 instance id> <part. 2 instance id> 0XEEEEEEEE



# bytes 1 2 2 4 4 4

This record contains the information to link two instances participating in an association. The referenced instances must have been loaded previously in this Stream.

Link Existing Instances Record

value 3 <domain #> <assoc #> <part. 1 instance id

attribute value> <part. 2 instance id

attribute value> 0XEEEE # bytes 1 2 2 4 4 4

This record contains the information to link two instances participating in an association that were not necessarily part of this Stream. Each participant must have an indentifying attribute. The referenced instances must have been created before this record is processed.

Population Complete Record

Value 4 <domain #> 0XEEEEEEEE# bytes 1 2 4

This record indicates all instances to be loaded via binary instance data have been completed. The domain loader may deallocate the instance id-to-pointer table. After the receipt of a Population Complete Record another population may be loaded by sending a new Population Record.

Single Domain, Whole Record Blocking

It is generally assumed that a Block (file or message buffer) containing binary instance data records will only contain records for a single domain. It is also assumed to have whole records. That way code reading/receiving binary instance data records does not have to concern itself with trying to detect if a complete record has been read/received.

3. Marking Rules

Checking of the BinaryInstanceData markings ensures

No association has BinaryInstanceData == “TRUE” unless both participants have BinaryInstanceData == “TRUE”.

o If a supertype class has BinaryInstanceData == “TRUE” then BinaryInstanceData is set to “TRUE” automatically for all subtypes

4. Creating and Using Binary Instance Data



The suggested approach for creating binary instance data files is to build a Windows deployment of the target domain, load in the required instance data via XML and use the generated domain and class methods to create the binary instance data file.

Establishing the Instance Population

Use the standard XML instance file markings and run-time loading mechanisms to create the instances needed.

Instance Selection

Each class with has BinaryInstanceData == “TRUE” gets a Boolean data member writeBinary_ . This is set to TRUE by default when the instance is created. It is used in the domain method BinaryWriteCurrentPopulation.

File Creation

Each domain with one or more classes marked with BinaryInstanceData generates a static method BinaryWriteCurrentPopulation(String file_name). This method can be invoked at any time and writes out a complete binary instance data stream of all instances of class with BinaryInstanceData == “TRUE” and writeBinary_ == TRUE. It also writes link records for all associations with BinaryInstanceData == “TRUE” that have instances of both participants.

Binary Instance Data Transmission

To support the transmission of Binary Instance Data the new class PfdBinaryInstanceData is used. It is a subtype of PfdSerializable. This SW domain uses this to make messages containing binary instance data.

5. Feature Implementation Impact Areas

System Router

A system router method is generated when any class has its BinaryInstanceData property set to “TRUE”. It gets a Binary Instance Data Block, checks the Domain # in the first record and routs the block in its entirety to the appropriate domain.

Domain Loader

A domain loader method is generated when any of its classes has its BinaryInstanceData property set to “TRUE”. It gets the Binary Instance Data Block and steps through it one record at a time. For a Population Record it allocates an instance id-to-pointer table. Class Instance Records are passed into the appropriate class instance factory advancing the next record pointer appropriately. The pointer to the newly created instance is put in the instance id-to-pointer table. Link Records are decomposed by the loader and the appropriate instance pointers are passed into



link methods for each participant. A Population Complete Record triggers the deallocation of the instance id-to-pointer table.

Class CreateFromBinaryData Method

If a class has its BinaryInstanceData property set to “TRUE” a class gets an additional method: CreateFromBinaryData (message_buffer_t *binary_data_buffer, int * binary_data_buffer_length). This method retrieves the attribute values from the binary_data_buffer which has them in their serialized forms (but without the type identifiers since they are known), retrieving them with the get_<type>_from_buffer serialization utility functions. The binary_data_buffer pointer is incremented and the byte count parameters is decremented for the data consumed.

After all attribute values are consumed, the binary_data_buffer is checked for an EOR marker. If the marker is not found or any other error is encountered an error is reported, the new instance is deleted and NULL is returned.

Linking Methods

Standard instance linking methods are used.

File Reader

See IDS domain below.

Interprocess Message Routing

See IDS domain below.

pfdsocket.hpp

Extend sw_socket_msg_type_e with SW_SOCK_MSG_BINARY_INSTANCE_RECORD.

6. ISSUES, QUESTIONS AND OPEN DISCUSSION

SW Domain

We will add these services to the SoftwareMechanisms domain:

LoadBinaryInstanceData(String binary_instance_data_file_name);

SendBinaryInstanceData(DestinationHandle destination, String binary_instance_data_file_name);

We may also want to include ReadXMLInstanceData.

Technology

Binary Instance Loading