Nokia Convergent Mediation Engine

Aggregation Agent 4.1

Copyright © 2006 Nokia

Version 1.39

1


The contents of this document are subject to revision without notice due to continued progress in methodology, design, andmanufacturing.

Nokia shall have no liability for any errors or damage of any kind resulting from the use of this document.

Nokia® and Convergent Mediation Engine™ are registered trademarks or trademarks of Nokia. All other trade names andmarks mentioned herein are the property of their respective holders.

2


Table of Contents1. Introduction ............................................................................................................... 4

1.1. Prerequisites .................................................................................................... 41.2. Convergent Mediation Engine™ Documentation .................................................... 41.3. Abbreviations and Terms ................................................................................... 4

2. Aggregation Agent ...................................................................................................... 52.1. Session UDR Type ........................................................................................... 52.2. Profile ............................................................................................................ 5

2.2.1. Storage Tab .......................................................................................... 52.2.2. Association Tab ..................................................................................... 6

2.3. Agent Configuration ......................................................................................... 82.3.1. APL Code Tab ....................................................................................... 82.3.2. Error Handling Tab ................................................................................ 92.3.3. Timeout Tab ........................................................................................ 102.3.4. Aggregation Flow Chart ........................................................................ 10

2.4. Transaction Behavior - Batch Workflow .............................................................. 112.4.1. Emits ................................................................................................. 112.4.2. Retrieves ............................................................................................ 11

2.5. Introspection .................................................................................................. 122.6. Meta Information Model .................................................................................. 12

2.6.1. Publishes ............................................................................................ 122.6.2. Accesses ............................................................................................. 12

2.7. Node Message Events ...................................................................................... 122.8. Aggregation Cache and Memory Footprint .......................................................... 12

3. APL Extensions ........................................................................................................ 143.1. Function Blocks ............................................................................................. 14

3.1.1. initialize, deinitialize, drain, beginBatch, end-Batch, cancelBatch ............................................................................. 143.1.2. consume ........................................................................................... 143.1.3. command ........................................................................................... 143.1.4. sessionInit ................................................................................... 143.1.5. timeout ........................................................................................... 14

3.2. Built-In Variables ........................................................................................... 153.3. Built-In Functions ........................................................................................... 15

3.3.1. sessionRemove ............................................................................... 153.3.2. sessionTimeout ............................................................................. 15

3.4. Aggregation Command .................................................................................... 164. Aggregation Session Inspection ................................................................................... 17

4.1. The Search Sessions Dialog. ............................................................................. 175. Example - Association of IP Data ................................................................................. 19

5.1. Session Definition .......................................................................................... 195.2. Association - Radius UDRs .............................................................................. 205.3. Association - Netflow UDRs ............................................................................. 215.4. The APL Code ............................................................................................... 21

3


1. IntroductionThis document describes the Aggregation agent, being part of the standard agents available on theNokia Convergent Mediation Engine™ platform.

1.1. PrerequisitesThe user of this document has to be familiar with:

• The Convergent Mediation Engine™ Platform

• UDR structure and contents

• Analysis Programming Language

1.2. Convergent Mediation Engine™ DocumentationThe Analysis Programming Language description and syntax is listed in the Convergent MediationEngine™ Analysis Agent - User Guide.

The Ultra Format Definition Language is described in the Convergent Mediation Engine™ UltraFormat Management - User Guide.

1.3. Abbreviations and TermsAbbreviations and terms used in the Convergent Mediation Engine™ user documents, are describedin the Convergent Mediation Engine™ Core - User Guide.

4


2. Aggregation AgentThe Aggregation agent allows related UDRs from single or multiple sources to be aggregated or con-solidated into one UDR. Related UDRs are grouped into sessions based on configurable data fieldsfrom each UDR. When a UDR arrives, available sessions are evaluated to see if the UDR is associatedwith any of them. If a suitable session is found, the user defined APL code is executed.

The agent stores the session data on the file system through a storage server, usually running on theexecution context. If UDRs are to be stored, a list of UDRs has to be defined as a session variable.This list is then populated with each incoming UDR through the APL code.

An Aggregation node is configured in three steps. First a session UDR type has to be defined in Ultra,refer to the Convergent Mediation Engine™ Ultra Format Management - User Guide, for further in-formation. Then the profile is defined, followed by configuration of the node.

Take special precaution when changing, updating or renaming formats. If the updated formatdoes not contain all the fields of the historical format, which UDRs may already reside in theECS or Aggregation, data is lost. When a format is renamed, it will still contain all the fields.The data, however, cannot be collected.

2.1. Session UDR TypeEach aggregation profile stores sessions of a specific Session UDR Type, defined in Ultra. Refer toConvergent Mediation Engine™ Ultra Format Management - User Guide, for details on how commonUltra formats are defined.

A Session UDR Type is defined in the same way as internal Ultra types, but uses the keyword sessioninstead of internal, see the following example:

Example 1.

session SessionUDRType { int intField; string strField; list<drudr> udrList;};

2.2. ProfileThe Aggregation Profile Editor is accessed from the Configuration menu on the desktop. Severalworkflows may be configured to use the same Profile, but only one of the workflows may be activeat a time.

2.2.1. Storage Tab

The Storage tab holds information about what Session UDR Type to use and where the session datawill be stored.

5


Figure 1. Aggregation Profile Editor, Storage tab.

What Session UDR Type to use.Session UDR Type

What Execution Context (or Platform for some High Availability solutions) theaggregation workflow will run on.

Storage Host

The directory where the aggregation data will be stored.Directory

A parameter used for performance tuning. The sessions are saved on disk accord-ing to the following creation scheme:

Partial File Count

1. A reference file containing all the sessions.

2. A number of partial files, used for comparison.

3. A new, updated reference file containing all the sessions.

The Partial File Count parameter decides how many partial files are allowed.There are two aspects to consider:

• Startup - during startup all files are read. It takes longer if there are a lot ofpartial files (large Partial File Count). This has to be considered especiallyin a High Availability solution.

• Transaction commitment - When the transactions are committed, small files(large Partial File Count) will increase performance.

The maximum number of sessions to keep in memory cache.Max Cached Ses-sions

This is performance tuning parameter deciding the memory usage of the aggreg-ation agent. This value should be set low enough that there is space for the cachein memory, but not too low, since this will give poor performance. For detaileddiscussion on how to decide on a good value see Section 2.8, “Aggregation Cacheand Memory Footprint”.

2.2.2. Association Tab

The Association tab holds information about the rules used to match each incoming UDR to a session.For each UDR Type a set of rules may be defined which are processed in order. For most cases onlyone rule per incoming UDR type will be defined.

A primary expression may be used to filter out UDRs that are candidates for a specific rule. If the UDRis filtered out by the primary expression, it is matched to the existing sessions by using one or severalID Fields as a key.

For UDRs whose ID Fields match an existing session, an additional expression may be used to specifyadditional matching criteria. For instance, if dynamic IP addresses are handed out to customers basedon time intervals, the field containing the IP address could be used as ID Field while the actual timecould be compared in the Additional Expression.

6


Figure 2. Aggregation Profile Editor, Association tab.

Type of UDRs that are expected by the node. Each UDR Type may have a list ofrules attached to it. Selecting the UDR type will display its rules in separate tabs.

UDR Types

An optional expression to be evaluated before the ID Fields are evaluated. If itevaluates to false the rule is ignored and the evaluation continues with the nextrule.

Primary Expres-sion

The syntax follows APL where the input variable is available for use.

List of fields that, together with the Additional Expression, are used to determineif a UDR belongs to an existing session. If the contents of the selected fields match

ID Fields

the contents of a session, and possibly an Additional Expression evaluated totrue, the UDR belongs to the session.

It is of major importance that the fields selected are of the same type and inthe same order for all rules defined for the node.

An optional expression to be evaluated together with the ID Fields.Additional Ex-pression

The syntax follows APL where the input and session variables are availablefor use.

The Additional Expression is useful when having several UDR Types to be con-solidated and they differ in number of ID fields. Having several UDR Types requiresthe ID fields to equal in number and type. If one of the types requires additionalfields which do not have any counterpart in the other type/types, this must be handledvia the Additional Expression. Save the field contents as a session variable, andcompare new UDRs against it. Refer to Section 5.2, “Association - Radius UDRs”for an illustration.

If no matching session is located this option determines if a new session will becreated or not. Note, if a primary expression is present and evaluates to false, therule is ignored and no new session will be created.

Create Sessionon Failure

If UDRs are expected in a particular sequence, in order for a session to be created,this option must be enabled on the UDR type/field to be considered master and isexpected to come first. If this is the case, a slave UDR type/field will be target forthe error handling if it happens to arrive before its master.

7


If the option is enabled on all rules, the order of the arriving UDRs are of no import-ance. A new session will be created no matter what UDR comes first. At least oneof all defined rules must have this option enabled or else no session would ever becreated.

Adds a new rule (tab) for the selected UDR Type. In most cases only one rule isrequired but in situations where a session is kept on IP number, and this may be

Add Rule

stored in either a target or source IP field, two rules are required with the source IPfield as ID Field in the first rule and the target IP field as ID Field for the second.

Removes the currently displayed rule.Remove Rule

2.3. Agent ConfigurationThe use and setting of private threads for an agent, enabling multi-threading within a workflow, isconfigured in the Thread Buffer tab. Please refer to Convergent Mediation Engine™ Core - UserGuide, for further details.

2.3.1. APL Code Tab

The code manages the detailed behavior of the Aggregation node and is written using the AnalysisProgramming Language (APL) with some limitations and some additional functionality. Refer toSection 3, “APL Extensions” for details.

The most important function block of the code is the consume block. This block is called whenevera UDR has been associated with an existing or new session.

A timeout block should normally be used to handle sessions that have not been successfully closeddue to that the final UDR has not arrived.

Figure 3. Aggregation agent configuration window, APL Code tab.

The selected Aggregation Profile.Profile

Text area where the APL code, used for UDR processing, is entered. Below the textarea there are line, column and position indicators, for help when locating syntaxerrors.

Code Area

8


For input assistance the area offers a pop-up menu, displayed when right-clickingand selecting

• UDR Assistance.... This option displays the UDR Internal Format Browserfrom which the UDR Fields may be inserted into the code area.

• MIM Assistance.... This option displays the MIM Browser from which theavailable MIM Resources may be inserted into the code area.

For further information on these options, please refer to Convergent Mediation En-gine™ Core - User Guide.

Compiles the entered code to check for validity. The status of the compilation isdisplayed in a dialog. Upon failure, the erroneous line is highlighted and a message,including the line number, is displayed.

CompilationTest...

2.3.2. Error Handling Tab

The tab contains information on what to do if no timeout is set in the code and what to do with un-matched UDRs.

Figure 4. Aggregation agent configuration window, Error Handling tab.

Action to take if no timeout for a session is set in the APL code using session-Timeout. The setting is evaluated after each consume or timeout functionblock has been called (assuming the session has not been removed).

Action on Miss-ing Timeout

Options are:

• Ignore - Do nothing. Note that this may leave sessions forever in the systemif the closing UDR does not arrive.

• Abort - Abort the node execution. This option is used if timeout must be setat all times. Hence, a missing timeout is considered being a configuration error.

• Use Default Timeout - Allow the session timeout to be set here instead fromwithin the code. If enabled, a field becomes available where the timeout, inseconds, may be specified.

If a UDR arrives that does not match any session and Create Session on Failureis disabled, then this option determines what the agent will do with the UDR.

Action for Un-matched UDR

• Ignore - Silently throw away the UDR.

• Abort - Abort the node execution. This option is used if all UDRs are associatedwith a session, then this error case means a configuration error.

• Route - Sends the UDR on the route selected from the on list.

9


A list of outgoing routes on which the UDR can be sent. Activated only in caseRoute is selected as Action for Unmatched UDR.

on

2.3.3.Timeout Tab

The Timeout tab is available for Realtime workflows only. Since there are no natural batch boundariesactivating an evaluation of outdated sessions, this is handled by setting a regular interval when thiswill be done. Note, this will be performed at any time when the workflow is active, whether data isbeing processed or not.

In this tab it is also specified when the changes to the aggregation data are written to file. Note thatdata that has not yet been committed to storage will not be visible in the session inspector.

Figure 5. Aggregation agent configuration window, Timeout tab.

How often, in seconds, the timeout block will be activated forall outdated sessions.

Check Session Timeout Everyn seconds

How often, in seconds, the in memory data will be saved to fileson disk.

Commit Storage After nseconds

How often, counting the number of Processing Calls, the in memorydata will be saved to files on disk. A 'Processing Call' is an execu-tion of any of the blocks consume, command or timeout.

Commit Storage After nProcessing Calls

If both this option and the Commit storage after seconds is con-figured, commits will be made when any of them is fulfilled.

2.3.4. Aggregation Flow Chart

Figure 6, “Aggregation flow chart.” illustrates how an incoming UDR is treated when it enters theAggregation node. If the UDR leaves the workflow without having called any APL code, it is targetedfor error handling.

10


Figure 6. Aggregation flow chart.

If several matching sessions are found, the first one is updated. If this occurs, redesign the workflow.There must always be zero or one matching session for each UDR.

2.4.Transaction Behavior - Batch Workflow

2.4.1. Emits

The agent itself does not emit Cancel Batch messages. However, if the code containsa call to the method cancelBatch this will make the agent emit a Cancel Batch.

Cancel Batch

If the code contains a call to the method hintEndBatch, this will make the agentemit a Hint End Batch. Note, only some collectors can act upon such a request. Referto the user guide of the respective collection agent, for further information.

Hint End Batch

2.4.2. Retrieves

When a Begin Batch message is received the agent calls the beginBatch functionblock, if present in the code.

Begin Batch

When an End Batch message is received the agent calls the endBatch function blocks,if present in the code.

End Batch

Prior to End Batch, possible timeouts are called. Thus, when a timer elapses, thetimeout function block will not be called until the next End Batch arrives. If theworkflow is in the middle of a data batch or is not currently receiving any data at all,this could potentially be some time after the configured timeout.

11


When a Cancel Batch message is received the agent calls the cancelBatch functionblock, if present in the code.

Cancel Batch

2.5. IntrospectionThe agent produces UDR or bytearray types, depending on the code since UDRs may be dynam-ically created. It consumes any of the types selected in the UDR Types list.

2.6. Meta Information Model

2.6.1. Publishes

Number of sessions in storage on the file system.Online Session Count

Type int, defined as MIM type global.

Number of sessions cached in memory.Session Count

Type int, defined as MIM type global.

2.6.2. Accesses

The agent does not itself access any MIM resources. However, APL offers the possibility of bothpublishing and accessing MIM resources and values.

For a listing of general Convergent Mediation Engine™ MIM resources, refer to ConvergentMediation Engine™ Core - User Guide.

2.7. Node Message EventsThe agent does not itself produce any events. However, APL offers the possibility of producing events.

2.8. Aggregation Cache and Memory FootprintThis section describes how the aggregation cache works and what considerations should be taken whendetermining the memory usage of the aggregation agent.

The maximum size of the cache will be determined by the Max Cached Sessions parameter in theAggregation Profile (see Section 2.2, “Profile”) and the average size in memory of a session. It isdifficult to estimate the exact memory consumption exception through testing. The following shouldbe considered when implementing an aggregation workflow.

1. Try to keep the session data small. Specifically, do not use large maps or lists in the sessions.These will take up a lot memory.

2. If memory issues are encountered, try decreasing the Max Cached Sessions. In order to find outif the cache size is overdimensioned, the memory statistics can be studied for the ExecutionContext hosting the workflow.

12


To avoid a large aggregation cache causing out of memory errors, the aggregation cache detects thatthe memory limit is reached. Once this is detected, sessions will be moved from the memory cache tothe file system. Note that this has a performance impact, since the agent will have to read these sessionsfrom the file system if they are accessed again. The aggregation agent will log information to the Exe-cution Context's log file in case the memory limit has been reached and the size of the cache needs tobe adapted.

While the aggregation cache will never cause the Execution Context to run out of memory, it is stilla good idea to set the Max Cached Sessions low enough that there is enough space for the full cachesize in memory. This will increase system performance.

13


3. APL ExtensionsThe aggregation APL code may be called when a node initializes and deinitializes, on Begin and EndBatch, when a session is created, when a timeout elapses and when a matching session for a UDR hasbeen found or has been created.

3.1. Function BlocksThe following function blocks are available for use in the aggregation APL code. Note that only theblocks defined here are permitted.

3.1.1.initialize, deinitialize, drain, beginBatch, end-Batch, cancelBatch

Called as explained in the Convergent Mediation Engine™ Analysis Agent - User Guide.

The session variable is not available within these function blocks.

3.1.2. consume

Called each time a UDR arrives that matches a session or causes a new session to be created. Thus,the consume block is not necessarily called for each arriving UDR. The UDR will be available throughthe input variable and the session through the session variable.

3.1.3. command

Called for all flushed sessions. The flush action is initiated from the Workflow Manager or throughthe commandline tool mzsh. Within this block the instruction variable is available and containsa string, optionally sent on by the user through the command. For further information, please refer toSection 3.4, “Aggregation Command” and Section 3.2, “Built-In Variables”.

3.1.4. sessionInit

Called each time a new session is created. The UDR that caused the session to be created will beavailable through the input variable and the new session through the session variable. The ses-sionInit block is optional, but it is recommended to use since it makes the code easier to follow.

3.1.5. timeout

The timeout block handles sessions that for some reason has been left hanging in the system. Forinstance, if the stop record was never received from the Radius agent before the timeout was reached.

Between drain and endBatch for each batch passing the aggregation node (batch workflow) or asdefined in the node configuration (Realtime workflow), the timeout block is called for all outdatedsessions. The outdated session will be available through the session variable.

Initially, make sure the code in the timeout block handles the session properly, then there are twoalternatives on how to handle the session.

• If the session is not needed anymore, remove the session with sessionRemove.

• If relevant UDRs are still expected, set a new timeout with sessionTimeout.

14


If these are ignored, outdated sessions that may never be used again will remain in the system foreverand will always be active for comparison against incoming data, which could have a negative impacton performance.

If there are old sessions with no timeout set, correct the timeout block and then open the AggregationSession Inspector. If there is no use of the sessions they may be deleted, otherwise set a new timeoutfor these sessions and they will be handled by the timeout block as soon as a new batch is processedby the Aggregation node.

3.2. Built-In VariablesThe following built-in variables are specific to the Aggregation node.

A reference to the current session to be used to access variables defined in theStorage tab. Note that a session remains in the database until manually removed.A route to ECS or an alternative route will not remove it.

session

The variable is available in the consume, sessionInit and timeout functionblocks.

An optionally inserted string, belonging to the currently flushed session.instruction

The variable is available in the command function block only.

3.3. Built-In FunctionsThe following functions may be utilized from the aggregation APL code. All functions described inthe Convergent Mediation Engine™ Analysis Agent - User Guide are also available for use.

3.3.1. sessionRemove

Removes the current session.

This function must be called when a session is considered to be closed and a new UDR has beenemitted, or when a session is considered to be closed due to an expired timeout.

sessionRemove(session)

session is a reserved word and cannot be replaced.

3.3.2. sessionTimeout

Sets a timer to call the timeout function block or, if not available the consume block, within thenumber of seconds specified or on an exact time. Only one timeout per session may exist, thus if asubsequent call to this function is made, the old timeout is cancelled.

Timeouts are not always called at the exact time stated. Refer to Section 2.4, “Transaction Behavior- Batch Workflow” for details on when the timer is actually invoked.

The timeout for a session is cleared each time the consume or timeout function blocks are called.

sessionTimeout(session, int timeoutSeconds)


Parameters:

15


Number of seconds. If set to 0(zero) or null, the timeout is removed. This isuseful if it is desired to reset the timeout to the default timeout.

timeoutSeconds

If set to a negative value, the session will be outdated.

sessionTimeout(session, date timeoutDate)


Parameters:

An exact date. If set to null, the timeout is removed. This is useful if it is desiredto reset the timeout to the default timeout.

timeoutDate

If set to a passed date, the session will be outdated.

3.4. Aggregation CommandTo flush stored sessions the commandline tool mzsh or the Command tab, available in monitor modeon the aggregation agent, can be used. There is a choice of applying the commmand on all sessions oronly on those with a timeout set. There is also an option of attaching information in a string to beavailable within the command function block in the Aggregation node.

Note that the command function block works the same way as the timeout function block, i.e. codehas to be written to route the session result, and to remove the session.

Syntax to run the command function block from mzsh:

mzsh wfcommand <Workflow Name> <Node Name> <"true"|"false"> <instruction>

Parameters:

Name of the workflow which contains the Aggregation node.Workflow Name

Name of the Aggregation node in the workflow.Node Name

Specifies what sessions to apply the command block on. "true" means applyon all sessions. "false" means apply only on sessions with timeout set. To have

"true"|"false"

the sessions removed, the command block must issue sessionRemove(ses-sion).

An optional string. If specified, the string will be available within the commandfunction block in the Aggregation node through the instruction variable.Please refer to Section 3.2, “Built-In Variables” for further information.

instruction

A simple APL example of command:

command { OutputUDR finalUDR = udrCreate( OutputUDR ); finalUDR.user = session.user; finalUDR.IPAddress = (string)session.IPAddress; finalUDR.downloadedBytes = session.downloadedBytes; finalUDR.uploadedBytes = session.uploadedBytes; if (instruction == "extra") { finalUDR.extraBytes = session.extraBytes; } udrRoute( finalUDR ); sessionRemove(session);}

16


4. Aggregation Session InspectionThe Aggregation Session Inspector enables the user to view and edit the existing sessions. The datais displayed in the form of a table where each row represents a session, with the session data orderedin configurable columns. It is also possible to edit the contents of the sessions, that is, to change thetimeout and the values of the session variables.

The Aggregation Session Inspector is accessed from the Inspection menu on the desktop.

Figure 7. The Aggregation Session Inspector.

Displays the Search Session dialog where search criteria may be defined toidentify the group of sessions to be displayed.

Edit menu Search...

Displays a new window where the session variables may be viewed andpossibly edited, if Read Only was disabled in the Search Session dialog.An example of a UDR Viewer window:

Edit menu ExploreSession

Note, the window also appears if the Index column of the session is double-clicked.

Allows a more detailed view of the UDRs in the session list. For further in-formation on UDR Views, please refer to Convergent Mediation Engine™Ultra Format Management - User Guide.

Edit menu Views

4.1.The Search Sessions Dialog.The group of sessions to view is selected in the Search Session dialog, displayed when selectingSearch... from the Edit menu.

17


Figure 8. Search Sessions dialog.

Select the profile that corresponds to the data of interest.Profile

Option to search for data from a certain timeout period.Timeout Period

Disable Read Only if content of the sessions need to be altered. Exclusive accessto the repository is required to alter the sessions, meaning that if a currently running

Lock Handling

workflow is using the selected profile, the workflow needs to be stopped to be ableto get exclusive access.

18


5. Example - Association of IP DataTo illustrate the features available through the Aggregation agent, an association example accordingto the following workflow setup is presented. The workflow is handling IP traffic data, and will groupinformation from routers and the corresponding network access servers.

Figure 9. An example where an Aggregation node is used to associate IP data.

The Netflow agent collects router data logging the interacting network elements' addresses and amountof bytes handled, while the Radius agent keeps track of who has initiated the connection, and for howlong the connection was up. Thus, each user login session will consist of two Radius UDRs (start andstop), and one or several Netflow UDRs. The Aggregation agent is used to associate this data fromeach login session. These additional rules apply:

• A Radius UDR belonging to a specific login session must always arrive before its correspondingNetflow UDRs. If a Netflow UDR arrives without a preceding Radius UDR, it must be deleted.

• Within a Netflow UDR, the user initiating the session may act as a source or destination, dependingon the direction of data transfer. Thus, it is important to match the IP address from the RadiusUDRs with source or destination IP from the Netflow UDRs.

The Radius specific response handling will not be discussed in this example. Please refer to theConvergent Mediation Engine™ Radius Agent - User Guide, for this purpose.

5.1. Session DefinitionFor each session, all the necessary data must be saved. A suggestion of useful variables for this scenarioare described in the following. Note, the input UDRs are not stored. Information from the UDRs isextracted and saved in the session variables.

The Ultra definition of the session type

session ExampleSession { string user; string IPAddress; long sessionID; long downloadedBytes; long uploadedBytes;};

19


The user initiating the network connection. This value is fetched from thestart Radius UDR.

user

The IP address of the user initiating the network connection. This value isfetched from the start Radius UDR.

IPAddress

A unique ID grouping a specific network connection session for the specificuser. This value is fetched from the start Radius UDR.

sessionID

The amount of downloaded bytes according to information extracted fromNetflow UDRs.

downloadedBytes

The amount of uploaded bytes according to information extracted from NetflowUDRs.

uploadedBytes

5.2. Association - Radius UDRsThe Radius UDRs are the Aggregation session-initiating units. They may be of two types in this ex-ample; start or stop.

Note the use of the Additional Expression. The fields associating the start and stop Radius UDRs areframedIPAddress and acctSessId. However, since there is no field matching the latter withinthe Netflow UDRs, this field cannot be entered in the ID Fields area.

Figure 10. The Aggregation Profile, Association tab - Radius UDRs.

This is how arriving Radius UDRs are evaluated when configured according to Figure 10, “The Ag-gregation Profile, Association tab - Radius UDRs.”:

1. Initially, the UDR is evaluated against the Primary Expression. If it evaluates to false, allfurther validation is interrupted and the UDR will be deleted without logging (since no more rulesexist). Typically, it is desired to delete invalid UDRs. In this case, only the UDRs of type start(acctStatusType=1) or stop (acctStatusType=2) are considered.

2. If the Primary Expression evaluation was successful, the fields entered in the ID Fields area, to-gether with the Additional Expression are used as a secondary verification. If it evaluates totrue, the UDR will be added to the session, if not - see subsequent step.

3. Create Session on Failure is the final setting. It indicates if a session will be created if nomatching session has been found according to step 2.

20


5.3. Association - Netflow UDRsAs previously mentioned, the IP address to match against in the Netflow UDRs depends on if data isbeing uploaded or downloaded. This results in the session initiator being either the source or destination.Hence, both these fields need to be evaluated in the Aggregation node:

Figure 11. The Aggregation Profile Editor, Association tab - Netflow UDRs.

This is how arriving Netflow UDRs are evaluated when configured according to Figure 11, “The Ag-gregation Profile Editor, Association tab - Netflow UDRs.”:

1. If the SourceIP (1st rule, ID Fields area) does not match any existing session, no new session iscreated. If a match is found, the UDR is associated with this session.

2. Regardless of the outcome of the 1st rule, all rules are always evaluated. Hence the 2nd rule isevaluated. If the DestinationIP (1st rule, ID Fields area) does not match any existing session,no new session is created. If a match is found, the UDR is associated with this session.

Since Create Session on Failure is not enabled for any of the rules, the UDRs which do notfind a matching session will be deleted and cannot be retrieved.

5.4.The APL CodeFrom the APL code (the node configuration window), all actions related to both initiating and matchinga session are defined. When a session is considered associated, the session variables are saved in anew UDR Type (outputUDR(out)) containing fields with the same name as the variables. Note, thetimeout of a session is set to five days from the current date. Outdated sessions are removed and theirdata is transferred to a UDR of type outputUDR, which is sent to ECS.

import ultra.Example.Out;

timeout { // Outdated sessions are removed, and a resulting UDR is sent on. OutputUDR finalUDR = udrCreate( OutputUDR ); finalUDR.user = session.user; finalUDR.IPAddress = (string)session.IPAddress; finalUDR.downloadedBytes = session.downloadedBytes; finalUDR.uploadedBytes = session.uploadedBytes; udrRoute( finalUDR );

21


sessionRemove(session); }

sessionInit { Accounting_Request_Int radUDR = (Accounting_Request_Int) input; session.user = radUDR.User_Name; session.IPAddress = radUDR.framedIPAddress; session.sessionID = radUDR.acctSessionId;}

consume { /* Radius UDRs. If a matching session is found, then there are two Radius UDRs and the session is considered completed. Remove session and route the new UDR. */

if (instanceOf(input, Accounting_Request_Int)) { Accounting_Request_Int radUDR = (Accounting_Request_Int)input;

if (radUDR.acctStatusType == 2 ) { OutputUDR finalUDR = udrCreate( OutputUDR ); finalUDR.user = session.user; finalUDR.IPAddress = (string)session.IPAddress; finalUDR.downloadedBytes = session.downloadedBytes; finalUDR.uploadedBytes = session.uploadedBytes; udrRoute( finalUDR ); sessionRemove(session); return; } }

/* Netflow UDRs. Depending on if the user downloaded or uploaded bytes, the corresponding field data is used to update session variables. */

if (instanceOf(input, V5UDR)) { V5UDR nfUDR = (V5UDR)input;

if ( session.IPAddress == nfUDR.SourceIP ) { session.downloadedBytes = session.downloadedBytes + nfUDR.BytesInFlow; } else { session.uploadedBytes = session.uploadedBytes + nfUDR.BytesInFlow; } }

// A session will be considered outdated in 5 days. date timer=dateCreateNow(); dateAddDays( timer, 5 ); sessionTimeout( session, timer );}

22


Analysis Agent 4.1


Version 1.42

1





2

Analysis Agent 4.1


1.1. Prerequisites .................................................................................................... 71.2. Abbreviations and Terms ................................................................................... 7

2. APL Code Editor ........................................................................................................ 83. Analysis Agent ......................................................................................................... 10

3.1. Configuration - Batch Workflows ...................................................................... 103.2. Configuration - Realtime Workflows .................................................................. 113.3. Assignment and Cloning .................................................................................. 113.4. Input and Output Types .................................................................................... 143.5. Transaction Behavior - Batch Workflows ............................................................. 15

3.5.1. Emits ................................................................................................. 153.5.2. Retrieves ............................................................................................ 16

3.6. Introspection .................................................................................................. 163.7. Meta Information Model .................................................................................. 163.8. Node Message Event ....................................................................................... 16

4. The Analysis Programming Language Guide .................................................................. 174.1. Syntax Description ......................................................................................... 17

5. APL - Language Characteristics ................................................................................... 185.1. Function Blocks ............................................................................................. 185.2. Variables and Variable Scope ............................................................................ 18

5.2.1. Final Variables ..................................................................................... 195.2.2. Persistent Variables ............................................................................... 19

5.3. Function Declarations ...................................................................................... 205.3.1. The synchronized Keyword .............................................................. 21

5.4. Data Types .................................................................................................... 215.4.1. Reference Vs. Value Semantics ............................................................... 225.4.2. Type Casting ....................................................................................... 235.4.3. Access of UDR Fields ........................................................................... 245.4.4. Access of Input UDR ............................................................................ 24

5.5. Control Flow ................................................................................................. 245.6. Constants ...................................................................................................... 25

5.6.1. Predefined Constants ............................................................................ 255.6.2. Integer Literals .................................................................................... 255.6.3. Floating-Point Literals .......................................................................... 255.6.4. String Literals ..................................................................................... 25

5.7. Separators ..................................................................................................... 255.8. Operators ...................................................................................................... 265.9. Global Code .................................................................................................. 27

5.9.1. Using Global APL Code ........................................................................ 275.9.2. Using Global UFDL Code ..................................................................... 27

6. APL - Workflow Blocks ............................................................................................. 296.1. initialize ............................................................................................... 296.2. beginBatch and endBatch ......................................................................... 296.3. consume ..................................................................................................... 29

6.3.1. Built-In Variables ................................................................................. 296.4. drain ......................................................................................................... 306.5. cancelBatch ............................................................................................. 306.6. deinitialize ........................................................................................... 306.7. Function Block Example .................................................................................. 31

7. APL - Built-In Functions ............................................................................................ 327.1. Bytearray Related Functions ............................................................................. 32

7.1.1. baAppend ........................................................................................... 327.1.2. baCreate ............................................................................................. 327.1.3. baGet ................................................................................................. 327.1.4. baHexDump ........................................................................................ 33

3

Analysis Agent 4.1

7.1.5. baSet ................................................................................................. 337.1.6. baSize ................................................................................................ 33

7.2. Bit Operation Related Functions ........................................................................ 347.2.1. bsCreate ............................................................................................. 347.2.2. bitIsSet and bitIsCleared ........................................................................ 347.2.3. bsSet and bsClear ................................................................................. 34

7.3. Date Related Functions .................................................................................... 357.3.1. dateGet* ............................................................................................. 357.3.2. dateGetAsMilliseconds ......................................................................... 357.3.3. dateGMTOffset ................................................................................... 367.3.4. dateInDaylightTime .............................................................................. 367.3.5. dateIsLeapYear .................................................................................... 367.3.6. dateHas* ............................................................................................ 367.3.7. dateCreateNowMilliseconds ................................................................... 377.3.8. dateCreate* ......................................................................................... 377.3.9. dateSet* ............................................................................................. 387.3.10. dateAdd* .......................................................................................... 397.3.11. dateDiff ............................................................................................ 40

7.4. IP Address Related Functions ............................................................................ 407.4.1. getHostName ...................................................................................... 407.4.2. getIPAddress ....................................................................................... 407.4.3. ipToAddress ........................................................................................ 417.4.4. ipToHostName .................................................................................... 417.4.5. ipAddressString ................................................................................... 417.4.6. ipFromAddress .................................................................................... 417.4.7. ipFromHostName ................................................................................. 427.4.8. ipLocalHost ........................................................................................ 427.4.9. ipIsV6Address ..................................................................................... 42

7.5. Log Related Functions ..................................................................................... 427.5.1. debug ................................................................................................. 427.5.2. log* ................................................................................................... 437.5.3. dispatchEvent ...................................................................................... 447.5.4. dispatchMessage .................................................................................. 44

7.6. List Related Functions ..................................................................................... 457.6.1. listAdd ............................................................................................... 457.6.2. listClear ............................................................................................. 457.6.3. listCreate ............................................................................................ 457.6.4. listFindIndex ....................................................................................... 467.6.5. listGet ................................................................................................ 467.6.6. listInsert ............................................................................................. 477.6.7. listRemove .......................................................................................... 477.6.8. listSet ................................................................................................ 477.6.9. listSize ............................................................................................... 487.6.10. listSort ............................................................................................. 48

7.7. Map Related Functions .................................................................................... 497.7.1. mapClear ............................................................................................ 497.7.2. mapContains ....................................................................................... 497.7.3. mapCreate .......................................................................................... 507.7.4. mapGet .............................................................................................. 507.7.5. mapKeys ............................................................................................ 507.7.6. mapRemove ........................................................................................ 517.7.7. mapSet ............................................................................................... 517.7.8. mapSize ............................................................................................. 517.7.9. mapValues .......................................................................................... 51

7.8. MIM Related Functions ................................................................................... 527.8.1. mimGet .............................................................................................. 527.8.2. mimIsNull .......................................................................................... 527.8.3. mimPublish ........................................................................................ 52

4

Analysis Agent 4.1

7.8.4. mimSet .............................................................................................. 537.8.5. MIM Example ..................................................................................... 54

7.9. Audit Related Functions ................................................................................... 547.9.1. auditAdd ............................................................................................ 547.9.2. auditSet .............................................................................................. 55

7.10. String Related Functions ................................................................................ 557.10.1. String Concatenation ........................................................................... 557.10.2. strEndsWith ...................................................................................... 557.10.3. strEqualsIgnoreCase ........................................................................... 567.10.4. strIndexOf ......................................................................................... 567.10.5. strInsert ............................................................................................ 567.10.6. strLength .......................................................................................... 577.10.7. strREContains ................................................................................... 577.10.8. strREIndexOf .................................................................................... 577.10.9. strREReplaceAll ................................................................................ 587.10.10. strReplaceChars ............................................................................... 597.10.11. strSplit ............................................................................................ 597.10.12. strStartsWith .................................................................................... 607.10.13. strSubstring ..................................................................................... 607.10.14. strToLower ...................................................................................... 607.10.15. strToUpper ...................................................................................... 617.10.16. strTrim ........................................................................................... 61

7.11. UDR Related Functions ................................................................................. 617.11.1. udrAddError ...................................................................................... 617.11.2. udrClearErrors ................................................................................... 627.11.3. udrClone ........................................................................................... 627.11.4. udrContainsError ................................................................................ 627.11.5. udrCreate .......................................................................................... 637.11.6. udrDecode ........................................................................................ 637.11.7. udrEncode ........................................................................................ 647.11.8. udrForceDecode ................................................................................. 647.11.9. udrGetErrorsAsString ......................................................................... 657.11.10. udrGetValue .................................................................................... 657.11.11. udrIsPresent ..................................................................................... 657.11.12. udrRoute ......................................................................................... 667.11.13. udrSetValue ..................................................................................... 667.11.14. udrUnsetPresent ............................................................................... 67

7.12. Type Conversion Related Functions .................................................................. 677.12.1. baToBigInt ........................................................................................ 677.12.2. baToHexString ................................................................................... 677.12.3. baToStr ............................................................................................ 687.12.4. dateToString ...................................................................................... 687.12.5. strToBA ............................................................................................ 697.12.6. strToBigInt ........................................................................................ 697.12.7. strToDate .......................................................................................... 697.12.8. strToDouble ...................................................................................... 707.12.9. strToFloat ......................................................................................... 707.12.10. strToInt ........................................................................................... 717.12.11. udrToString ..................................................................................... 71

7.13. Database Table Related Functions .................................................................... 727.13.1. sqlExec ............................................................................................ 727.13.2. tableCreate ........................................................................................ 727.13.3. tableRowCount .................................................................................. 737.13.4. tableGet ............................................................................................ 737.13.5. tableLookup ...................................................................................... 737.13.6. tableCreateIndex ................................................................................ 75

7.14. Misc Functions ............................................................................................. 767.14.1. abort ................................................................................................ 76

5

Analysis Agent 4.1

7.14.2. crc32 ................................................................................................ 767.14.3. hintEndBatch ..................................................................................... 767.14.4. instanceOf ........................................................................................ 777.14.5. mailNotify ........................................................................................ 787.14.6. cancelBatch ....................................................................................... 78

6

Analysis Agent 4.1

1. IntroductionThis document describes the Analysis agent being part of the standard agents available on the NokiaConvergent Mediation Engine™ platform.

The Analysis agent can be part of both batch and real-time workflows. Differences in the configurationare described in Section 3.2, “Configuration - Realtime Workflows”.

This document also describes the Analysis Programming Language, APL, utilized by the Analysis andAggregation Agents, the UDR Bulk Editor, and within UFDL (Ultra Format Definition Language).

If not stated elsewhere, Oracle database is assumed.




• Basic programming

1.2. Abbreviations and TermsAbbreviations and terms, used in the Convergent Mediation Engine™ user documents, are describedin the Convergent Mediation Engine™ Core - User Guide.

7

Analysis Agent 4.1

2. APL Code EditorGeneric APL code, to be used in the node configurations for the Analysis or Aggregation agents, isentered in the APL Code Editor. This code may be included from the node configuration specificcode, using the import keyword:

import apl.<folder name>.<code definition>

If generic code is modified in the APL Code Editor, it will be automatically reflected in all nodesthat contain this code the next time each workflow is executed.

Example 1.

An APL code definition, saved as MyGenericCode in the Default folder, is utilized froma node's APL code by:

import apl.Default.MyGenericCode;

APL commands that are related to workflows, such as udrRoute, cannot be utilized withinthe APL Code Editor. Also, no MIM related functions can be used.

The editor is displayed when APL Code... is selected from the Configuration menu on the desktop.

Figure 1. The APL Code Editor.

Compiles the current apl code definition to check for validity. The status ofthe compilation is displayed in a dialog. Upon failure, the erroneous line ishighlighted and a message, including the line number, is displayed.

Edit menu Validate

The text is color coded according to the following definitions:Code Definition

Brown - Strings.

Blue (dark) - Functions.

Blue (light) - Constants.

Green - Types.

8

Analysis Agent 4.1

Orange - Own (user defined) types.

Purple - Key words.

Red - Comments.

User interface items not listed, are considered to be of general type. Refer to Convergent MediationEngine™ Core - User Guide, for further information.

9

Analysis Agent 4.1

3. Analysis AgentThe Analysis agent offers the rich programming language - Analysis Programming Language (APL)- enabling examination, alteration, routing and cloning of each UDR passing the agent. The languagesupports most standard features of a programming language, plus interfaces to the Convergent MediationEngine™ platform for manual logging of events and errors etc. It is also possible to load dynamicplug-in functions, to enable advanced processing.

The Analysis agent is not only a pure processing agent. It may also be the final destination for a UDR;a sort of Forwarding agent. Leaving out the routing of UDRs (the udrRoute command), will makethe outgoing connection point disappear from the window, disabling connection to a subsequent node.In addition, it may also replace the Decoder and Encoder agents (using the udrDecode andudrEncode commands).

3.1. Configuration - Batch WorkflowsThe Analysis agent configuration window is displayed when the node in a workflow is double-clickedor right-clicked, selecting Configuration....

Figure 2. Analysis agent configuration window, Analysis tab.

Text area where the APL code, used for UDR processing, is entered. Below thetext area there are line, column and position indicators, for help when locatingsyntax errors.

Code Area

For input assistance the area offers a pop-up menu, displayed when right-clickingand selecting

• UDR Assistance.... This option displays the UDR Internal Format Browserfrom which the UDR Fields may be inserted into the code area.

• MIM Assistance.... This option displays the MIM Browser from which theavailable MIM Resources may be inserted into the code area.

10

Analysis Agent 4.1

For further information on these options, please refer to Convergent MediationEngine™ Core - User Guide.

Compiles the entered code to check for validity. The status of the compilation isdisplayed in a dialog. Upon failure, the erroneous line is highlighted and a message,including the line number, is displayed.

CompilationTest...

Enables selection of UDR Types. One or several UDR Types that the agent expectsto receive may be selected. Please refer to Section 3.4, “Input and Output Types”for a detailed discussion.

UDR Types

Automatically selects the UDR Type distributed by the previous node.Set To Input

When the Analysis agent configuration window is confirmed, a compilation is performed inorder to extract the configuration data from the code. Complex code and formats may requiresome time.

For further details about the pop-up menu in the Code Area and the UDR Internal FormatBrowser, please refer to Convergent Mediation Engine™ Core - User Guide.

The use and settings of private threads for an agent, enabling multi-threading within a workflow,is configured in the Thread Buffer tab. Please refer to Convergent Mediation Engine™ Core- User Guide, for further details.

3.2. Configuration - Realtime WorkflowsAn Analysis node may be part of Batch as well as a Realtime workflows. The desktops are identical;hence look at the previous Section 3.1, “Configuration - Batch Workflows”, when configuring thenode. However, there are some restrictions to be considered when designing a Realtime workflow:

• No Thread Buffer tab exists (no multi-threading for an individual node is available).

• APL plug-ins with transaction logic are not allowed. Note, the user must keep track of what typeof plug-ins are called. The node will not perform any validation during workflow configuration -the workflow will abort upon activation of illegal code.

• Function blocks related to batch handling cannot be used. This includes drain, beginBatch,cancelBatch, and endBatch.

• Published MIMs may only be of global type.

• Functions must be preceded with the synchronized keyword, in order to be thread-safe. Thiswill make it possible to alter global variables. It is possible to read global variables from anyfunction block. However, to avoid race conditions with functions updating the global variables,they must only be accessed from within synchronized functions.

• Synchronized functions cannot utilize the udrRoute command.

Please refer to Section 4, “The Analysis Programming Language Guide” for further details on thespecific commands.

3.3. Assignment and CloningAll agents handling UDRs will forward references to a UDR on all outgoing links from the agent.Thus, the same instance of the UDR will be referred to from all agents in a workflow. This is accepted

11

Analysis Agent 4.1

if all agents only read the contents of a UDR. If an agent alters the contents of a UDR, it effectivelyalters the contents for all other agents that will receive the UDR. To avoid this behavior the UDR maybe cloned. Note, cloning is a costly operation in terms of performance, therefore it must be used withcare.

12

Analysis Agent 4.1

Example 2.

In the following configuration, the Analysis node alters the contents of the received UDR andthe same UDR travels on all links.

Case 1:

Based on the earlier discussions, if the Analysis node alters the contents of an incoming UDR,it may also alter the contents of the same UDR propagated on r_3. This is probably not the desiredbehavior.

Figure 3. Assignment case 1.

Code in the Analysis agent, affecting the UDR for all agents in the workflow:

input.MyNumber=54;udrRoute(input);

If desired to alter a UDR but still keep its original values in other nodes of a workflow, the UDRmust be cloned.

Code in the Analysis agent, creating, amending and routing a cloned UDR, leaving the originalUDR untouched:

input=udrClone(input);input.MyNumber=54;udrRoute(input);

Case 2:

Cloning is also possible when a UDR is routed from an Analysis node. To get the expectedresult, that is, not to affect the UDR routed on r_3, the workflow and the code in the Analysisnode could be reconfigured according to the following:

13

Analysis Agent 4.1

Figure 4. Assignment case 2.

Initially the UDR is to be cloned and routed on to r_3. Then it could be amended and linked tor_2.

Code in the Analysis agent; routing a cloned and untouched UDR on to r_3, amending androuting the original UDR on r_2:

udrRoute(input,"r_3",clone);input.MyNumber=54;udrRoute(input,"r_2");

3.4. Input and Output TypesUDRs entering an Analysis node are referred to as input types, while UDRs leaving the node are referredto as output types. The input types must be specified on the desktop, while the output types are calculatedfrom the input types and the APL code.

Note that input is a built-in variable in APL, and must be referred to for all UDRs entering the node.

14

Analysis Agent 4.1

Example 3.

Suppose there is a workflow with one Analysis node, one input route streaming two differentinput types (typeA and typeB), and two output routes. The two output routes take two differentUDR types - the first equaling one of the input types (typeA), and the second is a new UDRtype (typeC) which is created out of information fetched from the other input type (typeB).

Figure 5. Several UDR types can be streamed to and from an Analysis node.

The APL code:

if (instanceOf(input, typeA)) { udrRoute((typeA)input,"r_2");else { typeC newUDR = udrCreate(typeC); newUDR.field = ((typeB)input).field; // Additional field assignments... udrRoute(newUDR, ,"r_3");}

Note that for the first udrRoute statement we explicitly cast to the typeA type, while for thesecond we need no cast. This is because the input variable does not have a known type (it canbe either typeA or typeB), while newUDR is known by the compiler to be of typeC.

If we had not done this cast, then the output type on r_2 would have been reported as drudrand the workflow would not be valid.

3.5.Transaction Behavior - Batch Workflows

3.5.1. Emits

The agent itself does not emit End Batch, but it can trigger the collector to do so bycalling the hintEndBatch method. Note that not all collectors can act upon sucha request, causing the workflow to abort if hintEndBatch is called.

End Batch

The agent itself does not emit Cancel Batch messages. However, if the code containsa call to the method cancelBatch this will make the agent emit a Cancel Batch.

Cancel Batch

If the code contains a call to the method hintEndBatch this will make the agentemit a Hint End Batch. Note that not all collectors can act upon such a request. Referto the user guide for the respective Collection agents.

Hint End Batch

15

Analysis Agent 4.1

3.5.2. Retrieves

When a Begin Batch message is received, the agent calls the beginBatch functionblock, if present in the code.

Begin Batch

When an End Batch message is received, the agent calls the drain and endBatchfunction blocks, if present in the code.

End Batch

When a Cancel Batch message is received, the agent calls the cancelBatch functionblock, if present in the code.

Cancel Batch

3.6. IntrospectionProduced types are dependent on input type and the APL code. The agent consumes bytearrays andany UDR type, selected from the UDR Types list.

3.7. Meta Information ModelThe agent does not itself publish or access any additional MIM resources. However, APL offers thepossibility of both publishing and accessing MIM resources and values.

For a listing of general Convergent Mediation Engine™ MIM resources, refer to Convergent MediationEngine™ Core - User Guide.

3.8. Node Message EventThe agent does not itself produce any events. However, APL offers the possibility of producing events.

16

Analysis Agent 4.1

4.The Analysis Programming Language Guide

4.1. Syntax DescriptionAll APL functions are described according to the following:

returnType functionName ( type1 parameter1, type2 parameter2 )

Parameters:

A description of parameter1. If necessary, an example is given.parameter1

A description of parameter2. If necessary, an example is given.parameter2

A description of what the function returns. If necessary, an example is given.Returns

Example 4.

If there is a need to explain a function in relation to other functions and/or agents, this is donein an 'Example' clause.

type myVar;

consume { // example code }

Where applicable, a framed text exists, giving information on limitations to the use of thefunction. For instance if the function can be used in the APL agents (Aggregation and Analysis)only, or for batch workflows only.

17

Analysis Agent 4.1

5. APL - Language CharacteristicsThe language syntax resembles the Java or C programming language. It uses a number of predefinedfunction blocks to be executed at different stages of data processing. User defined code may be insertedinto each block.

5.1. Function BlocksThe code is divided in different function blocks where each is executed at different occasions. Afunction block body is enclosed in { and } characters. Please refer to Section 6, “APL - WorkflowBlocks” for further information.

5.2. Variables and Variable ScopeVariables may be either global (declared outside any function block) or local (declared inside a functionblock). Variables may also be given an initialization value with general assignment syntax such as:int myInt = 42;

The declaration order of global variables is relevant for the initialization in the sense that earlier variablesmay be used to initialize later variables but not the other way around.

Example 5.

// Legalint f1 = 1;int f2 = f1 + 1;

// Illegal! The following will not compile!int f3 = f4 + 1;int f4 = 1;

The scoping rules are as follows:

• Global variables are available in all blocks. Note, they cannot be accessed from another node orby an external APL script.

• All function blocks and local function declarations defines a separate scope.

• Any block statement (statements within a {} block) defines a scope.

'Variable hiding' is not allowed. For instance, if a variable myVar has been declared in global scope,it is not permitted to declare another variable myVar anywhere else in the code.

18

Analysis Agent 4.1

Example 6.

int myVar;

consume { if ( true ) { int myVar2; // Separate block gives separate scope }

int myVar2; // Ok int myVar; // illegal! Will not compile

if ( true ) { int myVar2; // Illegal! Will not compile }}

5.2.1. Final Variables

Constants in APL work the same way as in Java. Hence, to prevent variables from being changed, thefinal keyword is used:

final int myVar = 100;

Only numeric and string types are supported for final variables.

5.2.2. Persistent Variables

By default, all variables are reset between workflow activations. To save the value of a global variablebetween each activation, the persistent keyword can be used upon variable declaration:

persistent int myVar = 100;

Persistent variables are read from the database between initialize and beginBatch. Corres-pondingly, the variable value is saved between endBatch and deinitialize. Consequently, anyassignment to the persistent variable in initialize only works the first time, since once you havea persistent value this will overwrite the value from initialize. As with all other variable types,persistent variable names are unique for each node within a workflow.

19

Analysis Agent 4.1

Example 7.

persistent int counter;

initialize { counter = 0; debug( "Value in initialize: " + counter );}

beginBatch { counter = counter + 1; debug( "Value in beginBatch: " + counter );}

The debug output from initialize will read zero each time, while the debug output frombeginBatch will read the actual value.

First time activated (one input batch):

Value in initialize: 0

Value in beginBatch: 1

Fifth time activated (one input batch):

Value in initialize: 0

Value in beginBatch: 5

Applicable for batch workflows only.

5.3. Function DeclarationsFunctions may be declared and called within the APL code using normal C syntax.

Example 8.

int addNumbers( int a, int b ) { return a + b;}

consume { debug( "1 + 1 = " + addNumbers( 1, 1 ) );}

To declare functions with no return value, the void keyword is used.

20

Analysis Agent 4.1

5.3.1.The synchronized Keyword

In order to support multi-threading, functions updating global variables within Realtime workflowsmust be made thread-safe. To achieve this, the function name is preceded with the synchronizedkeyword:

Example 9.

synchronized int updateSeqNo() { seqNo = seqNo +1; return seqNo;}

It is possible to read global variables from any function block. However, to avoid race conditions withfunctions updating the global variables, they must only be accessed from within synchronizedfunctions.

The synchronized keyword is ignored when used in batch nodes.

5.4. Data TypesThe following table describes the different data types available in APL. The second column states ifthe data type is a primitive, or an object. The third column indicates the default value a non-initializedvariable will obtain.

ExampleDescriptionDefval

P/OData type

string aCode;String values.nullOstring

int aNumber;Integer value.0Pint

char aCharacter;A single character value.'\0'Pchar

short aNumber;Short integer value.0Pshort

byte aNumber;Single byte integer value.0Pbyte

long aNumber;Long integer value.0Plong

float aValue;double aValue;

Single/double-precisionfloating point.

0Pfloat/double

bigint aNumber;Big integer. May hold in-teger values of any size.

0Obigint

boolean aBool;Boolean value (true orfalse).

falsePboolean

date aDate;Holds date and time, includ-ing the time zone.

nullOdate

bytearray anAr-ray;

Bytearray type.nullObytearray

bitset aBitset;bit set type.nullObitset

21

Analysis Agent 4.1

ExampleDescriptionDefval

P/OData type

MyDefinedType aUDR;drudr anyUDR;

Holds a reference to a UDR.UDRType must be adefined format or compila-

nullOUDRType

tion will fail. The specialformat drudr is availablefor the generic drudr type.

list<int> intL-ist;

If a list of lists isto be declared the">" charactersmust be separatedby at least onespace, not to con-flict with the">>" operator.That is,list<list<int> >intLists;

List of objects of the spe-cified type.

nullOlist<type>

any aVar;aVar = 1;aVar = "foobar";

"any" type that can be as-signed any value of anytype.

nullOany

map<string, int> aMap = mapCreate( string, int);

Hash maps allowingkey/value pairs to bedefined.

nullOmap

table aTable = tableCreate("Default", anSQL);

Special type used to holdtable data used by the tablecommands.

nullOtable

ipaddress anIPa = ipLocalHost();

Holds IP address informa-tion. Both IPv4 and IPv6 aresupported.

nullOipaddress

5.4.1. Reference Vs. Value Semantics

The variable types follow the Java standard when it comes to object reference vs. value semantics. Alltypes use value semantics when compared through the '==' and '!=' boolean operators and objects usereference semantics for all other operators, like assignments '='.

22

Analysis Agent 4.1

Example 10.

Two date variables (objects) and two long variables (primitives) are used to illustrate the referencesemantics. In the case of date variables, they will always contain the same value no matter ifthey are updated via myDate_A or myDate_B since they point to the same object.

date myDate_A = dateCreateNow(); date myDate_B = myDate_A; long myLong_A = 10; long myLong_B = myLong_A;

if (myDate_A == myDate_B ) { // true } if (myLong_A == myLong_B) { // true }

// Change value on A variables dateAddHours(myDate_A, 22); if (myDate_A == myDate_B) { // still true, points to same object } myLong_A = 55; if (myLong_A == myLong_B) { // false, primitives always hold their own values }

5.4.2.Type Casting

It is possible to cast between different types with normal Java/C++ syntax. The instanceOf functionis used for checking dynamic types since an incorrect type conversion will cause the workflow to abortwith a runtime error.

Example 11.

/* APL code for two UDR types */consume { if( instanceOf( input, UDRType1 ) ) { UDRType1 udr1 = (UDRType1) input; // Handle UDRType1 ... } else { UDRType2 udr2 = (UDRType2) input; // Handle UDRType2 ... }}

As an extension to the normal conversions, all types may be casted to string type.

Note that some types are subtypes of other types and can be directly assigned. Examples are:

23

Analysis Agent 4.1

• All types are subtypes of the any type.

• int is a subtype (narrowing) of bigint.

• All UDR types are subtypes of the drudr type.

• Some UDR types are subtypes of other UDR types (depending on Ultra definitions).

There is also a number of built-in type conversion functions (see the following section).

5.4.3. Access of UDR Fields

UDR fields are accessed with the '.' separator. For instance, to access the field myField within theUDR type variable theUDR, the syntax would be:

theUDR.myField = 5;debug( theUDR.myField );

In case of substituting the field name with a variable, getUDRValue/setUDRValue is used.

Access of non-present fields will return the default value for the specific type. Field presence is checkedwith udrIsPresent, see Section 7.11.11, “udrIsPresent”.

5.4.4. Access of Input UDR

The input UDR can be accessed in the consume function block by using the special variable input.The instanceOf function may be useful in case of dealing with several UDR types. See Sec-tion 7.14.4, “instanceOf”.

5.5. Control Flow

ExampleDescriptionKeyword

if ( myVar == "ABC" ) {Used to get conditionalparts of the code to be ex-ecuted.

ifelse if // do_thiselse } else {

// do_that}

if ( ANumber == null ) {Stops the execution of thecurrent function block.return

return;}

while( i < 20 ) {Loops while condition istrue.while

// do_stuff}

24

Analysis Agent 4.1

5.6. Constants

5.6.1. Predefined Constants

ExampleDescriptionConstant

if ( NumFld == null )Can be used to checkif a field has beenassigned a value.

null

myBool = true;Value of a booleanvariable.

true & false

Workflow name.WORKFLOW_NAME

Node name.NODE_NAME

5.6.2. Integer Literals

Integer literals are of the normal Java/C syntax, for instance: 42, 0x2A

An integer literal is always of the most narrow integer type that can contain the value. This normallyworks as expected, since numeric types can be implicitly converted to any wider numeric type.

5.6.3. Floating-Point Literals

Floating-point literals (that is, float and double values) follow the Java standard, for instance:2.3, 1e-12

5.6.4. String Literals

String literals have the normal Java/C quoting rules, for instance: \n for linefeed, \\ for backslash.

Some string comparison functions use regular expressions. If special characters like "*", "?"etc. are to be used as characters in the regular expression, they must be prefixed with twobackslashes in the APL since these strings will be parsed twice.

For instance, strREIndexOf("Am I right?", "\\?") will return the index of thequestion mark in the string.

5.7. Separators

ExampleDescriptionSeparator

NumFld = (NumFld2 + 2) * 3;Used to enclose logical ex-pressions and to overrulestandard precedence.

( )

if ( ACode == "ABC" )Used to enclose blocks.{ }

{ // do_this // and_this

25

Analysis Agent 4.1

ExampleDescriptionSeparator

}

NumFld = NumFld + 1;Used to end each statement.;

input.myField = 5;Separates fields in UDRvalue accesses.

.

5.8. Operators

ExampleDescriptionOper-ator

Arithmetic operators:

string1 = string2 + string3;Addition (numeric types) orstring concatenation (if leftoperand is of string type).

+

NumFld = NumFld + 1;

Arithmetic operators (only numeric types):

NumFld = NumFld - 2;Subtraction-

NumFld = NumFld * 3;Multiplication*

NumFld = NumFld / 4;Division/

NumFld = NumFld % 10;Modulus%

Bit operators (only integer types):

NumFld = NumFld & 1;Bitwise AND&

NumFld = NumFld | 2;Bitwise OR|

NumFld = NumFld << 1;Shift bits left<<

NumFld = NumFld >> 1;Shift bits right>>

Boolean operators:

if (Fld1 == 1 && Fld2 != 4)Logical AND&&

if (Fld1 == 1 || Fld2 != 4)Logical OR||

if (Fld1 == 1 && Fld2 != 4)Equals to==

if (Fld1 == 1 && Fld2 != 4)Not Equals to!=

if (Fld1 <= 5)Less than or equal to<=

if (Fld1 < 5)Less than<

if (Fld1 >= 5)Greater than or equal to>=

if (Fld1 > 5)Greater than>

if (! BoolFld)Not!

ExampleDescriptionChar-acter

NumFld = NumFld +1;Assignment.=

26

Analysis Agent 4.1

ExampleDescriptionChar-acter

TeaFld = 2; // Tea for twoComment until end of line.//

/* This is a comment Start of comment, end ofcomment.

/* , */

over several lines */

5.9. Global CodeConvergent Mediation Engine™ Global Code can be either APL code, as entered in the APL CodeEditor; or UFDL code, as entered in the Ultra Format Language Editor. Global APL Code is calledpackages and is referred to through the name under which it was saved. Correspondingly, Global UFDLCode is called modules.

5.9.1. Using Global APL Code

Before Global APL Code can be used in the Analysis and Aggregation nodes, it must be imported:

import apl.<folder>.<package>;

The package is the name under which the Global APL code was saved. Note, single function blocksfrom within a package may not be imported - the whole package must be imported.

The import command may also be used to include UFDL modules, that is, UDR types (see followingsection).

5.9.2. Using Global UFDL Code

Lookup of type names is slightly different in APL than in Ultra definitions. In APL you must eitherexplicitly specify the module or make sure that the module is imported.

You can do explicit module import by

import ultra.<folder>.<module>;

The module is the name under which the Ultra format was saved.

The modules of any entries in the UDR types section of the Analysis node are implicitly imported.For instance, if the input type format(myModule) is configured, the complete module myModuleis imported.

27

Analysis Agent 4.1

Example 12.

/* APL code without import statement */

consume { myFolder.module1.MyUDR newUDR = udrCreate( myFolder.module1.MyUDR );

newUDR.number = input.seqNum; newUDR.sub = udrCreate( myFolder.module1.MySubUDR ); newUDR.sub.subNum = 17;

udrRoute( newUDR );}

/* APL code with import statement */import ultra.myFolder.module1;

consume { MyUDR newUDR = udrCreate( MyUDR );

newUDR.number = input.seqNum; newUDR.sub = udrCreate( MySubUDR ); newUDR.sub.subNum = 17;

udrRoute( newUDR );}

28

Analysis Agent 4.1

6. APL - Workflow BlocksAPL code is divided into different workflow blocks, making it possible to control when to executewhich code. For instance at workflow start up, at the beginning of each batch processed, etc.

6.1. initialize

Executes every time the node is loaded - that is once for each invocation of the workflow.

6.2. beginBatch and endBatch

Executed at the beginning and end of each batch. Rules are that the beginBatch function is calledwhen a batch collection node emits a Begin Batch. This occurs either at file start, or when a hintEnd-Batch call is received from any node capable of utilizing APL code.

The endBatch function is called every time a batch collection node emits an End Batch. This occurseither at file end, or when a hintEndBatch call is received from any node capable of utilizing APLcode.

Example 13.

Executed APL function blocksEvent

beginBatchThe collector emits Begin Batch

-An APL node emits hintEndBatch

drain, endBatchThe collector emits End Batch

beginBatchThe collector emits Begin Batch


6.3. consume

Executed for each UDR or bytearray passing the node. Here, validation, modification and routing canbe performed. Each UDR or bytearray is referenced by the special input variable.

6.3.1. Built-In Variables

ExampleDescriptionVariable

input.ANumber = 1234567;udrRoute(input);

Read-only variable containing thecurrent UDR. Only available in theconsume function block.

input

When dealing with several types of UDRs entering the Analysis node, the APL code must first determinewhat type is currently handled, then cast it to the correct type. For an example, please refer to Sec-tion 5.4.2, “Type Casting”.

29

Analysis Agent 4.1

6.4. drain

This block is executed right before an endBatch, and must be treated as a final consume block.For instance, if a batch containing ten UDRs is processed by the node, consume will be executed tentimes before the drain block is called. This is useful, for instance when collecting statistical data fora batch which is to be routed as a special UDR. The advantage with drain is that all consume features(except for the input built-in variable) are accessible, as opposed for the endBatch block.


Example 14.

Example on how to use drain:

int UDRCounter; consume { UDRCounter=UDRCounter+1; udrRoute(input); }

drain { myFolder.myUFDLFile.myTrailerFormat myTrailer; myTrailer=udrCreate(myFolder.myUFDLFile.myTrailerFormat); myTrailer.closingdate=dateCreateNow(); myTrailer.numberOfUDRs=UDRCounter; myTrailer.sourceFileName=(string)mimGet("Disk_1", "Source Filename"); udrRoute(myTrailer); }

6.5. cancelBatch

Executed if a Cancel Batch is emitted anywhere in the workflow. Note that End Batch and CancelBatch are mutually exclusive - only one per batch can be executed.

If the cancelBatch block is called and the Cancel Batch behavior is set to Abort Immediately,the cancelBatch execution block will not be called. This block is only called when the preferencesare set to Abort After or Never Abort.


6.6. deinitialize

Executed right before the node is unloaded.

30

Analysis Agent 4.1

6.7. Function Block Example

Example 15.

int file_count;int UDR_count;

initialize { file_count = 0;}

beginBatch { UDR_count = 0;}

consume { udrRoute( input ); UDR_count = UDR_count + 1;}

endBatch { file_count = file_count + 1; debug( "Number of UDRs in file:" + UDR_count );}

deinitialize { debug(" Number of executed files:" + file_count );}

31

Analysis Agent 4.1

7. APL - Built-In Functions

7.1. Bytearray Related Functions

7.1.1. baAppend

Concatenates two bytearrays and returns the result. None of the input arrays are changed.

bytearray baAppend ( bytearray array1, bytearray array2 )

Parameters:

A bytearray.array1

Another bytearray to append to the end of array1.array2

The concatenated bytearray.Returns

7.1.2. baCreate

Creates a new bytearray of the specified size.

bytearray baCreate( int size )

Parameters:

Size of the new array.size

A bytearray of the specified size.Returns

7.1.3. baGet

Retrieves a byte value from a bytearray. The index must be in the range 0<=index<baSize(array)or the workflow will abort with a runtime error.

int baGet ( bytearray array, int index )

Parameters:

A bytearray.array

Index of the element to retrieve.index

A decimal representation of the indexed byte.Returns

Example 16.

Consider a bytearray with the following hex dump "00000000: 484a 4c HJL". Accessing thefirst element according to the example code, will return "72" - the decimal value of 0x48.int myVar = baGet( myBA, 0 );

32

Analysis Agent 4.1

7.1.4. baHexDump

Converts a bytearray into a readable hex dump string, useful for debugging.

string baHexDump( bytearray array )

Parameters:

A bytearray.array

A string containing the hex formatted bytearray.Returns

Example 17.

Consider a bytearray created with the code in this example.

bytearray myBA = baCreate( 3 );baSet( myBA, 0, 72 );baSet( myBA, 1, 74 );baSet( myBA, 2, 76 );string myDump = baHexDump(myBA);

Using the bytearray as input to baHexDump will output a hex dump with the following appear-ance:

"00000000: 484a 4c HJL"

7.1.5. baSet

Sets the value of a byte in the bytearray. The index must be in the range 0<=index<baSize(array)or the workflow will abort with a runtime error.

void baSet ( bytearray array, int index, int value )

Parameters:

A bytearray.array

Index of the byte to set.index

The new value of the byte. The actual value set is (value & 0xFF).value

Nothing.Returns

7.1.6. baSize

Returns the size of an array.

int baSize( bytearray array )

33

Analysis Agent 4.1

Parameters:

A bytearray.array

The size of the array.Returns

7.2. Bit Operation Related Functions

7.2.1. bsCreate

Creates an empty bitset.

bitset bsCreate()

None.Parameters:

An (empty) bitset.Returns

7.2.2. bitIsSet and bitIsCleared

Checks whether an individual bit is set or cleared. The check can be done against either an int, or abitset value. For int type, these are convenience functions that can be used instead of the normal logicaloperators.

boolean bitIsCleared ( int|bitset value, int bitnumber )

boolean bitIsSet ( int|bitset value, int bitnumber )

Parameters:

Value to check.value

The bit to check. 0 (zero) is the least significant bit, 31 is the most significant.bitnumber

false or true.Returns

7.2.3. bsSet and bsClear

Sets or clears a bit in a bitset.

void bsClear ( bitset bitset, int bitnumber )

void bsSet ( bitset bitset, int bitnumber )

Parameters:

34

Analysis Agent 4.1

The bitset to modify.bitset

The bit to check. 0(zero) is the least significant bit, 31 is the most significant.bitnumber

Nothing.Returns

7.3. Date Related FunctionsAll date related functions conforms to the standard Java date functions. Years, entered with two digits,will refer to 1900 if they are between 70 and 99 and refer to year 2000 if they are between 00 and 69.

7.3.1. dateGet*

Given a date, the following functions return the year number, month number, day number (Monday=1,Tuesday=2, etc), hour (24 hour clock), minute, second, or millisecond.

int dateGetYear( date d )

int dateGetMonth( date d )

int dateGetDay( date d )

int dateGetDayOfWeek( date d )

int dateGetHours24( date d )

int dateGetMinutes( date d )

int dateGetSeconds( date d )

int dateGetMilliseconds( date d )

string dateGetTimeZone( date d )

Parameters:

The date to convert.d

Integers stating year/month/day/hour/second etc for all functions except dateGet-TimeZone, which returns the time zone as a three letter string (GMT, PST, etc).

Returns

Example 18.

dateGetTimeZone always returns the abbreviation of the current time zone. The followingdebug output can give CET, GMT, etc.

debug( dateGetTimeZone( dateCreateNow() ) );

7.3.2. dateGetAsMilliseconds

Given a date, returns the total number of milliseconds since 1 Jan, 1970. This function is the inverseof dateCreateFromMilliseconds.

35

Analysis Agent 4.1

long dateGetAsMilliseconds( date d )

Parameters:

The date to examine.d

The total number of milliseconds since 1 Jan, 1970.Returns

7.3.3. dateGMTOffset

Returns the number of milliseconds diverging from GMT.

long dateGMTOffset( date d )

Parameters:


Number of milliseconds diverging from GMT.Returns

7.3.4. dateInDaylightTime

Given a date, true is returned if the date is a daylight-saving date.

boolean dateInDaylightTime( date d )

Parameters:


true or false.Returns

7.3.5. dateIsLeapYear

Given a date, a leap year evaluation is performed.

boolean dateIsLeapYear( date d )

Parameters:



7.3.6. dateHas*

Given a date the methods return true if the date includes a date or time part.

boolean dateHasDate( date d )

boolean dateHasTime( date d )

36

Analysis Agent 4.1

Parameters:



7.3.7. dateCreateNowMilliseconds

Gives the current date in milliseconds of the operating system.

long dateCreateNowMilliseconds()

None.Parameters:

The current date in milliseconds.Returns

7.3.8. dateCreate*

Creates a full date based on current timestamp of the host, or a given date and/or time:

date dateCreateNow()

date dateCreate ( int year, int month, int day, int hour, int minute, int second )

Create dates where either the date or time part is set:

date dateCreateFromDate ( int year, int month, int day )

date dateCreateFromTime ( int hour, int minute, int second )

date dateCreateFromMilliseconds ( long milliseconds )

Creates a copy of a date:

date dateCreateCopy( date d )

Parameters:

37

Analysis Agent 4.1

A date/year/month/day...d/year/month/day

A date.Returns

7.3.9. dateSet*

Sets different parts of a supplied date:

void dateSetYear ( date d, int year )

void dateSetMonth ( date d, int month )

void dateSetDay ( date d, int day )

void dateSetHours24 ( date d , int hours)

void dateSetMinutes ( date d, int minutes )

void dateSetSeconds ( date d , int seconds)

void dateSetMilliseconds ( date d, int milliseconds )

void dateSetTimeZone ( date d, string timeZone )

Sets the complete date or time part of a date:

void dateSetDate ( date d, int year, int month, int day )

void dateSetTime ( date d, int hour, int minute, int second)

Parameters:

38

Analysis Agent 4.1

A date/year/month/day...d/year/month/day

The component to be set needs to represent a valid value in order notto throw an exception at a later stage. For instance; Month= 1-12,Hours=0-23, Minutes=0-59, Milliseconds=0-999.

Also, the date needs to represent a valid date. For instance February31 is invalid.

A string stating the time zone to set. Two different formats are allowed: ab-breviation or offset from GMT:

Example 19.

The time zone for Pacific Standard Time can be given in any of thefollowing formats:

• "PST"

• "GMT-08:00"

Note that if an invalid time zone format is entered, no error is returned.Instead, the time zone is automatically set to "GMT".

timeZone

Nothing.Returns

7.3.10. dateAdd*

Adds a specified number of years, months, days, hours, minutes, seconds, or milliseconds to a datevalue.

void dateAddYears ( date d, int years )

void dateAddMonths ( date d, int months )

void dateAddDays ( date d, int days )

void dateAddHours ( date d , int hours)

void dateAddMinutes ( date d, int minutes )

void dateAddSeconds ( date d , int seconds)

39

Analysis Agent 4.1

void dateAddMilliseconds ( date d, int milliseconds )

Parameters:

A date/year/month/day...d/years/months/days

Nothing.Returns

7.3.11. dateDiff

Calculates the difference in milliseconds between two dates. The return value is date1 - date2which means that the result may be negative.

long dateDiff ( date date1, date date2 )

Parameters:

The date to subtract from.date1

The date to subtract with.date2

The difference in milliseconds.Returns

7.4. IP Address Related FunctionsThere is a set of built-in commands which may be utilized when performing, for instance, a host namelookup. All functions support both IPv4 and IPv6, but the results from some functions depend on theunderlying system configuration.

7.4.1. getHostName

Returns the fully qualified domain name of the host the workflow is executing on. Best effort method,meaning we may not be able to return the FQDN depending on the underlying system configuration.

string getHostName()

None.Parameters:

The host name as a string.Returns

7.4.2. getIPAddress

Returns the IP address of the local host.

string getIPAddress()

40

Analysis Agent 4.1

None.Parameters:

The raw IP address as a string.Returns

7.4.3. ipToAddress

Returns the numerical presentation of an IP address as elements of a bytearray. For instance; 10.0.3.22will be turned into [ 10 0 3 22 ].

bytearray ipToAddress ( ipaddress ipa )

Parameters:

An IP address.ipa

A bytearray representation of the IP address.Returns

7.4.4. ipToHostName

Extracts the fully qualified domain name part from a variable of type ipaddress. The host runningthe EC must have DNS setup. Note that this method may be time consuming.

string ipToHostName( ipaddress ipa )

Parameters:

An IP address.ipa

The host name corresponding to the given IP address.Returns

7.4.5. ipAddressString

Extracts the IP address part from a variable of type ipaddress. The result is a string according to"10.0.3.22".

string ipAddressString(ipaddress ipa )

Parameters:

An IP address.ipa

The IP address as a string.Returns

7.4.6. ipFromAddress

Converts the bytearray representation into an ipaddress type.

ipaddress ipFromAddress ( bytearray address )

Parameters:

41

Analysis Agent 4.1

The IP address as a bytearray. IPv4 address byte array must be 4 bytes long and IPv6 bytearray must be 16 bytes long.

address

The IP address as an ipaddress type.Returns

7.4.7. ipFromHostName

Given a string containing the IP address (numerical presentation), or host name, a variable of typeipaddress is returned. Note that this method may take a lot of time if, for instance, a DNS has tobe accessed.

ipaddress ipFromHostName ( string host )

Parameters:

The name of the host.host

The IP address.Returns

7.4.8. ipLocalHost

Returns the IP address of the local host.

ipaddress ipLocalHost()

None.Parameters:

The IP address.Returns

7.4.9. ipIsV6Address

Checks if the IP address represents an IPv6 address.

boolean ipIsV6Address()(ipaddress ipa )

Parameters:

An IP address.ipa

true if the address is IPv6, false otherwise.Returns

7.5. Log Related FunctionsThe following functions may be used for debugging APL code, or logging user defined messages andevents.

7.5.1. debug

Prints the supplied argument to the output target specified in the workflow Preferences dialog, Mon-itor tab. Valid options are none, event, or file. If file is selected, the debug is saved in the temporarydirectory as stated in $MZ_HOME/etc/config.xml (parameter mz.ec.tmpdir). The file is to

42

Analysis Agent 4.1

be called debug/<workflow name>. Each time the workflow is activated resulting in new debuginformation being output, the existing file is overwritten.

void debug(arg)

Parameters:

Argument to write to debug output. Could be any type. Note that printing a UDR type willdump all the field values, which may be a large amount of data. Similarly, the debug outputfor a table or list type may be very large.

arg

There is a special case if arg is a bytearray. In this case, the output string will be the hexdump returned from the baHexDump built in function.

For all other variable types the output is the direct string conversion so that debug ( arg )is the same as debug( (string) arg ).

Nothing.Returns

7.5.2. log*

Logs a message string to the System Log of type error, warning or information. The entry will fallunder the Workflow category where workflow name will be the name of the current workflow andnode name will be the name of the node logging the message.

void logError ( string message, string parameterName_n, // Optional string|int parameterValue_n, // Optional ... )

void logInformation ( string message, string parameterName_n, // Optional string|int parameterValue_n, // Optional ... )

void logWarning ( string message, string parameterName_n, // Optional string|int parameterValue_n, // Optional ... )

Parameters:

A main message appearing in the log.message

Name of an optional parameter. If declared, parameterValue_n mustbe declared as well.

parameterName_n

Value of an optional parameter. If declared, parameterName_n must bedeclared as well.

parameterValue_n

Nothing.Returns

43

Analysis Agent 4.1

Example 20.

The following code example logs a warning message, which when displayed in the System LogInspector will look like the following snapshot:

logWarning( "UDR failed validation", "ANUMBER IS ", input.anum, "BNUMBER IS ", input.bnum, "DURATION IS ", input.duration);

Figure 6. System Log Inspection.

7.5.3. dispatchEvent

A user can define a customized event type. This is done via an event UDR, optionally extended withuser defined fields. This event UDR can be populated with any information by utilizing APL code,and then be sent on, using the dispatchEvent function, to be caught by the Event Notifier. Forfurther information on the Event Notifier, refer to the Convergent Mediation Engine™ Core - UserGuide.

void dispatchEvent( UltraEvent eventUDR)

Parameters:

The name of the event UDR.eventUDR

Nothing.Returns

7.5.4. dispatchMessage

The dispatchMessage method is used to produce user defined messages belonging to predefinedEvent Categories (see the Event Notification Editor). For instance, an Event Category could benamed 'DISASTER', and be configured to cause a mail to be sent to the System Administrator. Thenan APL node could send a detailed description with the dispatchMessage function whenever thiserror situation is detected.

Another usage is mapping the value to a Dynamic Column to be used in the Workflow Manager.The mapping is made from the Preferences window of the workflow. This is useful for instance whenwanting to log some kind of progress report to be visible from the Workflow Manager. For furtherinformation, please refer to the Convergent Mediation Engine™ Core - User Guide.

void dispatchMessage

44

Analysis Agent 4.1

( string string, string <Event Category> )

Parameters:

Value/message to append to the <Event Category>.string

Name of a User Defined Event as defined from the Event Notification Ed-itor. This event must be defined in the Event Notification Editor in orderfor the APL code to compile.

<Event Category>

Nothing.Returns

7.6. List Related Functions

7.6.1. listAdd

Adds an element to a list.

void listAdd ( list<type> list, any value )

Parameters:

A list.list

Value to add to the list.value

Nothing.Returns

7.6.2. listClear

Removes all elements from a list.

void listClear ( list<type> list )

Parameters:

The list to remove.list

Nothing.Returns

7.6.3. listCreate

Creates a list of any type, and (optionally) populates it.

list<type> listCreate ( type,

element1, //Optionalelement2, //Optional

45

Analysis Agent 4.1

... )

Parameters:

The type of the elements comprising the list.type

The value of an element to be added to the list. Optional parameters: if none is entered,the list will be empty.

element<n>

The list.Returns

7.6.4. listFindIndex

Finds an element in a list.

The condition is tried for each element in the list until there is a match. On match, the index of thematching entry is returned. If no match is found, -1 is returned. The first index is 0 (zero).

int listFindIndex ( list<type> list,

varSpec, Condition,

int startIndex ) //Optional

Parameters:

A list.list

Variable specification. Defines a variable to hold the list data for the condition eval-uation. Can be either just a valid variable name in which case the variable type is the

varSpec

same as the list element type, or a type can be explicitly set on the variable in whichcase the list elements are cast to this type and then assigned to the variable. An incor-rect type will cause a runtime ClassCastException.

Condition for matching an item in the list.Condition

(Optional) index to start to search the list on. If startIndex is not specified, thelist is searched from the beginning.

startIndex

The position of the matching element. If several are found, the index for the firstmatch is returned. If none matches, -1 is returned.

Returns

Example 21.

Say there is a list of integers, where the index for the element being equal to 4 is to be found.The following code will output 1 for the list [50, 4, 152, 34, 1].

debug( listFindIndex( myList, i, i == 4 ) );

7.6.5. listGet

Retrieves an element from a list. The index must be in the range 0<=index<listSize(list)or the workflow will abort with a runtime error.

type listGet ( list<type> list,

46

Analysis Agent 4.1

int index)

Parameters:

A list.list

Index of the value to get.index

The value given at the defined index.Returns

7.6.6. listInsert

Inserts a new element in a list.

void listInsert ( list<type> list, int index, type value )

Parameters:

A list.list

Where to insert the new value.index

The new value.value

Nothing.Returns

7.6.7. listRemove

Removes an element from a list. The index must be in the range 0<=index<listSize(list) orthe workflow will abort with a runtime error.

void listRemove ( list<type> list, int index )

Parameters:

A list.list

Index of the value to remove.index

Nothing.Returns

7.6.8. listSet

Replaces an existing element in a list.

void listSet ( list<type> list, int index, type value )

Parameters:

47

Analysis Agent 4.1

A list.list

The position of the element to be replaced.index

The new value to replace the existing with.value

Nothing.Returns

7.6.9. listSize

Gets the size of a list.

int listSize ( list<type> list)

Parameters:

A list.list

The size of the list.Returns

7.6.10. listSort

Sorts a list of any type, including UDR lists, based on a field in the UDR.

For all lists except for UDR lists:

void listSort ( list<type> list,

order ) //Optional

For UDR lists:

void listSort ( list<UDRtype> UDRlist, string field,

order, //Optionaludrtype ) //Optional

Parameters:

A list.list

A list of UDRs.UDRlist

Name of the UDR field to sort on. The field must be of one of the primitive types(boolean, byte, int), one of the string types, or date.

field

Sorting order (optional). Could be one of ascending (default) or descending.order

Parameter to determine UDR type if the list is declared to hold generic UDRs. In the fol-lowing bullets, the first is generic and must have the udrtype specified. The seconddoes not require the udrtype since the UDR type is known from the declaration.

udrtype

1. list<drudr>MyList;

2. list<MyUDRType>MyList;

48

Analysis Agent 4.1

If a type name is used and the UDR cannot be converted to this type this will be a runtimeerror.

Nothing.Returns

Example 22.

list<MyUDRType> myList;

// Assume we have added items to the list

listSort( myList, SequenceNumber, descending );

7.7. Map Related FunctionsMap functions allow the user to operate through hash maps; create a set of variables, keys, and associateeach of them with a specific value. Hence, link a variable to another variable. All types are allowedfor both the keys and values.

7.7.1. mapClear

Removes all entries in a map.

void mapClear ( map<key,value> myMap )

Parameters:

The map to clear.myMap

Nothing.Returns

7.7.2. mapContains

Checks if a key is present in a map.

boolean mapContains ( map<key,value> myMap, keyType key )

Parameters:

The map to check.myMap

The key to look for.key


49

Analysis Agent 4.1

7.7.3. mapCreate

Creates a new, empty map.

map<key,value> mapCreate ( keyType key, valueType value )

Parameters:

The type of the key.key

The type of the value to associate with the key.value

An empty map.Returns

Example 23.

An example of a map definition, including assignment of the first key/value pair.

map<string,int> myMap = mapCreate( string, int);mapSet( myMap, "Failed", 22 );

7.7.4. mapGet

Gets the value of a specified key in a map.

valueType mapGet ( map<key,value> myMap, keyType key )

Parameters:

The map to fetch a value from.myMap

The name of the key holding the value to fetch.key

Any, depending on type of value.Returns

7.7.5. mapKeys

Returns a list of all keys present in a map. Note that the order of the elements in the list is not defined.

list<key> mapKeys( map<key,value> myMap )

Parameters:

The map to fetch all keys from.myMap

An unsorted list of keys according to: [keyA, keyB, keyX...]Returns

50

Analysis Agent 4.1

7.7.6. mapRemove

Removes a key and its corresponding value.

valueType mapRemove ( map<key,value> myMap, keyType key )

Parameters:

The map to remove a key from.myMap

The name of the key to remove.key

The value corresponding to the removed key.Returns

7.7.7. mapSet

Sets or updates a key's associated value.

void mapSet ( map<key,value> myMap, keyType key, valueType value )

Parameters:

The map to update.myMap

The name of the key to set/update.key

The corresponding value to set/update..value

Nothing.Returns

7.7.8. mapSize

Returns the number of keys present in a map.

int mapSize( map<key,value> myMap)

Parameters:

The map to examine.myMap

The size of the map.Returns

7.7.9. mapValues

Returns a list of all values present in a map. Note that the order of the elements in the list is not defined.

list<value> mapValues( map<key,value> myMap)

Parameters:

51

Analysis Agent 4.1

The map to fetch all values from.myMap

A list of all values (unsorted):Returns

[valueA, valueB, valueX...]

7.8. MIM Related FunctionsThe MIM functions allows the agent to publish its own MIM values or to read MIM values from othernodes.

The MIM functions cannot be used within global APL scripts, that is in code saved in the APLCode Editor.

7.8.1. mimGet

Returns the value of a MIM resource available in the workflow.

any mimGet ( string category, string mimResourceName )

Parameters:

Name of the node or workflow owning the MIM resource.category

Name of the MIM resource whose value is to be returned.mimResourceName

MIM value as any type. The result always needs to be type casted or elsecompilation fails.

Returns

7.8.2. mimIsNull

Evaluates if a MIM resource is defined. Available to check MIMs of non-object types, such as int.

boolean mimIsNull ( string category, string mimResourceName )

Parameters:

Name of the node or workflow owning the MIM resource.category

Name of the MIM resource to look for.mimResourceName


7.8.3. mimPublish

Publishes a MIM resource. The method call can only be present outside a function block.

void mimPublish ( assigned,

52

Analysis Agent 4.1

string name,mimType )

Parameters:

Keyword that identifies when the MIM resource is assigned a value. Can be one of:assigned

• batch - Value is assigned from the consume block.

• header - Value is assigned in the beginBatch block.

• trailer - Value is assigned in the endBatch block.

• global - Value is assigned from any block.

MIM resource name. Must be a string constant.name

Data type of MIM resource. May not be a UDR type.mimType

Nothing.Returns

Realtime workflows can only publish MIMs of type global.

7.8.4. mimSet

Assigns a value to a user defined MIM resource. The function may be called at any time. Note that itis the responsibility of the user to make sure the MIM value is available in accordance with the specifiedassigned type (see MIMPublish).

void mimSet ( string name, any value )

Parameters:

Name of the MIM resource to assign a value.name

Value to assign - must match the published MIM type.value

Nothing.Returns

53

Analysis Agent 4.1

7.8.5. MIM Example

Example 24.

Note that mimSet is made from a drain block. This to make sure the MIM exists when calledfrom any other node. The calls will be made at endBatch (for trailer MIMs), which alwaysoccurs after drain.

mimPublish(trailer, "Total duration", int);int duration;

beginBatch { duration = 0;}

consume { duration = duration + input.CallDuration;}

drain { mimSet( "Total duration", duration );}

7.9. Audit Related FunctionsLog Audit Data (the Preferences dialog of a workflow) has to be selected in order for all audit relatedfunctions to take effect.

The key or combination of keys are only unique within a batch. If a second batch is executed,a new row looking exactly the same as one from the previous batch might be received. Thepurpose of the key is to be able to save more than one row per batch.

7.9.1. auditAdd

The auditAdd function increases or decreases a column value in any table specified. Note that thetable must be configured in the Audit Profile Editor window first, and the column must be definedas a 'Counter'.

void auditAdd ( string profile, string tableAlias, string columnAlias, any value, any key1, //Optional any key2 ) //Optional

Parameters:

The name of a defined Audit Profile. This declaration is case-sensitive.profile

54

Analysis Agent 4.1

The profile declaration must include the folder name: myFolder.myProfile

Name of the table to update. Must exist in the specified Audit Profile.tableAlias

The declaration must be uppercase and include the table owner: ADMIN.MYTABLE.

Name of the column to update. The declaration must be uppercase: MYCOLUMN.columnAlias

The value to increase/decrease the existing value with.value

The key columns in the order as entered in the Audit Profile Editor.key<n>

Nothing.Returns

7.9.2. auditSet

The auditSet function sets a column value in any table specified. Note that the table must be con-figured in the Audit Profile window first, and the column must be defined as being of type 'Value'.

void auditSet ( string profile, string tableAlias, string columnAlias, any value, any key1, //Optional any key2 ) //Optional

Parameters:

The name of a defined Audit Profile. This declaration is case-sensitive.profile

The profile declaration must include the folder name: myFolder.myProfile

Name of the table to update. Must be defined in an Audit Profile first.tableAlias

The declaration must be uppercase and include the table owner: ADMIN.MYTABLE.

Name of the column to update. The declaration must be uppercase: MYCOLUMN.columnAlias

The value to set.value

The key columns in the order as entered in the Audit Profile.key<n>

Nothing.Returns

7.10. String Related Functions

7.10.1. String Concatenation

Strings are concatenated with the arithmetic '+' operator:

string str1 = string str2 + string str3...

7.10.2. strEndsWith

Returns true if str ends with substr, otherwise false.

boolean strEndsWith ( string str,

55

Analysis Agent 4.1

string substr)

Parameters:

String to examine.str

String to look for.substr


7.10.3. strEqualsIgnoreCase

Compares two strings and returns true if they are equal, and false if they are not. This comparisonignores case.

boolean strEqualsIgnoreCase ( string str1, string str2 )

Parameters:

A string to compare.str1

Another string to compare.str2


7.10.4. strIndexOf

Returns the first position where substr can be found in str. If substr is not found, -1 is returned.The position starts to count from 0 (zero), which is the first character.

int strIndexOf ( string str, string substr, int startIndex ) //Optional

Parameters:



Index where to start the string search. If startIndex is not specified, the string issearched from the beginning.

startIndex

The position of the first character of substr within str. If not found, -1 is returned.Returns

7.10.5. strInsert

Inserts a string into another at a specific position, and returns the result string.

string strInsert ( string str1, int position,

56

Analysis Agent 4.1

string str2 )

Parameters:

The string that str2 is inserted into.str1

The string to insert.str2

Position where str2 will be inserted. Position 0 (zero) indicates prior to the first char-acter of str1.

position

The result string. Note, the str1 and str2 are not modified.Returns

7.10.6. strLength

Returns the number of characters in a string.

int strLength( string str )

Parameters:


The number of characters in the string.Returns

7.10.7. strREContains

Returns true if a string contains a substring, else false. The function is case sensitive.

boolean strREContains ( string str, string regexp )

Parameters:


The substring to look for. Regular expressions are allowed.regexp


Regular expressions according to Java syntax applies. For further information, please refer to

http://java.sun.com/j2se/1.5.0/docs/api/java/util/regex/Pat-tern.html

7.10.8. strREIndexOf

Returns the first position where a regular expression can be found in a string. If the regular expressionis not found, -1 is returned. The position starts to count from 0 (zero), which is the first character.

int strREIndexOf ( string str, string regexp )

57

Analysis Agent 4.1

Parameters:


Regular expression to be used when examining str, using case sensitive andcontains (not matches).

regexp

The position of the first character of regexp within str. If not found, -1 is returned.Returns

Example 25.

strREIndexOf( "Hello There!", "[Tt]he" ) // Returns 6



7.10.9. strREReplaceAll

Replaces existing substrings within a string with new values.

string strREReplaceAll ( string str, string old, string new )

Parameters:

String to change.str

The substring to replace. Regular expressions are allowed.old

The new substring to replace the old. If an empty string is entered, the old will be removedwithout inserting any new value.

new

The result string.Returns

Example 26.

string strREReplaceAll ( string flower, string low, string orm )

This call will change the string "flower" to the string "former". If the new parameter was leftempty ("orm" in this case), the resulting string would have been "fer".



58

Analysis Agent 4.1

7.10.10. strReplaceChars

Replaces characters in a string at a given position with another string and returns the new string. Ifstr2 is longer than str1, then the resulting string will be expanded to fit the complete str2.

string strReplaceChars ( string str1, int position, string str2 )

Parameters:

The base string.str1

Position where str2 will start replacement in str1. Position 0 (zero) indicates priorto the first character of str1.

position

String to use for replacement.str2

The result string. Note, the str1 and str2 are not modified.Returns

Example 27.

The following example returns the string: Hi Walter

strReplaceChars("Hi Sister", 3, "Walt");

7.10.11. strSplit

Splits a string containing delimiters into a list of strings, not including the delimiters.

list<string> strSplit ( string str, string delimiter )

Parameters:

The base string.str

The delimiter which is to be used to split elements. The delimiter can be more thanone character, for instance "\r\n".

delimiter

A list of strings, without the delimiter.Returns

Example 28.

strSplit("one,two,three", ",");

Will return the following list:

[0] one[1] two[2] three

59

Analysis Agent 4.1

7.10.12. strStartsWith

Returns true if str starts with substr, otherwise false.

boolean strStartsWith ( string str, string substr )

Parameters:




7.10.13. strSubstring

Returns a substring from a given string. The extracted substring is from position start to end.

string strSubstring ( string str, int start, int end )

Parameters:

The base string.str

Position where to start the substring extraction. Position 0 (zero) points to the first characterof str.

start

Position where to end the extraction. That is, the index of the letter after the last letter in thesubstring.

end

The substring.Returns

Example 29.

The following call returns the string "the".

strSubstring("hi there", 3, 6);

7.10.14. strToLower

Turns all letters in a string to lower-case.

string strToLower( string str )

Parameters:

The base string.str

The result string. Note, the str is not modified.Returns

60

Analysis Agent 4.1

7.10.15. strToUpper

Turns all letters in a string to capital letters.

string strToUpper( string str )

Parameters:

The base string.str


7.10.16. strTrim

Removes leading and trailing spaces from a string and returns the result.

string strTrim( string str )

Parameters:

The base string.str


7.11. UDR Related Functions

7.11.1. udrAddError

Adds an error description message to the UDR, which will be visible from the Error CorrectionSystem Inspector window, Error Code column. A UDR can have several error descriptions.

void udrAddError ( drudr myUDR, string <ECS Error Code>, string myString )

Parameters:

The UDR to add error message to.myUDR

An ECS Error Code as defined from the Error Correction System Inspect-or.

<ECS Error Code>

Any string to associate with the ECS Error Code. This parameter comprisesthe Error Case found in the ECS window. Optional.

myString

Nothing.Returns

61

Analysis Agent 4.1

Example 30.

The following example appends error text to a UDR, which when displayed in ECS will looklike the following snapshot:

udrAddError( input, "CALL ID ERROR", "The callId: " + input.callId + " Calling number: " + input.anum );

Figure 7. To display the Error Viewer, double-click the Error Code element for the specificUDR.

7.11.2. udrClearErrors

Removes all error descriptions, added to a UDR. This is useful when collecting UDRs from ECS fora new validation which may send them back to ECS. Adding a new error description, if one alreadyexists, will make it impossible to auto-assign the UDR to a reprocessing group when it arrives to ECS.

void udrClearErrors( drudr myUDR )

Parameters:

The UDR to clear errors from.myUDR

Nothing.Returns

7.11.3. udrClone

Clones a UDR. Is used when having multiple routes leaving the node, where any of the routes changesthe contents of a UDR. For complex UDR structures, this can be a heavy operation, and must be usedwith care.

drudr udrClone( drudr myUDR )

Parameters:

The UDR to clone.myUDR

A clone of myUDR.Returns

7.11.4. udrContainsError

Returns true if an error description has been added to the UDR.

62

Analysis Agent 4.1

boolean udrContainsError( drudr myUDR )

Parameters:

The UDR to check for error descriptions.myUDR


7.11.5. udrCreate

Creates a new UDR of the specified type. All fields are included, except for the optional.

drudr udrCreate( type UDRType )

Parameters:

A defined UDR type.UDRType

A UDR of the specified type.Returns

7.11.6. udrDecode

Decodes a bytearray into a list of UDRs, and returns an error message if the operation fails. To routethe UDRs to a subsequent node, loop through the resulting list routing each element individually.

string udrDecode ( string decoderName, list<drudr> UDRlist, bytearray indata, boolean fullDecode ) //Optional

Parameters:

The name of a defined decoder, according to the syntax <folder_name>.<mod-ule_name>.<decoder_name>.

decoderName

The list in which the resulting, decoded UDRs will be saved.UDRlist

The input data to decode.indata

States if Full Decode will be applied. By default, this parameter is set to true. Forfurther information on the option, please refer to the Convergent Mediation Engine™Ultra Format Management - User Guide.

fullDecode

A string containing nothing (null) if the operation succeeded, and an error messageif it failed.

Returns

63

Analysis Agent 4.1

Example 31.

A list must be created, not only declared, before used by udrDecode:

list<drudr> myList = listCreate(drudr);

if (udrDecode("myFormat.myDecoder", myList, input) == null) { // Do something} else { // Error handling}

7.11.7. udrEncode

Encodes a UDR.

bytearray udrEncode ( string encoderName, drudr myUDR )

Parameters:

The name of a defined encoder, according to the syntax <folder_name>.<mod-ule_name>.<encoder_name>.

encoderName

The UDR to encode.myUDR

A bytearray.Returns

7.11.8. udrForceDecode

By default, a Decoder only evaluates record sizes and field termination marks. It does not read thefield values to check for validity. This is done for each field when it is actually accessed, either froma node utilizing APL code, or from an Encoder.

The udrForceDecode function will decode each field within a UDR. If a field is not valid, the nodewill abort. This function is equal to the Full Decode option in the Decoder window. Note that thefunction has a negative impact on performance, and must be used mainly for testing purposes.

string udrForceDecode ( drudr myUDR, boolean abortOnFail ) //Optional

Parameters:

The UDR to fully decode.myUDR

An optional argument. If not stated, true is assumed. If set to false, the workflowdoes not abort if decoding fails.

abortOnFail

If decoding succeeds, null is returned. If not - and abortOnFail is set to false- an error message is returned.

Returns

64

Analysis Agent 4.1

7.11.9. udrGetErrorsAsString

Returns a string containing all error information that has been added to the UDR.

string udrGetErrorsAsString( drudr myUDR )

Parameters:

The UDR.myUDR

A string with all error information added to the UDR.Returns

7.11.10. udrGetValue

Returns the value of fieldName in the myUDR.

any udrGetValue ( drudr myUDR, string fieldname )

Parameters:

The UDR containing the field of interest.myUDR

A string, exactly matching the name of an existing field. If the field is OPTIONALand is not present, the function returns null.

fieldname

Constructed field names, that is subUDR.subField are allowed. If a field in the path isnot present, null is returned in this case as well.

Any, depending on the field type.Returns

7.11.11. udrIsPresent

Returns true if the UDR field is present. This function will not compile if the field is not optional.

boolean udrIsPresent( identifier field )

Parameters:

The full field name including the UDR name. For instance, input.IN.moRecord.field


65

Analysis Agent 4.1

Example 32.

The function is recursive. If the optional field myField is to be accessed, and it is nested inthree levels: fieldA.fieldB.myfield, and all levels are optional, only one if-statementis required:

if ( udrIsPresent( input.fieldA.fieldB.myField ) ) { // Some kind of operation.}

7.11.12. udrRoute

This method provides several functionalities. In relation to just routing the current UDR on a specificroute, the functions also offers the possibility to clone a UDR. Refer to section Section 7.11.3, “udr-Clone” for further details.

Note that if no call to the route methods is made, the node will not have any outgoing routes in theWorkflow Designer environment - it will not be possible to link to a subsequent node.

The function routes a UDR to all the output routes, or a named output route. If the clone keywordis used, the UDR is cloned before output.

The udrRoute function cannot be used within global APL scripts, that is in code saved in theAPL Code Editor.

void udrRoute ( drudr myUDR, string routeName, //Optional

clone ) //Optional

Parameters:

The UDR to send on.myUDR

The route on which to send the UDR.routeName

The UDR is cloned before routing on the output/outputs.clone

Nothing.Returns

The udrRoute function cannot be used within global APL scripts, that is in code saved in theAPL Code Editor.

7.11.13. udrSetValue

Sets the field named fieldName in the UDR myUDR to value.

void udrSetValue ( drudr myUDR, string fieldName,

66

Analysis Agent 4.1

any value )

Parameters:

The UDR, for which to set the field value.myUDR

A string, exactly matching an existing field. Constructed field names in the form ofsubUDR.subField are allowed. Trying to set a non-existing field will cause runtimeerror.

fieldName

A value depending on the field type.value

Nothing.Returns

7.11.14. udrUnsetPresent

Sets an optional field to not present. By default, the Encoder maps null fields as empty but present, inthe output files. This can be avoided by clearing a flag, Is Present, using the udrUnsetPresentfunction. This causes the empty fields not to be present at all in the Encoded files.

void udrUnsetPresent( identifier field )

Parameters:

The full name of the optional field including the UDR name. For instance, input.IN.moRe-cord.

field

Nothing.Returns

7.12.Type Conversion Related FunctionsThe following functions are used to perform conversions between different types.

7.12.1. baToBigInt

Converts a bytearray to a big integer (two complement).

void baToBigInt ( bigint bigintVar, bytearray baValue )

Parameters:

Bigint variable to set.bigintVar

Bytearray value.baValue

Nothing.Returns

7.12.2. baToHexString

Converts a bytearray to a string containing the hexadecimal values.

67

Analysis Agent 4.1

string baToHexString(bytearray baValue)

Parameters:

Bytearray value.baValue

A hexadecimal string.Returns

7.12.3. baToStr

Converts a bytearray to a string.

string baToStr ( bytearray array, string charEncoding ) //Optional

Parameters:

The bytearray to convert.array

The character encoding to use. If no encoding is specified, ISO-8859-1 (Latin 1)is used.

charEncoding

The result string.Returns

7.12.4. dateToString

Converts a date to a string and assigns the result to a string variable (or field). Returns true if theconversion succeeded. The only time the conversion can fail, is if the date is null or the format is in-valid.

boolean dateToString ( string stringVar, date dateValue, string format ) //optional

Parameters:

String identifier to set.stringVar

The date value.dateValue

If no format string is specified, the default system format is used as specified in$MZ_HOME/etc/config.xml, the mz.server.dateformat parameter.

format

The date formatting is based on standard Java syntax. For further information,please refer to

h t -tp://java.sun.com/j2se/1.5.0/docs/api/java/util/regex/Pat-tern.html


68

Analysis Agent 4.1

7.12.5. strToBA

Converts a string to a bytearray and assigns the result to a bytearray variable (or field).

void strToBA ( bytearray array, string stringValue, string charEncoding) //Optional

Parameters:

The resulting bytearray.array

The string value.stringValue

The character encoding to use. If no encoding is specified, ISO-8859-1 (Latin 1)is used.

charEncoding

Nothing.Returns

7.12.6. strToBigInt

Converts a string to a big integer. If the conversion was successful true is returned.

boolean strToBigInt ( bigint bigintVar, string stringValue )

Parameters:

Big integer variable to set.bigintVar

String value.stringValue


7.12.7. strToDate

Converts a string to a date and assigns the result to a date variable (or field). Returns true if theconversion succeeded.

boolean strToDate ( date dateVar, string stringValue, string format, //Optional string timeZone ) //Optional

Parameters:

Date identifier to set.dateVar

Date value.stringValue

If no format string is specified, the default system format is used as specified in$MZ_HOME/etc/config.xml, the mz.server.dateformat parameter.

format

69

Analysis Agent 4.1

The date formatting is based on standard Java syntax. For further information,please refer to

h t -tp://java.sun.com/j2se/1.5.0/docs/api/java/util/regex/Pat-tern.html

An optional string stating the timezone to set. Two different formats are allowed - ab-breviation or offset from GMT:

timeZone

Example 33.

The time zone for Pacific Standard Time can be given in any of the followingformats:

• "PST"

• "GMT-08:00"

Note that if an invalid time zone format is entered, no error is returned. Instead,the time zone is automatically set to "GMT".


7.12.8. strToDouble

Converts a decimal string to a double and assigns the result to a double variable (or field). Returnstrue if the conversion succeeded.

boolean strToDouble ( double doubleVar, string stringValue )

Parameters:

Double identifier to set.doubleVar



7.12.9. strToFloat

Converts a decimal string to a float and assigns the result to a float variable (or field). Returns trueif the conversion succeeded.

boolean strToFloat ( float floatVar, string stringValue )

Parameters:

Float identifier to set.floatVar

70

Analysis Agent 4.1



7.12.10. strToInt

Converts a decimal or hexadecimal string to an integer and assigns the result to an integer variable (orfield). Returns true if the conversion succeeded.

If one of the keywords dec or hex are specified, the string is assumed to be decimal or hexadecimalrespectively. Decimal is the default.

boolean strToInt ( int intVar, string stringValue, dec|hex ) //Optional

Parameters:

Integer identifier to set.intVar



Example 34.

Provided that the field CallTime is a string containing a hexadecimal string, it converts itscontents to an integer.

int a;int b;strToInt( a, CallTime, hex );strToInt( b, "12345" ); /* dec is default */

7.12.11. udrToString

Converts a UDR to a string. Each UDR will be preceded with the internal class name, and each of itsfield will be preceded with the field name.

string udrToString( drudr myUDR )

Parameters:

The UDR to convert.myUDR

A string containing the UDR fields. The output will be in the following format:Returns

Field values for: <internal UDR reference>field1: <value>

71

Analysis Agent 4.1

field2: <value>

7.13. Database Table Related FunctionsA special table type is used to handle database table lookup with optional caching in memory. All ofthe following column references may be either numeric (column index, starts at 0) or string (columnname).

The table lookup only supports int, string and date types.

7.13.1. sqlExec

The sqlExec function is used when updating and inserting data into tables. It returns an integer valuestating how many rows were updated/inserted. SQL errors will cause runtime errors (the workflowaborts).

int sqlExec ( string dbProfile, string sqlQuery )

Parameters:

Name of the database where the table resides.dbProfile

SQL query to send to the database. Note that SQL statements must not be ended with';'. Calls to stored procedures must be embedded in blocks. Examples valid for Oracledatabases:

sqlQuery

"<SQL query>""BEGIN <procedure>; END;"

An integer equaling the number of rows updated or inserted.Returns

7.13.2. tableCreate

Returns a table that holds the result of a database query. SQL errors in the table lookup will causeruntime errors (workflow aborts).

table tableCreate ( string dbProfile, string sqlQuery )

Parameters:

Name of the database where the table resides.dbProfile

SQL query to send to the database. Note that SQL statements must not be ended with';'. Only columns of type number, date and string are supported.

sqlQuery

A table.Returns

72

Analysis Agent 4.1

Example 35.

To avoid performance problems, the table must be read from the database as seldom as possible.For instance when a workflow is activated.

initialize { table myTab = tableCreate("myFolder.myProfile", "select user from subscribers" );}

7.13.3. tableRowCount

Returns the number of rows in a table.

int tableRowCount ( table tableValue )

Parameters:

Table object.tableValue

An integer stating the number of rows found.Returns

7.13.4. tableGet

Returns the value of a table entry. The value is returned as an any object, which means that this valuewill typically have to be cast to a more useful type like int, string or date.

any tableGet ( table tableValue, int row, int|string column)

Parameters:

A table object.tableValue

The row index. The first row is indexed 0 (zero).row

Column index or name. The first column is indexed 0 (zero).column

Any depending on the column type.Returns

7.13.5. tableLookup

Returns a table containing all the rows of the original table matching the specified column value(s).At least one pair of (column, operator, value) group must be specified.

table tableLookup ( table tableValue, int|string column1, string operator1, any value1,

73

Analysis Agent 4.1

any value1a, //Optional ... int|column columnN, string operatorN, any valueN, any valueNa ) //Optional

Parameters:


Column index or name. The first column is indexed 0 (zero).columnN

Operator specifying the range of the requested values. Possible operators are:operatorN

• "="

• "!="

• ">"

• "<"

• "<="

• ">="

• "between"

• "not between"

• "starts with"

The "between" and "not between" operators require two operands, (columnN,operatorN, valueN, valueNa).

The "starts with" operator can only be applied to string columns.

Value matching columnN and operatorN.valueN

A table matching the query.Returns

74

Analysis Agent 4.1

Example 36.

The following example looks for a specific entry in the in-memory table returned by the precedingtableCreate command.

table myTab;

// To avoid performance problems, the table is read // from the database once for each time the// workflow is activated.

initalize { myTab = tableCreate( "myFolder.myProfile", "select user from subscribers" );}

consume { table user = tableLookup( myTab, "user", "=", input.anum );

if ( tableRowCount( user ) > 0) { // At least one entry found. } else { // No entry found. }

}

7.13.6. tableCreateIndex

Creates a hash index on one or several columns of a table. This will greatly increase the efficiency ofsubsequent tableLookup calls on these columns using the equals operator. If the column alreadyhas an index, this function has no effect.

void tableCreateIndex ( table tableValue, int|string column1, ... int|string columnN )

Parameters:


A column index (starting with 0 for the first column), or name on which to create anindex. At least one column must be specified.

columnN

Nothing.Returns

75

Analysis Agent 4.1

Example 37.

initialize { table myTab = tableCreate("myFolder.myProfile", "select id, user from anum" ); tableCreateIndex( myTab, "id" );}

7.14. Misc Functions

7.14.1. abort

Aborts the workflow, and logs a user defined message to the System Log Inspector.

void abort( string message )

Parameters:

A message (of type error) which is logged to the System Log Inspector when the workflowis aborted.

message

Nothing.Returns

The only APL code to be executed after an abort call is the deinitialize function block.

7.14.2. crc32

Function that computes the CRC-32 of supplied argument.

long crc32( any source )

Parameters:

The argument used to compute the CRC-32. Support types are string, bytearray andnumber types.

source

The CRC-32 value.Returns

7.14.3. hintEndBatch

Large input files can be split into smaller output files using hintEndBatch. The function will sendan End Batch message to the other nodes in the workflow, possibly causing a split of the current batchbeing processed. How each agent acts upon such a request can be found out in their respective userguides; the Transaction Behavior section. If the workflow does not contain any node capable of actingupon the request, the workflow will abort.

76

Analysis Agent 4.1

The hintEndBatch() function is only supported for workflows containing one of the following:

• Database Collection agent

• Disk Collection agent

• FTP Collection agent

void hintEndBatch()

None.Parameters:

Nothing.Returns

A split batch cannot be sent to ECS. Calling cancelBatch after hintEndBatch, will resultin workflow abort. However, the original batch can be sent to ECS to make sure to evaluate ifthe batch will be cancelled before it is split.

7.14.4. instanceOf

Returns true if the value is of the specified type.

boolean instanceOf ( any any, type type )

Parameters:

The value to check. Can be of any type, even primitive types.any

Any type to check against.type


Example 38.

consume{ if ( instanceOf( input, module1.type1 ) ) { udrRoute( ( module1.type1 )input, "link_1" ); } else if ( instanceOf( input, module2.type2 ) ) { udrRoute( ( module2.type2 )input, "link_2" ); } else { // handle error case }}

77

Analysis Agent 4.1

7.14.5. mailNotify

Sends an email to a configured recipient. In order to operate, the system must have an email remitterand an SMTP mail server defined in $MZ_HOME/etc/platform.xml.

If used within the consume block, make sure that conditional expressions guarantees that thisfunction does not get called for each UDR.

boolean mailNotify ( string address, string subject, string message )

Parameters:

Email address to the recipient on the form: "[email protected]".address

Text ending up as the email subject.subject

Message string ending up as the body of the email.message

true or false, depending on if the call succeeded.Returns

7.14.6. cancelBatch

Emits a Cancel Batch that aborts the processing of the current batch and possibly continues with thenext file (depending on the workflow configuration).

void cancelBatch ( string message, drudr errorUDR ) //Optional

Parameters:

A message (of type error) which is logged to the System Log when the batch is cancelled.message

It is possible to send an error UDR containing any information found useful to the ECSwhere the batch ends up (if configured to do so). Note that an error UDR may be defined

errorUDR

from the Preferences window as well. In this case, the one APL code overrules thewindow configuration.

This parameter is optional.

Nothing.Returns

78

Analysis Agent 4.1

Archiving 4.1


Version 1.28

1





2

Archiving 4.1



2. Archiving Agents ........................................................................................................ 52.1. Archive Profile ................................................................................................ 5

2.1.1. Directory Template Tokens ...................................................................... 62.1.2. Archive File Naming Algorithm ................................................................ 7

2.2. Configuration .................................................................................................. 72.2.1. Archiving Agent - Global ........................................................................ 72.2.2. Archiving Agent - Local .......................................................................... 9

2.3. Transaction Behavior ....................................................................................... 102.3.1. Emits ................................................................................................. 102.3.2. Retrieves ............................................................................................ 10



2.6. Node Message Events ...................................................................................... 113. Archive Inspector ...................................................................................................... 12

3.1. Searching the Archive ...................................................................................... 134. Maintaining Archives ................................................................................................. 14

3

Archiving 4.1

1. IntroductionThis document describes the Global Archiving Batch agent, Local Archiving Batch agent and theArchive Inspector, being part of the functionality available on the Nokia Convergent Mediation Engine™platform.




4

Archiving 4.1

2. Archiving AgentsConvergent Mediation Engine™ offers the possibility of archiving data batches for a configurableperiod of time. The Archiving agents can be configured to archive all received data batches. Each databatch is saved as a file in the specified repository. The global archiving agent also saves a correspondingreference in the database enabling the Archive Inspector browsing and purging of these files.

Depending on the selected profile, the Archive services are responsible for naming and storing of eachfile and to purge outdated files at a regular basis. Utilizing the Directory Templates and base direct-ories specified in the Archive Profile window, directory structures are dynamically built when filesare stored.

The system administrator defines what structure is suitable for each profile. For instance, set the dir-ectory structure to be changed with respect to collecting node name on a daily basis. The Archiveservices will automatically create all directories needed in the base directory or directories.

2.1. Archive ProfileProfiles containing storage, naming scheme and lifetime for targeted files are configured in the ArchiveProfile Editor window, displayed when Archive Profile... is selected from the Configuration menuon the desktop.

Figure 1. Archive Profile Editor.

If several base directories are configured, the switch policy determines for howlong the Archive services will populate each base directory before starting to

Switch Policy

populate the next one (daily, weekly, or monthly). When the last base directory isreached, the archiving wraps to the first again.

5

Archiving 4.1

One or several base directories that can be used for archiving of files. For consid-erable amounts of data to be archived, there might be a need for several base dir-ectories located on different disk partitions.

Base Directory

List of tokens that, in run-time, builds subdirectory names appended to one of thebase directories. The tokens could be either special tokens or user defined values.

Directory Tem-plate

Subdirectories on any level, can be constructed by using the special token Directorydelimiter.

If enabled, files older than the entered value will automatically be deleted fromthe archive. If Keep Files is enabled as well, only the reference is removed (validonly for the global archiving agent). If disabled, files will never be deleted.

Remove Entries(days)

• The Global archiving agent:

The cleanup of outdated files is mastered by the Archive Cleaner task.It removes the reference in Convergent Mediation Engine™ , as wellas the file itself from its archive directory. Consequently, the data storageis also dependent on the setup of the task scheduling criteria.

The timeout parameter is set in the Archive Profile window (RemoveFiles (days)).

The Archive Inspector allows files to be removed manually.

• The Local archiving agent:

The cleanup of outdated files is mastered by the workflow. It removesthe file from its archive directory.

If enabled, only the references in the database are removed while the files remainon disk. To remove the actual files, this option needs to be disabled.

Keep Files

2.1.1. Directory Template Tokens

Figure 2. Add Directory Template window.

Tokens to be used as part of the directory name.Specialtoken

• Year - Inserts four digits representing the year the file was archived.

• Month - Inserts two digits representing the month the file was archived.

• Day - Inserts two digits representing the day of the month the file was archived.

• Hour - Inserts two digits representing the hour (24) of the day the file was archived.

• Node directory name - Inserts the MIM value(s) defined in the Node DirectoryName list in the Archiving agent configuration window.

6

Archiving 4.1

• Day index - Inserts a day index between zero and the value entered in RemoveEntries (days). This number is increased by one every day until (Remove Entries(days) number - 1) is reached. It then wraps back to zero. Day index may not be usedin the template if Remove Entries (days) is disabled.

• Directory delimiter - Inserts the standard directory delimiter for the operating systemit distributes files to. This way a sub-directory is created.

If enabled, the token is entered from the text field. When disabled, the token is selectedfrom the Special token list.

Text

2.1.2. Archive File Naming Algorithm

The full path of each filename to store in the archive is completely dynamic and is determined by threeparameters.

AAA/BBB/CCC

Where:

Represents one of the base directories specified in the Base Directory list in the ArchiveProfile Editor window. If several base directories exist, this value will change according tothe frequency selected from the Switch Policy list.

AAA

The system automatically appends a directory delimiter after this name.

This part is constructed from the Directory Template. If the template contains one or severalDirectory delimiters this part will enclose one or several directory levels itself.

BBB

For instance, if the template contains Month, Directory delimiter, Day this will yield newdirectories every day, named 03/01, 03/02 ... 03/31, 04/01, 04/02 ... 04/30 etc. Inthis example, files are stored in a directory structure containing all months, which in turncontains directories for all days (which in turn will contain all files from that day).

The system automatically appends a directory delimiter after this name.

This is the name the file will get. It is defined on each archiving node using configurationsfrom the Filename Template tab in the Archiving agent configuration window.

CCC

2.2. Configuration

2.2.1. Archiving Agent - Global

The Global archiving agent configuration window is displayed when the node in a workflow is double-clicked, or right-clicked, selecting Configuration....

7

Archiving 4.1

Figure 3. Archiving agent configuration window, Archiving tab.

The following options are available in the Archiving agent configuration:

Name of the user defined profile to be used when determining the attributes ofthe target files. Profiles are created in the Archive Profile Editor window.

Profile

Compression type of the target files. Determines if the agent will compress thefiles before storage or not.

Compression

• No Compression - the agent will not compress the files.

• Gzip - the agent will compress the files using gzip.

No extra extension will be appended to the target filenames, even ifcompression is selected. The configuration of the filenames is managedin the Filename Template tab, only.

Possibility to select one or more MIM resources to be used when naming a sub-directory in which the archived files will be stored.

Node DirectoryName

If more than one MIM resource is selected, the values making up the directoryname will automatically be separated with a dot.

If at least one Node Directory Name is selected in Directory Templatesin the Archive Profile Editor window, this directory field is used.

MIM values to be logged as meta data along with the file. This is used foridentification of the files. The meta data is viewed in the Archive Inspector.

Logged MIM Data

If enabled, files are produced although containing no data.Produce EmptyFiles

8

Archiving 4.1

The names of the created files are determined by the settings in the Filename Template tab.Please refer to Convergent Mediation Engine™ Core - User Guide, for further details.


2.2.2. Archiving Agent - Local

The Local archiving agent configuration window is displayed when the node in a workflow is double-clicked, or right-clicked, selecting Configuration....

Figure 4. Archiving agent configuration window, Archiving tab.

The following options are available in the Archiving agent configuration:

Name of the user defined profile to be used when determining the attributes ofthe target files. Profiles are created in the Archive Profile Editor window.

Profile


Compression

• No Compression - the agent will not compress the files.


No extra extension will be appended to the target filenames, even if com-pression is selected. The configuration of the filenames is managed in theFilename Template tab, only.

Possibility to select one or more MIM resources to be used when naming a sub-directory in which the archived files will be stored.

Node DirectoryName

If more than one MIM resource is selected, the values making up the directoryname will automatically be separated with a dot.

9

Archiving 4.1

If at least one Node Directory Name is selected in Directory Templatesin the Archive Profile Editor window, this directory field is used.

If enabled, files are produced although containing no data.Produce EmptyFiles



2.3.Transaction Behavior

2.3.1. Emits

None.

2.3.2. Retrieves

Creates a new file in the archive.Begin Batch

The currently processed file is closed and committed into the archive.End Batch

The currently processed file is removed from the archive.Cancel Batch

2.4. IntrospectionThe agents consumes bytearray types.


2.5.1. Publishes

A parameter containing the name of the target filename, as defined in FilenameTemplate.

Target Filename

Type string, defined as MIM type trailer.

2.5.2. Accesses

Various MIM resources are accessed depending on the MIM value selection in the Node DirectoryName and Logged MIM Data lists. The MIM values are read at End Batch.


10

Archiving 4.1

2.6. Node Message EventsReady with file: name

A message event is reported along with target filename each time a file is archived.

11

Archiving 4.1

3. Archive Inspector

Note this chapter is only valid for the global archiving agent.

To locate files in the archive, the Archive Inspector is used. The access group user is permitted tolaunch and purge these files. Once a file is located, it is treated as a regular UNIX file using regularUNIX commands to view or copy it.

It is not encouraged to alter or remove a file from the archive using UNIX commands. If alteringis desired, make a copy of the file. If removal is desired, use the Archive Inspector.

To display the Archive Inspector, select Archive Inspector... from the Inspection menu on thedesktop.

Figure 5. The Archive Inspector.

Initially, the window is empty and must be populated with data using the corresponding Search Archivedialog. See Section 3.1, “Searching the Archive” for details. Each row represents information about adata batch (file).

Displays the Search Archive dialog where search criteria may be definedto limit the entries in the list. See Section 3.1, “Searching the Archive”for details on setting the filter for this dialog.

Edit menu Search...

If Keep Files is disabled in the Archive Profile, all selected files are re-moved from the archive, including their corresponding references in the

Edit menu Delete...

database. If Keep Files is enabled, only the references are removed whilethe files are still kept on disk.

Shows the raw data content for the selected file.View menu ViewArchive...

If the query resulted in a match larger than the Archive page size, this listtoggles between the result sets.

Show Archives

Holds the index of the rows in the archive.ID

Name of the archiving workflow.Workflow

Name of the archiving node.Node

Full pathname to the file as stored on disk.Filename

Time when the entry was inserted in the archive.Timestamp

12

Archiving 4.1

When double-clicking the more... text, a dialog is displayed where valuesfor the adherent MIM resources are displayed. Adherent MIM resources

MIM Values

are defined as Logged MIM Data in the Archiving agent configurationwindow (see Figure 6, “MIM Resources Dialog”).

Figure 6. MIM Resources Dialog

Name of the profile used to archive the file.Profile

User interface items not listed are considered to be of general type. Refer to Convergent MediationEngine™ Core - User Guide, for further details.

3.1. Searching the ArchiveThe Search Archive dialog allows the user to limit the number of rows displayed in the Archive In-spector. Search conditions may be defined to display either all entries, a specific entry or entriesarchived between two dates.

The Search Archive dialog is displayed when Search... is selected in the Archive Inspection window.

Figure 7. The Search Archive dialog.

Select the profile that corresponds to the data of interest. If no profile is selected archiveentries for all profiles will be shown.

Profile

Option to narrow the search with respect to what workflow and node that archived thefile.

Workflow

Option to search for data archived during a certain period.Period

13

Archiving 4.1

4. Maintaining ArchivesThe Archive Cleaner task removes outdated archives that has has expired according to the Purgecriteria in the Archive Profile window. The Archive Cleaner task is accessed from the Task Managerin the Tools menu on the desktop. The Archive Cleaner is only configurable with respect to schedulingcriteria. The cleanup of outdated files is mastered by removing the reference in Convergent MediationEngine™ as well as the file itself from the archive directory.

14

Archiving 4.1

Core 4.1


Version 1.116

1





2

Core 4.1



2. System Overview ........................................................................................................ 72.1. Access Zone .................................................................................................... 72.2. Execution Zone ................................................................................................ 72.3. Control Zone ................................................................................................... 8

2.3.1. Code Server .......................................................................................... 82.3.2. Other Servers ...................................................................................... 10

2.4. Software Environment ..................................................................................... 102.4.1. Directory structure ............................................................................... 11

3. Workflow Fundamentals ............................................................................................. 133.1. Realtime Workflows ........................................................................................ 143.2. Batch Workflows ............................................................................................ 143.3. Configuration ................................................................................................ 153.4. Data Streams ................................................................................................. 153.5. Transactions .................................................................................................. 163.6. Execution and Monitoring ................................................................................ 163.7. Commands .................................................................................................... 173.8. Introspection .................................................................................................. 173.9. Meta Information Model .................................................................................. 173.10. Version Management ..................................................................................... 19

4. General Desktop Features ........................................................................................... 204.1. Login/Logout ................................................................................................. 20

4.1.1. Changing Desktop User ......................................................................... 214.1.2. Exiting the Convergent Mediation Engine™ Desktop .................................. 21

4.2. Change Password ........................................................................................... 224.3. Locking ........................................................................................................ 224.4. Reconnect ..................................................................................................... 224.5. Encryption .................................................................................................... 224.6. Folders ......................................................................................................... 23

4.6.1. Configuration Naming .......................................................................... 234.7. Meta Data ..................................................................................................... 234.8. Visualization and Edit-Ability ........................................................................... 234.9. Menu and Toolbar Items .................................................................................. 244.10. Scheduling .................................................................................................. 26

4.10.1. Schedule Criteria - Day Plans ............................................................... 274.10.2. Schedule Criteria - Event Triggers ......................................................... 29

4.11. UDR Browser .............................................................................................. 304.11.1. Selection Modes ................................................................................. 31

4.12. MIM Browser .............................................................................................. 314.13. Text Editor .................................................................................................. 324.14. List Editor ................................................................................................... 334.15. Filename Character Substitution ...................................................................... 344.16. Date and Time Format Codes .......................................................................... 34

5. Configuration Menu .................................................................................................. 365.1. Audit Profile .................................................................................................. 36

5.1.1. Adding and Editing a Table Mapping ....................................................... 365.1.2. An Example ........................................................................................ 37

5.2. Database Profile ............................................................................................. 415.3. Event Notification Editor ................................................................................. 41

5.3.1. Notifier Setup ...................................................................................... 435.3.2. Population Types ................................................................................. 465.3.3. Event Setup ........................................................................................ 485.3.4. Event Category .................................................................................... 55

3

Core 4.1

5.4. Workflow Editor ............................................................................................. 555.4.1. Edit Mode .......................................................................................... 565.4.2. Monitor Mode ..................................................................................... 665.4.3. Agent Services ..................................................................................... 71

6. Inspection Menu ....................................................................................................... 776.1. Configuration Browser .................................................................................... 77

6.1.1. Folder ................................................................................................ 786.1.2. Configurations ..................................................................................... 786.1.3. Configuration Tracer ............................................................................. 796.1.4. Configuration Permissions ..................................................................... 79

6.2. Pico Viewer ................................................................................................... 806.2.1. Tool-Tip Information ............................................................................ 81

6.3. System Log ................................................................................................... 816.3.1. Searching the System Log ...................................................................... 836.3.2. Printing the System Log ........................................................................ 84

6.4. System Statistics ............................................................................................ 856.4.1. Host Statistics ..................................................................................... 856.4.2. Workflow Statistics .............................................................................. 856.4.3. Pico Instance ....................................................................................... 866.4.4. Viewing the System Statistics ................................................................. 866.4.5. Searching System Statistics .................................................................... 88

6.5. Workflow Manager ......................................................................................... 896.5.1. Deactivation Issues ............................................................................... 916.5.2. Workflow Views .................................................................................. 92

7. Tools Menu .............................................................................................................. 967.1. Access Controller ........................................................................................... 96

7.1.1. Users Tab ........................................................................................... 967.1.2. Access Groups Tab ............................................................................... 97

7.2. Pico Manager ................................................................................................ 987.2.1. Adding Pico Hosts ................................................................................ 99

7.3. System Exporter ........................................................................................... 1007.4. System Importer ........................................................................................... 1027.5. Task Manager ............................................................................................... 103

7.5.1. Task Configurations ............................................................................ 1058. The Command Line Tool mzsh ................................................................................. 109

8.1. Startup ........................................................................................................ 1098.1.1. Startup Options .................................................................................. 109

8.2. Command Availability ................................................................................... 1108.3. Commands Always Available .......................................................................... 1108.4. Commands Available Only When Control Zone is Running ................................... 112

9. Execution Context Web Interface ................................................................................ 1159.1. Startup ........................................................................................................ 1159.2. Functionality ................................................................................................ 115

9.2.1. Login ............................................................................................... 1159.2.2. Logout ............................................................................................. 1159.2.3. Execution Context Monitor .................................................................. 115

10. System Administration ............................................................................................ 11710.1. File System Permissions ............................................................................... 11710.2. Starting the Convergent Mediation Engine™ System ......................................... 11710.3. Stopping the Convergent Mediation Engine™ System ........................................ 11710.4. Starting Execution Contexts .......................................................................... 11810.5. Stopping Execution Contexts ......................................................................... 11810.6. Adding an Execution Context to the System ..................................................... 11810.7. Adding a Desktop to the System ..................................................................... 11910.8. Clearing the Pico Cache ............................................................................... 11910.9. Preventive Maintenance Procedures ................................................................ 119

10.9.1. Online Backup ................................................................................. 11910.9.2. Restore ........................................................................................... 121

4

Core 4.1

10.9.3. Cloudscape Backup and Restore .......................................................... 12211. Network Security ................................................................................................... 124

11.1. RCP Encryption, Using TLS ......................................................................... 12411.1.1. TLS Configuration Properties .............................................................. 12411.1.2. TLS Standard Setup .......................................................................... 12511.1.3. TLS Setup - Client Authentication Enabled ............................................ 125

11.2. HTTP Encryption, Using TLS or SSL ............................................................. 12611.2.1. Standard Setup ................................................................................. 12611.2.2. Configuration Properties .................................................................... 127

12. High Availability ................................................................................................... 12912.1. The HA Server ............................................................................................ 13012.2. Retrieval - Platform, Node 1 ...................................................................... 13112.3. Retrieval - EC2, Node 2 ................................................................................ 13212.4. Configuration and Installation ........................................................................ 133

12.4.1. Configuration of HA Cluster Daemon ................................................... 13312.4.2. Pico Clients ..................................................................................... 13312.4.3. The HA Server Interface .................................................................... 135

13. Terminology ......................................................................................................... 13613.1. Abbreviations ............................................................................................. 13613.2. Terms ........................................................................................................ 136

5

Core 4.1

1. IntroductionConvergent Mediation Engine™ is a data mediation foundation, based on a distributed real-time archi-tecture on which any type of mediation functionality may be deployed.

The system is based on workflow technology, where mediation processes may be modeled in agraphical user interface. Workflow activities are performed by software agents, which are linked intoflows providing the required mediation functionality.

1.1. PrerequisitesThe user has to be familiar with:

• Operating systems and processes.

• Databases.

• Distributed systems.

1.2. Abbreviations and TermsAbbreviations and terms, used in the Convergent Mediation Engine™ user documents, are describedin the Section 13, “Terminology”

6

Core 4.1

2. System OverviewThe Convergent Mediation Engine™ system architecture is designed to be completely distributed andscalable, capable of executing on various operating systems and hardware platforms.

All code is written in pure Java using a proprietary protocol to communicate between the servers. Thefollowing illustration describes the different zones that the system may be divided into, and what isavailable in each zone.

GUI Client Web Interface Command Line

ExecutionContext

ExecutionContext

ExecutionContext

CodeServer

EventServer

WorkflowServer

StatisticsServer

NotificationServer

ManagementServer

Access Zone

Control Zone

Execution Zone

DB

Figure 1. Convergent Mediation Engine™ architecture.

2.1. Access ZoneThe Access Zone contains Convergent Mediation Engine™ desktop user interfaces, web interfacesand command line interfaces (the mzsh) used to configure and monitor the system.

Access Zones are normally installed on PCs running Windows or Linux but may co-exist in the ControlZone or Execution Zone if desired.

2.2. Execution ZoneThe Execution Zone consists of one or many hosts, each running an Execution Context (EC), responsiblefor executing and supervising workflows on demand of the workflow engine. Each Execution Contextwill at startup register itself as available in the system. There are two kinds of Execution Contexts,one that is able to execute any type of workflow and one that is restricted in terms of what logic it canexecute but can keep workflows active even if Control Zone is absent.

Execution Zones may be installed on all from high-end servers to PCs running, for instance, Linuxand may co-exist with the other servers in the Control Zone, if desired.

7

Core 4.1

Figure 2. Three execution contexts.

2.3. Control ZoneThe Control Zone contains the Convergent Mediation Engine™ servers (platform) and the database.A central unit in the Control Zone is the Code Server, which is vital for all pico-started applications.Refer to Section 2.3.1.1, “Pico-Start” for further details. The Code Server is not a standalone entity.It runs on the same instance as all other servers in Control Zone.

The Control Zone is usually installed on reliable, high-end machines running Unix like Solaris, butoperates just as well on, for instance, Linux.

This section holds the core sub-systems of the Control Zone. Other sub-systems may be added on topof Core. Please refer to the user guide for the specific component for further information.

2.3.1. Code Server

The Code Server is the first server that is started on the Control Zone. It is responsible for maintainingthe code repository, located in the Convergent Mediation Engine™ database. On request from Pico-started clients, it will hand over the latest version of all code needed for execution.

The Code Server repository is populated with code via the installation or the command line interface.

2.3.1.1. Pico-Start

Pico-start is a utility that automatically downloads new code from a central repository maintained bythe Code Server.

A client that is pico-start compliant needs only the code for the pico-start utility. When initializing, itwill establish contact with the Code Server and dynamically request the latest version of all codeneeded. To gain performance, the client will store all code that it downloads on the local file system(in the pico cache). Next time it initializes, it will compare the code version in the local cache withwhat is stored in the Code Server. If nothing is new, it uses the code in the cache.

Not all code is transferred at once. Pico-start will request the server each time the pico-started clientdetects a class that it does not have. For instance, if a new agent package has been inserted into theConvergent Mediation Engine™ system, the next time a Workflow Editor window is opened on thedesktop, this new code is loaded.

8

Core 4.1

2.3.1.2. Software Packages

Convergent Mediation Engine™ groups all software code into packages that are inserted into the systemvia the Code Server by the installation scripts, or mzsh.

Each package is keyed by a name. The name and a version name are encapsulated in the package filetogether with code, images, property files etc. When a package is inserted, the Code Server uses thepackage name to determine if the package is new or if one is to be updated. If the package alreadyexists, the existing code is fully replaced with the code from the new package.

The version name is free to use by the vendor of the package, to identify the version of the package.Preferably, the version should relate the official version of the code. The version name is a plain string.

The installed packages and their versions may be viewed in the About window displayed when About...is selected from the Help menu.

Figure 3. About Convergent Mediation Engine™ window.

2.3.1.3. Code Environment

All Java code in the Code Server is stored in one of three levels.

A. Platform code.

Java code related to the core platform including some user interface plug-ins (DevelopmentToolkit).

Updates to code in the platform level, is only propagated to pico clients when they are re-started.

B. Execution code.

Agent related code, including Analysis agent plug-ins, Tasks, and Events. Updates to code in theexecution level and is propagated to clients on demand.

C. Generation code.

Code that is automatically generated from user configuration. This includes implementation codefor Ultra formats and APL scripts.

9

Core 4.1

2.3.2. Other Servers

This section contains other Core servers running in the Control Zone.

Workflows and user defined tasks may be automatically activated in twoways. Either at a specified time (Day Plans) or as a result of another activity

Activation Manager

within the system (Event Triggers). The Activation Manager handles allactivations.

Server that receives audit log information from running workflows and feedsthis to corresponding audit tables for different databases.

Audit Log

Responsible for compiling format and APL definitions into correspondingJava code. The resulting Java software is committed to the Code Server.

Compile Server

Logging within the Convergent Mediation Engine™ system, both of inform-ative type and error type is referred to as Events. The Event Server receivesall events, and hands them out upon requests from various applications.

Event Server

The Execution Context Trader holds information on available ExecutionContexts, and is pulled for information each time a workflow is activated.

Execution ContextTrader

The Management Utilities handles miscellaneous database related operations,such as management of persistence of the configuration framework, andcaching of database meta data.

Management Utilities

Events in the Convergent Mediation Engine™ system may be forwarded toany notifier. The mapping is made in the Event Notification Editor window,and is stored and handled by the Notification Server.

Notification Server

Configurations may be related to each other. For instance, an agent may de-pend on a format or a database profile. The Reference Manager keeps trackof such relations.

Reference Manager

Collects and manages statistics from the running Convergent MediationEngine™ system. This information is, among other things, used for balancingthe load of workflows. It may be viewed in the System Statistics Inspector.

Statistics Manager

Contains services for storing, browsing, and deleting entries in the System.It may be viewed in the System Log Inspector.

System Log

Any code or SQL statement may be scheduled to be activated at regular in-tervals using the Task Manager.

Task Manager

The User Server handles user authentication and permissions.User Server

This server is among other things taking care of a workflow's transactionstate.

Workflow Persistence

The Workflow Server loads all data belonging to a workflow, both referentialand configurational, upon workflow activation. It is also responsible formaking sure a workflow is up and running.

Workflow Server

2.4. Software EnvironmentThis section describes directories, files and environment variables installed by Convergent MediationEngine™ . Depending on the type of installation (Platform, Desktop, EC or ECSA) there may be lessdirectories and files than described here.

Paths are displayed as Unix paths but are the same on Windows with the exception of a "\" instead ofa "/".

10

Core 4.1

2.4.1. Directory structure

After a default installation of the platform, the following directory structure is created. By defaultMZ_HOME refers to:

Unix: /opt/mz

Windows: C:\Program Files\Nokia\Convergent Mediation Engine 4.1

Figure 4. Convergent Mediation Engine™ directory structure.

2.4.1.1. $MZ_HOME/backup

Contains backup files from the System Backup task and from the System Import window.

2.4.1.2. $MZ_HOME/bin

Contains the command line tool, mzsh.

2.4.1.2.1.The Command Line Tool - mzsh

Via the Command Line Tool it is possible to interact with Convergent Mediation Engine™ and performsystem related tasks such as shutdown, startup, import and export of configuration. Some desktop relatedtasks, such as starting and stopping workflows, are possible as well.

The command line tool is used to start the parts of the system such as the Platform or an ExecutionContext. Those entities are running in their own Java Virtual Machine and are referred to as a startable.Every startable has its own definition in a configuration file located in the $MZ_HOME/etc directory.

Please refer to Section 8, “The Command Line Tool mzsh” for further information.

2.4.1.3. $MZ_HOME/documentation

Contains the documentation in PDF format for all installed packages.

2.4.1.4. $MZ_HOME/etc

Contains various property files utilized by the servers and the user interface during startup.

Properties specific to mzsh.commandline.xml

Contains properties, common for all startables. The properties in thisconfiguration file, may be overridden in the configuration file for agiven startable.

common.xml

Properties for the Platform, which are made available to all otherstartables in the system.

config.xml

11

Core 4.1

Properties for the platform.platform.xml

Properties specific for Execution Contexts.executioncontext.xml

Properties specific to the user interface.desktop.xml

The license file for the Convergent Mediation Engine™ installation.mz.license

MIB definition file used by the SNMP trap utility.mz_trap_mib.txt

2.4.1.5. $MZ_HOME/ha

Contains a compressed file which in turn contains scripts used for High Availability. Please refer toSection 12, “High Availability” for further information.

2.4.1.6. $MZ_HOME/lib

Contains the JAR-files necessary for the non pico-started servers. Since all servers, the desktop, andthe ECs are pico-started, the executable code is never fetched from here during run-time. Instead it isfetched from the pico cache. The exception from this is the Code Server and the pico-start utility,which obtain their code from this directory.

2.4.1.7. $MZ_HOME/log

Contains various logs. If operation system on the installation machine provides so-called job control,this directory also contains process identification files (so called pid files *.pid) utilized by mzshin order to identify all processes related to the Convergent Mediation Engine™ system execution.

Logs from the pico client, that is, from the execution contexts, platform serversor desktop.

<pico name>.log

Process identification number.<pico name>.pid

2.4.1.8. $MZ_HOME/pico-cache

Pico cache used by all servers and clients running on this host.

2.4.1.9. $MZ_HOME/tmp

A directory used for temporary storage by various subsystems.

12

Core 4.1

3. Workflow FundamentalsWorkflows are the cornerstones of the Convergent Mediation Engine™ system. A workflow is identifiedas a set of nodes connected to each other to represent a data flow. Each node operates on the data flowin different ways, depending on what type of agent it represents and how it is configured.

A workflow by definition retrieves some kind of data from some kind of source. The data is forwardedto other agents, that may operate or act upon the data or distribute it to other systems. Having identifiedthis, the workflow can be said to contain three types of agents:

• Collection agents

• Processing agents

• Forwarding agents

A Collection agent is responsible for gathering data from external systems or devices and passing iton into the workflow. An example of a simple Collection agent could be one that reads a file from diskand sends the file contents into the workflow.

A Processing agent expects to be fed with data and to deliver data on one or several outgoing routes.A Processing agent could be as simple as a counter that counts the throughput. It could also be morecomplex, for instance, depending on the processed result it delivers data on different routes. Amongstthe Processing agents,there are agents responsible for translating an incoming byte stream into a UDRobject or the opposite. These are called Decoding and Encoding agents.

A Forwarding agent is responsible for distributing the data from the workflow to other systems ordevices. An example of a Forwarding agent could be one that creates a file from a data stream andtransfers it to another system using FTP.

Figure 5. The different agent types.

Workflows may be of three types: Normal, Template, and Instances. These in their turn are classifiedas Batch or Realtime workflows. All types are set when a new workflow is created from any of thefollowing alternatives in the File menu of the Workflow Editor window:

Sets the window in Batch mode; only valid batch agents are visible. Usingthis option will create a normal batch workflow.

New Batch Workflow

Sets the window in Batch mode; only valid batch agents are visible. Usingthis option will create a batch workflow template.

New Batch Template

Sets the window in Realtime mode; only valid real-time agents are visible.Using this option will create a normal Realtime workflow.

New Realtime Workflow

Sets the window in Realtime mode; only valid real-time agents are visible.Using this option will create a Realtime workflow template.

New Realtime Template

Creates a workflow of type Instance. A new dialog is opened from whichan existing template may be selected.

New Template Instance

13

Core 4.1

3.1. Realtime WorkflowsA Realtime workflow collects and processes unsafe data instantly as it arrives, without taking anytransaction boundaries into consideration. Instead, the processing entity consists of a UDR, makingno use of the batch transaction logic. This means that all processing is done in memory, and data maybe lost in case the workflow is deactivated.

The order of the UDRs processed is not guaranteed, due to the possibility of having multiple threadsexecuting in parallel. For Batch workflows, the order of UDRs leaving a workflow will be the sameas entered, no matter the level of multi-threading.

Realtime workflows consist of real-time agents; for instance Analysis and Aggregation Processingagents or Radius and Netflow Collection agents. Once activated, a Realtime workflow will stay activeawaiting arrival of new data until aborted or manually deactivated.

The output from a Realtime workflow is forwarded with an Inter Workflow agent or any customizedreal-time agent, designed with the Convergent Mediation Engine™ Development Toolkit. Refer tothe Convergent Mediation Engine™ Development Toolkit - User Guide, for further information.

Inter Workflow forwarded data is collected in a Batch workflow, containing an Inter Workflow Col-lection agent. The Inter Workflow Forwarding node will automatically introduce batch boundaries(according to settings in its configuration window) to the real-time data forwarded.

Figure 6. The Inter Workflow agents make communication between Realtime and Batch workflowspossible.

3.2. Batch WorkflowsA Batch workflow operates on data with some sort of batch boundary, for instance, standard Call DetailRecord (CDR) files. This means that each batch is identified as a separate entity, making it possibleto refer to it. It is, for instance, possible to send complete batches to the Error Correction system (ECS).

UDRs within a collected batch will still be forwarded within the same batch in the original order,provided that nothing else is stated (large batches may be split into smaller entities, UDRs may be

14

Core 4.1

routed to be grouped into new batches, etc). If something goes wrong during processing of a batch, alloperations on the current batch are rolled back.

3.3. ConfigurationAlthough workflows may be designed, configured and saved step-by-step, they are not valid to be ac-tivated until fully configured. A fully configured workflow contains three types of configuration data:

• Workflow data: General information related to the workflow as a whole, for instance, error handling.

• Workflow structure data: Contains the nodes and their connections (routes) to other nodes. A routeindicates the flow of data depending on the name of the route and the internal behavior of its sourceagent.

• Node specific data: Each agent has a different behavior. Thus, each node representing an agentrequires different configuration data in order to operate.

For Realtime workflows, multiple Collection nodes may be used within the same workflow (asopposed to Batch workflows). Also, there are collectors allowing circular configurations, forinstance, the Radius Collection agent, which can both send and receive UDRs from anotherConvergent Mediation Engine™ agent.

Once activated, a Realtime workflow stays active until manually deactivated.

3.4. Data StreamsInside a workflow, data propagates between agents as streams, that is, as a flow of bytes or UDRs.Most Collection agents have the responsibility of collecting a data stream from the source and insertingit into the workflow. This could be opening a file and reading it byte-by-byte. Forwarding agents dothe opposite. Given a data stream it creates, for instance, a file.

Forwarding nodes often need a set of boundaries in order to, for instance, close a file or commit adatabase transaction. To satisfy this, Convergent Mediation Engine™ introduces Begin Batch and EndBatch messages to propagate in the stream on demand by some of the Collection agents or the Aggreg-ation and Analysis agents. Different agents act differently on these messages. The data transferredbetween a Begin Batch and an End Batch message is referred to as a data batch.

For file based collection the natural place to insert a Begin Batch is at the start of the file and the EndBatch after the file has been read and inserted into the workflow. At any time between a Begin andEnd Batch request, any agent in the workflow may emit a Cancel Batch request that will not be partof the stream, but is broadcasted to all agents in the workflow. Cancel Batch messages indicate datacorruptness and, depending on the workflow configuration, it may cause the current data batch to becompletely removed from the workflow and instead be sent to ECS. The Convergent Mediation Engine™system will never recollect such data unless configured to do so.

Realtime workflows have no batch transaction logic. The processing entity is a UDR.

Erroneous data, both in form of individual UDRs as well as complete batches, can be sent to ECS. Ifthe workflow aborts as a result of configuration in the Preferences dialog of the workflow, it is alsopossible to abort a workflow immediately without sending the erroneous data to ECS. Refer to theConvergent Mediation Engine™ Error Correction System - User Guide, for further information. A

15

Core 4.1

Realtime workflow cannot send anything to ECS. It aborts immediately upon detection of erroneousdata.

3.5.TransactionsDue to the fact that a workflow operates on a data stream, Convergent Mediation Engine™ suppliesa transaction model where persistent synchronization is made on the workflow data and agent specificcounters etc. In theory this synchronization could be performed continuously on a byte level, but inpractice this would drastically decrease the performance of the system.

The Convergent Mediation Engine™ transaction model is based on the premises that Collection agentsare free to initiate a transaction to the Transaction server. At this point in time, the complete workflowis frozen and the Transaction server saves the state of the workflow data that is queued for each agent.In practice, agents indirectly emit a transaction when an End Batch is propagated. When all data issecured, the workflow continues the execution.

3.6. Execution and MonitoringA workflow is created and started on an Execution Context according to a configured distributioncriteria. Such criteria can be to distribute the workflow based on machine load or to explicitly statethe Execution Contexts where the workflow is allowed to execute.

There are two kinds of Execution Contexts. One that can execute any type of workflow and one thatcan run stand-alone. The stand-alone version only works with Realtime workflows and when suchworkflow is configured to not depend on external entities to run. The purpose with a stand-aloneworkflow is to allow it to run without relying on the platform. For instance in environment where thenetwork is unreliable or the workflow must guarantee uptime even if the platform for some reason hasterminated. If the platform is down a stand-alone Execution Context keeps track of all events that oc-curred and once the platform is up and running again, these events are propagated to the platform. Theonly events that are not remembered are Debug Event and internal events for statistics. An ExecutionContext features a web interface that shows what workflows that it runs, this web interface shouldonly be used in case that the platform process for some reason is unavailable or the user is unable todeactivate a workflow due to that the communication between the platform and the Execution Contextis down.

All running workflows are monitored by the workflow server in the platform. When the workflowserver fails to establish a connection to a running workflow the workflow will enter a state called un-reachable. When the workflow server successfully established a connection to the workflow it willreturn to the active state again. Convergent Mediation Engine™ will always set the workflow state tounreachable if unable to connect to the workflow even in the case when the Execution Context processhas terminated improperly. It is the user's responsibility to tell the workflow server that an unreachableworkflow is actually not running anymore by manually stopping it or in case the platform has noconnection the Execution Context where the workflow is active the Web Interface for the EC shouldbe used.

When starting the platform, the workflow server will automatically activate workflows. Workflowsthat were active or unreachable will be unreachable and workflows that were scheduled will bescheduled.

A Batch workflow that detects an error will abort the workflow and the detected error will be shownas part of the abort reason and also inserted into the System Log. A Realtime workflow handles errorsby only sending them to the System Log. The only time a Realtime workflow will abort is when theinternal processing limit has been reached or when an internal error has occurred. It is therefore import-ant to pay attention to the System Log or subscribe to workflow error events to fully understand thestate of a Realtime workflow.

16

Core 4.1

3.7. CommandsA workflow agent may support execution of commands while it is executing. Such commands areagent specific and can be invoked from either the command-line tool mzsh or the Workflow Monitor.A command will in most cases affect the data in the workflow in some way, for instance by flushingan internal cache of data to downstream agents.

3.8. IntrospectionThe type of data an agent expects and delivers is in Convergent Mediation Engine™ referred to asIntrospection. A Processing agent receives one or several types, and delivers one or several types.

Within a workflow, all agents operate on a specific type of data. They all send a data stream betweenthemselves, however, most agents need to know the structure of the stream in order to operate properly.For an agent that simply counts the number of bytes, it would expect to receive an array of bytes.However, if the agent were to count the number of lines it would probably expect the stream to containline feed characters every now and then. In the same way, an agent that operates on fields of a UDRexpects the stream to contain UDR data, structured in a predefined way.

The introspection type of an agent can be either static or dynamic based on how it is designed or con-figured. If the byte and line counters were introduced as two different agents, then their introspectiontypes would be static. However, if they were introduced as a single agent where the user could configureit to count bytes or lines, then its introspection type would vary depending on its current configuration.

A Collection agent must have one or several outgoing introspection types and Forwarding agents musthave one or several incoming introspection types. The Workflow Editor validates that the introspectiontypes between two connected nodes are compatible.

Realtime Collection agents may also be receivers within the workflow. This is called bidirec-tional capability and is used when the collector must respond back to the network element.

3.9. Meta Information ModelSome nodes in a workflow need information from the workflow or other nodes in order to operate.For instance, an agent that produces a file might need the source filename and the number of processedUDRs to be used in the outgoing filename. In order to satisfy these requirements, Convergent MediationEngine™ uses a model called Meta Information Model (MIM).

MIM is based on the fact that individual nodes may supply information during run-time that othernodes may need to use. The MIM information is used in various parts of Convergent Mediation Engine™, for instance, when selecting what variables to use in a filename or when selecting what data toidentify UDRs delivered to ECS.

17

Core 4.1

Figure 7. A MIM tree example.

MIM resources for each agent will have their values assigned at any time, depending on type. As anexample, the Disk Collection agent publishes the MIM resource Source Filename which is set at BeginBatch. The agent will put the name of the collected file in this resource before it starts collecting thefile.

There are in total four types of MIMs:

Batch MIMs are dynamic values, populated during batch processing. An example of such avalue is Outbound UDRs.

Batch

A global MIM value can be accessed whenever during the execution phase of a workflow.For instance static values such as Node Name.

Global

Header MIM values are populated when a batch is received for processing (a node emitsBegin Batch). For example Source Filename (published by the Disk collection agent).

Header

Trailer MIM values are populated after a batch is processed (a node emits End Batch). Anexample of such a value is Target Filename (published by the Disk forwarding agent)

Trailer

By default all nodes may publish the following MIM resources depending on their introspection types.

The name of the node as defined in the Workflow Editor. All nodes publishthis resource. The value is set when the workflow starts executing. Definedas MIM type global.

Node Name

The number of UDRs (or Bytes) routes on the link by the node. The valueis updated continuously during batch processing. This MIM is not validfor Forwarding nodes. Defined as MIM type batch.

<route name> UDRs (orBytes)

Number of incoming UDRs (or Bytes) since last Begin Batch. The valueis updated continuously during batch processing. This MIM is not validfor Collection nodes. Defined as MIM type batch.

Inbound UDRs (or Bytes)

18

Core 4.1

Number of outgoing UDRs (or Bytes) since last Begin Batch. The valueis updated continuously during batch processing. This MIM is not validfor Forwarding nodes. Defined as MIM type batch.

Outbound UDRs (orBytes)

Note that some MIM values will not be available until the node to which they belong has been con-figured.

There are also workflow specific MIM resources representing information of a running workflow.These are:

A boolean MIM stating if the current batch has been cancelled. Defined as MIMtype header.

Batch Cancelled

A unique sequence number that will be associated with each batch processed bythe workflow. This value is increased by 1 up to 263, and is saved between work-flow invocations. Defined as MIM type header.

Batch Count

The time it took to process a batch. This value is updated during processing.Defined as MIM type batch.

Batch Duration

End time for the processing of a batch. Defined as MIM type trailer.Batch End Time

Start time for the processing of a batch. Defined as MIM type header.Batch Start Time

Name of the Execution Context hosting the current workflow. Defined as MIMtype global.

Execution Context

Date and time when the workflow was started. The value is set when the workflowis activated. Defined as MIM type global.

Start Time

The value used for calculation of throughput. Convergent Mediation Engine™provides built-in capability of finding the data source used for throughput calcu-

Throughput

lation. A user defined data source can be defined in case Convergent MediationEngine™ is choosing the wrong data source or the user wants to measurethroughput on an alternate route in the workflow. Defined as MIM type global.

Each batch closed by all Convergent Mediation Engine™ workflows will receivea unique transaction ID. Cancelled batches as well. Defined as MIM type header.

Transaction ID

Name of the current workflow. The value is set when the workflow is activated.Defined as MIM type global.

Workflow Name

Realtime workflows can only have MIMs of type Global.

3.10. Version ManagementThe user may save new versions of a workflow at the same time that they are running or scheduled.In that case the workflow will be reloaded to the most recent version upon activation or when it finishedthe current execution. In case the most recent workflow version is invalid, the workflow will abortwhen trying to load that version.

Changes to the schedule criteria will be reflected as well. If the schedule criteria has been removedfor the most recent version it will be removed for the running workflow as well. This means that theworkflow will not be scheduled once it finishes the last run.

A Realtime workflow does not usually fall back to scheduled mode and therefore will not automaticallypick up changes made to the workflow. Some Realtime agents can be modified while they are running.Such agents can be modified in the Workflow Monitor and when saved will result in a new versionof the workflow.

19

Core 4.1

4. General Desktop FeaturesThe Convergent Mediation Engine™ desktop is a graphical user interface, enclosing all interactionswith the Convergent Mediation Engine™ system in an easy to use environment. This section describescomponents and other features that are used at various places within the Convergent Mediation Engine™desktop.

The desktop consists of several windows used to configure and monitor the system, where the centralunit is the Workflow Editor window. In the Workflow Editor workflows are created, executed andmonitored. The Workflow Manager window allows the user to control several workflows simultan-eously.

Windows within the desktop are divided in three categories:

• Configuration - Windows used to create and maintain all configuration in the system, for instancethe Workflow Editor. Please refer to Section 5, “Configuration Menu” for further information.

• Inspection - When workflows are executed, the agents may generate various kinds of data, suchas logging errors in the System Log, or sending erroneous data to the Error Correction System(ECS). The Inspection menu features windows that allow the user to view such information. Pleaserefer to Section 6, “Inspection Menu” for further information.

• Tools - Holds different tools such as the System Exporter and Access Controller windows. Pleaserefer to Section 7, “Tools Menu” for further information.

4.1. Login/LogoutWhen the Convergent Mediation Engine™ desktop application is started a login window is presented.In order to gain access to the desktop, the user has to be authenticated by supplying a User Name anda Password.

Figure 8. Login window.

Once the User Name and Password have been successfully entered, the desktop will be available.Depending on the logged in user, access is granted to different parts of the system. Note, all parts ofthe desktop are visible for all users, no matter the privileges set. Configuration and operation optionsmay however be disabled.

The name of the logged on user and the name of the Convergent Mediation Engine™ system that thedesktop has connected to are available in the status area at the bottom of the desktop.

The same Convergent Mediation Engine™ desktop application may be configured to run against sev-eral Convergent Mediation Engine™ Systems. In that case, a list will appear from which a specificinstance is selected.

To be able to do this the desktop.xml file must be updated to describe what systems that areavailable. First open the desktop.xml file in a text editor. Duplicate the configuration element, onebelow the other. There will now be two configuration elements.

In the second element, edit the name of the configuration (that is, the config name parameter withinthe configuration element) to match the new system. Add two new properties to define where the dif-ferent systems are installed to the contents of the config elements:

20

Core 4.1

<property name="pico.rcp.server.host" value="first/second-system-hostname"/><property name="pico.rcp.server.port" value="first/second-system-port"/>

Example 1.

<configlist prompt="true"> <config name="Desktop1"> . . . <property name="pico.rcp.codeserver.host" value="10.0.1.33"/> <property name="pico.rcp.codeserver.port" value="4111"/> </config> <config name="Desktop2"> . . . <property name="pico.rcp.codeserver.host" value="10.0.1.33"/> <property name="pico.rcp.codeserver.port" value="4192"/> </config></configlist>

The properties that may be added are defined in the common.xml file.

Figure 9. Several Convergent Mediation Engine™ instances may be accessed.

4.1.1. Changing Desktop User

If the logged in user wishes to change its identity to another user, the workflow desktop does not haveto be exited. Instead Logout... is selected from the File menu. This will display the log-in windowagain.

4.1.2. Exiting the Convergent Mediation Engine™ Desktop

By selecting Exit... from the File menu the user is logged out and the desktop is closed, removing itfrom the screen.

21

Core 4.1

4.2. Change PasswordAny user can change its own password. This is done from the File menu on the desktop, selectingChange Password....

4.3. LockingAll configurations are locked when being edited by a user. Another user can still open the configurationbut is only allowed to view it. When opening a locked configuration a dialog is shown stating the nameof the user owning the lock and when the lock was created. Read-only configuration will may only besaved using a new name.

Figure 10. A locked configuration.

Locks are not persistent. If the system is restarted, all locks will be forgotten.

4.4. ReconnectIn case the desktop runs on an unreliable network or the platform process for some reason is not re-sponding, Convergent Mediation Engine™ will detect that the platform is absent. When this happensthe desktop will block user input until a connection has been established to the platform again. Oncethe desktop is properly connected to the platform the user should close all opened windows. In case aconfiguration window contains unsaved changes these changes are possible to save before closing thewindow.

Figure 11. A disconnected desktop.

4.5. EncryptionA configuration in Convergent Mediation Engine™ is persisted using XML format and therefore moreor less available in a readable form to any user. Some configurations may be sensitive and possiblycontain descriptions that are proprietary and must be protected. To protect such configurations Conver-gent Mediation Engine™ features the ability to encrypt configurations using a pass phrase. A config-uration will thereby only be readable given that the pass phrase is known by the user. In case the passphrase is lost the configuration should be considered lost as well.

There are configurations that generates information to the system, for instance the Ultra Format thatrenders UDRs. A user can have access to the UDRs without knowing the pass phrase for the configur-ation source by setting the execute permission for the user group. A user can also import a format oranalysis package given that the execute permission is set those configurations.

22

Core 4.1

Encrypted configurations that are exported will be encrypted in the exported data as well. When im-ported on another system the configurations will be imported as encrypted and the user must still knowthe original pass phrase to be able to see the configuration content.

4.6. FoldersFolders allows the user to categorize configurations and ease maintenance and operation. Folders couldfor instance be created based on traffic type, such as all collection workflows or decoding for a specificnetwork element, as well as being based on geographic location etc.

Convergent Mediation Engine™ includes a system folder named Default. This folder cannot be renamednor removed.

4.6.1. Configuration Naming

Convergent Mediation Engine™ is not very restrictive on naming of configurations. A configurationname is considered a duplicate if contains the same configuration type and exists in the same folder.It is not recommended to create several configurations of different types with the same name in thesame folder, as it could confuse the user and make the configuration less easy to understand.

Convergent Mediation Engine™ has an internal key for every configuration. It uses this key toidentify the configuration. Renaming a configuration will not change this key. The key is built usingthe system name and the date when the configuration was created. In some places the configurationkey could be exposed to the user but this will not occur under normal circumstances, except for in theconfiguration tracer.

Only a-z, A-Z, 0-9, "-" and "_" are valid characters for naming of a configuration.

4.7. Meta DataThe desktop stores some information related to what selections a user has made in different windows.When the window is shown the next time this selection will be displayed according to the last selection.The following selections are remembered:

• Desktop size - The size of the desktop window.

• Recent configurations - Added to the File menu of configuration windows. Contains the last fiveopened configurations for the user.

• Selected folder - The last selected folder in the Configuration Browser.

• Workflow options - Display and layout configuration for the Workflow Editor.

• Font size - The size of the font used in the general text editor.

4.8. Visualization and Edit-AbilityWhen a configuration is opened or saved, the visualization and edit-ability is updated according to thefollowing:

• The opened version of the configuration is stated in the title bar. Every time the configuration issaved, this number will be updated since a new version has been created.

23

Core 4.1

Figure 12. Typical title bar for a configuration.

• A configuration that is not fully configured, will have a red cross drawn on top of the applicationicon. This means that the configuration is considered invalid.

Figure 13. Title bar for an invalid configuration.

• If a configuration is already opened for editing by another user, the configuration will be openedin read-only mode.

Figure 14. Title bar for a read-only configuration.

4.9. Menu and Toolbar ItemsThe following list holds explanations of general menu items.

Drops the current configuration and initializes a new.File menu New

Displays a dialog where previously saved configurations may be selectedand loaded into the desktop.

Figure 15. Configuration Selection window.

File menu Open...

Saves the current configuration in the database. The user is given anopportunity to enter a comment describing the configuration change.

File menu Save

Validation of configuration integrity is performed every time a config-uration is saved. If the configuration is not fully configured it is savedas invalid. A configuration may be invalid if it has not been fully con-figured, or if an external dependency is invalid or missing. Invalidconfigurations are marked with a red cross drawn on top of their applic-ation icon.

24

Core 4.1

Saves the configuration with a new name. A dialog is displayed wherea unique name must be entered and a folder must be selected. Refer toSave for further information.

File menu Save As...

Only a-z, A-Z, 0-9, "-" and "_" are valid characters for namingof a configuration.

Displays a dialog where the user may print the configuration.File menu Print...

Closes the window. If the current configuration is not saved, the userwill be asked to do so.

File menu Close

Displays a history of the five last configurations, opened in the config-uration window.

File menu <ConfigurationName>

Displays a dialog where the owner of the configuration may be set aswell as what groups to have Read, Write and Execute permissions for

Edit menu Set Permissions...

the selected configuration. Please refer to Section 7.1, “Access Control-ler” for further information.

Figure 16. Setting permissions for a configuration.

Deletes selected items.Edit menu Delete...

Cuts selections to the clipboard buffer.Edit menu Cut

Copies selections to the clipboard buffer.Edit menu Copy

Pastes the clipboard contents.Edit menu Paste

Displays a dialog where one or several criteria to be search for is entered.Edit menu Find

Repeats the search for the last entered string.Edit menu Find Again

Each time a configuration is saved, a new version is created. Manyversions of a configuration may exist but only the last version may be

View menu Show History

modified and executed. The old versions are kept for log and rollbackreasons. The historic configurations may be examined with the History

25

Core 4.1

Configurations panel which appears at the bottom of the configurationwindow, when selected from the View menu on the desktop.

The panel holds a list of all versions. Arrow buttons are used to stepback and forward between the different versions. Rollback to an oldversion of a configuration is handled by opening and saving the oldversion. A comment is automatically added, stating that the currentversion was created from a historic one.

Figure 17. History for a configuration.

All configurations have references. For each configuration, a ReferenceViewer shows all references to it and what configurations it utilizes.

View menu Show Refer-ences...

This overview is useful especially in case of wanting to remove a con-figuration, since it is not possible to remove a configuration unless allits references have been removed.

Figure 18. Reference Viewer for a configuration.

Updates the window according to the current settings/criteria.View menu Refresh

Shows the help topics for the configuration window.Help menu Help...

4.10. SchedulingWorkflows and tasks require some sort of scheduling criteria, in order to be executed automatically.Scheduling may be configured based on day plans, triggers, or both. A trigger may, for instance, be

26

Core 4.1

tied to termination and state change of workflows, logging of specific node messages, or selected useractions. A combination of several types is also possible.

A scheduled workflow must be activated manually in order for the scheduling criteria to takeeffect.

Selecting Activate once, will prepare the workflow for execution - it will only become activewhen the scheduling criteria is fulfilled. Selecting Activate a second time at this point, will ac-tivate the workflow immediately. However, the defined scheduling criteria will still be stored.

4.10.1. Schedule Criteria - Day Plans

Scheduling of activities in Convergent Mediation Engine™ may be based on day plans. An activitymay be scheduled to run every day, a specific weekday or a specific day of the month. On each day,the activity may be set to run only once or periodically at different intervals between different times.

Each day plan type has a priority that instructs the scheduler to pick the correct day plan. For instance,if an activity is scheduled to run every day with a 15 min interval and on Sundays with a 1-hour interval,the weekday schedule type has a higher priority.

The priority order is:

1. Last day of month

2. Day of month (1-31)

3. Weekday

4. Daily

More than one day plan for the same day may exist, provided that the time intervals specified does notoverlap.

4.10.1.1. Schedule Configuration

The scheduling window is located within the Workflow Preferences window (accessed from the Editmenu in the Workflow Editor), and it has the following appearance:

Figure 19. Scheduling window.

List of selected day plans that apply to the scheduling. The list is sorted according to thepriority described in previous section; day of month or weekday and the time of the day.Thus, it is easy to follow what day plans apply to a given day.

Day Plan

Displays the Execution Calendar window, where the currently defined day plans executionpattern is shown.

Show...

27

Core 4.1

The Add/Edit Day Plan dialogs allow the user to add a new day plan.

Figure 20. Add Day Plan dialog.

Target day. Valid options are:Day

• Every day

• A specific weekday

• A specific day of the month (1-31)

• The last day of the month

Indicates that there will be no execution at all on the selected Day.Day Off

Start time for the first execution.Start At

End time for execution. If left empty, 23:59 is set by default.Stop At

How often the execution will take place between the start and stop times. Could bedefined in either minutes or hours.

Repeat Every

If left empty, there will be only one execution at the configured Start At time.

4.10.1.2. Viewing an Execution Plan

When the day plan list has been populated, select Show... to display the Execution Calendar window.

Figure 21. Execution Calendar window.

The calendar displays a monthly overview for the next five years. Each column represents a day ofthe selected month and each row represents an hour of that specific day. If a cell is colored green, atleast one execution is planned for that hour. Clicking on the cell will display at what minutes of thehour the execution will start.

28

Core 4.1

Figure 22. Hourly execution plan dialog.

4.10.2. Schedule Criteria - Event Triggers

It is possible to configure tasks and workflows to be activated as a result of a specific trigger beingfired. An activation can, for instance, be related to user actions, workflow states or workflow nodemessages. It is also possible to define a chain of events.

Figure 23. Event triggers tab - Add Event Chain Trigger dialog.

In the Add Event Trigger Chain dialog, event types and matching values are selected. All selectionsin the trigger chain need to be fulfilled in order for the trigger to be fired.

Figure 24. Event triggers tab. A chain of events can only exist within a row.

If any listed event, or chain of events within a row, has occurred, the trigger is fired. Note, all criterialisted within a row have to be fulfilled in order for that event to be considered as occurred.

If a chain of events has been configured, the events fulfilled are kept in memory. If the ConvergentMediation Engine™ system is restarted, this information is lost and nothing in the event chainwill be considered executed. The same thing applies in case a workflow, with such schedulingcriteria, is restarted.

4.10.2.1. Add Event

Selecting Add will display a dialog where any of the following Event Types may be selected andconfigured:

29

Core 4.1

Figure 25. Add Event Selection window.

A list of available Event Types. Selecting one, will populate the Event Filter areawith the valid Event Field Names. Refer to Section 5.3.3.1, “Event Fields” forfurther information on the different Event Types.

Event Type

Configurable parameters within this area may vary depending on the Event Typeselected. A matching value can be a regular expression allowing, for instance, toactivate the workflow if any workflow matching GSM_.* is executed.

Match Value(s)

There are no referential constraints for Event Triggers, nor any possibilities to find out relationsbetween workflows triggered by one another. For instance, if workflow A is defined to be activ-ated when workflow B is activated - workflow B may be deleted without any warnings, leavingworkflow A still as a valid workflow. The reason for this is that value matching is based on aregular expression and not on a precise match.

4.11. UDR BrowserAt places where one or many UDR types or UDR fields will be selected, the UDR Browser is utilized.The UDR Internal Format Browser is accessed from the Analysis or Aggregation node, by right-clicking in the APL Code area and selecting UDR Assistance....

The browser contains all UDR types available in the system:

1. UDR types created in the Ultra Format Editor. These include events and sessions for the Ag-gregation sub-system.

2. UDR types installed with the system.

Refer to the Convergent Mediation Engine™ Ultra Format Management - User Guide for further in-formation on UDR types and fields.

30

Core 4.1

Figure 26. UDR Internal Format Browser dialog (Field input assistance mode).

List of available UDR types ordered in a tree structure according to: folder name -module name - internal/target internal types.

UDR Types

Displays the fields belonging to the selected UDR Type, ordered in a tree structure.UDR Fields

To ease identification, the fields are color coded where Convergent Mediation En-gine™ special fields are blue, optional fields are red and nested UDRs are gray. Ifa nested UDR is optional as well, it is colored red.

If enabled, fields declared as optional are displayed.Show Optional

If enabled, only fields matching the selected data type are displayed.Datatype

4.11.1. Selection Modes

The browser operates in five modes depending on where it is utilized.

1. Single selection of UDR Type. The UDR Type may be chosen either by double-clicking the UDRType or by selecting it followed by OK or Apply. OK and double-click dismisses the dialog.

2. Multiple selections of UDR Type. Many UDR Types may be chosen at once by selecting themfollowed by OK or Apply. OK or double-click dismisses the dialog.

3. Single selection of UDR fields. Same as for UDR Type, but fields are selected instead.

4. Multiple selections of UDR field. Same as for UDR Type but fields are selected instead.

5. Field input assistance. Fields may be inserted in the target text field by double-clicking them orselecting them followed by Apply. The OK button is not available.

4.12. MIM BrowserAt places where access to MIM resources is used, the MIM Browser is utilized. The MIM Browseris accessed from the Analysis or Aggregation node, by right-clicking in the APL Code area and selectingMIM Assistance....

The browser contains all MIM resources available in the workflow:

31

Core 4.1

Figure 27. MIM Browser dialog.

The available MIM resources are displayed, ordered in a tree structure. The MIM resource may bechosen either by double-clicking the MIM resource or by selecting it followed by Apply. Cancel dis-misses the dialog.

4.13.Text EditorAt various places in the Convergent Mediation Engine™ environment, editable text areas are availablein order to enter an extensive amount of text. To assist in composing the text, these areas host functionsavailable in a pop-up menu that is displayed when the right mouse button is pressed anywhere in thetext area.

Figure 28. Text editor pop-up menu.

The menu has the following options:

Sets the font size.Font Size

Imports the contents from an external text file into the editor. Note that the filehas to reside on the host where the client is running.

Import...

Exports the current contents into a new file to, for instance, allow editing in an-other text editor or usage in another Convergent Mediation Engine™ system.

Export...

This option exists for the APL and Ultra Format editors. It opens the Emacstext editor, where editing may be done. It is possible to set another default editor.This is done in the $MZ_HOME/etc/desktop.xml file:

Edit with Emacs

32

Core 4.1

mz.gui.editor.name = Emacsmz.gui.editor.command = emacs.exe

Displays a dialog where chosen text may be searched for and, optionally, replaced.

Figure 29. Find/Replace dialog.

Find/Replace...

Repeats the search for the last string entered in the Find/Replace dialog.Find Again...

See previous section.UDR Assistance...

Items not listed are considered to be of general type. Refer to Section 4.9, “Menu and ToolbarItems” for further information.

4.14. List EditorAt most places where multiple entries of the same type will be selected, the List Editor componentis used. The list shows one or several columns holding data representing an entered object. Each ListEditor has a corresponding dialog where data for one object (row) is entered or amended. When thedialog is confirmed the data is loaded into the list.

Figure 30. List Editor component.

Displays a dialog where data for a new object is entered. The dialog will be titled Add ...to indicate objects are added.

Add

Enabled if one row is selected in the list. Displays a dialog where data for the selectedrow may be modified. The dialog will be titled Update ... to indicate objects are updated.

Edit

Removes the selected rows.Remove

Moves a selected row up or down in the list.

Not all items described are visible in each place where the List Editor component is used.

33

Core 4.1

4.15. Filename Character SubstitutionSome named items in the Convergent Mediation Engine™ environment are used when constructingfilenames. To avoid potential conflicts in the file systems, Convergent Mediation Engine™ will convertthese unaccepted characters when constructing the filenames.

All characters but the following will be converted to underscores.

• a-z

• A-Z

• 0-9

• - (dash)

• _ (underscore)

• . (period)

4.16. Date and Time Format CodesIn various places in Convergent Mediation Engine™ , date formats are entered. The following listshows valid date and time format codes that may be combined with any characters that are not in theranges of 'a'-'z' and 'A'-'Z'. For instance, characters like ':', '.', ' ', '#' and '@' will appear in the resultingtime even if they are not embraced within single quotes.

The date syntax conforms to the Java class SimpleDateFormat. This section contains a summaryo n l y . F o r a f u l l d e s c r i p t i o n , p l e a s e r e f e r t o h t -tp://java.sun.com/j2se/1.5.0/docs/api/java/text/SimpleDateFormat.html.

ExampleDate or time componentLet-ter

ADEra designatorG

1996; 96Yeary

July; Jul; 07Month in yearM

27Week in yearw

2Week in monthW

189Day in yearD

10Day in monthd

2Day of week in monthF

Tuesday; TueDay in weekE

PMam/pm markera

0Hour in day(0-23)H

24Hour in day(1-24)k

0Hour in am/pm (0-11)K

12Hour in am/pm (1-12)h

30Minute in hourm

55Second in minutes

978MillisecondS

34

Core 4.1

ExampleDate or time componentLet-ter

PST; GMT-08:00Time zonez

-08:00Time zoneZ

Example 2.

The following examples show how date and time patterns are interpreted in the U.S. local. Thegiven date and time are 2001-07-04 12:08:56 local time in the U.S. Pacific Time time zone.

2001.07.04 AD at 12:08:56 PDT"yyyy.MM.dd G 'at' HH:mm:ss z"

Wed, Jul 4, '01"EEE, MMM d, ''yy"

12:08 PM"h:mm a"

12 o'clock PM, Pacific DaylightTime

"hh 'o''clock' a, zzzz"

0:08 PM, PDT"K:mm a, z"

02001.July.04 AD 12:08 PM"yyyyy.MMMMM.dd GGG hh:mmaaa"

Wed, 4 Jul 2001 12:08:56 -0700"EEE, d MMM yyyy HH:mm:ss Z"

010704120856-0700"yyMMddHHmmssZ"

35

Core 4.1

5. Configuration MenuThis section describes the non agent specific configuration editors, found in the Configuration menuon the desktop. Editors that belong to a certain agent are described in its respective user guide.

5.1. Audit ProfileConvergent Mediation Engine™ offers the possibility to output information to user defined databasetables. This means that several workflows may output information about the same batch to the sametable, which makes it possible to trace batches/UDRs between workflows.

To increase this traceability, it is highly recommended to add fields to the UDRs, to make it possibleto identify their origin. Useful values may be:

• Name of the switch.

• Name of the original filename.

• Timestamp of the original file.

In the Audit Profile Editor, the audit table column types are configured. Select Audit Profile... fromthe Configuration menu on the desktop to display the dialog.

Figure 31. The Audit Profile Editor.

Holds all available Database Profiles. Adding new databases is done via the DatabaseProfile Editor.

Database

Reloads the meta data for the tables residing in the selected database.Refresh

A list of selected audit tables. Refer to Section 5.1.1, “Adding and Editing a Table Mapping”for information on adding and editing tables.

Tables

5.1.1. Adding and Editing a Table Mapping

From the Add and Edit Audit Table Attributes dialogs, the existing table columns are mapped toConvergent Mediation Engine™ valid types.

36

Core 4.1

Figure 32. The Audit Profile.

A list from which the audit table is selected.Table

The name of the columns in the selected table.ColumnName

Right-clicking the cell, displays a list of valid Convergent Mediation Engine™ types.Each column must be mapped against a type. Valid types are:

Type

• Counter - A built-in sequence which is automatically incremented with the valuepassed on with the auditAddAPL function.

• Key - The user defined key sequence will uniquely identify all audit log entries tobe inserted per batch. This is not a database key and must be kept as small as possible.A value that is static during the whole batch must never be used as a key value.

Several keys may be present in the same table. Note that the key sequence enteredmust be entered in the same order when calling the APL commands.

• Value - A column holding any type of value to be set, except for Counter values.This is used in combination with the auditSet APL function. Another use ismapping against existing MIM values in the Workflow Preferences window.

• Transaction Id - To make sure entries are transaction safe, each table must containa column of type NUMBER and at least have the length twelve (or have no sizedeclared at all). Do not enter or alter any values in this column, it is handled auto-matically by the Convergent Mediation Engine™ system. The value -1indicatesthat the entry is committed and safe.

The Transaction Id should be indexed for best performance. The contentswill be of low cardinality and could therefore be compressed if supported.

• Unused - Used in case a column must not be populated, that is, set to null.

Each key in a table must have a sequence number in order to be identified when passedon as parameters to the APL audit functions. The first key is identified as 1, the secondas 2, and so on.

Key Se-quence

5.1.2. An Example

To illustrate how Audit may be used, consider a workflow with an Analysis node, validating androuting UDRs. Most of the UDRs will be sent on the "COMPLETE" route. The rest of the, incomplete,UDRs will be sent on the "PARTIALS" route. If there is a considerable amount routed to the latter,the batch is cancelled.

37

Core 4.1

Figure 33. A workflow example.

The output on each route is to be logged in a customized audit table, including information on cancelledbatches. An entry in the table will be made for each batch, and for each route. Hence two entries perbatch.

Figure 34. Audit information may be customized.

In this example only the destination key is needed, which will uniquely identify all rows to be insertedper batch. The name of the destination node is therefore selected. Note, it is not possible to update anexisting row in the table, only to add new rows. This to assure the traceability of data. In order to outputother information than MIM values (which may be mapped in the Workflow Preferences window),the workflow must contain an Analysis or Aggregation agent.

Setting up an Audit Profile involves the following steps:

1. Design the tables:

• One column (of type NUMBER) must be reserved for the Convergent Mediation Engine™transaction handling. This column should be indexed in order to achieve best performance.The contents will be of low cardinality and could therefore be compressed if supported.

• Consider which column/columns that contains tag information, that is, the key. A key mayconsist of one or several columns.

2. Create an Audit Profile. Refer to Section 5.1.2.1.1, “Adding the Table Mapping” for further in-formation.

3. Map parameters in the Workflow Preferences Audit tab to the Audit Profile. Refer to Sec-tion 5.1.2.2, “Workflow Preferences - Audit tab” for further information.

38

Core 4.1

4. Design APL code to populate the tables. Refer to Section 5.1.2.3, “Populating Audit Tables” forfurther information.

5.1.2.1. Audit Profile

In the Audit Profile Editor, the customized audit table column types are configured. Select AuditProfile... from the Configuration menu on the desktop to display the dialog. Select the database inwhich the table(s) reside, then select Add.

5.1.2.1.1. Adding the Table Mapping

From the Add and Edit Audit Table Attributes dialogs, the existing table columns are mapped toConvergent Mediation Engine™ valid types.

Figure 35. The Audit Profile.

The data to insert, will be put in the UDRs column. Setting it to type Counter, makes it possible touse the auditAdd function to increment the corresponding column value. If Value is used, theauditSet function can be used in order to assing a value.

5.1.2.2. Workflow Preferences - Audit tab

The Audit tab in the Workflow Preferences window defines the type of data entered in the table bythe workflow. This is either MIM types or anything sent on with the APL audit functions.

Figure 36. The Workflow Preferences window - Audit tab.

The DESTINATION and UDRs columns are populated using the APL audit functions. The CAN-CELLED column may be mapped directly to an existing MIM value.

5.1.2.3. Populating Audit Tables

There are two ways of populating audit tables; either by using the auditAdd function, which auto-matically increments the value of Counter columns, or by setting fixed values to columns of typeValue with the auditSet function. Note that Counter columns are automatically set to 0 (zero)when a batch is cancelled. This is not the case for Value columns.

In the following subsections, the differences of the case exemplified in Figure 34, “Audit informationmay be customized.” is discussed.

39

Core 4.1

In terms of performance, it does not matter how many times an audit function is called. Eachcall is saved in memory and a summary for each key is committed at End Batch.

5.1.2.3.1. Automatic Increment

Using the auditAdd function, the user does not have to keep track of the number to increment acounter column with. It is automatically handled by the system. At Cancel Batch, the value is set to 0(zero).

In the current example, each UDR is validated with respect to the contents of the causeForOutputfield. The statistics table is updated to hold information on the numbers of UDRs sent on the differentroutes.

Example 3.

int noPART;

beginBatch { noPART = 0;}

consume { if ( input.causeForOutput == "0" ) { udrRoute( input, "COMPLETE" ); auditAdd( "myFolder.count_PARTIALS", "ADMIN.PARTIALS_AUDIT", "UDRS", 1, mimGet( "TTFILES", "Source Filename" ), mimGet( "COMPLETE", "Node Name" ) ); } else { noPART = noPART + 1; if ( noPART < 300 ) { udrRoute( input, "PARTIALS" ); auditAdd( "myFolder.count_PARTIALS", "ADMIN.PARTIALS_AUDIT", "UDRS", 1, mimGet( "TTFILES", "Source Filename" ), mimGet( "PARTIALS", "Node Name" ) ); } else { cancelBatch( "Too many partials found." ); } }}

5.1.2.3.2. Fixed Values

Using the auditSet function for the same example as discussed in the previous section, means theuser has to keep track of the number of records in the APL code. Note that the Profile must be updated;the Counter column must be redefined to Value.

Value columns are not reset when a batch is cancelled. Hence there will be entries made in the tablefor the UDRs column for all batches.

40

Core 4.1

5.2. Database ProfileIn the Database Profile Editor, database aliases (profiles) to be used in various Convergent MediationEngine™ applications are created. For instance, in the Audit Profile Editor and the sqlExec APLfunction. Note that the standard Convergent Mediation Engine™ Platform database is not listed. Thisis because it is not intended to use for customization.

To display the window, select Database Profile... from the Configuration menu on the desktop.

Figure 37. Database Profile window.

The name identifying the database instance. This is typically the SID name.Database Name (SID)

The database host name or IP address. Type it exactly as when accessing itfrom any other application within the network.

Database Host

The database network port.Port Number

The database user name.User Name

The database password.Password

The type of database (ORACLE, SQL Server or Derby).Database Type

Tries the connection to the database. Use this to verify that all parametersare correctly configured.

Try Connection

5.3. Event Notification EditorThe Event Notification Editor offers the possibility to route information from events generated inthe system, to various targets. These targets include:

• Log file

• Database

• E-mail

• SNMP trap receivers

• System Log

An Event is an incident of importance that occurs in the Convergent Mediation Engine™ system.There are several different event types that all contain specific data about the particular event. Besidesbeing logged, events may be split up and selected parts may be embedded in user defined strings. Forinstance, consider an event originating from a user, updating an existing Notifier:

41

Core 4.1

userName: mzadmin3, userAction: Notifier AnumberEvents updated.

This is the default event message string for User Events. However, it is also possible to select parts ofthe information, or other information residing inside the event. Each type of event contains a predefinedset of fields. For instance, the event message previously exemplified, contains the userName anduserAction fields which may be used to customize event messages to suit the target to which they willbe logged:

An event message of type ‘User Event ’

categorycontents Username: mzadmin3, Act ion: Not i f ier AnumberEvents updatedeventName User EventipAddress 10.0.0.55sever i ty Informat ionuserAct ion Not i f ier AnumberEvents updated.userName mzadmin3

Mail contentUsername: mzadmin3, Act ion: Not i f ier AnumberEvents updated.

Log file entryUser Event repor ted on EC 10.0.0.55.

Database table

Error User

Informat ion mzadmin3

Figure 38. Events can be customized to suit any target.

The Category field is left out intentionally in the picture, since it does not have a value for thisspecific event. A category is user defined and is entered in the Event Categories dialog. It is astring which will route messages sent with the dispatchMessageAPL function.

The event types forms a hierarchy, where each event type adds its own fields and inherits all fieldsfrom its ancestors.

The event hierarchy is structured as follows:

• Base Event

• Code Manager Event

• User Event

• System Event

• Task Executor Message Event

• Task State Event

• Workflow Event

• Debug Event

• Ecs Insert Event

• Node Failure Event

• Node State Event

• Node Message Event

42

Core 4.1

• User Defined Event

• User Node Message Event

• Workflow State Event

• Node Event

Each event type and its fields are described in Section 5.3.3.1, “Event Fields”.

A notifier is a selected target, receiving event data when one or several selected event types are generatedin the system. In addition, filters may be applied for each selected event type. Notifiers are configuredin the Event Notification Editor window, displayed when selecting Event Notification... from theConfiguration menu on the desktop.

Figure 39. The Event Notification Editor.

Event streaming may be turned off for a specified notifier. Note, undirected eventsare not saved by the system, and can therefore not be retrieved.

Disabled

A Notifier is the target where event messages, configured in the Event Setup, aresent. For instance,to a database table, a log file or to the Convergent Mediation En-gine™ System Log.

Notifier Setup

The overall appearance of the message string is also defined in this tab.

In the Event Setup tab, events to catch are defined. If necessary, the message stringdefined in the Notifier Setup is also modified.

Event Setup

5.3.1. Notifier Setup

A Notifier is defined as the target where event messages are sent. A Notifier may output informationto any type of database table, text file, SNMP trap, mail or the standard Convergent Mediation Engine™System Log.

Duplicate messages may be suppressed from logging. If a duplicateevent arrives during the time interval specified, it will be ignored.Default is 0 (zero).

Duplicate Suppression (sec)

5.3.1.1. Notification Type - Database

Event fields may be inserted into database tables using either plain SQL statements or calls to storedprocedures.

43

Core 4.1

Figure 40. Event Notification Editor - Notification Type Database.

The name of the database in which the table resides. Databases are defined in theDatabase Profile Editor.

Database

Type in any SQL statement, using '?' for variables which are to be mapped againstevents in the Event Setup tab. Note that trailing semicolons are not used. In caseof running several statements, they must be embedded in a block.

SQL Statement

5.3.1.2. Notification Type - Log File

Messages may be routed to ordinary text files on the local file system.

Figure 41. Event Notification Editor - Notification Type Log File.

The path to the directory where the file, to append to, resides.Directory

The name of the file. In case the file does not exist, it will be created when the firstmessage for the specific event map arrives. New messages are appended.

Filename

The maximum size of the file. When this parameter is exceeded, the existing file is re-named and a new is created upon the arrival of the next event.

Size

The old file will receive an extension to the filename, according to<date_time_milliseconds_timezone>.

The maximum life time of a file before it is rotated. When this parameter is exceeded,the existing file is renamed and a new is created upon the arrival of the next event.

Time

The old file will receive an extension to the filename, according to<date_time_milliseconds_timezone>.

If both Size and Time are set, both behaviors apply.

Indicates how logged events are separated from one another. Valid options are:Separator

• Linefeed

• CR + LF (Carriage return + Linefeed)

44

Core 4.1

• Comma

• Colon

• (None)

Text to go into the log file.Log Line

Please refer to Section 5.3.2, “Population Types” for further information on the differentoptions.

Do not configure two different Event Notifiers to log information to the same file. Messagesmay be lost since only one notifier at a time can write to a file. Define one Event Notifier withseveral Event Setups instead.

5.3.1.3. Notification Type - Send Mail

It is also possible to send mails to one or several recipients when the specified events occur. Makesure the correct parameters, mz.mailserver and mz.notifier.mailfrom, have been configuredin $MZ_HOME/etc/platform.xml.

Figure 42. Event Notification Editor - Notification Type Send Mail.

Mail address to one or several recipients. Use comma to separate, if several. Select Addto select mail addresses, configured for available users.

Recipient

Please refer to Section 5.3.2, “Population Types” for information on how the text may beobtained.

The subject/heading of the mail. If Event Contents is selected, newlines will be replacedwith spaces to make the subject readable. If the string exceeds 100 characters, it is truncated.

Subject


The body of the mail message.Message


5.3.1.4. Notification Type - Send SNMP Trap

Events may be sent in form of SNMP traps to systems configured to receive such information. For theMIB definition, please refer to the $MZ_HOME/etc/mz_trap_mib.txt file.

45

Core 4.1

Figure 43. Event Notification Editor - Notification Type Send SNMP Trap.

The IP address of the target host.IP Address

The port on the target host defined for the SNMP communication.Port

The community name string used for the authentication process.Community

A list containing the supported SNMP versions.Version

A string which will be sent out as SNMP traps.User Message

Please refer to Section 5.3.2, “Population Types” for information on how the text maybe obtained.

5.3.1.5. Notification Type - System Log

Selecting this target will route messages, produced by the selected events, to the standard ConvergentMediation Engine™ System Log. The Contents field from each event will be used as the message inthe log.

Figure 44. Event Notification Editor - Notification Type System Log.

Do not route frequent events to the System Log. Purging a large log might turn into a performanceissue. If still doing that, keep the log size at a reasonable level by utilizing the System LogCleaner (configured in the Task Manager window).

5.3.2. Population Types

Some of the notifier types have one or many parameters that may be dynamically populated by datafrom an event. These parameters, if available, are configured in the Target Field Configurationpanel.

Depending on the parameter type, there will be one or several population types available in the listnext to it.

5.3.2.1. Manual

Selecting Manual allows the user to hard code a value, and thus gives no possibility to select any dy-namic values to be embedded in the message. The value entered will be assigned to the parameter exactlyas typed.

46

Core 4.1

Figure 45. Target Field Configuration - Log Line.

5.3.2.2. Event Field

Selecting Event Field allows the user to assign the value of one specific event field to the parameter.Refer to Section 5.3.3.1, “Event Fields” for further information on the different fields, valid for selectionfor each event type.

5.3.2.3. Event Contents

Selecting Event Contents assigns the event value of each event Contents field to the parameter. Allevent types have a suitable event content text. For instance, referring to the example in Figure 38,“Events can be customized to suit any target.”, the Event Contents string will be:

Username: mzadmin3, Action: Notifier AnumberEvents updated.

Another example; the following string is reported for a User Defined Event:

Workflow name: <name>, Node name: <name>, Message: <string>

The Message string originates from the dispatchMessage function. Note that nothing will belogged unless dispatchMessage is used. Also in this case, the Field Maps in the Event Setupwill be disabled.

The same result is achieved when selecting Event Field as Log Line, and then selecting Contentsas mapping (the Event Setup tab).

5.3.2.4. Formatted

Selecting Formatted allows the user to enter text combined with variable names, which are assignedevent field values in the Event Setup tab.


Log file entryUser Event message repor ted on EC 10.0.0.55.

categorycontents Username: mzadmin3, Act ion: Not i f ier AnumberEvents updatedeventName User EventipAddress 10.0.0.55sever i ty Informat ionuserAct ion Not i f ier AnumberEvents updated.userName mzadmin3

Figure 46. A Log Line may be a mix of hard coded strings and event fields.

Variable names must be preceded with a $, followed by a sequence of letters, digits or underscores.Note, the first character must be a letter.

47

Core 4.1

Figure 47. Target Field Configuration - Log Line.

This will be interpreted as containing the variables 'NO' and 'ANUM'.

5.3.3. Event Setup

In the Event Setup tab, parameters populated using the type Event Field, Formatted (with variables)or SQL, may be assigned their values from selected event fields. Also, the event types to catch areconfigured.

Figure 48. Event Notification Editor, Event Setup tab.

The Filter table allows the user to configure what event types to catch. For eachevent type, a filter may be defined to allow, for instance, one specific workflowand two specific event severities to pass.

Filter

Besides filtering on existing values (for instance workflow name) it is possibleto filter using regular expressions and hard coded strings. Regular expressionsaccording to Java syntax applies.

For further information, refer to

http://java.sun.com/j2se/1.5.0/docs/api/java/util/regex/Pattern.html

The contents of this column varies depending on the selected event type. Formore information on different events, refer to Section 5.3.3.1, “Event Fields”.

Event Field Name

Note that only fields of string type are visible.

The filtering criteria defining what events to catch. Double-clicking a cell willdisplay a list from which values may be selected. Note, these values are onlysuggestions. Entering hard coded strings and regular expressions are also allowed.

Match Value(s)

The default value for each of the components is All.

Maps variables against event fields. The Field Map table exists only if any ofthe parameters for the selected notifier type is set to Formatted, Event Fieldor SQL.

Field Map

48

Core 4.1

States the Notifier parameter available in the Notifier Setup tab.Notifier Field

If a specific parameter has more than one variable, it will claim one line pervariable.

The name of the variable, as entered in the Notifier Setup tab. If the parametertype is Event Field, this field will be empty.

Variable

Double-clicking a cell will display a list, from which the event field, to obtainvalues from, is selected.

Event Field

Displays a dialog from which an event type is selected. The new event type willbe added as a separate tab in the Event Setup.

Add Event...

Removes the currently selected event type tab in the Event Setup dialog.Remove Event...

Updates the Field Map table. Required if parameter population types orformatting fields have been modified in the Notifier Setup tab.

Refresh Field Map

5.3.3.1. Event Fields

An Event is defined as a set of parameters. These are fields containing the original event message, aset of standard workflow related information, and event specific fields which are basically the parametersthe original event message consists of:


categorycontents Username: mzadmin3, Act ion: Not i f ier AnumberEvents updatedeventName User EventipAddress 10.0.0.55sever i ty Informat ion

The event speci f ic f ie lds, which arealso par t of the ‘contents ’ f ie ld.

Fields inher i ted fromthe Base Event.

userAct ion Not i f ier AnumberEvents updated.userName mzadmin3

Figure 49. An event message contains information ordered in fields.

All Event types in the Convergent Mediation Engine™ system inherit fields from the Base Event type.Workflow related events inherit fields related to the workflow as well, such as nodeName. In additionto these, User Defined Events will receive any fields as defined by the user.

There are two types (hierarchies) of Events:

• User Defined Events.

• Events inherited from a Base Event (all other events), with additional information added.

The first type of event must be configured in the Ultra Format Editor. Other than the fields enteredby the user, Convergent Mediation Engine™ will automatically add basic fields. User Defined Eventsmay only be dispatched from an agent utilizing APL, that is, Analysis or Aggregation.

49

Core 4.1

Ultra Format Definitionevent myEvent {

string anum;string bnum;int errorNumber;int udrCount;

};

Analysis / AggregationmyEvent theEvent = udrCreate( myEvent );

theEvent.anum = theUDR.anum;theEvent.bnum = theUDR.bnum;theEvent.udrCount = count;

dispatchEvent( theEvent );

instance of myEvent

myEventanumbnumerrorNumberudrCount

categoryeventNameipAddressnodeNameseveritytimeStampworkflowName

Automaticallyadded fields

Figure 50. User Defined Events are sent from Analysis or Aggregation nodes.

Fields added by the user must be populated manually by using APL commands, while the basic fieldsare populated automatically. From the basic fields, only category and severity may be assigned values.The other basic fields are read only, hence it is not possible to assign values to them.

The following list describes all event types and their fields.

A Base Event is the parent of all events, except for the User Defined Events (describedin this table). Since it is the parent it means that all events will inherit the fields of theBase Event.

Base Event

Subscribing on Base Events is not recommended since it will match every eventproduced in the system, which may generate a high volume of events.

Base Events contain the following information:

• category - Not utilized for Base Events.

• contents - A hard coded string containing event specific information; the ori-ginal event message. For instance, for the ECS Insert Event, this string will containthe type of data sent to ECS, the Workflow name, the node name, and the UDRcount. Refer to the specific event types (this table) for information on the contentsfield.

• eventName - The name of the Event, that is, any of the types described in thistable. Base Event, Code Manager Event etc.

• ipAddress - The IP address of one of the following:

• Execution Context the workflow issuing the event is running on.

• Platform, if it is not a workflow event.

50

Core 4.1

• Desktop, if it is a User Event.

• severity - The severity of the event. May be any of: Information, Warning,Error or Disaster. The default value is Information.

• timeStamp - The date and time for the issuing of the event.

These messages are logged when code is changed, removed or added; when pico in-stances are added/removed; or when the code server reports an error. Other than theBase Event fields, the following fields are included:

Code Man-ager Event

• cmAction - The message stating what part of the system is affected.

• cmActionType - States if code was changed, removed or added.

• The contents field - Code: <message>

Dispatched when debug is used, if enabled in the Workflow Preferences window ofa workflow. The event is of workflow type and therefore defines the following fields:

Debug Event

• nodeMessage - Message issued by the node.

• nodeName - The name of the node issuing the event.

• workflowKey - The name of the internal Convergent Mediation Engine™workflow key.

• workflowName - The name of the workflow issuing the event.

These messages are logged each time data is inserted into ECS. Other than the BaseEvent fields, the following are included:

ECS InsertEvent

• ecsMessage - Message supplied when ECS batch data is inserted. This valueis always none for inserted UDRs.

• ecsMIMs - A comma separated list of the MIM values entered in the WorkflowPreferences window.

• ecsSourceNodeName - For batches, this value is equal to the node issuing thecancelBatch request. For UDRs, this will be equal to the node that sent theUDR to ECS.

• ecsType - Holds either the value UDR or batch.




• The contents field for UDRs - Type: UDR, Workflow: <workflowname>, Node: <node name>, UDR Count: <count>, MIMs:<MIMs>

• The contents field for batches - Type: batch, Workflow: <workflowname>, Node: <node name>, UDR Count: <count>, MIMs:<MIMs>

The event specific fields:Node Event

51

Core 4.1



• workflowName - The name of the workflow.

• The contents field - Workflow: <workflow name>, Node: <nodename>

A Node Failure message is not a failure literally speaking, but is reported when a nodeacts upon a predefined Error Case. For instance, when duplicates are found by the

Node FailureEvent

Duplicate UDR Detection agent. Not all nodes can issue these sort of events. Refer tothe user guide of the specific agent for further information. Other than the Base Eventfields, the following fields are included:

• nodeErrorMessage - Error message issued by the node.

• nodeName - The name of the node.



• The contents field - Workflow: <workflow name>, Node: <nodename>, Message: <errorMsg>

A Node Message is of informative type. An example of such a message is when aDatabase Collection agent starts collecting data, or when a Disk Forwarding agent isfinished with a batch.

Node Mes-sage Event

Not all nodes can issue these sort of events. Refer to the user guide of the specificagent for further information.

The event specific fields:





• The contents field - Workflow: <workflow name>, Node: <nodename>, Message: <message>

This event is dispatched when a node state changes. The event specific fields:Node StateEvent


• nodeState - The state of the node. The following are available: Aborted, Active,Created, Idle, Stopped.



52

Core 4.1

• The contents field - Workflow: <workflow name>, Node: <nodename>, State: <state>

Event generated upon unit failure. The Base Event is extended with one field:System Event

• systemMessage - The message string.

• The contents field - Exception message: <errorMsg>

This message type displays task activations and their result. Other than the Base Eventfields, the following fields are included:

Task Execut-or MessageEvent

• executorName - The name of the task for which information is to be displayed.

• executorMessage - Message supplied by the executor.

• taskName - The name of the task as defined in the Task Manager.

• The contents field - Executor: <executor name>, Message:<message>

This event is dispatched when a task state changes. The event specific fields:Task StateEvent

• executorName - The name of the task for which information is to be displayed.

• taskName - The name of the task for which status change messages are desired.Not selecting any in particular, will result in information for all defined tasks beingdisplayed.

• taskState - Listed options are; Started, Stopped, Running and Aborted.

• The contents field - Task: <task name>, State: Star-ted/Stopped/Aborted

This event is dispatched when the user changes something, for instance, updates aconfiguration. Other than the parameters of the Base Event, two additional parameterscan be selected:

User Event

• userName - The name of the user.

• userAction - Action performed by the user.

• The contents field - Username: <username>, Action: <action>,Workflow: <workflow name>

A user defined event is a basic event, extended with any variables entered by the user.It is configured in the Ultra Format Editor:

event myEvent {

<UserDefinedEvent>

ascii addedField1; int addedField2; };

A user defined event is of workflow type therefore includes workflow specific fields.

The basic fields are automatically included in myEvent, along with the typed in fields.Population of the fields is done via an APL utilizing node. Example code:

53

Core 4.1

myEvent x = udrCreate( myEvent ); x.addedField1 = aUDR.anum; x.addedField2 = aUDR.code; dispatchEvent( theEvent );

The fields of a User Defined Event:

• category - A user defined category, as entered in the Event Categories dialog.If utilized, this field is set manually in the APL code.

• eventName - the name of the Event, as defined in Ultra.

• ipAddress - the IP address of the Execution Context the workflow issuing theevent is running on.

• severity - The severity of the event. May be any of: Information, Warning,Error, or Disaster. The default value is Information. If another severity is required,this field must be set manually in APL to one of the strings: "Information","Warning", "Error", "Disaster".




• <any> - Any information, as stated in the format configuration.

• The contents field - Workflow name: <workflow name>, Nodename: <node name>

This event holds a value only if the APL function dispatchMessage has beenused. Other than the Base Event fields, the following fields are included:

User NodeMessageEvent





• The contents field - Workflow: <workflow name> Node name:<node name>, Message: <message>

This is the parent for all workflow events. Other than the fields defined in the BaseEvent, two additional fields are defined:

WorkflowEvent


• workflowName - A list indicating what workflow(s) information to select.

• The contents field - Workflow: <workflow name>

54

Core 4.1

This event is dispatched when the workflow state changes. Other than the fields of theBase Event, the following fields are defined:

WorkflowState Event

• abortReason - Desribes the abort reason for a state event of type aborted.


• workflowName - A list indicating what workflow(s) information to select.

• workflowState - A list from which the interesting states are selected. Validoptions are: Aborted, Active, Unreachable, Executed, Scheduled, Stopped.

• The contents field - Workflow: <workflow name>, State:<state>, Execution time: <time in milliseconds>, Abortreason: <error msg>

5.3.4. Event Category

An Event Category is defined from the Edit menu in the Event Notification Editor. The Event Categoryis used to send any kind of information to a Column, as opposed to direct mapping of a MIM resource.The Event Category name is a string which is used as argument in the dispatchMessage APLfunction to route messages to a Column. For further information of the usage of an Event Category,please refer to Section 6.5.2.4, “An Example”.

Figure 51. The Event Category window, where a user defined string is specified to be used asname for the category field in an event.

When the Event Category is defined it is mapped towards a suitable Column in the Workflow Prefer-ences window, Monitor tab.

5.4. Workflow EditorThe Workflow Editor window is the central unit of the Convergent Mediation Engine™ desktop.This is where all workflows are designed by adding nodes and connecting them to each other to forma data flow, and then configure them accordingly.

The window operates in two modes.

• Edit mode

• Monitor mode

In Edit mode, workflows are created and configured, while in Monitor mode workflows may be startedor stopped and the status of its individual nodes may be monitored.

55

Core 4.1

5.4.1. Edit Mode

In Edit mode, workflows are created and configured. The mode is selected using the button in the rightcorner of the toolbar area and is the default mode when the Workflow Editor window is opened.

Since many configurations include a selection of MIM resources and route information fromother available nodes in the workflow, it is recommended that all nodes and routes are addedto the workflow and named before the actual configuration of any of them begin.

Figure 52. Workflow Editor window in Edit mode.

5.4.1.1. Window Layout

In Edit mode the Workflow Editor window is divided in the following areas.

5.4.1.1.1. Menu Bar Area

The menu bar contains the following menus and options. Items not listed are considered to be of gen-eral type, refer to Section 4.9, “Menu and Toolbar Items” for further details.

Clears the work area including all configurations for a new Batchworkflow.

File menu New Batch Workflow

Clears the work area including all configurations for a new work-flow of type Batch template.

File menu New Batch Template

Clears the work area including all configurations for a new Real-time workflow.

File menu New Realtime Work-flow

Clears the work area including all configurations for a new work-flow of type Realtime template.

File menu New Realtime Tem-plate

Displays a new window where the template, from which the in-stance will inherit the configuration, is selected. Once selected,

File menu New Template In-stance

56

Core 4.1

the work area is populated with a copy of the template configura-tion.

Displays a dialog where a previously saved workflow may be se-lected and opened. The dialog also allows to filter on the differentworkflow types.

File menu Open...

Displays the Workflow Preferences dialog where workflow re-lated data may be configured. Refer to Section 5.4.1.4, “WorkflowPreferences” for further details.

Edit menu Preferences...

Contains an alternative way to drop nodes onto the work areaother than to drag them from the icon area.

Agents menu

Sets the Workflow Editor in Edit mode.Edit

Sets the Workflow Editor in Monitor mode.Monitor

5.4.1.1.2. Work Area

This is the area where workflows are designed by dropping nodes and connecting them to each other.Double-clicking a node displays its node configuration window. Right-clicking, displays a pop-upmenu from which the configuration window may be selected, as well as standard copy/paste/cut fea-tures.

Resting the cursor over a node, displays parts of its configuration in a tool-tip text.

5.4.1.1.3. Icon Area

Contains available agents, represented as icons, in three tabs named Collection, Processing and For-warding.

5.4.1.2. Workflow Templates

A workflow template is a base from which several workflows may be instantiated. The advantage withthis feature - besides the time saved for not having to configure workflows from scratch - is the possib-ility to make changes in the template that will have effect on all instances. A node may also be uniquefor each instance, in which case changes to the template will not have an effect on the existing instances- only new instances will inherit these changes.

This is accomplished by turning on a node's ability to be reconfigured in the instance workflow. TheEditable option is found in the node pop-up menu for workflow templates (right-click the node):

57

Core 4.1

Figure 53. A node tool-tip of a workflow template.

Note that an instance does not have to look exactly like its template. New nodes may be added, and ifa template node is set to Editable, this node may be reconfigured.

A workflow template cannot be activated.

5.4.1.2.1. Editing a Workflow Template

A Workflow template may be edited even if its instances are running, as long as it is not set in Editmode. Depending on the changes made to a template, the result could be invalid instances. SelectValidate... from the template to examine the state of the instances.

Figure 54. Workflow Instances Validation dialog.

Note, a new version of all instances is created each time Save is selected in the corresponding template.The user is expected to manually change any invalidated instance workflows.

The following rules apply:

Before a workflow template can be deleted, all its instances must bedeleted.

Removing a workflow template

If a non-editable node is removed from a workflow template, thenode will be removed from the instances as well. An editable nodewill, however, not be removed from the instances.

Removing a node

An added node will be inherited by all instances, whether it is edit-able or not.

Adding a node

The instances will automatically inherit the current template nodeconfiguration when it is saved.

Changing a node from editableto non-editable

The instance nodes will inherit the changes in case the node is non-editable. Editable nodes will keep their configuration.

Reconfiguring a node

58

Core 4.1

Make sure to always have unique names for all nodes. If conflicting node names exist for atemplate and instance Convergent Mediation Engine™ will automatically change the name ofthe node in the instance to <nodename>_T.

If the template changes any non-editable node, the historic versions for the instances are changedas well.

5.4.1.3. Normal and Instance Workflows

An instance workflow is an editable copy of a workflow template, while a normal workflow has norelations to any other workflow. The main difference between workflow templates, and the other twotypes (normal and instance), is that a workflow template cannot be activated.

Figure 55. Different ways to mark a node.

Nodes inherited from a workflow template are marked with circles. Yellow circles show that the nodemay be edited but not removed, while a red circle shows that the node cannot be removed nor edited.To change such a configuration, the workflow template has to be reconfigured. This is set through theEditable option in the workflow template.

Adding a node within an instance workflow will result in a normal node. Also, all nodes within normalworkflows are unmarked.

5.4.1.4. Workflow Preferences

Workflow specific data is configured in the Workflow Preferences dialog. From the Edit menu, selectPreferences... and the Workflow Preferences dialog will appear.

5.4.1.4.1. Execution Tab

The Execution tab contains configurations related to where the workflow will be executed and howoften it will execute.

59

Core 4.1

Figure 56. Workflow Preferences dialog, Execution tab.

A workflow executes on an Execution Context (EC). Specific ECs may be definedby the user, or the Convergent Mediation Engine™ system can handle it automatically.The following options exist:

Distribution

• Sequential - Valid only if Execution Contexts are defined. Starts the workflowon the first EC in the list. If this EC is not available, it will precede with the nextin line.

• Workflow Count - Starts the workflow on the EC running the fewest number ofworkflows. If the Execution Targets list contains any entries, only these ECswill be considered.

• Machine Load - Starts the workflow on the EC with the lowest machine load.If the Execution Contexts list contains any entries, only these ECs will be con-sidered. Which EC to select is based on information from the System Statisticssub-system.

Defines workflow scalability for a Realtime workflow.Scalability

The number of threads used for decoding and routing of records in a Realtimeworkflow.

Threads

The number of unprocessed entries (backlog) the workflow can buffer before thecollectors and I/O agents are slowed down. The limit will be reached when requests

Queue Size

are received in a faster pace than can be processed by the workflow and associatedback-end systems can. The records or decoding task will still be queued, but thequeuing is delayed to limit the backlog from growing too fast.

Determines on what EC(s) the workflow may execute. If several are entered, the se-lected Distribution is considered. If no EC is selected, Convergent Mediation En-

ExecutionTargets

gine™ will consider all available ECs as possible targets. A stand-alone workflowmust be configured to run on a stand-alone EC. Only one stand-alone EC can beconfigured.

60

Core 4.1

A host is added by selecting Add to present a list of available Execution Contexthosts.

For a stand-alone workflow agents will validate against the configured Execu-tion Context. If an agent depends on another host, the workflow will be invalidsince it in this case relies on another Execution Context and a network failurecould cause the workflow to abort.

Determines when the workflow will execute.Schedule Cri-teria

If nothing is entered (including no Event Triggers), the workflow will start executingupon manual activation. This setup will result in a one-time execution of the workfloweach time it is manually started.

For a Batch workflow this means that it will stop automatically when running out ofinput data. Note, the Inter Workflow Collection agent (and other agents of "push"type) will not stop, but wait for more data to process. A Realtime workflow, runsuntil aborted or manually terminated.

The workflow will be periodically started according to the configured day plan(s).Refer to Section 4.10.1, “Schedule Criteria - Day Plans” for details on how to config-ure periodic scheduling.

Day Plan

Activation of a workflow may be tied to events issued by other workflows, nodes ortasks etc. Refer to Section 4.10.2, “Schedule Criteria - Event Triggers” for details onhow to configure Event Triggers.

Event Trig-gers

Collectors within a Realtime workflow never stop executing. Scheduling such workflows toperiodically execute, may seem unnecessary. However, if the workflow does abort, then theworkflow is started again at the next planned schedule time. The same applies to Batch workflowscontaining Inter Workflow Collection nodes.

5.4.1.4.2. Error Tab

The Error tab contains configurations related to error handling of the workflow.

Note that only a short summary of the functionality description is given. Please refer to ConvergentMediation Engine™ Error Correction System - User Guide, for further information.

61

Core 4.1

Figure 57. Workflow Preferences dialog, Error tab.

Since the batch concept does not exist for Realtime workflows, this option isvalid for Batch workflows only.

Cancel Batch Be-havior

• Abort Immediately - If enabled, the workflow immediately aborts on thefirst Cancel Batch message from any node in the workflow. The erroneousdata batch is kept in its original place and must therefore be moved/deletedmanually before the workflow is started again.

• Abort After X Consecutive Cancel Batch - If enabled, the value of X indicatesthe number of allowed Cancel Batch calls, from any node in a workflow beforethe workflow will be aborted. The counter is reset between each successfullyprocessed data batch. Thus, if 5 is entered, the workflow will abort on the 6thdata batch in a row that is reported erroneous. All erroneous data batches, butthe last one, are removed from the stream and placed in ECS.

• Never Abort - The workflow will never abort. However, as with the othererror handling options, the System Log is always updated for each CancelBatch message, and data is sent to ECS.

Since the batch concept does not exist for Realtime workflows, this option isvalid for Batch workflows only.

ECS Batch ErrorUDR

When cancelling a batch, a UDR holding information on selected MIMs may beassociated with the batch. This is useful when reprocessing a batch from ECS,the fields of the Error UDR will appear as MIMs in the collecting workflow.

The batch UDR may be populated from Analysis or Aggregation nodes as well.This is useful in case of wanting to enter other values than MIMs.

An Error Code as defined in the Error Correction System Inspector.Error Code

The error UDR to be associated with the batch.Error UDR Type

A list of all the fields available for the selected Error UDR Type.UDR Field

A list of selected MIM resources to map to the Error UDR Type fields.MIM Resource

62

Core 4.1

Holds information on what MIM resources to be logged in the System Log whenthe workflow aborts or sends UDRs and batches to ECS. These values may alsobe viewed from ECS (the MIM column).

Logged MIMs

The most relevant resources to select are things that identifies the data batch, suchas the Source Filename, if available.

5.4.1.4.3. Audit Tab

The Audit tab in the Workflow Preferences window defines what data to enter in the user table(s)as defined in the Audit Profile. Either MIM values are entered from this window, or anything may besent on with the APL audit functions (from Analysis or Aggregation nodes). This tab is only valid forBatch workflows.

Figure 58. Preferences dialog, Audit tab.

Turns on audit for the workflow.Enable Audit

A list from which an existing Audit Profile is selected. Please refer to Sec-tion 5.1, “Audit Profile” for further information.

Profile

If enabled, audit is logged for cancelled batches. Note, counter column valueswill be reset for cancelled batches, while value columns will keep their valuethey had when the batch was cancelled.

Log Audit on CancelBatch

The name of the columns as defined in the selected Audit Profile.Column Name

The type of the columns as defined in the Audit Profile.Type

Indicates if the column will be populated using APL functions, or via config-uration using existing MIM values. Valid options are:

Method - The column will be populated through any of the APL functionsauditAdd or auditSet. The first is used on Counter columns, and thelatter on Value columns.

Assign By

MIM - The column will be populated with the MIM value selected in theMIM Resource column.

System - Reserved for the Transaction Id.

Valid only if MIM is selected in Assign By. Double-click the cell to displaya dialog from which existing MIM values may be selected.

MIM Resource

5.4.1.4.4. Monitor Tab

In the Monitor tab, monitoring options such as how debug is handled, user defined throughput calcu-lation and mappings for the Workflow Manager are defined.

63

Core 4.1

For Workflow Manager Views, MIM values and user defined events may be mapped to columns inthe Workflow Manager. These columns may contain useful real-time information, such as the nameof the file currently being processed or user defined messages.

Figure 59. Preferences dialog, Monitor tab.

A workflow containing agents that sends debug information, for instance using thedebug command in APL, can be configured to run without debug information orto forward it to either an event or a file.

Debug Settings

Debug Output • File - Debug information is logged to the file <pico.tmpdir>/debug/<workflow name>. If the file exists, it will be reset. The parameter< p i c o . t m p d i r > i s c h a n g e d i n$MZ_HOME/etc/executioncontext.xml.

• Event - Debug information is logged as an Event. The output is visible whenthe workflow is in Monitor mode.

Debug Default • On - Turns debug on.

• Off - Turns debug off.

It is highly recommended to turn off debug, other than for testing purposes,since it may have major impact on performance of the running workflow.

Convergent Mediation Engine™ has a built in algorithm to calculate the throughputof a running workflow. Initially it locates the first node that delivers UDRs which

ThroughputCalculation

is typically the decoder. It then counts the number of passed UDRs per second. Incase no UDRs are passing through the workflow, the first node delivering raw datawill be used.

It is possible to override the selection of what MIM value to be used to calculatethroughput. The MIM value should represent how much data that has entered theworkflow since start (for a batch workflow this means since the start of the currenttransaction). The MIM value must be of numeric type and be dynamic as it willchange as the workflow is running.

There are two different types; MIM Events or Event Categories. The latter must bedefined in the Event Notification window first.

Event Categories are assigned values during workflow execution from nodesutilizing APL code (Analysis and Aggregation). The function to use is dis-patchMessage.

Value

64

Core 4.1

The name of the column to which the Value will be mapped. Dynamic Columnsmust be defined in the Workflow Manager first. Refer to Section 6.5.2.3, “Columns”for further information.

Column

5.4.1.5. Displaying Validation Details for a Workflow

When a workflow is saved it will be silently validated and if some of its configuration is invalid ormissing, a dialog will state this and asks whether to still save the workflow or not. The workflowsymbol on the window border and in the Open dialog will be marked with a red cross.

If the erroneous configuration is not evident, the button Validate... in the View menu will display adialog that for each node describes details, such as if configuration data is missing, if routes are missingor if referenced MIM resources are no longer available. If configuration data is missing for a specificnode, the validation details may be seen by opening the node's configuration and press OK.

For performance reasons, the configuration of each node is not fully validated when Validate... (orSave) is selected. It expects each node to validate itself when its configuration window is confirmed.However, there are external entities that might have been changed since the node configuration windowwas confirmed, such as route names or node names. Therefore the Workflow Editor validates suchentities again.

5.4.1.5.1. Validating Workflow Templates

Validation of a workflow template is treated somewhat differently. Since there are most likely instancesdepending on the template configuration, they may be made invalid as a result from changes made tothe template. Selecting Validate... after changing a template will validate all its instances. The resultis displayed in a dialog. The user is expected to correct the instances' configurations manually.

5.4.1.6. Maneuvering Nodes

Agents are added to the Work Area either by dragging icons displayed in the tabs at the bottom of theeditor or by selecting them from the Agent menu. If dragged, they are positioned where the left mousebutton is released. Otherwise they will appear in the upper left corner of the Work Area. Once deployedin the Work Area, the agents are referred to as nodes since many nodes may represent the same agent.

Nodes are moved inside the Work Area using the drag technique. If routes are attached to a movingnode they will follow. A node is removed from the Work Area by selecting the node (clicking it) andselecting Cut from the Edit menu or by pressing the <Delete> key on the keyboard.

Nodes and their configurations may be copied using the Copy and Paste functions in the Edit menu.Select the source node followed by Copy and then Paste. This will deploy a copy of the selected nodeand its configuration in the editor.

5.4.1.6.1. Workflow Options Dialog

To change the outlook of the workflow, select Workflow Options... from the Edit menu to displaythe following dialog. These options are only esthetical and do not affect the functionality of theworkflow.

Figure 60. Workflow Options dialog.

Holds options on how to display the grid. Invisible is default.Grid Type

Changes the grid density. A large number will increase the distance betweennodes.

Grid Size

65

Core 4.1

Sets the style of the routes in the workflow. The Route Type may also bechanged for a specific route only, not affecting the entire workflow. This isdone by right-clicking the route in the workflow and selecting Route Types.

Default Route Type

5.4.1.7. Maneuvering Routes

To create a data flow, nodes need to be connected to each other. Two nodes are connected by firstclicking the center of the source node. Without releasing the left mouse button, the pointer is movedto the target node and released there. This maneuver will create a connection (route) between the twonodes indicating the data flow.

A route is removed by selecting the route and selecting Cut from the Edit menu or by pressing the<Delete> key on the keyboard.

5.4.1.8. Naming Nodes and Routes

When a node is deployed into the Work Area it receives a default name located underneath it. Thesame applies to routes when they are added. These names may be modified to ease identification inmonitoring facilities and logs.

A node name or a route name is modified by selecting the name (clicking on it) and typing a new name.Node names may also be edited in the configuration dialog, displayed when a node is double-clicked.Node names and route names must be unique within a workflow and may only contain the a-z, A-Z,0-9, "-" and "_" characters.

5.4.1.9. Sizing Workflows

The Workflow Editor toolbar contains a component to zoom a workflow. For very large workflowsthis is a good way to see the complete flow. The default value is 100 (%). Use the arrow up or downbuttons to enlarge/shrink the workflow. Note that the scaling will have no impact on the configurationwindows, or on the Workflow Editor itself.

5.4.1.10. Node Configuration Window

Each node in the workflow has a specific configuration window named after the agent type that isdisplayed when a node is double-clicked. The dialog contains configuration information located intabs.

The left-most tab contains configuration parameters that are unique for each agent while possible ad-ditional tabs are of a more generic kind and may be recognized in other agents windows. A descriptionof components in the first tab is available in the user guide of the respective agent, while the remainingtabs are described in Section 5.4.3, “Agent Services”.

5.4.2. Monitor Mode

The Workflow Editor in Monitor mode controls workflow execution and gives a detailed view ofwhat the workflow is doing. When a workflow is running, states and events may be monitored for eachnode. Having a workflow in Monitor mode does not imply exclusive rights to start or stop it. Therefore,the workflow may be activated and deactivated by another user while it is on display. The monitoralso allows the user to change configuration for agents that can be reconfigured while running, andthe ability to send commands to agents. A command can be telling an agent to flush or reset data ithas in memory. Please see the agent user guides for more information on what commands that areprovided.

To switch back to Edit Mode, select the button in the right corner of the toolbar area.

66

Core 4.1

Monitor mode can be enter if a workflow is valid, or an older version of the workflow is active.When switching to Monitor mode, the active version of the workflow is displayed.

Figure 61. Workflow Editor window in Monitor mode.

5.4.2.1. Window Layout

In Monitor mode the Workflow Editor is divided into the following areas.

5.4.2.1.1. Menu Bar Area

The menu bar contains the following menus and options. Items, not listed are considered to be ofgeneral type, refer to Section 4.9, “Menu and Toolbar Items” for further details.

The menu area, during run-time, also contains the IP address of the machine it is executing on, whichEC is used, and the workflow's current execution state, updated in real-time. Refer to Section 5.4.2.7,“Workflow States” for details on workflow states. When the workflow is not executing but has beenactivated, the next scheduled time is displayed in this area.

Activates the currently loaded workflow.Edit menu Activate

Refer to Section 5.4.2.2, “Activating and Deactivating a Workflow”for details.

Deactivates the currently executing workflow.Edit menu Deactivate

Refer to Section 5.4.2.2, “Activating and Deactivating a Workflow”for details.

Monitors events for selected nodes.Event menu Events for SelectedNodes

Refer to Section 5.4.2.3, “Viewing Agent Events” for details.

Monitors events for all nodes.Event menu Events for AllNodes

Refer to Section 5.4.2.3, “Viewing Agent Events” for details.

Updates a running workflow with new configuration that has beenentered in monitor mode. Agents that supports update of configura-

Dynamic Update

tion while they are running will be updated with the new configura-tion. Once the update has been introduced to the workflow the userwill be shown if anything was affected. An update of a workflow inthe monitor will result in a new workflow version.

67

Core 4.1

Turns on or off debug information for an active workflow. Thisfunction can only be used when the workflow is active.

Toggle Debug Mode On/Off

Sets the Workflow Editor back to Edit mode.Edit

5.4.2.1.2. Work Area

Contains the drawn workflow in view only mode, hence, it is not possible to modify or view any con-figurations. The state of each node is updated in real-time and is displayed above each node. Refer toSection 5.4.2.8, “Agent States” for details on agent states.

5.4.2.1.3. Event Area

Contains columns holding events generated by the nodes in the workflow. The maximum number ofdisplayed events is 1000. If an already executed workflow is loaded into the Workflow Editor inMonitor mode, the last row generated by each node will be visible in the node's corresponding column.If debug is used, this information is output in real-time in this area.

It is possible to resize or move a column by clicking and dragging either on the title or on the separatingline between the two columns.

Not all agents publish events. Refer to the User Guide for the respective agent for more inform-ation.

5.4.2.2. Activating and Deactivating a Workflow

If a workflow is not already running it may be started by selecting Activate from the Control menu.This action will schedule the workflow for execution, according to its scheduled configuration.

After activation of the workflow, the user may be asked to reload the version. This is the casewhen another user has modified the workflow or when the user entered Monitor mode and anolder version of the workflow was running.

If a workflow is running or is scheduled to do so, it may be stopped by selecting Deactivate from theControl menu. If this measure is not sufficient to stop the workflow, the execution context on whichit is running must be restarted. Note, this only happens in rare occasions, for instance when an infiniteloop has accidentally been introduced in an Analysis agent, or in case a network element connectedto a Collection agent has terminated in a bad state.

When selecting Deactivate for Batch workflows, a dialog is displayed asking the user to choose oneof two ways to terminate the workflow; Batch or Immediate. If selecting Batch, the workflow is ter-minated when the active batch has finished. If selecting Immediate, the workflow is terminated withoutfinishing the current batch.

Deactivating a workflow may result in the abortion of some nodes, depending on connectionsto external systems, such as the FTP Forwarding agent. For these nodes, the connection is takendown immediately, without waiting for acknowledgment from the connected network elements.When the workflow is activated again, the node will resume processing without losing or duplic-ating data.

5.4.2.3. Viewing Agent Events

While running, many agents report progress in terms of events. These events may be monitored in theevent area. The following steps have to be performed in order to view these events.

68

Core 4.1

1. Select the nodes in the workflow for which events are desired. This is achieved by clicking eachnode while holding down the <Ctrl> key on the keyboard.

2. Select Events for Selected Nodes from the Event menu.

3. If desired, rearrange the size and position of each column.

If events are desired for all nodes in the workflow, step 1 and 2 may be replaced with selecting Eventsfor All Nodes from the Event menu.

5.4.2.4. Agent Configuration

Some agents allows parts of them to be reconfigured while active. When double clicking the agent inmonitor mode, the agent's configuration is shown in the displayed window if the agent can be recon-figured. When the configuration has been updated it can be committed to the active workflow usingthe Dynamic Update button in the toolbar.

5.4.2.5. Agent Command

There are agents that can be controlled while active. Such agents may provide a user interface for in-teracting with the agent or just rely on the wfcommand in mzsh. If the agent provides a commanduser interface this is shown in window displayed when double clicking the agent. The Command tabholds the configuration for the command and Execute runs the command. The result of the commandwill then be shown once the command is complete.

5.4.2.6. Viewing Abort Reasons

In most cases if a workflow has aborted, one of its nodes will have the state Aborted displayed aboveit. Double-clicking such a node will display a dialog containing the abort reason. Also, the SystemLog holds valid information for these cases.

Figure 62. Status details for an aborted node.

Selecting Show Trace... will give detailed explanations suitable when consulting Nokia Global Support.

5.4.2.7. Workflow States

A workflow can have one of the following states:

The workflow is about to be distributed to an Execution Context and loaded.Loading

69

Core 4.1

The workflow has been loaded and initialized.Created

The workflow and at least one of its agents are running.Active

The workflow is probably running but Convergent Mediation Engine™ failed tocommunicate with it the last time. An Unreachable workflow may require manual

Unreachable

attention in case the workflow is not running any more. Please see Section 3.6,“Execution and Monitoring” for more information.

The workflow is waiting for scheduled criteria to commence. The workflow is notloaded and none of its agents are running. The state is suffixed with any of thefollowing information:

Scheduled: <cri-teria>

• Scheduled: <date>

• Scheduled: By trigger only

• Scheduled: <date>, and by trigger

This is an intermediate state reported before Stopped or Scheduled and indicatesthat the workflow has finished processing successfully. This state is meant to be

Executed

used when configuring event triggers to depend on other workflows finishing theirtasks first. Stopped and Scheduled cannot be used in this case, since an Abort(unsuccessful termination) may have preceded these states. If a user manually de-activates a workflow, no Executed state precedes Stopped.

The workflow has successfully stopped and will not be automatically started again.Stopped

At least one of the nodes in the workflow has aborted. The error reason may betracked either by double-clicking the aborted node, or by examining the SystemLog.

Aborted

In rare situations the workflow engine itself detects problems resulting in anabortion, that is, no node is causing the abortion. When locating such errors in theSystem Log, the system node must be examined instead of the workflow node.

Although a workflow has aborted, its scheduling will still be valid. Thus, if it is scheduled toexecute periodically, it will be automatically started again the next time it is due to commence.This since the cause of the abortion might be a lost connection to a network element, whichcould be available again later. Therefore, a periodically scheduled workflow, which has aborted,is treated as Active until it is manually deactivated.

5.4.2.8. Agent States

The state of a node is displayed above it. A node may have one of the following states:

The agent is starting up. No data may be received during this phase. This state only existsfor a short while during workflow startup.

Created

The agent is started, awaiting data to process. This state is not available for a Realtimeworkflow.

Idle

The agent has received data and is executing.Active

The agent has successfully finished processing data.Stopped

The agent has terminated erroneously.Aborted

The error reason can be tracked either by double-clicking the node or by examining theSystem Log.

70

Core 4.1

5.4.3. Agent Services

The node configuration windows are optionally equipped with additional tabs holding configurationsfor different common services. This section describes the general services that are available in Conver-gent Mediation Engine™ . The user guide for each agent, will in turn describe what services it uses.

5.4.3.1. Filename Sequence

The Filename Sequence service is available on Collection agents that may want to collect files con-taining a sequence number in the filename. The sequence number is expected to be found on a specificposition in the filename and will have a fixed or dynamic size.

Figure 63. Filename Sequence tab.

Determines if the service will be utilized or not.Enable Filename Se-quence

The offset in the filename where the sequence number starts. The first characterhas offset 0 (zero). In the example filename TTFILE0001 the start positionis 6.

Start Position

The length of the sequence number, if the sequence number has a static length(padded with leading zeroes). If the sequence number length is dynamic thisvalue is set to 0 (zero).

Length

Example 4.

TTFILE0001-TTFILE9999 Length: 4

TTFILE1-TTFILE9999 Length: 0

If the Filename Sequence service is enabled, a value must be specified onwhich the sequence will wrap. This number should be larger than or equal toNext Sequence Number and it must be larger than Wrap To.

Wrap On

Value that the sequence will wrap to. This value must be less than than theWrap On value and less than or equal to Next Sequence Number.

Wrap To

The starting point in the sequence. This value is then updated, holding thenext sequence number to be processed. The value must be larger than or equalto the Wrap To value and less than or equal to the Wrap On value.

Next Sequence Num-ber

71

Core 4.1

If enabled, the node will log an informative message to the System Log whendetecting an out of sequence file, before deactivating. The Collection node

Warn On Out Of Se-quence

will not continue to collect any files upon the next activation, until either themissing file is found, or the Next Sequence Number is set to a valid value.

5.4.3.2. Filename Sort Order

The Filename Sort Order service is available on Collection nodes that may want to sort matched filesbefore collection. The service is utilized in two versions. One called Filename Sort Order and anotherone called Sort Order. The only difference between these services is that the Sort Order service alsohas the ability to sort on file modification timestamp.

The sort pattern is expected to occur on a specific position in the filename or to be located using aregular expression.

Regular expressions according to Java syntax apply. For further information, please refer to


Figure 64. Sort Order tab.

Determines if the service will be utilized or not.Enable FilenameSort Order

Type of method to locate the item (part of the filename) to be the target forthe sorting. This could be either Position that indicates that the item is located

Value Pattern

at a fixed position in the filename or Regular Expression indicating that theitem will be fetched using a regular expression.

If Position is enabled, this value points out the offset in the filename wherethe sorting item starts. The first character has offset 0 (zero).

Start Position

If Position is enabled, this value holds the length of the sorting item (part ofthe filename) if it has a static length (padded with leading zeroes). If the length

Length

of the sorting item (part of the filename) is dynamic, the default value zero (0)will be used.

If the filename does not end with a digit, the Regular Expression is used in-stead of Position if Numeric sorting is used and the Length is undefined.

If enabled, the sorting item is extracted from the filename using the regularexpression.

Regular Expression

72

Core 4.1

Example 5.

This locates the first digit sequence in a filename.

FILEA_1354.log FILEB_23.log FILEC_1254.log

Using the \d+ command will sort the files according to the following(or the other way around, depending on the chosen Sort Direction):

FILEC_1254.log FILEA_1354.log FILEB_23.log

Type of sorting. Could be either Alphanumeric or Numeric.Sort Type

If enabled, the sorting is not case sensitive.Ignore Case

Indicates if the sort order will be Ascending or Descending.Sort Direction

5.4.3.3. Filename Template

The Filename Template service is available to Forwarding agents responsible for creating a file. Theconfiguration contains MIM resources for all available nodes in the workflow, whose values may beused when constructing a filename for the outgoing file.

Since this service includes a selection of MIM resources from available nodes in the workflow, it isadvised to add all nodes to the workflow before the filename template configuration is completed.

Figure 65. Filename Template tab.

The list contains MIM resources or user defined values that will make up the filename. The order ofthe items in the list controls the order of the items in the filename.

73

Core 4.1

Figure 66. Filename Template tab, Add or Edit dialog.

Determines if the Value will be selected from a MIM resource.MIM Defined

Determines if the Value will be a user defined constant entered in the text field.User Defined

Allocated characters in the filename for the selected MIM resource (or user definedconstant). If the actual value is smaller than this number the remaining space will be

Size

padded with the chosen padding character. If left empty the number of characters al-located in the filename is equal to the MIM value or the constant.

Character to pad remaining space with if Size is set. If Size is not set this value is ig-nored.

Padding

Left or right alignment within the allocated size. If Size is not set this value is ignored.Alignment

Separating character to add to the filename after the MIM value or constant.Separator

Adds a timestamp to the filename in the selected way.Date Format

74

Core 4.1

Example 6.

Assume a workflow containing a Disk Collection node named DI1 and two Disk Forwardingnodes named DO1 and DO2. The desired output filenames from both Forwarding nodes are asfollows:

A-B-C.DATWhere A is the name of the Disk Forwarding node, B is the name of the currentlyprocessed file, C is the number of UDRs in the outgoing file and .DAT is a customer suffix. Itis desired that the number of UDRs in the files takes up six characters and is aligned right.

The following filename template configuration applies to the first node:

The following filename template configuration applies to the second node:

If two files FILE1 and FILE2 are processed where 100 UDRs goes to DO1 and 250 goes to DO2from each file, the resulting filenames would be:

DO1-FILE1-000100.DAT DO2-FILE1-000250.DAT DO1-FILE2-000100.DAT DO2-FILE2-000250.DAT

5.4.3.4.Thread Buffer

5.4.3.4.1. Multi-Threading - Batch Workflows

By default, each workflow has one active thread at a time. In case of having complex workflows witha lot of nodes, there is a possibility to configure multiple threads to speed up the processing. This isdone by defining a buffer where records may be queued up. All agents that receive UDRs can utilizethis functionality. Open up the Configuration window of the node, and select the Thread Buffer tab.

Figure 67. The Thread Buffer tab, found for Batch processing and Batch forwarding agents.

Enables the multi-threading functionality.Use Buffer

Debug to be used when trying out a suitable place to use the Thread Buffer. Thebusy and idle percentage of the node, is logged in the event area of the workflow at

Print Statistics

the end of each batch. Please refer to Section 5.4.3.4.2, “Analyzing the Debug Output”for a discussion of how to interpret the results.

75

Core 4.1

A UDR may be queued up while another thread is busy processing a reference to it. Workflowsrouting the same UDR on several routes, and involving further processing of its data, mustconsequently be reconfigured to avoid this. A simple workaround is to route the UDR to anAnalysis node for cloning before routing it to the other nodes (one unique clone per route).

Also, routing bytearrays into an agent utilizing this service will throw an exception. The threadservice must only be used when the incoming data is UDRs.

5.4.3.4.2. Analyzing the Debug Output

By default, a workflow is designed to only utilize one thread. Configuring a record buffer storage fora node will make it possible for yet another thread to be created. One thread will be populating thebuffer, and another pulling it for data. Adding yet another buffer for another node will add yet anotherthread and so on.

NodeNode with buffer

RecordBuffer

DeliveringThread

ProcessingThread

Figure 68. One buffer in a workflow will result in one extra thread being generated.

Turning on debug (Print Statistics) will log the busy and idle percentage of the threads utilizing thebuffer; busy for the delivering thread and idle for the processing thread. The values must be as lowas possible, and as equal as possible. The latter may be hard to achieve, since the nodes may differ toomuch in processing complexity. If possible, add and configure another node to take over some of theprocessing steps from the most complex node.

76

Core 4.1

6. Inspection MenuThis section describes the non agent specific Inspectors, found in the Inspection menu on the desktop.Inspectors that belong to a certain agent are described in its respective user guide.

6.1. Configuration BrowserThe Configuration Browser gives a view of all configurations in Convergent Mediation Engine™ .The user can easily filter what configurations to be shown, by selecting configurations of a specifictype or owned by a specific user. By default all configurations are displayed. The browser supports aset of operations on a configuration, such as move, rename, copy, validate and set permissions.

From the Configuration Browser it is possible to launch a window where all active and historicconfigurations are shown. An active configuration is currently present in the system while a historicconfiguration has previously been deleted. Using the Configuration Tracer, the user can restore deletedconfigurations.

Figure 69. The Configuration Browser.

This option allows the user to create a new folder. A new dialog isdisplayed, where a name for the folder may be entered.

File menu New Folder...

This option is enabled when a configuration has been selected inthe browser. When selected, the corresponding configuration isloaded into the window.

Edit menu View/Edit Configur-ation...

Selecting this option will clone the configuration. If the configura-tion is currently locked this action is not permitted. The clone mustbe given a unique name for that folder.

Edit menu Copy...

Selecting this option deletes the selected configuration. Some con-figurations may have relational data associated. In that case the user

Edit menu Delete...

is asked if that data is to be left or removed from the system. Note,referenced objects may not be deleted.

77

Core 4.1

Figure 70. Delete Relational Data window.

Changes the name of the selected configuration. Take special pre-caution when renaming configurations. If, for instance, an APL

Edit menu Rename...

script is renamed, workflows that are using this script will becomeinvalid.

This will move the selected configuration to another folder.Edit menu Move...

Encrypts the selected configuration.Edit menu Encrypt...

Decrypts the selected configuration.Edit menu Decrypt...

Removes all historic entries for the selected configuration. Theversion number will not be affected by this operation.

Edit menu Clear ConfigurationHistory...

Alters the permissions for the selected configuration(s). Please referto Section 6.1.4, “Configuration Permissions” for details on thisoption.

Edit menu Set Permissions...

Validates the configuration. A validation message will be shownto the user.

Edit menu Validate...

Launches the Configuration Tracer. Please refer to Section 6.1.3,“Configuration Tracer” for details on this option.

Edit menu ConfigurationTracer...

This menu contains all Convergent Mediation Engine™ configura-tion types and allows the user to filter the current view in the Con-figuration Browser to only display a set of types.

View menu Configuration Types

This menu contains all Convergent Mediation Engine™ users andallows the user to only display configurations that are owned by aset of users.

View menu Owners

6.1.1. Folder

On the left side of the Configuration Browser all folders in the system are displayed. The contentsof the folder is shown in the configuration table. Right-clicking a folder, will display a pop-up menu,allowing the user to rename or delete the folder. The built-in Default folder cannot be modified.

6.1.2. Configurations

Displays all configurations for the selected folder. Right-clicking a configuration launches a pop-upmenu where the user may perform the same action as from the Edit menu.

Icon representing the application type.Type

The name of the configuration.Name

Displays if the configuration is currently locked.Lock

The permissions granted to the current user for the configurations is shown as R(Read), W (Write) and X (Execute). Also, if the configuration is encrypted, an E is

Perm

shown. For further information on permissions, please refer to Section 6.1.4,“Configuration Permissions”.

The name of the configuration owner.Owner

78

Core 4.1

The user that made the last modifications to the configuration.Modified By

The date when the configuration was last modified.Modified Date

6.1.3. Configuration Tracer

The Configuration Tracer displays current and historic (deleted) configurations. Active and Historicare shown in different views. The window allows the user to see more information than the Configur-ation Browser shows such as comment for the latest version and the key used internally to identifythe configuration. Note, the latest version of the configurations is shown.

Figure 71. The Configuration Tracer.

This option is enabled when in active mode and a configuration hasbeen selected. When selected, the corresponding configuration windowis launched and loaded.

Edit menu View/Edit Con-figuration...

If View Mode is set to Historic, the old configuration is restored. Whenthe configuration is restored it will also be validated and if it is invalidthe user must confirm that the configuration to be restored as invalid.

Edit menu Restore...

Active mode shows the same configurations as those displayed in theConfiguration Browser. Historic mode shows configurations that have

View Mode

been removed from the system. The user may select to restore such aconfiguration.

Icon representing the application type.Type

Name of the folder, containing the configuration.Folder

The name of the configuration.Name

The configuration version.Ver

The internal key used by Convergent Mediation Engine™ to identifythe configuration.

Key

The user that made the last modifications to the configuration.Modified By

The date when the configuration was last modified.Modified Date

The last comment when the configuration was saved.Comment

6.1.4. Configuration Permissions

To alter permissions on a configuration, select Set Permissions... from the Edit menu to display thefollowing dialog.

79

Core 4.1

Figure 72. The Permissions dialog.

The permissions only apply if the user is granted access to the affected applications. Permissions aregiven to a user group. That means users may be granted different access to the same configuration.

Granted ActionsXWR

Only allowed to view the configuration, given the useris granted access to the respective application.

--R

Only allowed to delete the configuration. This is donethrough the Configuration Browser.

-W-

Only allowed to execute the configuration. For workflowsthis is done using the Task Manager. All components

X--

used in a workflow need to have execute permissions inorder to run the workflow. This is common "Black Box"behavior.

Allowed to view the configuration, given the user isgranted access to the needed application, and delete theconfiguration.

-WR

Allowed to execute the configuration and to delete it.XW-

Allowed to view the configuration, given the user isgranted access to the needed application, and to executethe configuration if possible.

X-R

Full access.XWR

6.2. Pico ViewerThe Pico Viewer window displays a list of all pico clients currently online in the system. Pico clientsare grouped in pico-started instances. For instance, EC1 in the following figure is a pico-started instance,while its content, Batch Storage and Execution Context is referred to as pico clients.

The Pico Viewer is available from the Inspection menu on the desktop.

80

Core 4.1

Figure 73. Pico Viewer window.

Name of the Convergent Mediation Engine™ pico instance/clients. For each andevery of one the pico instances - Platform, Desktop, EC, etc - a JVM (Java VirtualMachine) is started.

Pico Instance

Allows the user to remove a stand-alone Execution Context from the system in caseit is unreachable. The platform will never automatically unregister such instancesince it is accepted that it can reside on an unreliable network.

The time the pico instance was started.Start Time

Used, available, and maximum memory on the hosting JVM.Memory

The time it took for the local desktop to invoke a ping on the pico instance.Response [ms]

6.2.1.Tool-Tip Information

Resting the mouse pointer on any of the objects in the Pico Instance column, will display informationon the OS and JVMs on which it is running.

Figure 74. The tool-tip of a Pico Instance.

6.3. System LogBy default, events and errors encountered in the Convergent Mediation Engine™ system are saved inthe System Log. The System Log handles duplicate events within a time frame. Therefore every eventand error has a first and last occurred date, as well as information of how many times it was repeated.The System Log Inspector allows the user to browse and purge this log.

81

Core 4.1

There are three types of categories:

• System related log entries, generated by the core platform servers.

• Workflow related log entries, generated by the agents.

• User related log entries, originating from user actions.

If, for instance, a workflow aborts, the reason for the abortion may be tracked through this utility.

To display the System Log Inspector , select System Log... from the Inspection menu on the desktop.

Figure 75. The System Log Inspector.

Initially, the window is empty and must be populated with data using the Search System Log dialog.Refer to Section 6.3.1, “Searching the System Log” for further information.

Selects all entries in the group currently displayed. A group is selected fromthe Show Entries list.

Edit menu Select All

Displays the Search System Log dialog where search criteria may be definedto limit the entries in the list. Refer to Section 6.3.1, “Searching the SystemLog” for further information.

Edit menu Search...

Displays the Stack Trace Viewer window. This information must alwaysbe included when contacting Nokia Support in cases involving error messages.

View menu ShowTrace...

Entries matching the search criteria are displayed in groups of 500. ShowEntries contains a list of all groups of 500, from which one is selected. Note

Show Entries

that the full content of the log messages for a group is fetched from the data-base once the group is selected. This to have as little impact on the overallperformance of the system as possible.

The severity of the message, could be any of the following:Sev

• I (information) - An informative message which is logged, for instance,when a user logs in or a workflow is activated.

• W (warning) - A warning message is also informative, but is consideredbeing slightly more serious than an information message. A warningmessage is logged, for instance, when a workflow sends data to ECS.

• E (Error) - An error is logged when any part of the system fails, for in-stance, when a workflow aborts. Double-clicking an error message willdisplay the Stack Trace Viewer. This information must always be in-cluded when contacting Nokia Support, in cases involving error messages.

82

Core 4.1

• D (Disaster) - Is usually never used, other than possibly for user definedagents.

The date when this message last occurred.Last Occurred

Shows how many times this message has occurred.Repeated

Indicates which part of the system the message originates from; user, systemor workflow.

Area

The name of the workflow/node from which the message originates.Workflow/Node

The message. Note that selecting an entry from the list will display its fullcontents.

Message

The date when this message first occurred.First Occurred

If an entry is selected from the list, further details about it is displayed in thisarea.

Message Area

6.3.1. Searching the System Log

In order to more easily track down specific entries, the Search System Log dialog offers the possibilityto constrain the entries that are displayed in the list.

The dialog is displayed by selecting Search... from the Edit menu.

Figure 76. Search System Log dialog.

83

Core 4.1

For an entry to be displayed in the list, it has to pass all of the following filters.

Which part of system that reported the entry. At least one must be enabled.Log Area

Period between what dates entries will be viewed. A few predefined options areavailable. If none is selected, all are considered.

Period

If User Defined is selected in the Period list, all entries reported after the selecteddate will match. If not, all entries before the To date will match.

From

If User Defined is selected in the Period list, all entries reported before the selecteddate will match. If not, all entries after the From date will match.

To

If neither From nor To is selected, all entries will match.

Type of severity. At least one type must be enabled.Severity Type

Contains options to filter out specific workflows and/or node names. If disabled, allworkflows/nodes will match.

Workflow

If enabled, all activities performed by the selected user will match. If disabled, alluser activities will match.

User

This only applies to events invoked by users, such as inserting, updating anddeleting data in the system. Thus, it does not apply to workflow ownership andcan therefore not be utilized to filter out a specific user's workflows.

Log entries may be scanned for occurrences of specific messages.Log Message

Text field where a string is entered. All messages containing this string will be extrac-ted.

Message

6.3.2. Printing the System Log

Detailed information, or a brief description of one or many System Log entries may be printed fromthe Print System Log dialog. Select the entries of interest, before selecting Print. Selections are madein one of the following ways:

• Single select - An individual row is selected by clicking on it.

• Browse select - A continuous number of rows are selected by first clicking it, then while holdingdown the <Shift> key clicking the last row.

• Extended select - Individual rows are selected by clicking them while the <Ctrl> key is held down.

By selecting either the Workflow, User or System folder, all log entries grouped under them will beprinted.

Selecting Print... from the Edit menu, displays the dialog.

Figure 77. Print System Log dialog.

Will print only a short summary of each selected entry. The printed informationis the same as displayed on each row in the browser.

Headers Only

84

Core 4.1

Will print detailed information about each selected entry, one page for each.The information printed is the same as displayed in the Message Area of theSystem Log Inspector.

Full Details

Will include the stack trace for the log entries where available (that is, Errortype messages).

Include Stack Trace

6.4. System StatisticsConvergent Mediation Engine™ constantly collects information from the different sub-systems andhosts in the system. This information is among other things used for load balancing. Using the SystemStatistics window collected statistics may be viewed.

There are three different kinds of statistics: Host, Workflow and Pico instances

6.4.1. Host Statistics

Convergent Mediation Engine™ collects statistics from the different machines hosting a Platform orExecution Context. For instance, the load of the CPU or the number of context switches. This is calledhost statistics.

Convergent Mediation Engine™ uses an external binary called vmstat to collect the information.This binary must be installed to have statistics collected and to perform load-balancing work forworkflows. The following list holds all values collected from each host. On newer operating systemssome of these may not be available for collection due to changes in the kernel of the operations system.

• CPU User Time - This value shows how much time was spent in non-kernel specific code. Thisvalue is in percentage. 100% means that all processing power is spent. See CPU System Time aswell.

• CPU System Time - This value shows how much time was spent in kernel specific code, such asscheduling of different processes or network transfer. This value is in percentage. 100% meansthat all processing power is spent.

• Context Switches - The number of context switches per second. A context switch occurs whenone thread hands over information to another thread. The more context switches, the less effectiveand scalable the system will be.

• Swapped To Disk - The amount of data that was swapped out. A large value indicates that thesystem does not have enough RAM to manage the memory requirements of the different processes.

• Swapped In From Disk - The amount of data that was read from swap.

• Processes Waiting For Run - Shows how many processes that are waiting to be run. A highnumber indicates that the machine is not fast enough to manage the load.

• Processes Swapped Out - Processes that has been persisted in swap due to not enough availablememory or due to aggressive management of the memory layer.

• Processes In Sleep - The number of processes that are presently not doing anything.

6.4.2. Workflow Statistics

When a workflow is running, Convergent Mediation Engine™ continuously collects information onhow much data that is processed. This (throughput) is measured per second, that is, with what speeddata is processed. In most cases this number will be UDR per second but in the case where there areno UDRs routed in the workflow it will instead be raw data per second. The throughput is sampledevery five second.

85

Core 4.1

Throughput is also defined as a MIM value for the workflow in case it would be convenient to delegatethis to external systems, or to generate alarms if the throughput falls to low. For further information,please refer to Throughput Calculation.

The System Statistics window also calculates the number of simultaneously running workflows.

6.4.3. Pico Instance

Every minute Convergent Mediation Engine™ collects memory information from the different Javaprocesses defining the platform and the Execution Context. This information shows how much memoryis used and how much memory that can be used for the running process

• Used Memory - Shows how much memory is currently allocated by the running process. As Javais a language utilizing garbage collection, this number may very well get close to the maximummemory limit without it being a problem for the running process. However, if used memory isclose to the maximum limit for a long time the process needs more memory. This value is in bytes.Please refer to the -Xmx and -Xms properties defined in the XML file defining the process.

• Maximum Memory - The amount of memory that the process can use.

• Garbage Collection Count - How many times the garbage collector ran since the last time statisticswas collected.

• Garbage Collection Time - For how long time the garbage collector ran since the last time statisticswas collected.

• Thread Count - The number of allocated threads.

6.4.4. Viewing the System Statistics

There are three different resolutions on which statistics are collected.

This is the most precise value. It is saved every minute.Minute

These values are calculated every hour and are a sum of the minute values for that hour.Hour

Day values are calculated by the corresponding statistics task and is a sum of minute valuesfor that day.

Day

There are three different types of values.

The lowest value that was sampled.Minimum

The average value that was sampled.Average

The highest value that was sampled.Maximum

To display the System Statistics window, select System Statistics... from the Inspection menu onthe desktop.

86

Core 4.1

Figure 78. The System Statistics window.

Initially, the window displays Host and Workflow statistics for the last seven days. Refer to Sec-tion 6.4.2, “Workflow Statistics” for further information.

Displays the Search System Statistics dialog where search criteria may be defined.Edit menuSearch...

Shows the start and end date of the current statistics.Date:<from> to<to>

A list of the different values that may be displayed.Host View -Value

A list of all hosts matching the search criteria. Hosts are shown in different colors. Ifthere are several matching hosts, all may be displayed at the same time.

Host View -Hosts

A list of the different values that may be displayed.Pico InstanceView - Value

87

Core 4.1

A list of all pico instances, matching the search criteria. Pico Instances are shown indifferent colors. If there are several matching instances, all may be displayed at thesame time.

Pico InstanceView - PicoInstance

A list of the different values that may be displayed.WorkflowView - Value

A list of all workflows matching the search criteria. Workflows are shown in differentcolors. If there are several matching workflows, all may be displayed at the same time.

WorkflowView - Work-flow

At the bottom of the window there is a scroll bar and a zoom in and out button. Thesecomponents allows the user to focus on a particular time window within the result.

Scroll, ZoomIn/Out

Using the scroll bar allows the user to step back and forth in time to study how thevalue changed.

Figure 79. System Statistics window - Stepping in time.

Determines how to present the selected values: minimum, average and maximum.Default is to show Average.

Value

6.4.5. Searching System Statistics

In the Search System Statistics dialog, search criteria may be defined in order to single out the Stat-istics of interest.

88

Core 4.1

Figure 80. Search System Statistics dialog.

Specifies what part of the statistics to be shown. For instance, to only view host statisticsjust select Host.

View Mode

Specifies what resolution to be used. Minute will give more precise values but will re-quire much more from the server when it locates all the statistics.

Resolution

Host - If enabled, the statistics originating from the host will be displayed.Criteria

Pico Instance - If enabled, the statistics originating from the pico instance will be dis-played.

Workflow - If enabled, the statistics originating from the Workflow will be displayed.

Period - If configured, only the values within the selected period will be displayed.

If several criteria are enabled, an absolute match will be displayed. For instance,if Host and Workflow is specified as well as Period, only the time for whichthere are both workflow measures and host measures is displayed.

6.5. Workflow ManagerThe Workflow Manager allows monitoring and maneuvering of multiple workflows. When a workflowis fully configured for execution and is saved, it will appear in the Workflow Manager. New workflowsare not automatically added to the list. To get the list refreshed, either the Workflow Manager windowmust be reopened or the refresh button must be pressed.

To display the Workflow Manager, select Workflow Manager... from the Inspection menu on thedesktop.

89

Core 4.1

Figure 81. The Workflow Manager.

Items not listed are considered to be of general type, refer to Section 4.9, “Menu and Toolbar Items”

Displays a dialog that, on confirmation, schedules selected workflows forexecution. If workflows that are already running, are invalid or locked areselected, they will not be part of the presented list.

Edit menu Activate

Displays a dialog that, on confirmation, will stop execution of the selectedworkflows.

Edit menu Deactivate

Displays the Workflow Editor window in Edit mode containing the selectedworkflow(s).

Edit menu WorkflowEditor...

Displays a dialog where user defined columns may be entered. These are se-lected when defining Views and assigned when configuring a workflow.Columns may be populated with any value.

Edit menu Columns...

Selecting Add will display a dialog where a suitable name for a column maybe added. Refer to Section 6.5.2.3, “Columns” for further information.

A selection of workflows and/or folders may be selected for managing, aswell as which information to monitor in the columns. These criteria defines

View

a Workflow View and is configured in the Workflow View Editor (selectWorkflow View... from the Configuration menu). Please refer to Sec-tion 6.5.2, “Workflow Views” for further information.

Name of executable workflows. In case of being an Instance workflow, it willbe prefixed with the name of the Workflow template (within [] brackets).

Workflow

Double-clicking it, will display the selected workflow in a Workflow Editor.

90

Core 4.1

State of the workflow. Refer to Section 5.4.2.7, “Workflow States” for detailson workflow states. If the workflow has a Scheduled state, the next plannedexecution time is displayed.

State

If the state of a workflow is double-clicked, a new window is displayed withmore detailed information.

Name of the host that the workflow is running on, and the numerical ID ofthe Execution Context (appended after ':'). This information is only displayedwhen the workflow is active.

Execution Context

The name of any user defined column - a so-called Column. Refer to Sec-tion 6.5.2.3, “Columns” for further information.

<any>

If a column is not applicable for an item in the view, the element will be left empty. For instance,a Column may be workflow specific.

6.5.1. Deactivation Issues

When a deactivation request is issued from the Workflow Manager (or directly from the WorkflowEditor), different dialogs appear depending on the type of workflow to deactivate.

Some agents are designed to wait for acknowledgment from sources they communicate with. Thus, astop request may take a while before acknowledged. In case of a network element connected to aConvergent Mediation Engine™ Collection agent has terminated in a bad state, causing the Collectionagent to hang, the Execution Context on which the workflow is running must be restarted.

6.5.1.1. Realtime Workflows

Realtime workflows are deactivated immediately, accepting no more input data. UDRs already withinthe workflow, are fully processed before the workflow deactivates.

In case an Inter Workflow Forwarding node is included in the workflow, the last file might be incom-plete. For these cases, the error handling is taken care of by the corresponding Inter Workflow Collectionnode.

6.5.1.2. Batch Workflows

Batch workflows have two termination possibilities, indicating if the End Batch will be waited for ornot. If the batches are large, and the batch has just being loaded by the workflow, the Immediate optionwill terminate the workflow without waiting for the current batch to finish.

Figure 82. Confirmation dialog when Batch workflows are deactivated.

Awaits the next End Batch before unloading the workflow, that is, when the current batchis fully processed.

Batch

Deactivates the workflow immediately, causing the current batch to be terminated. Thismay still take a while, however, it is still faster than the Batch termination option.

Immediate

91

Core 4.1

6.5.2. Workflow Views

Workflows may be grouped together into Views to ease handling and overview. The columns of theseviews may be customized to contain any static or real-time information, such as the name of the filecurrently being processed or the number of UDRs processed. Further, defined Views may be configuredto be part of other Views.

Workflow Views are handy when using the Workflow Manager. Other than being able to manageseveral workflows through one View, Inter Workflow relations and hierarchy may be illustrated.

To define a View, select Workflow View... from the Edit menu of the Workflow Manager.

6.5.2.1. Workflows/Folders Tab

The Workflows/Folders tab defines what workflows and/or folders of workflows to belong to theView.

Figure 83. Workflow View Editor - Workflows/Folders tab.

If enabled, folders will be created as a tree unit in the Workflow Manager.If disabled, all workflows will be shown as a common list.

Show Folder Hierarchy

A list of the selected workflows and folders belonging to the View. Selectinga workflow template will include all its instances.

Workflows or Folders

6.5.2.1.1. Add Workflows or Folders

Selecting Add will display a window containing all workflows and folders. Select Apply to populatethe View.

Figure 84. The Add Workflows or Folders dialog.

92

Core 4.1

6.5.2.2. Columns Tab

From the Columns tab, the information to be displayed in the View is selected. The columns are ofthree types; the predefined Workflow State, Execution Context or the user defined Column.

Figure 85. Workflow View Editor, Columns tab.

The type of column:Source

• Workflow State - Shows the state of the workflow. Refer to Section 5.4.2.7,“Workflow States” for further information on different states.

• Execution Context - Shows the IP address of the host and the EC number, on whichthe workflow is running.

• Column: <any> - The name of a user defined column. Refer to the subsequent section.

Sets the column alignment; left, right or center.Alignment

Indicates if the column will be visible or not.Disabled

6.5.2.2.1. Add Column

Selecting Add will display a window from which columns for the View are defined.

Figure 86. The Add Column dialog - Column selection.

The previously components are explained.

6.5.2.3. Columns

Columns allow the user to name own columns, where any information may be displayed. These columnsare populated with values in the Monitor tab of the Workflow Preferences window (Workflow Ed-itor). Mapping may be made to existing MIM resources, or to any user defined information by usingAPL code (the dispatchMessage function).

The order of action to populate a Column in a Workflow Manager View:

1. Create a Column.Select Columns... from the Edit menu of the Workflow Manager.

93

Core 4.1

2. Create a Workflow View.Select Workflow View... from the Configuration menu on the desktop. Save the View.

3. Open the View.In the Workflow Manager, select the View saved in the previous step.

4. Populate the Columns.Select Preferences... in the Edit menu of the workflow. Turn to the Monitor tab and select Addto access the Add Workflow Manager Columns dialog. Map MIM resources against the Columns.

Refer to following section for an example.

6.5.2.4. An Example

Another use of the dispatchMessage function, is to display information in a Column of a WorkflowManager View. This sort of behavior may be configured in the Workflow Preferences window,Monitor tab, where MIM resources are mapped against Columns. However, there is one restriction;batch MIMs are reported at End Batch only, which makes it impossible to view progress of batchvalues during UDR processing. For these cases, the dispatchMessage function is used.

Consider a View, where real-time information for the number of partials found within a file is displayed,incrementing with 1 for each occurrence.

6.5.2.4.1. Event Categories

In order to use the dispatchMessage function, an Event Category has to be defined. This is donefrom the Event Notification window by selecting Event Categories from the Edit menu.

Figure 87. An Event Category is a user defined string to be used as argument in thedispatchMessage function.

6.5.2.4.2. Columns

Columns are defined in the Workflow Manager. Select Columns... from the Edit menu to append asuitable value (that is, the column name).

Figure 88. A Column may hold any information. Choose a descriptive string to be displayed asthe column name in the View.

94

Core 4.1

6.5.2.4.3.The Workflow Preferences Settings

In the Workflow Preferences, the defined Column is mapped against the defined Event Category:

Figure 89. Besides Event Categories, MIMs may also be mapped against Columns.

6.5.2.4.4.The APL Code

The dispatchMessage function takes two arguments, the first being the string to be logged in theColumn, and the second being an existing Event Category.

Figure 90. Any string may be sent on with the dispatchMessage function.

95

Core 4.1

7.Tools MenuIn this section the general tools are described in detail, except the applications that belong to Ultra,please refer to Convergent Mediation Engine™ Ultra Management - User Guide, for further information.

7.1. Access ControllerIn order to operate in the Convergent Mediation Engine™ user interface, one have to be defined as auser in the system. Access to different Convergent Mediation Engine™ applications is managed ongroup level. For applications Execute permission means the user group is allowed to use the application.If there are altering functionality within the application, Write permission needs to be granted in orderfor the full application functionality to be available.

Only members of the Administrator group have access to the Access Controller, hence onlyadministrators may add users to the system. Only one user may use the Access Controller atthe time.

For further information on permissions, please refer to Section 6.1.4, “Configuration Permissions”

To display the Access Controller window select Access Controller... from the Tools menu on thedesktop.

Figure 91. Access Controller window.

7.1.1. Users Tab

At installation a default user is created, who is to be treated as the administrator of the system, calledmzadmin, and is also referred to as the Super User. The Super User has full access to the system nomatter the settings. A user of the system will also be a member of one or several user groups.

To add a new user to the system, select the Users tab and then New from the File menu or from thetoolbar.

96

Core 4.1

Figure 92. Access Controller window, Users tab / Add User.

If enabled,the user will be disabled in the system and therefore not acknowledgedwhen trying to log in. Disable the box to again allow the user its predefined accessrights.

Disable User

The name of the new user.

Valid characters are: A-Z, a-z, 0-9, '-' and '_'

User Name

A descriptive name of the user.Full Name

Email address for the user. This address will turn up automatically in applicationsfrom which emails may be sent.

Email

Adds/Edits a password for the user account.Set Password

A list of all groups that the user could be a member of. All users belong to group All.Group

If enabled, the user is registered a member of the specific group.Member

If enabled, this group is set as default group for the user. By default, this group willhave read, write and execute permissions for new configurations created by the user.

Default

Any user can change its own password. This is done from the File menu on the desktop, selectingChange Password....

7.1.2. Access Groups Tab

The only standard group available is Administrator. The Administrator group will always have fullauthority, no matter the permissions set.

To add a new group to the system, select the Access Groups tab and then select New from the Filemenu or from the toolbar.

97

Core 4.1

Figure 93. Access Controller window, Access Groups tab / Add Group.

The name of the new group.

Valid characters are: A-Z, a-z, 0-9, '-' and '_'

Name

Descriptive information about the group.Description

For each group, permissions to various parts of the system may be altered.This column holds all applications in the system.

Application

The members of the group are allowed to start an instance of this application.If disabled, the corresponding application menu item will be disabled in thedesktop menu.

Execute

The members of the group are allowed to use the application to add, view andedit configurations. If Write is selected, Execute is automatically selectedfor the application as well.

Write

A drop down menu that allowes the user to filter on application type. Optionsare All, Configuration, Inspection or Tools. The options corresponds toapplications found in the menus on the desktop.

Application Category

Enables Write (if applicable) and Execute for all permissions in the chosencategory.

Select All

Disables Write and Execute for all permissions in the chosen category.Deselect All

To alter permissions on configurations, please refer to Section 6.1, “Configuration Browser”

7.2. Pico ManagerThe Pico Manager displays all available hosts running pico clients, including their current permissions.

From the Pico Manager it is possible to deny specified clients access to the system. By default, allclients within the network will have access to the system, until the privilege is manually disabled. Thisfeature may be changed to automatically denying connection privilege, until configuration is changedfrom the Pico Manager window. The parameter, mz.dynamicconnections, is set in the$MZ_HOME/etc/platform.xml file. Note, the server must be restarted for changes to take effect.

98

Core 4.1

The default value is set to true, resulting in access beingautomatically enabled until manually disabled. Setting

mz.dynamicconnections

the value to false will give the opposite - access isautomatically disabled until manually enabled.

Execution Contexts must be registered before use, as opposed to desktops and mzsh. The regis-tration is done from the Pico Manager, refer to Section 7.2.1, “Adding Pico Hosts” for furtherinformation.

Figure 94. Pico Manager window.

The IP address of the pico client.IP Address

Shows if the client is authorized to connect to the Convergent Mediation Engine™ system.Access

7.2.1. Adding Pico Hosts

As described in the previous section, access may be denied/allowed for specified hosts. In case a hostis disabled while connected to the system, the change will take effect the next time the pico client triesto connect to the system.

Figure 95. The Add dialog of the Pico Manager.

The IP address of the pico client.IP Address

Indicates if the pico client will have access to connect to Convergent Mediation Engine™.

Deny Access

99

Core 4.1

The type of operating system hosting the pico. Each instance may in turn host severalinstances:

Instances

• commandline - The command line tool, $MZ_HOME/bin/mzsh, which makesit possible to operate the Convergent Mediation Engine™ from the command line.

• desktop - A user interface client.

• ec - An Execution Context.

• ecsa - An stand-alone Execution Context.

7.3. System ExporterWhen moving information from one Convergent Mediation Engine™ system to another, for instancebetween a test system and a live system, the System Exporter and System Importer applicationsmay be utilized. By default, dependencies are included. That is, when selecting a workflow, all config-urations it depend upon will automatically be selected as well.

The System Exporter also supports saving of the entries selected in the Available Entries tree tosimplify environments where it is always the same sub-set of all configurations to be exported.

The following must be evaluated before selecting System Exporter...:

• No historic configurations are exported for any application.

• No configuration changed during runtime is exported - the initial value is exported. This concernscollectors containing file sequence number parameters. When the system has been stopped forexport, take a note of this data.

• To avoid moving too much data to the new system, clean out as much data as possible. See theTask Manager.

To display the System Exporter, select System Exporter... from the Tools menu on the desktop.

100

Core 4.1

Figure 96. The System Exporter.

This setting aborts the export if something fails. By default it is not en-abled which means that an export file will be created regardless, but willlack the failing components.

Edit menu Abort On Error

If tagged, an entry's dependencies will be automatically selected for ex-port.

Edit menu Select Dependen-cies

The produced export file is a series of XML documents, packed into onezip file. Selecting Encryption will make the files password-encrypted.Note that UDR data, for instance archived data, is encrypted as well.

Edit menu Encryption

By default, the export file is packed as a Windows zip-file. This optionwill not pack the output, but save it as it is - a set of XML files structuredin directories according to the Available Entries view.

Edit menu Directory Out-put

Opens up a new window where a progress log for an on-going exportmay be examined.

View menu View Log...

An entry's dependency on other entries within the system, can be foundout by highlighting it, followed by Show Dependencies....

View menu Show Depend-encies...

The name of the file where the export data is saved.Output Target

101

Core 4.1

Lists all configurations available in the system, grouped in categories.Selecting a category, will select all its configurations.

Available Entries

Displays a progress bar for the on-going export.Progress

Selecting this option starts an export according to the current definitions.The button will change into an Abort button during export, which makesit possible to cancel the current export.

Export

7.4. System ImporterWith the System Importer utility, configuration data from another Convergent Mediation Engine™system may be reused, or backed up data from the current system may be reinserted. The file fromwhich import data is fetched, has to be created with the Convergent Mediation Engine™ System Ex-porter utility. Note, not all data in the file has to be imported, a selection may be made.

Every time something is imported in Convergent Mediation Engine™ , a backup is stored containingall the imported configurations. The backup file is saved on the machine hosting the platform under$MZ_HOME/backup/yyyy_MM where yyyy_MM is translated to current year and month. The im-ported backup file is named import_<date>_<filename>.zip where <filename> is the nameof the file being imported. The backup file is a ZIP file.

The Available Entries area is populated once the export file has been selected.

After the import has finished, and before the workflows are activated, the following is recommended:

• Update the dynamic configuration data, that is, the file sequence numbers in the collectors(manually noted down before export).

• Start all workflows configured with scheduling.

If there are existing configurations with identical names and configuration keys, a new versionof this configuration will be created. In a case where there already exists a configuration withthe same name, the user will be asked how to continue; replace the existing configuration anduse its configuration key, or create a new configuration by appending a suffix to the configurationname.

It is not possible to create new versions of configurations that are locked for editing by a userin the system.

To display the System Importer window select System Importer... from the Tools menu on thedesktop.

102

Core 4.1

Figure 97. The System Importer.

By default, this option is not enabled, resulting in all selected items, exceptfor the failing, being imported. Enabling it will abort the import immedi-

Edit menu Abort On Er-ror

ately, but not rollback the already successfully imported items. InvalidUltra and APL definitions will also be considered erroneous, and resultin abortion. Note, the failing item may be imported before termination.

In case the export file is not packed as a zip file, this option will make itpossible to select the root directory to where the export files reside.

Edit menu Directory In-put

Opens up a new window where a progress log for an on-going import maybe examined.

View menu View Log...

The name of the import file or directory.Input Source

Lists all configurations available in the selected file, grouped in categories.Selecting a category, will select all its configurations.

Available Entries

Displays a progress bar for the on-going import.Progress...

Selecting this option, starts an import according to the current definitions.The button will change into an Abort button during import, which makesit possible to cancel the current import.

Import

7.5.Task ManagerFrom the Task Manager window the state of all configured tasks within Convergent Mediation En-gine™ may be viewed. It also provides the possibility to configure and schedule user defined tasks tobe executed on a regular basis. There are two different types of tasks; Convergent Mediation Engine™internal tasks and user defined tasks. Internal tasks are included in the system and takes care of cleaningup old runtime data and logs. Convergent Mediation Engine™ provides two tasks which may be con-figured by the user; SQL Executor and Script Executor.

The Task Manager is displayed when selecting Task Manager... from the Tools menu on the desktop.

103

Core 4.1

Figure 98. The Task Manager.

Make sure to schedule cleanup tasks to run when the system load is as low as possible. Thissince cleaning out a lot of data has a performance degrading impact.

Activates selected tasks. Even if the task is scheduled to commence later, it willrun immediately when this button is pressed.

Edit menu Activ-ate

Selecting a task followed by Task Report... will result in a pop-up window beingdisplayed, containing information about the outcome of the last activation of thetask.

View menu TaskReport...

The name of the task. Predefined tasks, which are included by default, may notbe deleted. However, their scheduling may be changed.

Task Name

• Configuration Cleaner - This task takes care of cleaning up historic config-urations.

• Statistics Cleaner - This task is used to calculate daily statistics and clean outold statistics entries according to user defined days to keep value.

• System Backup - This task is used to take backups of all (current) configura-tions in the system. The task also supports cleaning of backed up files.

• System Log Cleaner - This task takes care of cleaning up the System Log. Itis configurable for how long data may be kept. It is also possible to use differentsettings for different levels of log severity (Error/Disaster, Warning, Informa-tion).

The date and time of the last execution.Last Executed

There are three types:Status

• OK - The task is terminated successfully.

• Running - The task is executing.

• Aborted - Returned if the task cannot execute at all, or if it has exited withstatus other than 0 (zero).

The latter status type is mainly meant to be used for user defined scripts and De-velopment Toolkit Java tasks. If a Convergent Mediation Engine™ task returnsNOK or Aborted, there is something seriously wrong with the system. This

104

Core 4.1

should not occur. Refer to the Convergent Mediation Engine™ DevelopmentToolkit - User Guide, for further details.

The criteria managing the execution of the task. Refer to Section 4.10, “Scheduling”for further information.

Next Execution

• <date> - If the task is scheduled to run at a regular basis, the next planneddate and time is displayed.

• By trigger only - Displayed if the task is activated by a trigger.

• <date>, and by trigger - A combination of date and trigger scheduling canbe used. For these cases, both criteria are displayed.

• None - Displayed for tasks with no scheduling criteria. These types can onlybe activated manually (the Activate button).

7.5.1.Task Configurations

User defined tasks may be added, removed and edited. Mandatory tasks may be reconfigured, exceptthe Archive Cleaner which is only configurable with respect to scheduling criteria. There is also atask called ECS Maintenance, which is described in Convergent Mediation Engine™ Error CorrectionSystem - User Guide.

For information on how to set up scheduling criteria, please refer to Section 4.10, “Scheduling”.

7.5.1.1. Configuration Cleaner

This task is responsible for the cleanup of configuration history. The configuration window offers thepossibility to configure if and how historic configurations will be cleaned for every configuration type.

Figure 99. Edit Task Configuration window - Configuration Cleaner.

An icon representation the configuration type.Type

Name of the configuration type.Name

Defines for how long configurations of this type will be kept. Possible values are:Keep

• Always - Configurations for this type is never deleted. This is the default value.

105

Core 4.1

• Days - The configuration is kept for a number of days.

• Versions - Only a number of versions are kept. For instance, only keep the last 10 versions.

This value defines how many days or versions that a configuration will be kept. It must bedefined for Days and Versions.

Value

7.5.1.2. Statistics Cleaner

This task is responsible for cleanup of old statistics, collected by the Statistics sub-system. The usermay specify how many days statistics will be kept for the different resolution levels - minute, hourand day.

Figure 100. Edit Task Configuration window - Statistics Cleaner.

Specifies for how many days a minute level record is kept.Minute Level Records

Specifies for how many days an hour level record is kept.Hour Level Records

Specifies for how many days a day level record is kept.Day Level Records

7.5.1.3. System Backup

This task is responsible for making a backup of all configurations in the system. The resulting backupfile is saved on the machine hosting the platform.

Backup files are stored under $MZ_HOME/backup/yyyy_MM where yyyy_MM is translated tocurrent year and month. A backup file is named backup_<date>.zip and is of ZIP format.

Cleaning of old backup files can also be configured. This is defined by setting for how many days abackup file will be kept.

Figure 101. Edit Task Configuration window - System Backup.

The user interface is divided in two parts, one for defining if backup will be done, and another onedefining how backup will be cleaned. System Backup holds of the following options.

106

Core 4.1

Turns the system backup functionality on/off. Default is on.Enable System Backup

If enabled, the backup will be encrypted using a password entered in thePassword text field.

Use Encryption

The password to be used when Use Encryption is enabled.Password

Cleanup consists of the following options.

Every time configuration is imported to the system using the System Importer,Convergent Mediation Engine™ stores a backup of the configurations that

Imported Files

were imported into the system. This field allows the user to define how manydays backed up imported files will be kept.

Defines how many days system backup files will be kept.System Backup Files

7.5.1.4. System Log Cleaner

The System Log Cleaner cleans the System Log at a regular interval. Its configuration window offersthe possibility to set for how long different types of messages will be kept.

Figure 102. Edit Task Configuration window - System Log Cleaner.

Defines for how many days messages of type Error and Disaster are kept. The max-imum value is 99, default is 30.

Error/Disaster

Defines for how many days messages of type Warning are kept. The maximum valueis 99, default is 20.

Warning

Defines for how many days messages of type Information are kept. The maximumvalue is 99, default is 10.

Information

7.5.1.5. Script Executor

A user may write a script to be executed regularly. Typically, this may involve cleaning up directoriesand other instances which need to be attended to periodically.

Figure 103. Edit Task Configuration window - Script Executor.

The name, including the full path, to the script performing the task.Script Name

Parameters expected by the Script Name. This field is optional.Parameters

107

Core 4.1

An after commit script cannot be designed to affect already committed batches.

7.5.1.6. SQL Statement

A user may write SQL statements or SQL scripts to be executed regularly. Typically, this may involvecleaning up tables which need to be attended to periodically.

Figure 104. Edit Task Configuration window - SQL Statement.

A list from which an existing database is selected. The databases are entered fromthe Database Profile dialog.

Database

The PL/SQL script, or SQL statements are entered in this window. Several SQLstatements must be embedded within a block.

SQL Statement

Note that if only one SQL statement is entered, it must not end with ';'.

108

Core 4.1

8.The Command Line Tool mzshThe Convergent Mediation Engine™ command line tool mzsh is a shell that is used for starting andstopping the system. It also provides some access to the Control Zone from the command line.

To start an application, that is execute a workflow, the user needs to have Execute permissionson the application and configuration. For further information, please refer to Section 6.1.4,“Configuration Permissions” and Section 7.1, “Access Controller”

8.1. Startupmzsh can both be used interactively or non-interactively. In the interactive mode, mzsh is started likeany other shell and uses the commands available. In the non-interactive mode, the command and itsarguments are given on the same invoking command line. To start mzsh interactively just give thecommand:

$ mzshMZ>

If the platform is up and running, a user name and password is requested.

To start mzsh non-interactively, do for example:

$ mzsh mzadmin/<password> wfactivate MyWF

Note, the Convergent Mediation Engine™ user name and password has also been given on the commandline as one parameter.

mzsh can also read commands from standard input. This means that commands to be executed maybe put in a file or simply let other commands produce the commands for it, for instance:

$ while read wf ; do echo wfactivate $wfdone < my-wfs.list | mzsh mzadmin/<password>

The previous command reads the names of the workflows to be started from the file my-wfs.listand creates startup commands for them. This output is then fed from standard input to mzsh.

8.1.1. Startup Options

mzsh accepts a number of general options.

-q Makes the shell quiet.

This option suppresses certain error messages, that normally would be printed on standard error. Forinstance, the following command will produce an error message if the platform has already been started:

$ mzsh startup platform

109

Core 4.1

Platform is already running.

Using -q will suppress that error message. This may be useful in a script where it is desired to makesure that the code server is running before issuing other commands and no particular error messagesare wanted.

-e Exit on error.

When using -e, mzsh will terminate when one of its commands return an error code. Normally mzshwill just continue with the next command (given on standard input). This is useful if feeding mzshwith commands from a file and it is desired to terminate on an error.

8.2. Command Availabilitymzsh is a tool that is used for starting up the system and it is also at the same time a client of theControl Zone. When used without running the Control Zone, mzsh provides commands for startingand stopping the system and doing other tasks that are not dependent of a running Control Zone.

If mzsh is started when the Control Zone is running, mzsh will become a pico client and extra com-mands, such as wfactivate and wfdeactivate, will be downloaded from the code server. mzshmay also be used in interactive and non-interactive mode as described earlier.

Commands that are downloaded from the Control Zone and require the user to be logged in, are onlyavailable if the user name and password has been given. This is either prompted for by mzsh if it isstarted interactively:

$ mzshUsername: mzadminPassword: ******

It may also be done by providing the user name and password as an argument to mzsh.

$ mzsh mzadmin/<password> wfactivate MyDisk

Note that some commands are not available if Convergent Mediation Engine™ is not running.

8.3. Commands Always AvailableThe following commands described, are always available whether logged into mzsh or not.

Starts the Convergent Mediation Engine™ desktop.desktop

[ -n ]echo

Outputs the arguments. n means output without newline.

[ N ]exit

Exits the command line tool. N is the final exit code returned to the calling process. Ifno N value is supplied the result of the previous operation will be returned to the callingprocess.

110

Core 4.1

[ command ]help

Displays information about all available commands or information on the given com-mand.

[ -l | List of running entities to be killed ]kill

Kills one or more startables. The -l option lists all running entities.

This command will be available on platforms where it is possible to pickup an externalreference to the started JVM (pid on Unix) and it will perform an unconditional termin-ation of the running startable without giving it a chance to terminate itself. This commandwill only be available for the $MZ_HOME owner.

<name> <version> <package-file> [ -level <default level>] [ -metadata <data> ] [ -hidden ] [[-level <level name>]<file-to-include>] ... ]

pcreate

<name> - the name of the package, <version> - the version string of the package,<package-file> - the resulting package, ( level - [ platform | execution ] ) Each filenameto include can optionally be preceded by a level.

Please refer to the Convergent Mediation Engine™ Development Toolkit - User Guide,for further information.

[ -f <filename> ] [ -p <property1> <property2> ... ]setpass-word

<filename> - File in which the password should be set. Default "common.xml". If nopath is specified, default path is $MZ_HOME/etc. <propertyN> - List of properties tobe updated. If no property is specified, a scan for all Convergent Mediation Engine™password properties is done.

[ -q ] <running entity> ...shutdown

Shuts down one or more startables. q means quiet, which will output less information.

This command shuts down a running startable. The startable will be shutdown in anorderly fashion, that is, it will do the required cleanup before terminating. The last thingthe startable does is to report back to mzsh that it is about to terminate. When this isdone mzsh will continue with the next startable or if there are no more startables given,the command will return. The command will timeout if there is no answer after a periodof time. This command will only be available for the $MZ_HOME owner.

[ -q ] <startable> ...startup

Startup of a defined startable(s). q means quiet, which will output less information.

This command will startup the system as specified by the arguments. This includesstarting new JVMs for the platform and all defined ECs. Each startable have a corres-ponding configuration file, even the command line tool, in which properties for thatparticular JVM may be specified.

This command will not terminate until a startable has reported back that it is up andrunning. If starting more than one startable, it will not continue with the next one untilthe previous has reported that it is up and running. Note that the platform must be startedbefore any other startable. This command will only be available for the $MZ_HOMEowner.

[ -q | -verbose ] <startable> ...status

This command will display the current status of all defined startables or for the startablesgiven as arguments. A startable can be in three states. It can be running, not running or

111

Core 4.1

it can also be in a state where the JVM process is running but the actual system is notanswering any more. This could be due to a stopped Platform or a faulty startable.

-q Means quiet, which will output less information.

-verbose Lists communication errors. This is only used for error searching purpose.

[ N ]quit

N is the final exit code returned to the calling process. If no N value is supplied theresult of the previous operation will be returned to the calling process.

[ -q ] <running startable>restart

This command will perform a shutdown directly followed by a startup. q meansquiet, which will output less information. Refer to description on shutdown and startuprespectively, for further details.

8.4. Commands Available Only When Control Zone isRunning

Changes the encrypted database password in the configuration files.changed-bpw

[ -n <N>] [ -b ] [ -s <I|W|E|D> ] [-d <date>] [-h <hour>][-f <filename>] [-q <C|S|D>]

dumpsys-log

This command will allow to display and to save entries from the Convergent MediationEngine™ System Log. The command accepts the following filter options:

Maximum number(N) of entries to dump. By default (no -b flag is given) themost recent entries in the System Log are shown.

-n

Will select entries from the beginning of the System Log (used in conjunctionwith -n). That is, the oldest entries instead of the most recent (default) entries.

-b

Severity types to display; Information (I), warning (W), error (E), disaster (D).-s

Filter for a certain date or date interval. Date is given on the form specified inconfig.xml and must be enclosed within quotation marks; "d1 d2" or "d1".

-d

If only one date is given, the display will hold entries for that date only (time00:00:00 to 23:59:59), unless the -h flag is set. If giving two dates, all entriesfrom date1 (time 00:00:00) two date2 (time 23:59:59) is displayed, unless the -hflag is set.

Filter for hour or time interval. Time is given on the form hh:mm:ss and mustbe enclosed within quotation marks; "h1 h2" or "h1". If only one time is given,

-h

the display will hold entries for that hour only (time 00:00:00 to 00:59:59). Ifgiving two times, all entries in the interval will be displayed.

If both date and time intervals are entered; -d "d1 d2" -h "h1 h2"the displayed entries are between date d1 at time h1 to date d2 at time h2.

Direct output to a file (including directory path).-f

Query mode; Number of entries (C), number of entries of each severity (S),earliest and latest date of entry (D).

-q

<package-file> ...pcommit

112

Core 4.1

Inserts a package into the system.

[ -name <package name> [ -archive [ <package archive name>] [ -class [ <java class name> ] ] [ -resource [ <javaresource name> ] ] ] ]

plist

Lists all or given packages installed in the system.

<package-name>premove

Removes the package from the system.

<file> [password]systemex-port

Exports configuration data from Convergent Mediation Engine™ and sends it to a filein a local repository in a compressed format (ZIP). The file argument is what theexport file will be named. If the optional password parameter is given, the exportwill be encrypted using this password.

The output log information from this command is shown during execution of thecommand. No log file is produced, however, the log information can be sent to a filemanually if required.

<export file> [password]systemim-port

This command will import a file or directory into Convergent Mediation Engine™ .For encrypted files, a password is required. If an import entry is conflicting with analready existing entry in the current system, it will not be imported. This command isonly available to the mzadmin user, therefore all entries are imported regardless ofdefined access permissions.

<workflow name> ...wfactiv-ate

This command will schedule one ore more workflows for execution. The commandaccepts standard wild cards, such as '*' and '?'. For instance, wfactivate my-Folder.D* will activate all workflows in folder myFolder that with the letter D.The workflow name must include the name of the folder.

<workflow> <agent | 'Workflow'> <instruction> ...wfcommand

Will send a command to a workflow. Certain agents and the workflow itself allows theuser to interact with the flow, for instance by telling an agent to flush all its data tosubsequent nodes. Please refer to the agent user guides for further details.

The workflow only supports one command at the moment and that is to enable or disabledebug. The following turns debug on for a workflow:

wfcommand <workflow> Workflow debug on

To turn debug off for a workflow, simply replace on with off.

[ -immediate ] <workflow name> ...wfdeactiv-ate

This command will deactivate one or more workflows. Using immediate will deactivatethe workflow without waiting for a batch to finish. This will not kill all processes, butsend a message to all agents to stop as soon as they can.

[ -invalid | -valid | -active | -inactive ] [ -long ] [ -loop [ N ] ]

wflist

Lists workflows.

This command lists workflows. Without any options it will list all workflows. Thecommand accepts the following options:

113

Core 4.1

Lists all active workflows.-active

Lists all valid workflows.-valid

Lists all invalid workflows.-invalid

Lists all inactive workflows.-inactive

Long format.-long

Sets a loop mode that runs the wflist command forever every Nseconds (default 10 and Ctrl-C will terminate).

-loop N

Combining -active and -long also lists EC and the executiontime for the workflow.

114

Core 4.1

9. Execution Context Web InterfaceAn Execution Context contains an internal web interface that can list all workflows deployed in theEC together with their state and throughput. The purpose of the monitor is to be able to check thatworkflows that must be up and running are active even in case that the platform for some reason isunavailable. In case the platform for some reason is unable to stop a workflow that is running the webinterface also allows the user to stop an active workflow.

The web interface does not allow the user to activate a workflow.

9.1. StartupThe EC web interface will start automatically in case that the ec.httpd.enabled property is true. Thehost and port that it will listen on can be configured using the ec.httpd.port property.

9.2. FunctionalityTo access the web interface type the following URL into the web browser:

http://<hostname>:<port>/

If encryption is enabled, the URL is https://<hostname>:<port>/

Please refer to Section 11, “Network Security” for further information on how to enable encryp-tion.

9.2.1. Login

The web interface does not use the standard Convergent Mediation Engine™ user authenticationmechanism. Instead an administrator password is used. This password is configured in the ec.httpd.pass-word property. Type this password to log in to the web interface.

9.2.2. Logout

Selecting this button will log out the user and display the Login page.

9.2.3. Execution Context Monitor

The monitor shows a list of all workflows deployed on the Execution Context.

115

Core 4.1

Figure 105. The Execution Context Monitor

The workflow name.Workflow Name

The workflow status, just like the status of the workflow in the Workflow Editorin monitor mode. Please see Section 5.4.2.7, “Workflow States” for more inform-ation.

Status

The throughput of the workflow. Please see Throughput Calculation for more in-formation.

Throughput

Allows the user to deactivate an active workflow. This function should only beused in case the platform has been unable to stop the workflow due to communic-ation failure that could occur in unreliable networks.

Activation

116

Core 4.1

10. System AdministrationThis section contains useful step-by-step procedures describing basic system administration of Con-vergent Mediation Engine™ .

The following procedures and commands described, expects the $MZ_HOME/bin to exist in the$PATH variable for mzadmin user.

10.1. File System PermissionsAll workflows will be executed in Execution Contexts started by the mzadmin user. If such a workflowcontains agents that reads or writes data directly from the file system (such as the Disk Collectionagent), it is important that all directories have the correct file permissions. The same applies to, forinstance, directories targeted from the platform, such as archiving directories and error directories.

10.2. Starting the Convergent Mediation Engine™ System1. Log in as mzadmin on the host where the Control Zone is installed.

2. Start the system by entering the command:

$ mzsh startup <startable> <...>

where <startable> is one or several names of the startables to be started. Note that the Platformmust be started before any Execution Context. For example:

$ mzsh startup platform ec1

If there are still Execution Contexts running in the system before the platform is started it isimportant to not start new Execution Contexts with the same name since these will block andterminate the already running Execution Contexts.

10.3. Stopping the Convergent Mediation Engine™ Sys-tem

1. Login as mzadmin on the host where the Control Zone is installed.

2. Stop the system by entering the command:

$ mzsh shutdown <startable> <...>

Note that the Platform must be the last of the startables to be shut down. The reason for this isthat the shutdown commands are issued through the Platform.

117

Core 4.1

Note that the kill command must only be used when the shutdown command is not working.It will force interruption of all processing, instead of waiting until the next End Batch for eachworkflow. The use of this command may be necessary, for instance, if the Platform is accidentallyshut down prior to the Execution Context.

10.4. Starting Execution Contexts1. Login as mzadmin on the host where the Execution Context is installed.

2. Start the Execution Context by entering the command:

$ mzsh startup ec1 ecsa1

This command will start two execution contexts named ec1 and ecsa1. They could both be definedin the configuration file executioncontext.xml or they could have their own configurationfiles.

If the Execution Context is installed in the Control Zone, it will be started together with theplatform servers. See Section 10.2, “Starting the Convergent Mediation Engine™ System”.

10.5. Stopping Execution Contexts1. Log in as mzadmin on the host where the Execution Context is installed.

2. Stop Execution Context ec1 by entering the command:

$ mzsh shutdown ec1

In case workflows were active on the shut down Execution Context and are still in unreachableor active state these may need manual attention to make sure that they are either activated/deac-tivated manually.

10.6. Adding an Execution Context to the System1. Make sure the new host is properly installed in the local area network. It has to be assigned a host

name and have the prerequisite software installed according to the Convergent Mediation Engine™Installation Instructions - User Guide.

2. Make an Execution Context only installation of Convergent Mediation Engine™ on the new host.Make sure to give the new Execution Context a unique name.

3. Register the new Execution Context in the Pico Manager. Make sure to enter the name exactlyas entered in the previous step.

4. Start the Execution Context on the new host by entering the command:

118

Core 4.1

$ mzsh startup ec2

where ec2 is the name given to the Execution Context.

10.7. Adding a Desktop to the System1. Make sure the new desktop host is properly installed in the local area network and have the pre-

requisite software installed according to the Convergent Mediation Engine™ Installation Instruc-tions - User Guide.

2. Make a Desktop the only installation of Convergent Mediation Engine™ on the new host.

3. Start the desktop on the new host by:

i. on Windows, clicking the Convergent Mediation Engine™ icon on the desktop

ii. on Unix, issuing the command:

$ mzsh desktop

10.8. Clearing the Pico CacheThe pico cache may be cleared manually by removing the cache directory. Make sure that no processesthat currently uses the cache, are running.

• Unix:

$ rm -rf $MZ_HOME/pico-cache

• Windows:Delete the directory pico-cache in the Convergent Mediation Engine™ installation directory.

The next time pico started processes are running, the cache will be created and populated.

10.9. Preventive Maintenance ProceduresPreventive maintenance procedures are recommended tasks to be carried out at a regular basis to avoidperformance degradation.

10.9.1. Online Backup

The procedure described in this section will produce a backup that may be used to restore ConvergentMediation Engine™ (including the Oracle database) in case of disaster. The backup contains, for apoint in time, a consistent snapshot of the Convergent Mediation Engine™ filesystem and the Conver-gent Mediation Engine™ database. The snapshot may be used to recover the system to the state it hadat the time of the backup.

119

Core 4.1

The database part of the backup is done as one full backup and a numbers of incremental changes. Forevery incremental backup of the database, a full backup has to be taken of the file system. It is importantto keep the file system and the database in a consistent state, since Convergent Mediation Engine™stores state data on files.

If disaster strikes non-critical parts of Oracle, a point of failure recovery is possible. For thistype of recovery, please refer to the Oracle documentation.

10.9.1.1.The Database

1. Before taking backups, Oracle needs to be set in Archive mode. Add the following toinitMZ.ora:

log_archive_start = truelog_archive_dest = /path_to_oracle/redologs_directorylog_archive_format = "T%TS%S.ARC"

2. Make a secure backup of initMZ.ora.

3. Run the following commands:

$ sqlplus "/AS SYSDBA"SQL> shutdownSQL> startup mountSQL> alter database archivelog;SQL> alter database open;SQL> quit;

4. Write down the DBID (Database Identifier). It is needed in case of making a recovery.

$ rman nocatalogRMAN> connect targetRMAN-06005: connected to target database: MZ (DBID=749276174)RMAN> exit;

5. Do a full backup.

6. Do one or several incremental backups. Together with the full backup, this creates a point in timebackup that may be used for recovery.

10.9.1.2.The File System

Taking a file system snapshot, is exemplified with the fssnap utility (included in Solaris 8 and morerecent versions).

1. Take a snapshot of the file system.

$ fssnap -F ufs -o maxsize=500m,bs=/opt/snap,unlink /export/home

120

Core 4.1

This creates an open file, changelogfile, in /opt/snap. The file contains all changes tothe file system made after the command is issued.

2. Mount the snapshot before the open file expires (that is, when filled up):

$ mount -F ufs -o ro /dev/fssnap/0 /opt/snap/backup

3. Make a backup of the file system snapshot using cp and tar.

4. To re-sync the file system (that is, to apply the changes collected in the changelogfile), issuethe following unmount command:

$ umount /opt/snap/backup

10.9.2. Restore

In case of disaster, the Convergent Mediation Engine™ system may be recovered with the backupfiles produced using the procedure described in the previous section. Note that TEMP table spaces arenot recovered.

1. Shutdown Convergent Mediation Engine™ .

2. Shutdown the Convergent Mediation Engine™ Oracle Instance.

3. Make a cold backup of the crashed Convergent Mediation Engine™ , including the Oracle instance.

4. Locate the backup to restore. That is, a directory containing a full backup and at least one incre-mental backup. The directory should contain one or several Oracle control files control.<id>,and a snapshot of the Convergent Mediation Engine™ file system MZFS_<id>.

5. Remove all Convergent Mediation Engine™ table space files, the corresponding archived redologs, and control files. If necessary, remove the initMZ.ora file as well, and replace it withthe backed up version.

6. Make a backup copy of the control files MZ_control1 and MZ_control2.

7. Replace the Convergent Mediation Engine™ dynamic directories with the corresponding backupdirectories. An example of a dynamic directory, is the storage handler directory, used by the InterWorkflow application.

8. Execute the following:

$ export BACKUP_DIR=/path_to_backup_to_use$ rman nocatalogRMAN> set dbid = <DBID>RMAN> connect target;RMAN> startup force mount;RMAN> run {RMAN> allocate channel MediationZoneRestore type disk format '$BACKUP_DIR/%U';RMAN> restore database;RMAN> restore archivelog all;RMAN> }

121

Core 4.1

The reply messages:

RMAN-08017: channel MediationZoneRestore: starting archivelog...RMAN-08022: channel MediationZoneRestore: restoring archivelogRMAN-08510: archivelog thread=1 sequence=20RMAN-08023: channel MediationZoneRestore: restore backup...RMAN-08511: piece handle=/tmp/backup/0feduv7h_1_1...RMAN-08024: channel MediationZoneRestore: restore completeRMAN-08031: released channel: MediationZoneRestore

Use the last RMAN-08510 to fill in 'set until logseq NR thread NR'

RMAN> run {RMAN> allocate channel MediationZone Restore type disk format 'BACKUP_DIR/%U';RMAN> set until logseq 20 thread 1;RMAN> recover database;RMAN> alter database open resetlogs;RMAN>}RMAN> exit;

9. Restart the database instance:

SQL> shutdown normalSQL> startup

10. Start Convergent Mediation Engine™ .

10.9.3. Cloudscape Backup and Restore

To produce a full online backup of the Convergent Mediation Engine™ platform database when runningCloudscape, the following mzsh command is used:

./mzsh <user>/<pw> cloudscapebackup <directory>

The Convergent Mediation Engine™ user name.<user>

Password, corresponding to the user name.<pw>

The directory where the backup is stored.<directory>

To restore the database the following needs to be performed:

1. Shutdown Convergent Mediation Engine™

2. Move the directory in the backup directory to the Cloudscape restore directory. Please refer tothe cloudscape.restore.path in the platform.xml file.

3. Startup Convergent Mediation Engine™

To prevent another restore to be made unintentionally; after a restore is done, the used directory is re-named upon the next startup. The new name consists of the old name with the _USED suffix attached.The startup procedure also creates an entry in the System Log, showing the outcome of the restore.

122

Core 4.1

A way to perform regular backups is to create a script that runs the cloudscapebackupcommand and passes it a date as directory name. The script may be called at a regular interval,for instance once a day, by adding and scheduling a script executor in the Task Manager.

123

Core 4.1

11. Network Security

11.1. RCP Encryption, Using TLSThe RCP (Nokia internal protocol) communication can be protected using TLS (Transport Layer Se-curity). To enable TLS, various configurations need to be done with respect to platform and pico-instancesettings. Further, TLS can be used with or without authentication.

11.1.1.TLS Configuration Properties

The TLS support uses a keystore file as generated using the Java standard tool keytool. Please referto SUN product documentation for futher information on how to use keytool.

The TLS support is configured through properties. It is recommended to set the properties incommon.xml on the corresponding Convergent Mediation Engine™ installation (client or platform).The available properties are:

• pico.rcp.tls.keystore

The keystore file to use. When this property is set on the platform, the system will use TLS for allRCP connections that are not from the local host. If this property is not set, TLS will not be usedat all.

• pico.rcp.tls.keystore.alias

If the keystore contains multiple private keys, RCP will prefer to use the key with this keystorealias. If it is not set and the keystore contains more than one private key, it is undefined which keyis used.

• pico.rcp.tls.keystore.password

Set to the password for the keystore, as chosen in keytool. This password is encrypted throughthe normal Convergent Mediation Engine™ password encryption.

To set this value, the mzsh command setpassword is used, please refer to Section 8.3, “Com-mands Always Available” for command syntax.

Example 7.

Keystore password update:

$ mzsh setpassword -f common.xml -p pico.rcp.tls.keystore.password

• pico.rcp.tls.key.password

Set to the password for the key, as chosen in keytool. By default this is the same as the keystorepassword. (This is also the default for keytool).

• pico.rcp.tls.require_clientauth

Set to value="true" if client authentication is required. Default value is false.

124

Core 4.1

11.1.2.TLS Standard Setup

The configuration of TLS is done in three steps if no client authentication is required. For instructionson how to include authentication, please refer to Section 11.1.3, “TLS Setup - Client AuthenticationEnabled”.

1. Create a keystore and private key on the platform server.

Example 8.

Example on keystore setup on the platform server:

$ keytool -keystore /opt/mz/keys/platform.keys -genkey -keyalg RSA -alias platform

2. Configure the keystore and keystore password through specifying the corresponding propertiesin common.xml (on the platform server).

Example 9.

<property name="pico.rcp.tls.keystore" value="/opt/mz/keys/platform.keys"><property name="pico.rcp.tls.keystore.password" value="">

3. Set the password property through the setpassword command

Example 10.

$ mzsh setpassword -f common.xml -p pico.rcp.tls.keystore.password

11.1.3.TLS Setup - Client Authentication Enabled

TLS can be set up to demand authentication from all clients running outside the local host. The platformwill then request valid certificates from each client.

After standard TLS configuration has been setup (see Section 11.1.2, “TLS Standard Setup”), the fol-lowing is needed:

1. Create keystore and key for each Convergent Mediation Engine™ client installation (externalexecution context or desktop installations), or reuse the same keystore for all clients.

This means, setup of TLS properties is required in each client XML file (common.xml) for thecorresponding Convergent Mediation Engine™ installation.

125

Core 4.1

2. Set the pico.rcp.tls.require_clientauth property in common.xml on the platformserver.

Example 11.

<property name="pico.rcp.tls.require_clientauth" value="true">

3. Make sure the clients can validate the server certificate.This can be done by importing the server certificate to the client keystore or signing the clientcertificates with a key recognized by the platform.

4. Make sure the platform can validate the client certificates.

11.2. HTTP Encryption, Using TLS or SSLThe HTTP communication can be protected using TLS (Transport Layer Security) or SSL (SecureSockets Layer), resulting in communication through the HTTPS protocol. Once the properties for securecommunication is configured, TLS or SSL (depending on the third party product) will automaticallybe switched on for the Execution Context Web Interface and the Platform Web Interface.

11.2.1. Standard Setup

The encryption requires a keystore file as generated using the Java standard tool keytool. Pleaserefer to SUN product documentation for futher information on how to use keytool.

1. Create a keystore and private key on the platform server or external EC.

Example 12.

Example on keystore setup on the platform server:

$ keytool -keystore /opt/mz/keys/platform.keys -genkey -keyalg RSA

The -keyalg flag is mandatory. If not used, HTTPS will not work.

2. Configure the keystore and keystore password through specifying the corresponding propertiesin common.xml (on the platform server).

126

Core 4.1

Example 13.

<property name="mz.httpd.security.keystore" value="/opt/mz/keys/platform.keys"><property name="mz.httpd.security.keystore.password" value="">

3. Set the password property through the setpassword command

Example 14.

$ mzsh setpassword -f common.xml -p mz.httpd.security.keystore.password

11.2.2. Configuration Properties

The encryption is configured through properties. It is recommended to set the properties incommon.xml on the corresponding Convergent Mediation Engine™ installation (client or platform).The available properties are:

• mz.httpd.security

Set to value="true" if encryption is required. Default value is false.

• mz.httpd.security.keystore

The keystore file to use. The absolute path is required.

• mz.httpd.security.keystore.password

Set to the password for the keystore, as chosen in keytool. This password is encrypted throughthe normal Convergent Mediation Engine™ password encryption.

To set this value, the mzsh command setpassword is used, please refer to Section 8.3, “Com-mands Always Available” for command syntax.

Example 15.

Keystore password update:

$ mzsh setpassword -f common.xml -p mz.httpd.security.keystore.password

• mz.httpd.security.key.password

127

Core 4.1

Set to the password for the key, as chosen in keytool. By default this is the same as the keystorepassword. (This is also the default for keytool).

128

Core 4.1

12. High AvailabilityThe Convergent Mediation Engine™ High Availability (HA) solution includes any number of physicalhosts, each hosting any, or a combination of any, of the following components of Convergent MediationEngine™ :

• Convergent Mediation Engine™ Platform

• Convergent Mediation Engine™ Database

• Convergent Mediation Engine™ Execution Context

A component or combination of components is called a package. Each package runs a HA monitorserver. A package is monitored via the Convergent Mediation Engine™ HA client and the executionof the HA client is wrapped by the cluster daemon. The cluster daemon is a 3rd Party Product likeVeritas, Sun Cluster or MC Service guard. Each package is identified by virtual IP-addresses to enableany physical host within the cluster to run it. The state of a package is controlled with the standardmzsh commands; startup, shutdown and kill.

As an illustration, an example containing two nodes is given in the following. The Convergent MediationEngine™ Platform and Database comprise one package called Platform, while the Execution ContextsEC1 and EC2 are configured as separate packages. The first physical host serves as primary node forPlatform and EC1, and as secondary node for EC2. Vice versa for the second physical host.

Disk Array

EC1- Execution Context

Platform- Platform- Database



Platform- Platform- Database


Node 1

Cluster

Node 2

SAN

Preferred runtimeAlternate runtime

Figure 106. An example of a Convergent Mediation Engine™ High Availability cluster.

The Execution Context packages within a Convergent Mediation Engine™ cluster are independententities and are not affected by one another in case of failure. The Platform package will not be affectedif an Execution Context package fails. However, if several Convergent Mediation Engine™ componentsare bundled in one package, the HA scripts have to be modified to take the mutual relationship withinthis package into account.

129

Core 4.1

For instance, consider a package containing both Platform and an EC; if the EC fails and the Platformcomponent does not, in most cases a switch of physical host is not necessary. For simplicity, suchcombinations within a package must be avoided.

The example described in this section will be referred to throughout the whole document.

12.1.The HA ServerThe HA Server is responsible for providing information on the availability of the pico clients withina package. Pico Clients belonging to the same package will be able to interface each other. Only oneinterface per package has to be considered by the HA surveillance script.

Each package has to contain the following components:

• List of Pico Clients.

• HA Server.

• Reserved port.

For each surveillant package, the cluster daemon must make use of the following surveillance scripts:

• Monitor script, $MZ_HOME/ha/monitor. This script is used when requesting status of the HAServer via its port.

• Start script, $MZ_HOME/ha/online. This script is used to start the components in the package.

• Stop script, $MZ_HOME/ha/offline. This script is used to gracefully stop the components inthe package.

• Kill script, $MZ_HOME/ha/clean. This script is used to terminate the components immediatelyand should be used if the offline script fails.

The scripts must be modified to use the correct arguments and exit codes depending on whichHA 3PP product is used.

A HA Server uses a port where the status of the components may be obtained. The access can only befrom local host (127.0.0.1). Executing a ping request to HA Server with the selected componentswill result in a response stating OK or FAILED depending on the availability.

130

Core 4.1

Ha ServerEC2

Ha ServerPlatform

Ha ServerEC1

Ha ServerPlatform

Ha ServerEC1

Ha ServerEC2

Monitor ScriptNode 1

Monitor ScriptNode 2

Cluster

Node directory hosting the Cluster Daemon

ClusterDaemon

OK

OK

FAILED

FAILED

FAILED

OK

Node 1 Node 2

Figure 107. The Daemon script interfaces the HA Server of each package, and receives OK orFAILED for all requested pico clients within the package.

The communication between the HA Server and the cluster daemon in detail:

1. The cluster daemon executes the monitor script with the configured arguments. The monitorscript executes the HA client to pull the HA Server for information on the availability of its picoclient(s).

$ ./monitor <resource> <java_home> <mz_home> \<unix_user> <port> <pico client/clients>

2. The HA Client pings all its pico clients and returns OK or FAILED to the monitor script thatexit with the appropriate exit code to the cluster daemon.

3. The cluster daemon acts upon this information. In case of failure the offline scripts executesand then the clean script on the failed node and the online script starts the package on a dif-ferent node.

12.2. Retrieval - Platform, Node 1When the cluster daemon receives a FAILURE from the Platform package which causes a switchof the physical hosts, the following behavior applies:

1. Make sure the Platform package is completely stopped by calling its offline script.

131

Core 4.1

2. The Platform package should be down, but to make sure it is completely down, call its cleanscript.

3. The Platform depends on the Database which is a part of the package. Therefore the databaseentity must be shutdown as well. Note that the Database entity and itsmonitor/online/offline functionality is not part of the Convergent Mediation Engine™HA solution Please refer to the 3rd party product recommended by the specific vendor to handlethe database.

4. The HA software moves the Platform package to an alternative host.

5. The Database entity, including its listener, must be be brought up before the Platform. The Con-vergent Mediation Engine™ Platform depends on the Database. The HA daemon executes theDatabase online script on the alternative host.

6. When the Database entity has responded to its first successful monitor request. The HA daemonexecutes the online script for the Convergent Mediation Engine™ Platform.

7. Restart the EC1 and EC2 packages by calling its offline and online scripts. This is done tomake sure the downtime is a little as possible. The EC1 and EC2 packages will find the Platformpackage within the time as defined in REF reconnect time, however to minimize downtime theyare restarted.

All scheduled workflows belonging to the Execution Contexts are brought up on the nextscheduled time. If needed, critical scheduled and unscheduled workflows may be restartedby a script making use of the mzsh. Please refer to Section 8.1, “Startup” for further in-formation.

12.3. Retrieval - EC2, Node 2When the cluster daemon receives a FAILURE from the EC2 package which causes a switch ofphysical hosts, the following behavior applies:

1. Make sure the package is completely down by calling its offline script. All workflows willbe brought down immediately, interrupting the current batches being processed.

2. The EC2 package should be down, but to make sure it is completely down, call its clean script.

3. The HA software moves the EC2 package to an alternative host.

4. Restart the EC2 package by calling its online script.

Workflows that are scheduled to be activated periodically will be brought up automatically thenext time they are due to start. The same applies to workflows scheduled by triggers. Note thatif chains of events are configured, the counting will restart. This since triggered events are keptin memory only.

Unscheduled workflows, for instance Realtime workflows, have to be attended to manually.For this purpose, modify the online script to log in to $MZ_HOME/bin/mzsh and restartthem.

132

Core 4.1

12.4. Configuration and Installation

12.4.1. Configuration of HA Cluster Daemon

Configuring the HA cluster daemon is beyond the scope of this manual. Please refer to the HA softwaredocumentation for this purpose. It is of importance to configure dependencies and availability of theresources within the cluster correctly. The HA cluster daemon must handle Network interfaces, Disks(NFS or SAN) and Database monitoring and switching, including dependencies.

12.4.2. Pico Clients

Set a standard Convergent Mediation Engine™ Platform / Execution Context installation to be run onthe preferred host. Make sure to only use the virtual IP addresses (that is, the Package addresses) whenrequested to enter any.

12.4.2.1. HA jar File

For all monitored instances in the HA cluster do the following: Copy the $MZ_HOME/ha/ha.jarfile to the directory $MZ_HOME/lib.

12.4.2.2. Parameters

Reserve a unique port for the communication for each package. Edit the files:$MZ_HOME/etc/executioncontext.xml and platform.xml

1. Modify the pico.bootstrapclass, add the HA Server.

2. Add the HA mz.ha.port property.

Example 16.

property_name="mz.ha.port" value="6801"property_name="pico.bootstrapclass"value="com.digitalroute..., com.digitalroute.ha.HAServer"

In order to set up Convergent Mediation Engine™ to an HA environment, various properties need tobe added in the following files:

1. commandline.xml

• pico.rcp.server.host

Set to value="0.0.0.0"

2. common.xml

• pico.rcp.codeserver.host

Set to the virtual IP address.



3. executioncontext.xml

133

Core 4.1

ec.httpd.host

Set to value="0.0.0.0".

•



Optionally, add any of the following HA properties. If multiple properties are specified the pingreturns FAILED if any of the properties have exceeded their threshold.

• mz.ha.interval to define the time interval in seconds for the counters specified below.

• mz.ha.min_number_of_transactions to define the minimum number of closedtransactions during the interval specified above.

• mz.ha.max_aborts to define the maximum number of aborts during the interval specifiedabove.

• mz.ha.min_activations to define the minimum number of activations during the in-terval specified above.

• mz.ha.min_activations_for_aborts to define the minimum number of activationsbefore the aborts is counted during the interval specified above.

• mz.ha.min_consumes to define the minimum number of consumes during the intervalspecified above.

• mz.ha.log_file to define the file where the debug and statistics should be saved, i.e allthe counters specified are saved in this file.

4. platform.xml

• mz.wi.serveraddress

Set to value="0.0.0.0".



12.4.2.3. HA Scripts

Unpack the files found in the $MZ_HOME/ha directory, and copy the files online, offline,clean and monitor to the cluster surveillance directory. Adjust them to suit the conditions of theHA Cluster daemon. Consider the following:

• Time the response from the HA Servers for different load conditions, to be able to distinguishbetween an unavailable package and a heavily loaded machine.

• If a Convergent Mediation Engine™ package containing the platform fails, all other packages willfail, so make sure to call their online and offline scripts after the Convergent MediationEngine™ Platform is safely up and running.

• When bringing up a failed Execution Context package, restart unscheduled workflows via$MZ_HOME/bin/mzsh.

The function provided by each script is:

• The monitor script pings a pico instance and returns OK in case the instance is up and runningcorrectly otherwise FAILED.

134

Core 4.1

• The offline script makes sure all parts of a pico instance are stopped. It must be executed whenthe platform availability evaluation fails.

• The online script starts a pico instance.

• The clean makes sure all parts of a pico instance are stopped. It must be executed when theoffline script takes too long to execute.

12.4.3.The HA Server Interface

Communication with the HA Server Server may be done via a telnet client. The following commandsare supported:

Lists the available commands.help

Checks the pico client(s) for availability. Returns OK if available, andFAILED if is not.

ping <pico_name>

Closes the connection.quit

Closes the connection.exit

If the machine is heavily loaded, the ping request may take a while before being answered.Consider this when designing the scripts.

135

Core 4.1

13.TerminologyThis section contains a glossary for all abbreviations and terms used throughout the Convergent Medi-ation Engine™ documentation.

13.1. Abbreviations

Analysis Programming LanguageAPL

Abstract Syntax Notation 1ASN.1

Call Event DetailCED

Execution ContextEC

Stand-alone Execution ContextECSA

Error Correction SystemECS

File Transfer ProtocolFTP

High AvailabilityHA

Home Data Clearing HouseHDCH

Home Public Mobile NetworkHPMN

Input/Output GroupIOG

Internet ProtocolIP

Inter WorkflowIWF

Java ArchiveJAR

Java Virtual MachineJVM

Management Information BaseMIB

Meta Information ModelMIM

Operating SystemOS

Public Mobile NetworkPMN

Return Account ProcedureRAP

Structured Query LanguageSQL

Transfer Account ProcedureTAP

Usage Detail RecordUDR

Ultra Format Definition LanguageUFDL

Visiting Public Mobile NetworkVPMN

Extensible Markup LanguageXML

13.2.Terms

A type of application, executing a specific task, in workflows.Agent

Programming language used by the Analysis and Aggregation agents,utilized to analyze or manipulate UDR fields.

Analysis ProgrammingLanguage

Start of a data batch to be fed into a workflow. Some Collection agentsemit Start Batch messages at the beginning of a data batch.

Begin Batch

Message initiated by any agent that identifies data corruptness.Cancel Batch

136

Core 4.1

An agent that collect and inserts data into workflows. This include filebased and UDR based collectors.

Collection agent

The data transferred through a workflow between a Start Batch and EndBatch message. In file based mediation, a data batch often contain acomplete file.

Data batch

End of a data batch fed into a workflow.End Batch

Execution Contexts are responsible for executing workflows. One or manyhosts in a Convergent Mediation Engine™ system can be configured asExecution Contexts, allowing workflow balancing.

Execution Context

An agent that distributes data from workflows.Forwarding agent

An identifier for a specific resource, published by the workflow or anagent. MIM resources are static during the workflow execution.

MIM resource

The current value of a MIM resource. MIM values can be assigned theirvalues either static, or at Start Batch, or at End Batch messages.

MIM value

Represents an agent once deployed in a workflow editor work area. Manynodes can represent the same agent in a workflow.

Node

An agent that process data in a workflow. In a workflow, a processingagent always has at least one incoming and one outgoing data stream.

Processing agent

Convergent Mediation Engine™ format management system.Ultra

Language used to describe formats in the Ultra system.Ultra Format DefinitionLanguage

A set of nodes connected to each other to represent a data flow. Workflowsare defined in the Workflow Editor.

Workflow

137

Core 4.1

Database Agents 4.1


Version 1.24

1





2

Database Agents 4.1



2. Database Collection Agent ............................................................................................ 52.1. Configuration .................................................................................................. 5

2.1.1. Source Tab ........................................................................................... 52.1.2. Assignment Tab ..................................................................................... 62.1.3. Condition Tab ....................................................................................... 72.1.4. Advanced Tab ....................................................................................... 8

2.2. Transaction Behavior ........................................................................................ 82.2.1. Emits .................................................................................................. 92.2.2. Retrieves .............................................................................................. 9

2.3. Introspection ................................................................................................... 92.4. Meta Information Model .................................................................................... 9

2.4.1. Publishes ............................................................................................. 92.4.2. Accesses ............................................................................................. 10

2.5. Node Message Events ...................................................................................... 103. Database Forwarding Agent ........................................................................................ 12

3.1. Configuration ................................................................................................ 123.1.1. Target Tab ........................................................................................... 123.1.2. Assignment Tab ................................................................................... 143.1.3. Advanced Tab ...................................................................................... 14




3.5. Node Message Events ...................................................................................... 164. General ................................................................................................................... 17

4.1. Inter-Workflow Communication, Using Database Agents ....................................... 174.1.1. Pending Transaction Table ..................................................................... 174.1.2. Exchanging Storable Data ...................................................................... 18

4.2. Configuration ................................................................................................ 184.2.1. Assignments ........................................................................................ 184.2.2. Value Types ......................................................................................... 19

4.3. Tables and Stored Procedures ............................................................................ 214.3.1. Working Table ..................................................................................... 214.3.2. After Collection Stored Procedure ........................................................... 214.3.3. Database Forwarding Target Stored Procedure ........................................... 224.3.4. Cleanup Stored Procedure ...................................................................... 234.3.5. After Forwarding Stored Procedure .......................................................... 23

4.4. SQL Server Considerations .............................................................................. 24

3

Database Agents 4.1

1. IntroductionThis document describes the Database Collection and Database Forwarding agents being parts of thestandard Batch agents available on the Nokia Convergent Mediation Engine™ platform.

If not stated elsewhere, this guide uses Oracle as the Convergent Mediation Engine™ standard database.



• SQL



4

Database Agents 4.1

2. Database Collection AgentThe Database Collection agent collects records from a database table on a remote or local databaseand inserts them into a Convergent Mediation Engine™ workflow.

On each invocation the agent will create an SQL select statement based on the user configuration,and retrieve all rows matching the statement. For each row a UDR is created and populated, accordingto the assignments in the configuration window.

The agent uses a transaction ID to utilize a rollback functionality. Additionally, based on configurations,the agent deletes the data in the table after it has been inserted into the workflow. When all thematching data has been successfully processed, the agent stops to await the next activation, scheduledor manual initialization.

2.1. ConfigurationThe Database Collection agent configuration window is displayed when the node in a workflow isdouble-clicked or right-clicked, selecting Configuration....

2.1.1. Source Tab

Figure 1. Database Collection agent configuration window, Source tab.

The Source tab contains configurations related to the placement and handling of the source databasetable and its data, as well as the UDR type to be created and populated by the agent.

Type of UDR to be created and populated.UDR Type

Name of the database that the agent will connect to and retrieve data from. Thelist is populated each time the configuration window is opened. Refer to the

Database

Convergent Mediation Engine™ Core - User Guide, for details on database profilesetup.

Refresh must be selected if changes have been made in the customer database.This will update the presented information in the Source tab.

Name of the working table in the selected Database, in which the data to be col-lected resides. The list is populated each time a new Database is selected. Refer

Table Name

to Section 4.3.1, “Working Table” for a further description and an example of aworking table.

5

Database Agents 4.1

Name of the column in the selected Table, which is utilized for the transactionID. The list is populated each time a Table Name is selected. The column mustbe of the data type number, with at least twelve digits.

Transaction IDcolumn

If enabled, this option will remove the collected data rows from the working table.Remove

If enabled, this option will assign the value -1 to the Transaction ID columnfor all the collected rows.

Mark as Collec-ted

If enabled, this option executes a user defined stored procedure that is responsiblefor the handling, most often removal, of the collected data.

Run SP

It is important that this procedure actually deletes the data or sets theTransaction ID to -1, to avoid the data being recollected.

Refer to Section 4.3.2, “After Collection Stored Procedure” for a further descriptionand an example of such a stored procedure.

If enabled, all collected data will remain in the table. The state of the data (thevalue of the Transaction ID) will remain unchanged, enabling the same data to be

Ignore

collected over and over again. This option is mainly intended to be used duringthe design and test phase of the workflow.

2.1.2. Assignment Tab

Figure 2. Database Collection agent configuration window, Assignment tab.

The Assignment tab contains the mapping of column values to UDR fields. The content and use ofthis tab is described in detail in Section 4.2.1, “Assignments”.

If the Source tab is correctly configured and the Assignment tab is selected, the table will automaticallybe populated, as if Refresh was clicked. If assignments already exist in the Assignment tab, thenRefresh must be manually clicked for the assignments to be updated with the configurations in theSource tab.

Potential changes in the database table will not be visible until the Refresh button for the data-base, in the Source tab, has been clicked.

Only the value types UDR Field, To UDR and NULL, described in Section 4.2.2, “Value Types”, areavailable for selection.

6

Database Agents 4.1

2.1.3. Condition Tab

Figure 3. Database Collection agent configuration window, Condition tab.

In the Condition tab, query constraints may be added to limit the selection of data. The statement mustfollow the standard SQL WHERE-clause syntax, except for the initial where and the final semi-colon(;) which are automatically appended to the entered condition statement. It is, for instance, possibleto include an order by statement to get the rows sorted.

The condition statement may contain dynamic parameters, represented by question marks that inrun-time will be replaced by a value. If the text area contains question marks, Assign Parameters...must be selected, to be able to assign values to these parameters. The assignments are made in theParameter Editor dialog.

Figure 4. Database Collection agent configuration window, Parameter Editor window.

In this dialog each parameter, represented as a question mark in the condition statement, appears asone row. The value types available are MIM Entry and Constant. Since constant values are alsopossible to be given directly in the condition statement, MIM Entry is most likely to be used here.

7

Database Agents 4.1

2.1.4. Advanced Tab

Figure 5. Database Collection agent configuration window, Advanced tab.

The Advanced tab contains a setting for performance tuning and allows viewing of the generated SQLstatement, based on the configuration in the Source, Assignment and Condition tabs.

The number of UDRs (rows) to be removed between each database commitcommand. This value is used to tune the performance. If tables are small and

Commit WindowSize

contain no Binary Objects, the value may be set higher than the default. Defaultis 1000.

Rows are only removed if Remove is enabled in the Source tab.

In this window the SQL statement to be used to query the database is shown.It may be used for debug purposes or for pure interest.

Generated SQLStatement

In order for the statement to appear, the Source and Assignment tabs have tobe properly configured. If not, information about the first detected missing orerroneous setting is displayed.

2.2.Transaction BehaviorThe Database Collection agent performs some extra maneuvers to ensure that data is not recollectedor lost if the workflow aborts before the collection of a batch has finished correctly .

1. A unique Transaction ID is retrieved for each new batch.

2. The pending transaction table is queried for all pending Transaction IDs, to be compared withthe transaction IDs in the working table, from which the node will collect.

3. The SQL query is built and executed, and all matching rows are collected. In addition to the userdefined condition, the agent adds some conditions to the query, to ensure that pending data, can-celled data and data marked as collected is not collected.

4. For each row that has been successfully converted to a UDR, the agent updates its TransactionID column to the Transaction ID retrieved in bullet 1.

5. When all rows matching the query have been successfully collected, the After Collection config-uration in the Source tab, is used.

a. If Remove, all rows with the given Transaction ID are removed in batches of the size con-figured as the Commit Window Size, in the Advanced tab.

8

Database Agents 4.1

b. If Mark as Collected, all rows with the given Transaction ID are updated with the reservedTransaction ID value -1.

c. If Run SP, the user defined stored procedure is executed. Refer to Section 4.3.2, “AfterCollection Stored Procedure” for details.

2.2.1. Emits

Emitted after the SQL select statement execution.Begin Batch

Emitted after the SQL select statement execution, when all possible matching rowshave been successfully inserted as UDRs in the workflow.

End Batch

If the SQL select statement does not return any data, Begin and End Batch will not be emitted.Not even if Produce Empty Files is selected in a Forwarding Disk agent.

2.2.2. Retrieves

All rows with the current Transaction ID are updated with the reserved TransactionID -2. If these rows are to be recollected, the Transaction ID column must first beset to 0(zero). If set to NULL this row cannot be collected.

Cancel Batch

The database row that issued the Cancel Batch request is written to the System Log.

If the Cancel Batch behavior defined on workflow level is configured to abortthe workflow, the agent will never receive the last Cancel Batch message. Inthis situation the rows will not be updated with the reserved Transaction ID-2.

An End Batch call will be issued, causing the original batch returned by the SQLquery to be split at the current UDR. The database commit command is executed,followed by a new select statement to fetch the remaining UDRs from the table.

Hint End Batch

2.3. IntrospectionThe agent produces the UDR types selected from the UDR Type list.


2.4.1. Publishes

A parameter containing the name of the database the agent is collecting from.Database

Type string, defined as MIM type global.

A parameter containing the name of the working table the agent is collecting from.Table


A parameter containing the name of the currently processed file, as defined at thesource.

Source Filename

9

Database Agents 4.1

Type string, defined as MIM type header.

2.4.2. Accesses

Various resources, if MIM parameter assignments are done in the Parameter Editor in the Conditiontab.


2.5. Node Message EventsStart collecting

Indicates that possible cleanup procedures are finalized and that the actual collection begins.

No rows selected from table tablename

Reported, along with the name of the working table, if no rows have been selected for collection.

Avoids reading the following pending txn ids: list of ids

Reported, along with a list of Transaction IDs, if the constructed SQL select statement finds anypending Transaction IDs in the pending transaction table. Rows marked with these transaction IDswill be excluded by the query.

Marking collected data as cancelled

Reported when a Cancel Batch is received.

Deleting collected data

Reported when collected rows are removed after collection, if Remove is selected in After Collection.

Has deleted n rows in tablename

Subevent to the Deleting collected data event, stating the number of rows that are removedby each SQL commit command. The maximum number depends on the Commit Window Size.

Marking collected data

Reported when collected rows are marked as collected, if Mark as Collected is selected in AfterCollection.

Has updated n rows in tablename

Subevent to the Marking collected data event, stating the number of rows that are markedas collected by each SQL commit command. The maximum number depends on the Commit WindowSize.

Taking care of collected data via SPname

Reported, along with the name of the stored procedure, when it is called after collection, if Run SP isselected in After Collection.

Try to force running Stored Procedure to Stop

10

Database Agents 4.1

Reported when stopping a workflow that has a Stored Procedure running.

Ready with table: tablename

Reported, along with the name of the working table, when all rows are collected from it.

11

Database Agents 4.1

3. Database Forwarding AgentThe Database Forwarding agent inserts UDR data into a database table, based on user defined mappingsbetween UDR fields and database table columns. Also, the agent offers the possibility of populatingcolumns with other types of data. The data is inserted either using a plain SQL statement, or througha call to a stored procedure that will be responsible of inserting the data.

The agent does not only map and forward the data. A special column in the target table is also assigneda unique Transaction ID, generated for each batch. In relation to this, a pending transaction table isutilized to indicate that a batch is open. The Database Collection agent also utilizes this table to preventproblems if collecting data from the target table.

3.1. ConfigurationThe Database Forwarding agent configuration window is displayed when the node in a workflow isdouble-clicked or right-clicked, selecting Configuration....

3.1.1.Target Tab

Figure 6. Database Forwarding agent configuration window, Target tab.

The Target tab contains configurations related to the target database table and the UDR Type that willpopulate it with data.

Type of UDR to populate the target database table.UDR Type

Profile defining the database that the agent is supposed to connect and forward datato. The list is populated each time the configuration window is opened. Refer to the

Database

Convergent Mediation Engine™ Core - User Guide, for details on database profilesetup.

Select Refresh if changes have been made in the customer database, to update thepresented information in the Target tab.

Determines if the insertion of data is to be performed directly into the target table,or via a stored procedure.

Access Type

• Direct - Insertion of data is performed directly.

• Stored Procedure - Insertion of data is performed via a stored procedure.

12

Database Agents 4.1

Depending on the selected Access Type, the target database table name, or the storedprocedure name, is selected. The list is populated each time a new Database or AccessType is selected.

Table Name orSP Name

Refer to Section 4.3.1, “Working Table” for a further description and an example ofa working table, and Section 4.3.3, “Database Forwarding Target Stored Procedure”for further information about the stored procedure.

Name of the column in the selected table, or the parameter from the selected storedprocedure, which is utilized for the Transaction ID. The list is populated each timea Table Name or SP Name is selected.

Transaction IDcolumn

The column must be of the data type number, with at least twelve digits.

If the selected Access Type is Stored Procedure, the agent does not automaticallyclean up the target table, in case of a workflow abortion (Cancel Batch). If that is

Cleanup SP

the case, the customer must supply a stored procedure that manages the clean up.The list is populated each time a new Database is selected.

Refer to Section 4.3.4, “Cleanup Stored Procedure” for a further description and anexample of a Cleanup Stored Procedure.

Name of the target table for the stored procedure. This field is only enabled if theAccess Type is Stored Procedure. The list is populated each time a new Databaseis selected.

SP TargetTable

If this node is chained with a Database Collection node in another workflow, bothnodes need to be aware of the mutual table. In the Collection node, a table to collectfrom is always selected. However, in the Forwarding node, it is possible to selectthe update of the table to be done via a stored procedure. If that is the case, the targettable for the stored procedure must be selected here. Refer to Section 4.1.1, “PendingTransaction Table” for further details.

The correct name of the SP Target Table must be selected, or else a DatabaseCollection node will be able to collect pending data that is not supposed to becollected. This may cause data duplication.

If enabled, this option causes a user defined stored procedure to be called when theforwarding process terminates. It will then receive the transaction ID for the forwardedrows as input.

Run SP

This option is used for transaction safety when the table is read from another system,to ensure no temporary rows are read. Rows are classified as temporary until EndBatch is reached. In case of a crash before End Batch is reached, the workflow needsto be restarted for the temporary rows to be expunged.

Convergent Mediation Engine™ specific database tables from the Platform database must neverbe utilized as targets for output. This may cause severe damage to the system in terms of datacorruption that in turn may make the system unusable.

13

Database Agents 4.1

3.1.2. Assignment Tab

Figure 7. Database Forwarding agent configuration window, Assignment tab.

The Assignment tab contains the assignment of values to each column or stored procedure parameter.The content and use of this tab is described further in Section 4.2.1, “Assignments”.

The Column Name column does not necessarily contain column names. If Stored Procedure is selectedas the Access Type, this column will hold the names of all incoming parameters that the stored procedureexpects.

If the Target tab is correctly configured and the Assignment tab is selected, the table will automaticallybe populated, as if Refresh was clicked. If assignments already exist in the Assignment tab, thenRefresh must be manually selected, for the assignments to be updated with the configurations in theTarget tab.

Potential changes in the database table will not be visible until the Refresh for the database, inthe Target tab, has been selected.

All Value Types, described in Section 4.2.2, “Value Types”, except for To UDR, are available forselection.

When using Function as Value Type, it is not allowed to use question marks embedded instrings. Convergent Mediation Engine™ will interpret a question mark as a parameter.

3.1.3. Advanced Tab

Figure 8. Database Forwarding agent configuration window, Advanced tab.

The Advanced tab contains a setting for performance tuning and viewing the generated SQL statement,based on the configuration in the Target and Assignment tabs.

The number of UDRs (rows) to be inserted or removed between each databasecommit command. This value may be used to tune the performance. If tables are

Commit WindowSize

small and contain no Binary Objects, the value may be set to a higher value thanthe default. Default is 1000.

14

Database Agents 4.1

Rows are inserted for each UDR that is fed to the node. All UDRs are stored inmemory between each database commit command, to enable rollback. Rows areremoved at the next workflow startup in case of a crash recovery.

In this window, the SQL statement that will be used to populate the database, isshown. This field may not be edited, however, it is useful for debug purposes orfor pure interest.

General SQLStatement

In order for the statement to appear, the Target and Assignment tabs have to beproperly configured, or else information about the first detected missing or erro-neous setting is displayed.

3.2.Transaction BehaviorThe agent utilizes a Transaction ID, unique for each batch, in two ways.

A. To make sure that inserted (distributed) rows are removed in case the batch is cancelled. This toavoid duplicated rows. To handle this, the agent inserts its batch Transaction ID in the assignedTransaction ID column. If the batch is cancelled, all rows matching the batch Transaction IDwill be removed again.

If a stored procedure is used to populate the table, the configured Cleanup SP must be able todo the same, or something similar, to avoid duplicates. Refer to Section 4.3.4, “Cleanup StoredProcedure” for further descriptions and an example of cleanup stored procedure.

B. To make sure that a potential Database Collection agent does not collect rows from the targettable, before the current batch is closed. To handle this, the agent populates a pending transactiontable with the current Transaction ID, database and table name in the beginning of the batch andremoves the entry in the end of the batch. Refer to Section 4.1, “Inter-Workflow Communication,Using Database Agents” for a detailed description on this behavior.

3.2.1. Emits

None.

3.2.2. Retrieves

Retrieves a Transaction ID and inserts an entry in the pending transaction table.Begin Batch

Deletes the pending Transaction ID row.End Batch

Removes the distributed rows with the current Transaction ID or calls the configuredCleanup SP. The pending Transaction ID row is deleted.

Cancel Batch

3.3. IntrospectionThe agent consumes the selected UDR type.


3.4.1. Publishes

A parameter containing the name of the database the agent is distributing to.Database

15

Database Agents 4.1


A parameter containing the name of the working table or the stored procedure the agent isdistributing to.

Table


3.4.2. Accesses

Various resources, if MIM parameter or function assignments are made in the Assignment tab.


3.5. Node Message EventsReady with table/SP: tablename

Reported when a stored procedure starts running, if Run SP is selected in the Database ForwardingTarget tab.

Rollback transaction data

Reported when the agent receives a Cancel Batch or when recovering after system abortion.

Has deleted n rows in tablename

Subevent to the Rollback transaction data event, stating the number of rows that are removedby each SQL commit command. The maximum number depends on the Commit Window Size.

Try to force running Stored Procedure to Stop

Reported when stopping a workflow that has a Stored Procedure running.

16

Database Agents 4.1

4. General

4.1. Inter-Workflow Communication, Using DatabaseAgents

Data may propagate between workflows or Convergent Mediation Engine™ systems by combining aDatabase Forwarding node with a Database Collection node, where the exchange point is a mutualdatabase table.

When using the same table, the collection node must make sure that it does not collect data that theforwarding node is simultaneously feeding with data from its current batch.

Transfer of UDRs between workflows is ideally handled using the Inter Workflow agents. TheDatabase agent approach is useful in case of wanting to change the contents of the UDRs. An-other use is when wanting to pass on MIM values and merge batches at the same time. In theInter Workflow agent case, only the MIM values for the first (Header MIMs) and last batch(Trailer and Batch MIMs) are considered. Using the Database agents, MIM values may bemapped into database columns.

4.1.1. Pending Transaction Table

The Convergent Mediation Engine™ database, hosts a table where pending transactions are registered.A pending transaction is an ongoing population of a table by a Database Forwarding node. The pendingtransaction continues from a Begin Batch to an End Batch. The purpose of this table is for DatabaseCollection nodes to avoid collecting pending data from the table that a Database Forwarding node iscurrently distributing to.

The pending transaction table holds database names and table names. Thus, before a collection sessionstarts, the collector evaluates if there are any pending Transaction IDs registered for the source databaseand table. If there are, rows matching the Transaction IDs will be excluded.

In the following figure, the Database Collection node will exclude all rows with transaction ID 187.

17

Database Agents 4.1

A Database Forwarding node may be configured to target a stored procedure, instead of a table directly.In such cases the user must specifically select the table that the stored procedure will populate (SPTarget Table). The reason for that is that the pending transaction table must contain the table name,not the SP name, so that the selected table name in the Database Collection node can be matched.

4.1.2. Exchanging Storable Data

All Convergent Mediation Engine™ UDRs have a special field named Storable. This field containsthe complete UDR description and all its data. If UDRs, having many fields or a complex structure tobe exchanged, it could be suitable to store the contents of the Storable field in the database. In thatway the table would only need one column. The database type of that column must be a RAW, LONGRAW or a BLOB.

The data capacity of the column types RAW, LONG RAW and BLOB differs. Consult thedatabase documentation. For performance reasons it is advised to use the smallest type possiblethat fits the UDR content.

When configuring the Database Forwarding node, the Storable field from the UDR is be assigned tothe table column in a straight forward fashion. However, when collecting that type of data the columnassignment must not be made to the Storable field. Instead To UDR is selected in the Value Typefield.

When the Database Collection node detects a mapping of type To UDR, the selected UDR type is notconsulted for what UDR type to create. The information about the UDR type will be found in the dataof the column itself. Thus, if the UDR stored in the column is of another type than selected in theSource tab, the type to be distributed by the Database Collection node is the type actually found.

4.2. Configuration

4.2.1. Assignments

The Database agents are designed to either collect data from a database column and assign it to a UDRfield, or vice versa. In their configuration they share the Assignment tab, where these mappings areconfigured. Due to the resemblance this configuration is described here.

Figure 9. Database Collection/Forwarding agents configuration window - Assignment tab.

Updates the table with all the columns or parameters from the selected table or storedprocedure (Database Forwarding agent, only).

Potential changes in the database table will not be visible until Refresh for thedatabase in the Source tab, has been selected.

Refresh

18

Database Agents 4.1

If rows already exist in the table, the refresh operation will preserve the configurationfor all rows with a corresponding column or parameter name. Thus, if a table has beenextended with a new column, the old column configurations will be left untouchedand the new column will appear when Refresh is selected.

The value type on each new column that appears in the table is automatically set toUDR Field.

Auto assignment:

All rows with no value assigned and with a value type of UDR Field will be targetedfor auto assignment in the end of the refresh process. If the selected UDR type containsa field whose name matches the column name, the field will be automatically assignedin the Value column. Matching is not case-sensitive and is done after stripping boththe column and field names from any characters, except a-z and 0-9.

Displays a list of all columns or stored procedure parameters (Database Forwardingagent, only) for the selected table or stored procedure, except the Transaction IDcolumn.

ColumnName

Displays the data type for each column as declared in the database table. If the columndoes not accept NULL this is displayed as: (NOT NULL).

ColumnType

Allows the user to select what type of value to be assigned to the column, or vice versa.Refer to Section 4.2.2, “Value Types” for details.

Value Type

The value to be assigned to the column, or vice versa. The technique of selecting avalue depends on the selected Value Type.

Value

It is important that the data type of the selected value corresponds to the datatype of the column. Most incompatibilities will automatically be detected,however, there are situations where validation is not possible.

4.2.2. Value Types

The Database agents offer six different types of values that may be assigned to a column, or vice versa.Depending on the agent, not all value types are applicable and will therefore not be available in thelist.

If selected, a UDR browser will be launched when the corresponding Value cell is selected.When a UDR field has been selected in the browser it will appear in the Value cell.

To save the user from launching the UDR browser for every cell to be assigned,the browser window may be kept on display. When a UDR field is selected andApply is selected or if a UDR field is double-clicked, the field will go into theValue cell of the selected row, provided that this row has a value type of UDRField.

The same rule applies when OK is selected in the browser, but then the browserwill be dismissed. It is possible to change target (Value cell) by selecting the desiredrow in the Assignment tab in the configuration window, while still keeping theUDR browser window open.

UDR Field

Whether data types of the selected UDR and the database column are compatible or not,is validated when the configuration dialog is confirmed.

Only applicable for the Database Forwarding agent.MIMEntry

19

Database Agents 4.1

If selected, a MIM browser will be launched when the corresponding Value cell is clicked.When a MIM resource has been selected in the browser it will appear in the Value cell.The previous Note for the UDR Field applies to this browser as well.

Whether data types of the selected UDR and the database column are compatible or not,is validated when the configuration window is confirmed.

Only applicable for the Database Forwarding agent.Constant

If selected, a text entry field will be available in the Value cell where any constant to beassigned to the column may be entered. The agent automatically appends possible quotesneeded in the SQL statement, based on the data type of the column.

Only applicable for the Database Forwarding agent.Function

If selected, a text entry field will be available in the Value cell where any database relatedfunction to be called may be entered. If the function takes parameters, these must bemarked as question marks. Selecting a cell containing question marks will display theFunction Editor window where each question mark is represented by a row.

Figure 10. Function Editor window.

The selection of parameter values follows the same procedures as for the assignment ofcolumn values but only Constant, UDR Field and MIM Entry are available as valuetypes.

If constants are entered in the Function Editor they must be quoted correctly sincethe agent has no way of knowing what data types they must have.

If selected, no value may be entered. In Database Collection nodes, NULL must be selectedfor all columns whose values are not mapped into a UDR Field. In Database Forwarding

NULL

nodes, NULL must be selected for columns that are populated with a NULL value orcolumns that, when inserted, will be populated by internal database triggers.

Only applicable for the Database Forwarding agent.FromUDR

It is selected if a complete UDR is to be stored in a binary column to later be collectedby a Database Collection node. The Database Forwarding node must populate the columnfrom the special field Storable, available in all UDR types. This is only applicable forcolumn types RAW, LONG RAW or BLOB.

Only applicable for the Database Collection agent.To UDR

It is supposed to be selected if a complete UDR has been stored in a binary column by aDatabase Forwarding node and that UDR will be recollected by the Database Collectionnode. The Database Forwarding node must have been populating the column from thespecial field Storable, available in all UDR types. If this value type is selected no otherassignments are allowed. If other columns exist their value types must be set to NULL.An evaluation to ensure the column type is actually a RAW, LONG RAW or BLOB, iscarried out.

20

Database Agents 4.1

4.3.Tables and Stored Procedures

4.3.1. Working Table

Following list holds information to be taken into consideration when creating the database table thata Database Collection node collects from, or a Database Forwarding node distributes to.

• The table must have a Transaction ID column, dedicated to the Database agent's internal use.The column could be named arbitrary but must be numeric with at least twelve digits. It must alsonot allow NULL.

• Reading from or writing to columns defined as BLOB will have a negative impact on performancefor both Database agents.

• It has proved inefficient to put an index on the Transaction ID column.

• Entries with Transaction ID column set to -1 (Mark as Collected) or -2 (Cancel Batch) mustbe attended to manually at regular intervals.

The following example shows a working table with a Transaction ID column named txn_id.

Example 1.

CREATE TABLE my_tab ( txn_id NUMBER(12) NOT NULL, a_num VARCHAR2(25) NOT NULL, b_num VARCHAR2(25) NOT NULL, duration NUMBER(10) );

4.3.2. After Collection Stored Procedure

If a Database Collection node has been configured to call a stored procedure after collection, it willbe called when each batch has been successfully collected and inserted into the workflow.

The procedure is expected to take one (1) parameter. The parameter must be declared as a NUMBERand the agent will assign the current Transaction ID to the parameter. The procedure must ensure thatthe rows with the supplied transaction ID are removed from the table, or their Transaction ID columnis set to -1.

The following example shows such a procedure that moves the rows to another table.

21

Database Agents 4.1

Example 2.

CREATE or REPLACE PROCEDURE my_move_sp (txn IN NUMBER) ISBEGIN --copy collected rows to another table INSERT INTO my_collected_data_tab (txn_id, a_num, b_num, duration) SELECT txn_id, a_num, b_num, duration FROM my_tab WHERE txn_id = txn; --now delete the rows DELETE FROM my_tab WHERE txn_id = txn;END;

It is recommended for the previously described stored procedure to use an internal cursor withseveral commits, not to overflow the rollback segments.

4.3.3. Database Forwarding Target Stored Procedure

If a Database Forwarding node has been configured to use a stored procedure as the Access Type theagent will call this procedure for each UDR that is to be distributed. The stored procedure must bedefined to take the parameters needed, often including a parameter for the Transaction ID. In the dialogthese parameters are assigned their values. When the procedure is called, the agent will populate eachparameter with the assigned value.

The following example shows a stored procedure that selects the number of calls made by thea_number subscriber from another table, calls_tab, and uses that value to populate the targettable.

Example 3.

CREATE OR REPLACE PROCEDURE my_insert_sp(a_num IN CHAR, b_num IN CHAR, txn IN NUMBER) ISBEGINDECLARE cnt_calls NUMERIC(5); BEGIN SELECT COUNT(*) INTO cnt_calls FROM calls_tab WHERE anumber=a_num;

INSERT INTO my_tab (from_num, to_num, txn_id, num_calls) VALUES (a_num, b_num, txn, cnt_calls); END;END;/

22

Database Agents 4.1

4.3.4. Cleanup Stored Procedure

If a Database Forwarding agent uses a stored procedure to populate the target table, a cleanup storedprocedure must be defined, that will remove all inserted entries in case of a Cancel Batch in theworkflow. The procedure is expected to take one parameter. The parameter must be declared as aNUMBER and the agent will assign the current Transaction ID to the parameter.

The following example shows such a procedure that removes all the entries with the current TransactionID.

Example 4.

CREATE OR REPLACE PROCEDURE my_clean_sp (txn IN NUMBER) ISBEGIN DELETE FROM my_tab WHERE txn_id = txn;END;/

4.3.5. After Forwarding Stored Procedure

The following example shows a stored procedure that marks the row as safe to read by another system.

Example 5.

CREATE TABLE billing_data ( customer_id varchar2(100) NULL, number_of_calls number(5) NULL, money_to_pay number(9) NULL, txn_id number(12) NULL, txn_safe_indicator varchar2(10) DEFAULT 'UNSAFE' NOT NULL);

CREATE or REPLACE PROCEDURE mark_billing_data_as_safe(txn IN number) ISBEGIN LOOP --updates 5000 rows at the time to spare rollback segments update billing_data set txn_safe_indicator = 'SAFE' where txn_id = txn and rownum <= 5000;

EXIT WHEN SQL%ROWCOUNT < 5000; COMMIT; END LOOP; COMMIT; END;

The billing system must avoid reading rows that contains 'UNSAFE' in the txn_safe_indicatorcolumn, to ensure no data is read that could be rolled back later on.

23

Database Agents 4.1

4.4. SQL Server ConsiderationsWhen distributing data to an SQL Server table, the following must be taken into consideration:

• The table must have at least one UNIQUE column.

• The column reserved for the Transaction ID must be of type bigint.

• Columns of type UNIQUEIDENTIFIER must be set with a function. Hence, map it to NULL inthe agent.

• Columns with an IDENTITY set, must be mapped to NULL in the agent.

24

Database Agents 4.1

Development Toolkit 4.1


Version 1.25

1





2



1.1. Prerequisites .................................................................................................... 51.2. Documentation ................................................................................................ 51.3. Abbreviations and Terms ................................................................................... 5

2. Contents ................................................................................................................... 62.1. Package Overview ............................................................................................ 6

3. Installation and Setup .................................................................................................. 73.1. Installing DTK 4.1 ........................................................................................... 73.2. Setting up the Environment ................................................................................ 73.3. Creating a User Defined jar File .......................................................................... 73.4. Creating a User Defined Package ........................................................................ 83.5. Committing a User Defined Package .................................................................... 8

4. Miscellaneous ............................................................................................................ 94.1. Environment ................................................................................................... 9

4.1.1. DREnvironment ................................................................................ 94.1.2. DRUIEnvironment ........................................................................... 104.1.3. DRWorkflowEnv ............................................................................... 104.1.4. DRRealtimeWorkflowEnv ............................................................... 104.1.5. DRBatchWorkflowEnv ..................................................................... 104.1.6. DRWfUIEnvironment ....................................................................... 104.1.7. DRServerExecEnv ........................................................................... 114.1.8. DRRealtimeServerEnv ................................................................... 114.1.9. DRBatchServerEnv ......................................................................... 11

4.2. Serialization .................................................................................................. 124.3. Exception ...................................................................................................... 12

4.3.1. Internationalization .............................................................................. 125. Agent Plug-Ins ......................................................................................................... 13

5.1. Realtime Vs. Batch ......................................................................................... 135.2. Standalone .................................................................................................... 145.3. Meta Information Model .................................................................................. 145.4. Agent User Interface ....................................................................................... 14

5.4.1. The User Interface Dialog ...................................................................... 155.4.2. Inspectable ......................................................................................... 185.4.3. Other User Interface Components ............................................................ 19

5.5. Agent Executable ............................................................................................ 205.5.1. DRAgent ........................................................................................... 215.5.2. DRObjectRouter ............................................................................. 215.5.3. DRBatchAgent ................................................................................. 225.5.4. DRBatchCollector ......................................................................... 245.5.5. DRBatchProcesssor ....................................................................... 245.5.6. DRRealtimeAgent ........................................................................... 255.5.7. DRRealtimeAsyncRouter ............................................................... 255.5.8. DRRealtimeThreadAgent ............................................................... 255.5.9. DRRealtimeCollector ................................................................... 255.5.10. DRRealtimeProcessor ................................................................. 26

5.6. Services ........................................................................................................ 265.6.1. Filename Template ............................................................................... 265.6.2. Filename Sequence ............................................................................... 265.6.3. Filename Sort Order ............................................................................. 275.6.4. File Sort Order .................................................................................... 275.6.5. ECS Batch .......................................................................................... 27

6. Additional Features ................................................................................................... 286.1. UDR ............................................................................................................ 28

6.1.1. DRUDR ............................................................................................... 286.1.2. DRUDRDesc ....................................................................................... 30

3


6.1.3. DRUDRField ..................................................................................... 306.1.4. DRUDRType ....................................................................................... 31

6.2. Events .......................................................................................................... 326.2.1. Historic Events .................................................................................... 326.2.2. Event Suppression ................................................................................ 326.2.3. DRBaseEvent ................................................................................... 33

6.3. Notifier Plug-Ins ............................................................................................ 336.3.1. DRNotifierTarget ......................................................................... 336.3.2. DRNotifierTargetUI ..................................................................... 34

6.4. APL Plug-Ins ................................................................................................. 356.4.1. DRAPLFunctionSignature ............................................................. 356.4.2. DRAPLExecutor ............................................................................... 35

6.5. Ultra field plugins ........................................................................................... 356.5.1. DRUltraFieldDecoder ................................................................... 366.5.2. DRUltraFieldEncoder ................................................................... 36

6.6. Task Plug-Ins ................................................................................................. 366.6.1. DRTaskExecutor ............................................................................. 366.6.2. DRTaskExecutorUI ......................................................................... 366.6.3. DRTaskExecutorMsgEvent ............................................................. 37

6.7. Desktop Window Plug-Ins ................................................................................ 376.7.1. DRDesktopPlugin ........................................................................... 37

4


1. IntroductionThis document describes the Development Toolkit (DTK), available in Convergent Mediation Engine™. With DTK it is possible to develop user-defined programs in Java to be incorporated into the Conver-gent Mediation Engine™ platform. This document will not include general Java programming languagedetails, but will instead describe the rules and recommendations that apply in order for the new plug-ins to work with the Convergent Mediation Engine™ platform.

This document holds a complete overview and description of all classes and interfaces in JavaDoc(HTML) format. In several places, only method and class names are referred, expecting the user tofind the full syntax details in the JavaDoc.

DTK also comes with examples for full-featured plug-ins. Therefore, this document provides limitedcode examples and the reader is instead encouraged to examine the examples.


• The Convergent Mediation Engine™ Platform.

• Java Platform Edition 5.0

1.2. DocumentationFor further information regarding Convergent Mediation Engine™ , please refer to Convergent Medi-ation Engine™ Core - User Guide.


5


2. ContentsThe Development Toolkit enables users to build their own plug-ins for the standard Convergent Medi-ation Engine™ platform. This includes agents, tasks, events, desktop features and additions to thestandard Analysis Programming Language.

2.1. Package OverviewThe following packages are available:

Contains APL plug-in support classes.apl

Contains classes for viewing and manipulation of UDR contents, including UDR relatedgraphical user interface tools, such as the UDR Browser.

drudr

Contains events and notifier classes, related to the Event Notification Editor.event

Contains exception handling classes.exception

Contains various classes which do not fit in other packages.misc

Contains classes related to Convergent Mediation Engine™ serialization.storable

Contains classes defining a Convergent Mediation Engine™ task.task

Contains all workflow related functionalities, for instance, agent development, serviceand MIM handling.

wf

Contains Ultra field plugin support classes.ultra

6


3. Installation and SetupThis document forms a part of the DTK 4.1. The files contained on the DTK 4.1 CD are:

This document.devkit.pdf

Installer for DTK 4.1.devkit_4.1.x.y.mzp

Installer for user doc and DTK online help.documentation_standard_devkit_4.1.x.y.mzp

The Java JDK is not included. Download version Java Platform Edition 5.0 from ht-tp://java.sun.com

3.1. Installing DTK 4.1First, copy the DTK install files to a folder in your tmp directory:

> mkdir /tmp/dtk41installer> cp <cdpath>/*.mzp /tmp/dtk41installer/> cd /tmp/dtk41installer

Finally, issue the commands below to install the DTK at the location of the MZ_HOME environmentvariable. The files will be copied to a new subdirectory called dtk41 and the online help and examplecode will be added to the system.

> java -Dmz.home=$MZ_HOME -jar devkit_4.1.x.y.mzp install> java -Dmz.home=$MZ_HOME -jar documentation_standard_devkit_4.1.x.y.mzp install

3.2. Setting up the Environment

Unpack Java JDK Platform Edition 5.0Required Files

Place devkit.jar in any directory.

CLASSPATH=<path>/devkit.jar:<path>/my_dev_dirEnvironment Variables

PATH=$PATH:/path/to/SDK/java/bin

3.3. Creating a User Defined jar FileCompile your classes, for instance

> javac -classpath $CLASSPATH com/mycompany/myagent/*.java

It is strongly recommended for the Java code not to use the default package context. All classesin Convergent Mediation Engine™ must have unique names.

Create a jar file:

> jar cvf my_agent.jar com/mycompany/myagent/*.class

7


It is also possible to add resources to a jar file, just append any resource name to your jar create com-mand. This allows you to, for instance, add images that your class depends on to the jar.

3.4. Creating a User Defined PackageIn order to insert the jar file into the system, a package containing the jar file must be created. Apackage gives the jar file a name and a version. This name and version will be available from theAbout window once the new package has been committed into the Convergent Mediation Engine™system.

A package is created using the pcreate option to the Convergent Mediation Engine™ shell:

> mzsh username/password pcreate \"My Agent" 1.0 my_agent.mzp -level execution \file=my_agent.jar -level execution file=my_agent_javahelp.jar

Please refer to Convergent Mediation Engine™ Core - User Guide, for details on the ConvergentMediation Engine™ shell and the pcreate option.

3.5. Committing a User Defined PackageNew/updated code is inserted into the system from the Convergent Mediation Engine™ shell:

> mzsh username/password pcommit my_agent.mzp

The name (My App from the previous section) is the package key. Thus if a new package withthe same name is inserted, then the old code for that package name is replaced.

8


4. Miscellaneous

4.1. EnvironmentMany DTK classes hold various types of "environment" objects. These objects contain useful inform-ation and utilities related to the whereabouts of the class. For instance, agent related classes have accessto the workflow environment where other workflow configurations may be obtained. There are alsoenvironment objects referencing data in the Convergent Mediation Engine™ system.

The environment classes can be divided into three categories:

• Common interfaces, including DREnvironment, DRWorkflowEnv, DRRealtimeWork-flowEnv and DRBatchWorkflowEnv

• Client interfaces, including DRUIEnvironment, DRWfUIEnvironment

• Server interfaces, including DRServerExecEnv, DRRealtimeServerEnv and DRBatch-ServerEnv

Common interfaces are utilized both in client and server side code and are mainly super-interfaces forthe other interfaces. Client interfaces are utilized in code related to the Convergent Mediation Engine™user interface. Server interfaces are utilized in code related to the execution of workflows.

The following figure illustrates the relationship and inheritance between all environment interfaces.

Figure 1. Environment overview

Below follows a description of each interface:

4.1.1. DREnvironment

This is the base interface for all other environments. It contains various methods to obtain system data.The method getDatabases returns a list of all database configurations that have been defined inthe system. If a database configuration key is provided, the method getDatabaseProfile returnsa DRDBProfile object, holding information about the specific database.

9


A list of all UDR types created in the Ultra Format Editor or created using the DTK, is availablethrough the getDRUDRDescs method. There are various other methods that return UDR descriptionsgiven UDR type names etc.

The Convergent Mediation Engine™ system name is obtained through getSystemName, propertyvalues are obtained with getProperty and, finally, events can be dispatched to the Event Serverusing the dispatchEvent method.

4.1.2. DRUIEnvironment

This interface is available in all user interface related DTK code.

The method reportException can be used to display the built-in exception viewer. If special userinterface coding is desired, the Java JDesktopPane can be obtained via getDesktopPane.

4.1.3. DRWorkflowEnv

Utilities from this interface are available in agent development.

The current workflow name is returned from the getWorkflowName method, the isRealtimemethod checks if the current workflow is a real-time or batch workflow, the getNodeOfTypemethod returns a list of node names of either all collection or processing nodes available in the workflow,and the getMIMTree method returns all MIM resources currently published in the workflow. Thistree may be used within configuration components or for validation purpose.

Given a node name, getInputNodeNames and getOutputNodeNames return a list of nodenames that are connected to or from the node, and getInputTypes and getOutputTypes returnthe data types that the node is configured to receive or pass on. In case of output, a route name has tobe specified as well. Also, given a node name, getRouteNames returns a list of outgoing routenames.

The getExecutionHosts method allows you to retrieve the execution hosts that the workflowhas been scheduled to execute on (if any). The isStandalone method checks if the current workflowis a standalone workflow, only real-time workflows can be standalone.

4.1.4. DRRealtimeWorkflowEnv

This interface is available in real-time workflows.

Currently there are no additional methods available in this interface.

4.1.5. DRBatchWorkflowEnv

This interface is available in batch workflows.

The method getCancelBatchBehaviour returns how a Cancel Batch directive will be treatedby the workflow. Please refer to Convergent Mediation Engine™ Core - User Guide, for further in-formation on Cancel Batch behaviors.

4.1.6. DRWfUIEnvironment

This interface is available when coding agent user interfaces. Note, it extends both DRBatchWork-flowEnv, DRRealtimeWorkflowEnv and DRWfUIEnvironment. Given a node name,readState returns the nodes last saved state (if any). This method is only applicable for batchworkflows.

10


4.1.7. DRServerExecEnv

This interface is available in all server related DTK code.

Node message events may be sent to the event server using the method dispatchMessage, whichcomes in two versions where one offers the possibility of both setting a node message and a type. Nodefailure events may be sent to the event server using logException. However, this method mustonly be used if the agent wants the exception to be silently logged, not causing the node to abort.

The execute method allows an agent to execute a Runnable in a new thread. This method mustbe used when a node wants to add a thread to the workflow. The workflow will be running until allcollection nodes and all threads added through this method are finished. The node that creates theRunnable is also responsible for stopping it when the workflow is stopped.

The method isDebugEnabled states if the debugging has been enabled for the workflow. To senda debug event the dispatchDebug should be used. A debug event will only be sent if isDebugEn-abled is true.

4.1.8. DRRealtimeServerEnv

This interface is available when coding real-time agents.

The interface includes methods for invoking and controlling simultaneously executing tasks.

It is also possible to control the internal workflow queue which distributes work to the workflowthreads. The queue has a soft limit which is set in the configuration. By calling getQueueLimitthe current limit can be inspected. The limit can be changed by calling setQueueLimit from theagent's initialize method.

To check the current size of the queue during run time, getQueueSize is used.

If the agent implements DRRealtimeAsyncRouter, the callback on changes in the queue size canbe controlled through setNotificationStep. This method can only be used in the agent's ini-tialize method.

The current notification step can be inspected with getNotificationStep.

4.1.9. DRBatchServerEnv

This interface is available when coding batch agents.

The method cancelBatch allows processing agents to cancel the batch if necessary. This will sendthe currently processed batch to ECS, if the workflow is not configured to abort. Based on the workflowconfiguration, the workflow may continue with the next batch or abort.

If a Processing agent for some reason requires the currently processed batch to be split, it may hint thecollector to do so using the hintEndBatch method. If the collector supports splitting batches, itwill immediately end the current batch and start a new one. If not, it will return an unsupported excep-tion, causing the workflow to abort. The following collectors support the operation: Disk, FTP, Databaseand ECS.

Each batch is assigned a unique transaction ID that may be obtained using the getCurrentTxnIDmethod.

11


4.2. SerializationData that is transferred between processes, persisted in the database, or exported to file, must be seri-alized. Convergent Mediation Engine™ implements its own serialization mechanism for input andoutput streams, where each object to be serialized is referred to as a Storable. The serialization alsoincludes a version control mechanism where an object can populate itself differently depending on theversion of the incoming stream.

In the DTK there is only one programmatic interface for handling object serialization. Classes for thisare located in the package devkit.storable. However, depending on the target , ConvergentMediation Engine™ handles serialization in two different ways:

• Plain streams

Plain streams are used in a running system. As with regular Java serialization, it is imperative thateach element is written and read in the exact same order. For performance reasons these streamtypes are used within a running system in inter-client communication.

• XML

In XML streams, each element is associated with a user defined name tag. The tag is used to locatethe value. XML streams are used for database persistence and when objects are exported to a file,imported into another Convergent Mediation Engine™ system and during upgraded. The XMLstream provides the ability to make use of a supplied version number to be able to automaticallyupgrade configuration in a running system.

An object to be serialized, must implement the DRStorable interface. This interface contains themethods readFrom that de-serializes the object, and writeTo that serializes the object. The methodsoperate on a DRInputStream and DROutputStream respectively, where data of various typesmay be obtained or set using a name tag.

Note, all storables that are to be persisted, in either the database or export/imported in ConvergentMediation Engine™ , must have names defined. Other storables may use null, however, it is encouragedto always use unique name tags.

The readFrom method is handed a version parameter. When storables are de-serialized, this parametermay be used to detect if the input stream is of the same version as when it was serialized. If not, differentactions may be taken to populate values correctly. Again, it is only on XML streams that this versionnumber is set. Streams supplied from the Convergent Mediation Engine™ run-time system will alwayscontain the constant LATEST_VERSION.

4.3. ExceptionDTK encapsulates exceptions in the DRException class. The class offers the possibilities of nestingexceptions and assigning a severity. Valid severities are defined in DRSeverity.

Exception related classes are available in the devkit.exception package.

4.3.1. Internationalization

DRExceptions can be created with a DRTextCode as argument. DRTextCode allows the userto define messages in a property file, from which messages will be extracted using the Java Resource-Bundle concept. Currently, DRTextCode only supports English locale, however, support for morelanguages will be added in the future. Until then, DRException also defines constructors that takea message string directly. The user is currently encouraged to use these constructors.

For an example on how a property file may be used, consult the examples directory.

12


5. Agent Plug-InsA Convergent Mediation Engine™ agent is defined by inserting a class that extends the DRAgentIn-spectable class into the code server. The servers will then instantiate the class and extract theneeded information about the agent. Most agent related classes are located in the devkit.wf package.

An agent is either a collector, responsible for feeding data into a workflow, or a processing agent re-sponsible for processing the incoming data. Processing agents that does not forward data to other agentsbut forwards data to down-stream systems are referred to as Forwarding agents. Agents are designedto operate either in a batch oriented or real-time oriented workflow.

To create a user defined agent, four Java classes are required. It is encouraged to name the files asshown in the following:

The agent configuration data must be kept in a class that extends DRAgentCon-figData. This class may also contain MIMs that are to be published to theworkflow. MIMs published will be accessible to other nodes.

<Agent>Config

This is the class responsible for the actual execution that implements the agentrun-time logic. The class to extend depends on what type of agent is developed:

<Agent>Exec

DRRealtimeCollector, DRBatchCollector, DRRealtimePro-cessor orDRBatchProcessor.

The agent definition class that contains references to the user interface and ex-ecutable implementations of the agent, as well as other information. This classmust extend DRAgentInspectable.

<Agent>Insp

The agent user interface dialog, used to collect and display configuration data,is defined in a class extending DRAgentUI.

<Agent>UI

5.1. Realtime Vs. BatchBatch agents are collectors that feed the workflow with one batch at the time, and processing agentsthat process one piece of data at the time and do not emit any transactions. The workflow logic is ad-apted to data executing sequentially and single threaded.

Real-time agents are collectors that feed the workflow record-by-record, and processing agents thathandle several records simultaneously. Workflow logic is adapted to several threads in parallel andstate-fullness on user session. In real-time there may not be any need at all for forwarding the processeddata, and a response must at times be sent back through the connecting agent that received the request.

The following table lists some of the differences between batch mode and real-time mode.

Real-timeBatch

Order is not guaranteed.Data is ordered.

Several collectors per workflowmay exist.

Only one collector per workflow isallowed.

The agent is able to handle multiplethreads.

Only one built-in thread in theagent is utilized.

Never stops processing.Stops to close transactions.

Never stops executing.Stops executing when no moredata.

Data may be routed back to collect-or.

Data flows one way.

Circular routing is allowed.No circular routing.

13


Only global MIM values.Different types of MIM values.

5.2. StandaloneA special type of workflow, called a standalone workflow, is available in Convergent Mediation En-gine™ . A workflow becomes standalone if it is scheduled to execute on a Standalone ExecutionContext (ECSA). Agents in standalone workflows must be validated by a stricter set of rules to ensurethat they can safely execute in a Standalone Execution Context environment.

To ensure that an agent is compatible with standalone execution it must adhere to the rules listed inthe following table:

Only real-time agents are candidates for standalone.

Remote communication can only occur in the agent's initialize and deinitial-ize methods.

Only local storage can be depended on.

5.3. Meta Information ModelConvergent Mediation Engine™ provides a data model that during workflow execution will supplyinformation, published by agents and the workflow, to all other agents. This data model is referred toas the Meta Information Model (MIM). In DTK the DRMIMEntry class describes a specific MIM.MIMs are often used to communicate configuration of runtime information between agents as well asadding audit and ability to trace errors. To request the value of an entry, or to retrieve a list of all existingentries, the DRMIMTree method is utilized.

Agents can define when a MIM's value is available, using four constants in the DRMIMEntry class.These four types include:

• Global - An entity that spans over many batches. If this entry is cacheable it will be constant forthe execution of a workflow. Note, only real-time agents can utilize this MIM type.

• Batch - An entity that can be updated during a batch. If this entry is cacheable (DRMI-MEntry.isCacheable()) the tree delegating MIMs will keep the value of this entry duringthe current batch.

• Header - An entity available in the header of a batch. This value is, once requested, cached inthe MIM tree for the given batch. Mostly this MIM is assigned its value in the beginBatch call.

• Trailer - An entity available in the trailer of a batch. This value is, once requested, cached inthe MIM tree for the given batch. Mostly this MIM is assigned its value in the endBatch call.

5.4. Agent User InterfaceThe user interface of an agent is displayed when a node in a workflow is double-clicked. The WorkflowEditor is responsible for calling different methods in the user interface classes, and the implementationsare expected to respond properly. To be able to enter and display data, the DRAgentUI class mustbe extended. This class will further be referred to as the GUI class.

An extension of DRAgentInspectable is used to notify the Workflow Editor about the currentconfiguration when the user interface is displayed. This class will further be referred to as the In-spectable class. The common data exchange point is an extension of DRAgentConfigData.

14


The Workflow Editor will provide the GUI and Inspectable classes with information about theworkflow as a whole in the DRWfUIEnvironment class.

5.4.1.The User Interface Dialog

The GUI class is used to collect and display the configuration data required for the agent to operate.The user interface will be implemented in a JPanel that the Workflow Editor will put in a dialog.The Workflow Editor automatically adds the OK and Cancel buttons.

The programmer must extend the DRAgentUI class, that in turn extends the JPanel class. TheWorkflow Editor will create an instance of this class when a node is dropped or loaded into theworkflow. The instance will be kept until the node is deleted or the workflow is re/unloaded.

5.4.1.1. Defining the User Interface

The first method to call is the constructor. Here it may be suitable to create all the user interface com-ponents to be used in the dialog. However, when the GUI class is constructed, all variables that maybe useful are not yet available. These include, the node name and the Workflow Editor environment.If this information is required the initialize method must be used.

5.4.1.2. Initialization and Deinitialization

The initialize method will be called once after the class is constructed. In this method the nodesname and the Workflow Editor environment is available.

The deinitialize method is called once prior to the destruction of the class.

5.4.1.3. Displaying Configuration Data

The displayConfig method is called every time the node is double-clicked prior to the dialogdisplay. The method is handed a DRStorable configuration object that is either null, if the nodehas not yet been configured, or contains data if it has. The data has to be type casted to the appropriatetype before usage.

It is important that all components in the user interface are cleared in this method before they getpopulated with the supplied data. If not, old data may reside in the Java components if they were pre-viously filled and Cancel was pressed.

A hint is to populate, for instance, combo boxes with data in this method, if they contain environmentaldata such as database names or route names. That way these will get updated each time the user interfacedialog is displayed. If they were populated only in the constructor, the workflow would have to be re-loaded or a new node needs to be dropped in order to get these components refreshed.

5.4.1.4. Collecting Data

The collectConfig method collects the configuration from the user interface when OK is pressed,provided that a previous call to validateInput returned null. The method will expect the datato be valid and will thus never validate the entered data, since that is done in the previously mentionedmethod.

The method must not return null or data that is invalid (non executable).

5.4.1.5. Validation of User Data

Data entered in the user interface is validated once OK is pressed. The validation must guarantee thatthe configuration is sufficient to be able to execute the agent. This includes the validation of valueranges, missing values etc.

15


It is important that references to Workflow Editor environment entities, such as route names andMIM resources, are also validated. The recommended way of designing, for instance route names se-lection, is to populate a user interface component in the displayConfig method. That way, theuser cannot select one that is invalid. However, when the user interface dialog has been validated andconfirmed, the user may change names on routes and on nodes invalidating the previous validation.To prevent this, such external entities must be validated again in the Inspectable class.

The validateInput method is called when OK is pressed in the dialog. If this method returnsnull the Workflow Editor will call the collectConfig method and the user interface dialogwill be dismissed. If it returns a string, a validation error dialog will be displayed containing the returnedstring as its message and the dialog will not be dismissed.

5.4.1.6. Online Help

DRAgentUI provides support for documentation in JavaHelp format. The JavaHelp jar file must beinstalled with the Convergent Mediation Engine™ package. There are many vendors that supportgenerating JavaHelp for different source formats. DTK does not provide a documentation frameworkbut just the ability to show the online help. Please see the JavaHelp user guide for more information.

Once the JavaHelp jar file has been added to the system the getHelpIdentifier should be imple-mented and return a String array where the first element points out the help set name, for instancemyagent.hs, and the second element is optional and points to an identifier within the help package.The files in the help package must be unique within the system. Therefore it is recommended to putthe html and image files in a directory named like the agent. The help set file should also be namedlike the agent.

The current version of Convergent Mediation Engine™ does not provide APIs to add a DTKdeveloped package with online-help to be part of the list of help packages in the About menu.

5.4.1.6.1. User Guide Contents

Documentation of an application should follow a certain structure to be consistent with existing userdocumentation, provided in Convergent Mediation Engine™ . There is a set order of sections wherethe information should be presented in the following way:

Section 1 - Intro-duction

• Contains information about what type of agent (realtime/batch, standard/ex-tension) and that it is available in the Convergent Mediation Engine™ platformor as a Convergent Mediation Engine™ plugin etc.

• Section 1.1 - Prerequsities

• Section 1.2 - Documentation (if applicable)

• Section 1.3 - Abbreviations and Terms (if applicable)

This section is optional and will specify needed procedures in order to be able toconfigure the application as declared in the following sections.

Section 2 - Installa-tion/FurtherInfo/Setup (if ap-plicable)

If there is a profile that needs configuration, it should be described before thenode configurations are described.

Section 3 - Profile(if applicable)

There must be a snapshot of the GUI provided, with all the GUI-items listed anddescribed below. The items are listed in two columns with items to the left andthe description to the right. Also, if applicable, suggested values or examples maybe provided.

16


If the GUI contains configurations in separate tabs, each tab must be describedin its own subsection, each providing a snapshot of GUI and item listing.

This section, including all subsections, are repeated for each agent in the userguide; Collection, Processing, Forwarding etc.

Section 4 - Agent

• Section 4.1 - ConfigurationThere must be a snapshot of the GUI provided, with all the GUI-items listedand described below. The items are listed in two columns with items to theleft and the description to the right. Also, if applicable, suggested values orexamples may be provided.

If the GUI contains configurations in separate tabs, each tab must be describedin its own subsection, each providing a snapshot of GUI and item listing.

• Section 4.1 - Transaction BehaviorPossibly a description of transaction behavior if not self explanatory.

• Section 4.1.1 - EmitsList the possible transaction methods the agent is sending out. Else, type"None." in this section.

• Section 4.1.2 - RetrievesList the possible transaction methods the agent can receive. Else, type"None." in this section.

• Section 4.2 - IntrospectionDescribe what types the agent produces and what types the agent consumes.

• Section 4.3 - Meta Information Model

• Section 4.3.1 - PublishesList the MIM Resources that are produced by the agent itself, including ashort description. Else, type "None." in this section.

• Section 4.3.2 - AccessesList the MIM Resources that are fetched by the agent itself, including ashort description. Else, type "None." in this section.

Also, leave a reference to the Convergent Mediation Engine™ Core - UserGuide, for a listing of general MIM resources.

• Section 4.4 - Node Message EventList the Node Message Events produced by the agent, including a short de-scription of when they occur. Else, type "None." in this section.

If there is an inspection tool available, it should be described following the nodeconfiguration descriptions.

Section 5 - Inspec-tion (if applicable)

There must be a snapshot of the GUI provided, with all the GUI-items listed anddescribed below. The items are listed in two columns with items to the left andthe description to the right. Also, if applicable, suggested values or examples maybe provided.

If there is a Search option to limit the inspection, this GUI must be described inits own subsection, providing a snapshot and item listing.

Where the usage of an application is complex it is sometimes helpful to providean example of a "real" situation, including corresponding configurations. If thisis necessary, the section is placed at the end of the user guide.

Section 6 - Ex-ample (optional)

17


5.4.1.7. User Interface for Commands

To create a command to interact with an active workflow, the programmer extends the DRCommandUIclass. The inspectable class must return the class name in getCommandUIClassName.

All methods defined in the User Interface for Commands must be implemented in the same way asthe corresponding methods in DRAgentUI.

5.4.2. Inspectable

The Inspectable class is used both by the Workflow Editor and by the workflow engine to obtaininformation about the agent and its configuration status.

Inspectable classes must extend DRAgentInspectable. The Workflow Editor queries the codeserver for such instances in order to determine which agents are available.

5.4.2.1. Agent Details

To obtain the user interface configuration panel, the agent name and its representing icon, the WorkflowEditor uses the getUIClassName, getCommandUIClassName, getName, and getIconName.The workflow engine uses getExecClassName to obtain the executable class. The class namesand icon name must contain the full package name.

If the agent publishes any MIM resources, these will be returned in the getMIMEntries method.If so, the Workflow Editor will publish them for use by other nodes.

If the agent supports update of configurations, the method supportsUpdate must be implementedand return true. other nodes.

5.4.2.2. Incoming and Outgoing Data Types

The Workflow Editor automatically compares and validates output types from one node with the inputtypes on a target node by querying the methods getInputTypes and getOutputTypes. Forsome agents these types may be static, while others are dependent on the configuration. Since differentoutput types may be distributed on different outgoing routes, the name of each outgoing route is sentalong as a parameter.

Collection agents never have any input types. If a Processing agent has no output types, it is classifiedas a Forwarding agent in the Workflow Editor.

5.4.2.3. Validation

A user interface dialog validates everything in its validateInput method. However, some itemsreferred to, such as route names, might have changed since the dialog was confirmed. The Inspect-able class has its own validate method, whose purpose is to catch things that may have changedafter the dialog was confirmed.

If the node has any configuration data, the validate method is called every time a workflow issaved and every time the Validate... option is selected from the Edit menu of the Workflow Editor.The validate method of a node is called during startup of a workflow execution as well, to makesure that the workflow is still valid.

A workflow is considered valid if it contains at least one input node, if all nodes are properly connected,if all nodes have a configuration, if there are no data type compatibility problems between connectionsand if all nodes return null from their validate method. Note, the Workflow Editor takes careof validating the incoming and outgoing data type compatibility. Thus, this does not have to be donethrough the validate method.

18


The default implementation of validate calls getExpectedMIMNamedEntries, getExpec-tedRouteNames, getExpectedExecutionContextName, getInputTypes andgetOutputTypes and checks that the expected MIMs and routes really exist and that input andoutput types match with the workflow configuration.

If used in standalone workflows, agents that depend on storage can override the getExpectedExe-cutionContextName method to return the location where their storage is configured. This allowsvalidate to ensure that the location of the storage matches the scheduling of the workflow, thuspreserving the workflow's standalone integrity.

As for the GUI class, the validate method will return null if the node configuration is valid,otherwise a string describing the problem is returned.

5.4.2.4.Tool-Tip

When the mouse pointer rests on a node, the Workflow Editor displays a tool-tip text, aiming tocontain a subset of the most important configuration values and, optionally, the current transactiondata of the agent. The latter is only necessary if it is useful to the user.

The involved method is getConfigTooltip. This must be implemented either in this class or inthe configuration data class extending DRAgentConfigData. Otherwise the tool-tip displays NotConfigured in the tool-tip, although it may be configured. Both methods return an array list con-taining string pairs as label, value, label, value, etc.

5.4.3. Other User Interface Components

The DTK holds some useful components that may be utilized in the user interface classes. These classesinclude:

• UDR selection

• MIM selection

5.4.3.1. UDR Selection

Two components exist where UDR Types and UDR fields may be selected; DJUDRTypeSelectorand DJUDRFieldSelector. They both operate in single or multiple selection mode. In singlemode, only one UDR Type or UDR field may be selected, while in multiple mode several may be se-lected.

An instance of the DJUDRBrowser is used to select the UDR Types or UDR fields. In single modethe browser, by default, contains a Clear button which when pressed will clear the selected value.

The selection components hold various get and set methods where UDR Types and fields may be setand obtained. Since the launched browser is specific for each selection component, it is advised to seta unique title by calling the method setUDRBrowserTitle and to use a combination of the nodename and the component name in the title.

19


Figure 2. Example of DJUDRFieldSelector

5.4.3.2. MIM Selection

The DJMIMBrowser is a dialog where a MIM value may be selected. It has two corresponding selec-tion components; DJMIMSelector and DJSingleMIMSelector, where one or multiple MIMvalues may be selected.

Figure 3. Example of DJMIMBrowser.

5.5. Agent ExecutableAn executable is the class performing the execution part of an agent. It routes data into the workflowif it is a Collection agent and consumes data from previous agent if it is a Processing (or Forwarding)agent. There are two branches for agent executables, one for Batch and one for Realtime workflows.The different types will be explained separately. They both share the same abstract class DRAgent.

20


5.5.1. DRAgent

DRAgent is the base class for all agent types. The getEnv method returns the run-time environmentthat will be a DRRealtimeWorkflowEnv if the workflow is a Realtime workflow or aDRBatchWorkflowEnv for Batch workflows.

The environment also exists as a protected variable called _env. The getIdentity method returnsan identity of the node, which contains the workflow name and the node name.

The identity also exists as a protected variable called _identity. The getRouter method returnsthe router object responsible for routing raw data or UDRs into the workflow.

The router also exists as a protected variable called _router that is used if many router invocationsare expected. The router is explained further in the following. These methods can be overridden:

• consume

Called when data is routed to the agent. This function is not applicable for batch Collection agents.The method is implemented for Processing and Forwarding agents as well as for Realtime Collectionagents with response capability.

• getMIMValue

This method is called for agents that publish one or many MIM values. The entry whose value isto be returned, is given as an argument and the agent must return the value for the given entry.Depending on the workflow and entry type, this function can be called frequently. The agent musttherefore consider defining its MIM entries as static final to be able to use direct referencecomparison in this method. The result must match the types defined in the UDR Type system.

• deinitialize

Called when the workflow has been stopped and no more calls to consume will be performed.The agent must free all resources in this method, such as file and database references.

• executeCommand

Called when the operator wants to interact with the workflow. Only agents supporting the DynamicCommands functionality need to implement this method.

• updateConfig

Called when a new version of the workflow is deployed. Currently this is only supported for real-time workflows. This must be implemented if the inspectable indicates support for updated config-urations.

5.5.2. DRObjectRouter

An object router exists on Collection and Processing agents. In case several outgoing routes are defined,the object router manages them all. The router defines consume methods for DRUDR and byte[].

In case an agent will route specific data on different routes, the consume method that takes a routeindex, must be used. The route index can be resolved using the getRouteIndex method that takesthe name of the route as argument. The object router also keeps internal counters of how manyUDRs/bytes that have been forwarded. These may be used to collect statistics within the agent and arealso available as MIM values.

21


5.5.3. DRBatchAgent

The batch agent adds transaction points to a DRAgent. It is the super class for all batch agents. Thefollowing methods are defined:

• initialize

The initialize (DRStorable, DRInputStream) method is called during startup to givethe agent its configuration and optionally a stream, where the state is read from. The config ar-gument is the agent configuration class, populated with the configuration saved in the user interface.The state argument is an input stream that if not null must be read in the same order as writtenin the writeState method.

• beginBatch

This method is called at the beginning of a batch and it is fed the transaction id. Agents must preparethemselves to be fed with data at this point.

• endBatch

This method is called when all data for the current batch has been processed by the workflow.Agents must prepare themselves for a commit of the current transaction at this point.

• writeState

This method is invoked after endBatch and allows the agent to persist transaction specific in-formation. In case of a crash during commit/rollback, the state will be used at startup to apply thecorrect logic for commit/rollback of the transaction.

• rollback

Called if the current transaction will be rolled back, that is, removed. This method is only calledafter initialization in case of a previous crash.

• commit

Called when the current transaction will be committed. The transaction is closed at this point andthe produced data is considered safe.

• cancelBatch

Invoked when a batch is being cancelled. A collection agent should route the incoming file to theerror service. Other agents should make sure temporary data that has been persisted in interfacedsystems is removed.

• getTransactionResources

If the agent depends on resources that must take an active part of the workflow transaction in termsof prepare/rollback/commit, it must return a reference to these resources in this method. These re-sources must be defined in the initialize method. An example is to return the DRECSBatch-ServiceExec if it is dependent on the transaction.

5.5.3.1.Transaction Order

An important feature of a mediation system is transaction safety. Convergent Mediation Engine™provides a two-phase commit transaction model, with the possibility to save a transaction state withthe current transaction. The Collection agent controls a transaction, since it feeds the incoming batches.

22


The DTK defines a method that can be called to hint the Collection agent to close a transaction. Thismethod is called hintEndBatch and is defined in DRBatchServerEnv. The collector may chooseto ignore this call.

The following examples shows possible method invocation orders for Processor agents.

Example 1.

A batch is successfully processed. The Collector agent begins the batch, feeds the data into theworkflow, and then ends the batch.

Invocation order: beginBatch -> consume* -> drain -> endBatch -> writeState-> commit

The flag isRecover to commit will be false.

Example 2.

A batch is committed at startup. The commit invocation started the last time the workflow wasrunning. Before the commit was finished, the workflow aborted.

Invocation order: initialize -> commit

The state passed to initialize is the state written in writeState just before the workflowaborted. Since one or more agents may already have received commit before the crash, anagent must always be prepared to receive more than one invocation to this method.

The flag isRecover to commit will be true. This indicates that the method may have beeninvoked earlier with the same transactionID.

Example 3.

A batch is rolled back at startup. The commit invocation never started the last time the workflowwas running. Before reaching commit, the workflow aborted.

Invocation order: initialize -> rollback

The state passed to initialize is the state written in writeState during the last success-fully committed batch.

NOTE: If there was no successful batch processed before the workflow aborted, the statewill be null.

The flag isRecover to rollback will be true. This indicates that the method may havebeen invoked earlier with the same transactionID.

23


Example 4.

A batch is cancelled. It can happen at any time after beginBatch and before endBatch.

Invocation order: beginBatch -> consume* -> drain? -> cancelBatch ->writeState -> commit

A cancelled batch is still treated as a processed batch and therefore the transaction will becommitted.

Any Processor agent may call cancelBatch in the environment during an invocation ofconsume. The thread will eventually be unwound and call execute on the Collector agentagain. An agent calling DRObjectRouter.consume must ensure that its internal state isnot corrupted when the thread is unwound. Note that cancelBatch, writeState, andcommit may be called before the the thread is unwound.

Any Processor agent may also call cancelBatch in the environment during an invocation ofdrain.

5.5.4. DRBatchCollector

In Convergent Mediation Engine™ the collector is the controller of the workflow. It controls the mainexecution thread, and when the thread returns from the collector, the workflow will stop. DRBatch-Collector defines the execute method, where the workflow thread lives, as well as transactioncontrol methods. These are used to propagate Begin Batch and End Batch calls.

• execute

When called, the collector is given the workflow thread. The collector must keep this thread untilthe workflow will terminate, for instance, when the stop method has been called. The executemethod typically emits beginBatch, routes the batch into the workflow and then emits end-Batch.

• beginBatch

Propagates a new transaction to the transaction controller and all the workflow nodes.

• endBatch

Propagates a transaction end to the transaction controller and all the workflow nodes.

• endBatchHinted

Called when another node has called hintEndBatch in DRBatchServerEnv. The collectormay choose to ignore this method, that is, not to implement it in case it cannot be supported.

• splitBatch

When invoked, it will propagate to the other agents that the input batch will be split (DRBatch-Processor.splittingInfile).

F o r a b a t c h C o l l e c t i o n a g e n t e x a m p l e , p l e a s e r e f e r t ocom.digitalroute.devkit.examples.diskcollection.*

5.5.5. DRBatchProcesssor

This is the class to be used for Processing and Forwarding agents. Since they do not differ more thanfrom an output perspective, Convergent Mediation Engine™ uses the same class for both types. This

24


agent is fed data through the consume method, applies the logic on the data and then either routes itforward in the workflow (processing) or to external system (forwarding).

• drain

Called before the current batch ends. The agent must flush all internal buffers to make sure allpending data has been processed before the transaction is ended.

• splittingBatch

Called when the collector has split the input batch. This is the result of an agent callingDRBatchServerEnv.hintEndBatch. If the agent keeps internal buffers to be flushed differ-ently depending on the nature of the transaction, this method serves as a hint to the drain call.

Fo r t wo b a t c h P r o c e s s i n g a g e n t ex a m p l e s , p l e a s e r e f e r t o :com.digitalroute.devkit.examples.maxmin.* andcom.digitalroute.devkit.examples.diskforwarding.* (forwarding).

5.5.6. DRRealtimeAgent

This class defines no new functions on top of DRAgent. It is the superclass for all real-time agents.A real-time agent receiving raw data can create a task from it and schedule it for decoding. Refer tothe real-time environment.

Fo r a r e a l - t i m e P r o c e s s i n g a g e n t ex a m p l e , p l e a s e r e f e r t o :com.digitalroute.devkit.examples.tcpcollection.*

5.5.7. DRRealtimeAsyncRouter

Interface that indicates that records, routed out from the agent, will be turned over to a global workflowthread for further processing. The interface also defines a notification method to be called when thebacklog of unprocessed records grows over a certain limit, as defined in the Workflow Preferencesas Limit.

The notification can also be called before the limit is reached. To control this, see DRReal-timeServerEnv.

An important limitation is that an agent, implementing this interface, is not allowed to route raw datato other agents. Only records are allowed. If raw data needs to be routed, encapsulate it in a record.

5.5.8. DRRealtimeThreadAgent

Interface that defines the execute method. An agent must use the thread that calls this method forI/O operation throughout the entire execution of the workflow. This method will only be called once.The workflow will stop (or abort) if this method returns (or throws an exception).

The DRRealtimeThreadAgent extends the DRRealtimeAsyncRouter. The purpose of thisis that the thread, entering the execute method, never leaves the agent by routing records to anotheragent.

5.5.9. DRRealtimeCollector

The real-time collector implements DRRealtimeThreadAgent.

As previously described, the real-time collector may choose to implement the consume method tohandle request/response mechanisms. To tell the Workflow Editor that this capability exists, theInspectable for the collector must return the response UDR Type in the getInputTypesmethod.

25


5.5.10. DRRealtimeProcessor

Defines no new methods on top of DRRealtimeAgent. Refer to the description on the DRBatch-Processor for the differences between a Processing and a Forwarding agent.

A real-time processor may choose to implement DRRealtimeThreadAgent to process incomingdata asynchronously. This is the case when a separate thread is used to handle the communication toa back-end entity. Note, the developer must handle the turnover of records entering the consumemethods to the executing agent thread.

If it prefers to process that data synchronously, but to route the result asynchronously, it must insteadimplement DRRealtimeAsyncRouter. This is the case when the communication with the back-end is handled by a service of which the agent developer has no thread control, for instance, RMI orCORBA.

5.6. ServicesPlug-in agents may utilize built-in services for various purposes. Some services comes with separateuser interfaces in order to be configurable. Services with a graphical user interface are added as anextra tab to the agent user interface. To use a service, the inspectable class name of the service mustbe returned from the getServices method in the Inspectable class of the agent.

At run-time the agent executable must gather the service's executable, using the getServicemethod in either DRBatchServerEnv or DRRealtimeServerEnv, based on the type of workflow.The getService method takes the service inspectable name as argument and returns the executablefor the service. During run-time, each service executable acts differently, that is, the executable forthe specific service must be examined.

5.6.1. Filename Template

The Filename Template service is a service that file-based Forwarding agents may use. It is a servicewith a user interface where the user defines how the filename will be built, based on MIM or userdefined values with the possibility to pad, align etc.

To use this service, the DRFNTServiceInsp class name must be returned in the inspectable of theagent. The Filename Template executable is called DRFNTServiceExec and it provides one methodto get the generated filename. Since the service may depend upon MIM values published in drain, thismethod must be invoked at endBatch to get the generated filename.

5.6.2. Filename Sequence

This service allows the user to define a part of a filename to contain a sequence. This is a service witha user interface that is also possible to turn off. The service keeps track of what the next sequencenumber is. It is also possible to change what the next sequence number will be in its user interface.The executable class is called DRFNSServiceExec.

At run-time the agent will check whether the service is enabled before invoking methods on it (exceptfor the state handling). The service executable, defines a method called isServiceConfiguredthat returns whether is has been configured. The agent must manage the state of this service as wellas saving the result from the getSequenceNoChk and getNextSequenceNo methods.

The isSequenceNoChangedByGUI must be invoked at startup to find out if the user has changedthe next sequence number. The arguments supplied to the method are the ones saved with the state,that is, the ones returned by getSequenceNoChk and getNextSequenceNo. If the result istrue, the agent must consider an update to its state.

26


To find out if the next file(s) to collect have correct sequence numbers, the isNextFileToRetrieveor getConsecutiveSequenceNumbers are used. The isNextFileToRetrieve returnstrue if the given file has the next sequence number. The getConsecutiveSequenceNumbersmethod takes a list of filenames and returns them sorted according to the sequence number. If there isa gap in the sequence an error is returned.

The agent must manually tell the service to update the sequence number, since services are not partof the transaction. This must be done prior to calling End Batch.

5.6.3. Filename Sort Order

This is a service with a user interface that could be used by Collection agents. The Filename Sort Orderservice allows the user to define the order of how files will be collected. The user may define sortingbased on a segment of the filename, or by using a regular expression. This service may also be turnedoff. An agent that uses both the Filename Sequence service and this service must make sure that notboth services are turned on at the same time.

At run-time the executable of the agent must check whether the service is configured, using the is-ServiceConfigured method. The service defines two methods for sorting the files and the agentmay use either, based on an array of strings, or based on a list of strings.

5.6.4. File Sort Order

This service adds the option to sort on file modification time, on top of the functionality of the FilenameSort Order service previously described. The only difference is that the sort methods operate on theJava File object instead of String.

5.6.5. ECS Batch

To support cancellation of batches in batch based collectors, the ECS Batch service must be used. Theservice writes raw data to a temporary file and sends it to ECS when the agent receives a cancelBatchcall. Since services are not aware of the transaction state, the agent must call the methods on the batchservice according to the transaction state.

When starting collection of a new batch, the begin method must be called. For every block routedinto the workflow, the consume method must be called with the same data. If the file has been correctlyprocessed by the workflow, that is the agent goes to commit, the cancel method must be called onthe service. This removes the temporary file from the file system.

If the workflow goes to cancelBatch the commit method must be called. In case of a crash, thatis commit nor cancel has been called on the service, it is recommended to call cancel in thedeinitialize method as well.

If the workflow has been configured to abort immediately, the service manages this by ignoring calls,which means that the agent does not need to consider this optimization.

27


6. Additional Features

6.1. UDRData sent between agents in workflows is either represented as UDRs (Usage Detail Records) or asraw data. Raw data is sent as bytearrays, while UDRs are to be treated as individual records.

Due to the various formats of incoming data, Convergent Mediation Engine™ does not provide anyhard-coded formatting rules. Instead, it provides the Ultra subsystem where nearly any format may bedefined and handled using the Ultra Format Definition Language (UFDL). Such definitions will auto-matically compile into classes, implementing the DRUDR interface. In addition, the DTK provides thepossibility of coding such classes directly in Java.

UDR related classes are located in the package com.digitalroute.devkit.drudr

6.1.1. DRUDR

UDRs developed using the DTK must extend the DRAbstractUDR class, which uses Java reflectionto implement all DRUDR methods except for the actual field data manipulation. A DRUDR class is in-tended to be a data carrier and thus it will require methods for setting and retrieving user defined values,as it walks its way through a workflow. Such values are referred to as fields.

By implementing method names prefixed with the get_ and/or set_ keywords, the UDR sub-systemuses Java reflection mechanisms to obtain information about available fields, such as their access typeand data type. DRUDRs contain a list of possible error codes that may be useful if UDRs are routed toECS. Error codes may be added using addErrorCode, examined via getErrorCodes and removedby clearErrors. Finally, a description of the UDR Type and its fields may be obtained usinggetDRUDRDesc.

28


Example 5.

A UDR is created to contain the three fields: name, age and isChild. Note, it is not thevariable names but the method names that determine the field names. The isChild field willnot be possible to populate since it does not have a set_ method.

public class MyUDR extends DRAbstractUDR { private String _name; private int _age;

public void set_name(String newName) { _name = newName; } public String get_name() { return _name; } public void set_age(int newAge) { _age = newAge; } public int get_age() { return _age; } public boolean get_isChild() { // Read only field return _age < 18; } // writeTo and readFrom methods are omitted}

The above UDR class can be instantiated and used as follows from DTK code.

MyUDR udr = new MyUDR();udr.setValue("name", "Karen");udr.setValue("age", 10);udr.setValue("isChild", true); // Will cause exceptionSystem.out.println( "Name: " + udr.getValue("name") + "Age: " + udr.getValue("age") + "Child: " + udr.getValue("isChild"));

or used as follows in APL code:

MyUDR udr = udrCreate(MyUDR);udr.name = "Karen";udr.age = 10;udr.isChild = true; // Will cause APL compilation errordebug( "Name: " + udr.name + "Age: " + udr.age + "Child: " + udr.isChild);

29


The Ultra format system allows the definition of nested UDRs. That is, for instance, a UDR withinanother UDR. Addressing fields in such UDRs is done by so called qualified names where a dot (".")separates each level.

F o r t w o D R U D R p l u g - i n e x a m p l e s , p l e a s e r e f e r t o :com.digitalroute.devkit.examples.udr.*

6.1.2. DRUDRDesc

The DRUDRDesc class contains information about a specific UDR Type. An instance of this classmay be obtained from a DRUDR and a list of all available UDR Types in the system from the DREn-vironment. The class offers methods such as getFields, which returns a list of DRUDRFieldinstances that describe each field.

DRUDRDesc extends the DRUDRType class, where each returned field's type may be examined. Areturned field could be a primitive field but also a nested UDR (DRUDRDesc), a list (DRUDRList-Desc), a map (DRUDRMapDesc), a table (DRUDRTableType) or a byte array (DRUDRRawData-Desc).

Convergent Mediation Engine™ automatically creates DRUDRDesc instances for all UDRs definedin the system.

6.1.3. DRUDRField

A DRUDRField represents an element within a UDR. The getFieldName method returns theelement name and getType returns the type of the element. The returned type could be a primitivefield, such as a string or integer, but also a composite type, such as another DRUDRDesc or any otherDTUDRType subtype.

If traversing a nested DRUDRDesc, the returned field's type has to be examined and if it is of typeDRUDRDesc the a further examination needs to be performed.

30


Example 6.

The content of a DRUDRDesc will be printed. Note, no lists, tables or maps will be examined.

{ ... printDesc(aDesc); // Initial call ...}public void printDesc(DRUDRDesc desc) { System.out.println("Desc: " + desc.getTypeName()); DRUDRType type; DRUDRField[] fields = desc.getFields(); for (int i=0 ; i < fields.length ; i++) { type = fields[i].getType(); if (type instanceof DRUDRDesc) { printDesc((DRUDRDesc) type); // Dig deeper } else { System.out.println("Field: " + fields[i].getFieldName()); } }}

6.1.4. DRUDRType

To be able to support type validation and automatic code generation, Convergent Mediation Engine™implements a type description system separate from standard Java class handling. The base class forall type descriptions is DRUDRType, and various UDR types are described by subclasses of DRUDR-Desc.

The following table lists the available types in Convergent Mediation Engine™ , their type descriptionsand their run-time implementation classes.

Java Object TypeType DescriptionConvergent Mediation En-gine™ Type

ObjectDRUDRType.PT_ANY()any

BigIntegerDRUDRType.PT_BIGINT()bigint

DRBitSetDRUDRType.PT_BIT_SET()bitset

BooleanDRUDRType.PT_BOOLEAN()boolean

byteDRUDRType.PT_BYTE()byte

byte[]DRUDRRawDataDescbytearray

charDRUDRType.PT_CHAR()char

DRDateDRUDRType.PT_DATE()date

floatDRUDRType.PT_FLOAT()float

doubleDRUDRType.PT_DOUBLE()double

intDRUDRType.PT_INT()int

DRIPAddressDRUDRType.PT_IPADDRESS()ipaddress

31


longDRUDRType.PT_LONG()long

shortDRUDRType.PT_SHORT()short

StringDRUDRType.PT_STRING()string

DRTableDRUDRTableTypetable

DRListDRUDRListDesclist type

MapDRUDRMapDescmap type

DRUDR subtypeDRUDRDescGeneral UDR type

DRAbstractUDR sub-class

DRUDRClassDescDevKit UDR type

6.2. EventsAn event is an object containing information related to an event in Convergent Mediation Engine™ .As an example there are Workflow State Events that are emitted when the state of a workflow ischanged and there are User Events that are emitted when a user logs in or modifies data in the system.

All events are forwarded to a platform feature, named Event Server. Units interested in events, subscribeon specific events with the Event Server. An example of such units are the Workflow Editor, thatshows workflow and node states, the Event Notification Editor, that subscribes and routes events tonotifiers etc.

The DTK allows new event types to be introduced and emitted to the Event Server. These events maybe collected and parsed in the Event Notification Editor and forwarded to one of the existing notifiers,such as the Log File or a user defined notifier created with DTK. Please refer to Section 6.3, “NotifierPlug-Ins” for details on notifier plug-ins.

Example 7.

For an event plug-in example, please refer to:

com.digitalroute.devkit.examples.event.TAPErrorEvent

Event related classes are located in the package com.digitalroute.devkit.event

6.2.1. Historic Events

After an event is processed by the Event Server it is disposed of by default. Thus, a new subscriberwill not receive a matching event until the next one arrives. For subscribers like the Workflow Editorthat needs to know each node's state when a workflow is loaded, this is not sufficient. To solve thisproblem, an event can be defined as historic. If such an event arrives to the Event Server, it will cachethis event after processing it, instead of throwing it away. Only the last event for each key will besaved. An event defined as clearing history will remove historic events sharing the same group key.

6.2.2. Event Suppression

The Event Notification Editor (not the Event Server) offers a feature of suppressing duplicate events.For instance if duplicate suppression is configured to 10 seconds, the first arriving event will be for-warded while duplicates, arriving within 10 seconds, will be discarded. To handle this the duplicatekey may be defined in the event. This is referred to as the suppress key.

32


6.2.3. DRBaseEvent

User defined events are introduced by extending the DRBaseEvent class. DRBaseEvent itselfextends DRAbstractUDR so that it can benefit from the introspection mechanism introduced in theConvergent Mediation Engine™ UDR sub-system. The Event notification Editor utilizes this whenallowing automatic filtering on or selecting data from individual event fields. The base event containssome common fields that will be available in all events. These include severity, IP-address andtimestamp.

The following methods must be overloaded for new events. The method getEventName will returna string representation of the event. This name will be displayed in the list of available events whenselecting which events to subscribe to in the Event Notification Editor. The getValue method,called by the toString method, will return an array of name/value pairs that describe the currentvalue of the event. The toString implementation arranges these name/value pairs in a commonlayout.

As previously mentioned, the UDR introspection mechanism automatically displays fields for allmethods whose names begins with get_, that is methods such as: String get_FirstName().These will, in the Event Notification Editor, allow filtering and value collection from a field namedFirstName. Note, only fields with return type String will show up for filtering in the Event Notific-ation Editor.

To serialize the data, the writeTo and readFrom methods must be implemented, and if an eventwill be cached by the Event Server, in order for a new subscriber to immediately obtain a value (withoutwaiting for the next event), then the methods isHistoryEvent, getKey, isClearingGroupand getGroupKey must be overloaded. For further information, please refer to Section 6.2.1, “His-toric Events”.

6.3. Notifier Plug-InsAll Convergent Mediation Engine™ events and user defined events may be selected, filtered androuted to notifiers in the Event Notification Editor. Convergent Mediation Engine™ supplies a fewstandard notifiers where event information may be directed to a Log File, a database table, by email,by SNMP or to the System Log. If required, new notifiers can be developed and introduced into thesystem using the DTK.

Notifier related classes are located in the package com.digitalroute.devkit.event wherethe class DRNotifierTarget and possibly DRNotifierTargetUI and DRAbstractStorablemay be extended.

Example 8.

For a notifier plug-in example, please refer to:

com.digitalroute.devkit.examples.event.DynamicLogFile*

6.3.1. DRNotifierTarget

A new notifier plug-in is defined by extending DRNotifierTarget. The Code Server will locateall such classes and make them available for configuration in the Event Notification Editor. Theconfiguration of a notifier can be obtained in two ways (or through a combination of both):

• Static User Interface - This is configuration entered in a user defined user interface, introduced byextending the DRNotifierTargetUI class. If available, it will appear in the Base Configurationframe of the Notifier Setup in the Event Notification Editor.

33


• Dynamic Fields - These are input fields that the Event Notification Editor automatically displaysin the Target Field Configuration frame in the Notifier Setup tab. Such fields do not require a userdefined interface and are also possible to populate using formatted input, where values from theincoming events may be picked. Please refer to Convergent Mediation Engine™ Core - User Guide,for information on how such fields may be populated.

The getName method will return the name to be visible in the Notification Type combo box in theEvent Notification Editor. If a static user interface is desired, then the getUIClassName methodmust return the full class name of a class that extends DRNotifierTargetUI. If dynamic fieldsare desired, then getDynamicFields must return an array of DRUDRFields. The name of theDRUDR field will be displayed as the label and the data type (DRUDRType) will determine what inputcomponent to use, what validation rules that will apply and how the field will be populated. Valid datatypes are:

DRUDRType.PT_SHORTDRUDRType.PT_INTDRUDRType.PT_LONGDRUDRType.PT_FLOATDRUDRType.PT_DOUBLEDRUDRType.PT_CHARDRUDRType.PT_STRINGDRUDRType.PT_BOOLEANDRUDRType.PT_DATEDRUDRType.PT_IPADDRESS

For all data types, apart from PT_STRING, the only population options will be manual input and directmapping from an event field, whose data type must correspond. In addition, the PT_STRING offersformatted input where many event fields may be mapped into the value. The methods isDynam-icFieldMandatory will, for each field, determine if it will validate that input is entered, and is-DynamicFieldMultiline will determine if the input component will be a single-line or multi-line field.

When a notifier configuration is saved, the initialize method is called. If the notifier was previouslysaved, the deinitialize method is called first. When the Event Notification Editor receives anevent that the notifier has subscribed on, the notify method is called. The method is handed theevent itself and, if dynamic fields are used, a hash map keyed by the dynamic field name and a corres-ponding object. The corresponding object will hold the assigned field value. If the value has beenmanually entered or assigned directly from an event field, then the object will have the same data typeas the dynamic field. If the value used formatted input (PT_STRING only) then this value will containthe formatted string, that means any variables will have been substituted by an event field value.

6.3.2. DRNotifierTargetUI

If the notifier requires a static user interface for user configuration, the DRNotifierTargetUIclass must be extended. The class extends the Java JPanel that is the main container for the config-uration components. When a notifier is selected in the user interface, this panel will be displayed inthe Base Configuration frame.

When the panel is displayed, the method reset is called to reset all components, immediately followedby displayConfig. The latter hands over a previously configured configuration object that has toimplement DRStorable, and has to be casted into the appropriate type. If this data is not null thenthe data will be displayed.

When a notifier is saved in the Event Notification Editor, the validateInput method is called.If this method returns a string, then this string is displayed as an error message. If it returns null, the

34


collectConfig method is called to create and populate a configuration object. The platform takescare of storing this configuration.

6.4. APL Plug-InsA new APL plug-in is defined by extending DRAPLPlugin. The code server will locate all suchclasses and make their functions available within the APL language. This class will be instantiated bythe APL compilation engine to extract information about the available APL functions from getFunc-tions and the executor class, which is responsible for executing the function, from getExecutor-Class.

If the plug-in routes UDRs into the workflow, this should be declared by implementing getOutput-Types and returning all types that the plug-in may route for each outgoing route. If the plug-in expectsany hard coded route names to be present in the workflow, it must also implement getExpected-RouteNames. The presence of these names on outgoing routes will then be validated.

6.4.1. DRAPLFunctionSignature

A DRAPLFunctionSignature object describes each new function. The function name is returnedfrom getName. The number of parameters and their data types are returned from getArgument-Types and the return type from getReturnType. If the function supports dynamic parametersisDynamic will return true. In this case the function must have an additional parameter of Javatype List that, when called, will contain the additional parameters.

The APL compiler will automatically make the necessary syntax validations regarding the name,parameters and return type when APL code is compiled. If additional validation is required, the val-idate method must be implemented. It is essential that the executor class has a function that exactlymatches the name, the return type and the number of parameters stated by this object.

6.4.2. DRAPLExecutor

When a workflow is executing, the APL function call will be delegated to the corresponding functionin the APL plug-ins executor object. An executor class must implement the interfaces DRAPLExecutoror DRAPLTxnExecutor, where the latter allows the APL plug-in to interact with the ConvergentMediation Engine™ transaction handling.

Executor classes will have their initialize method called once, prior to any invocation of thefunction(s) it holds. Transaction executors may implement the methods from DRBatchTransac-tionResource, such as cancelBatch or commit.

To route UDRs from the plug-in, a router must first be obtained by calling getRouter on the envir-onment.

6.5. Ultra field pluginsIt is possible to define user defined Ultra field types for use with an external block. A new Ultrafield plugin is defined by extending DRUltraFieldPlugin. The code server will locate all suchclasses and make the field types available within the Ultra format system.

The plugin class defines which classes to use for encoding and decoding. Both encoding and encodingare optional operations, but if no decoder class is supplied, it will be impossible to decode the fieldtype (and vice versa).

35


It is important to note that both the field decoder and encoder classes must be stateless to work properly.The reason is that each UDR decoder or encoder only keeps a single instance of the correspondingfield decoder or encoder class.

6.5.1. DRUltraFieldDecoder

A DRUltraFieldDecoder object is instantiated by the UDR decoder to handle all decoding offields of the corresponding type.

6.5.2. DRUltraFieldEncoder

A DRUltraFieldEncoder object is instantiated by the UDR encoder to handle all encoding offields of the corresponding type.

6.6.Task Plug-InsTasks are utility programs, executed on a periodic basis or when triggered by some event. ConvergentMediation Engine™ includes some clean-up tasks, such as the System Log Cleaner, and more generictasks that may execute, for instance, scripts. These tasks are maintained in the Task Manager. If re-quired, new tasks may be developed and introduced into the system using the DTK.

Task related classes are located in the package com.digitalroute.devkit.task where theclass DRTaskExecutor and possibly DRTaskExecutorUI and DRAbstractStorable maybe extended.

Example 9.

For a task plug-in example, please refer to

com.digitalroute.devkit.examples.taskmanager.*.

6.6.1. DRTaskExecutor

A new task plug-in is defined by extending DRTaskExecutor. The Code Server will locate all suchclasses and make them available for configuration in the Task Manager. The platform will automat-ically execute the task as configured.

The getName method returns the name to be visible in the Executor combo box in the Add TaskConfiguration dialog. The getUIClassName method will return the full class name of the GUIclass that is used for configuration of the task, please refer to Section 6.6.2, “DRTaskExecutorUI”.This class will be instantiated and displayed when the executor is selected in the dialog. The allow-MultipleTask method determines if multiple instances of this task are allowed to run simultaneously.For example the Convergent Mediation Engine™ System Log Cleaner would return false whilethe Script Executor would return true.

Prior to execution the initialize method will be called where configuration data is supplied. Thetask executor has to cast this data to the appropriate type. When the executor is due to commence theexecute method is called. Finally, each executor object will return a status report from the last exe-cution through the getExecReport method. This report will be displayed when the executor instanceis selected in the Task Manager window and the Task Report is selected.

6.6.2. DRTaskExecutorUI

If the task requires user configuration, the DRTaskExecutorUI class must be extended. The classextends the Java JPanel that is the main container for the configuration components. When a task

36


executor is selected in the user interface, this panel will be displayed. User interface components couldbe added either in the constructor or in the plain initialize method, called after an instance iscreated.

When the panel is displayed, the method clear is called to reset all components, immediately followedby displayConfig. The latter hands over a possible previously configured object, that has to extendDRStorable and has to be casted into the appropriate type. If this data is not null then the userinterface will display this data.

When OK or Apply is selected in the Add Task Configuration dialog, the validateInputmethod is called. If this method returns a string, it will be displayed as an error message. If it returnsnull, then collectConfig is called to create, populate and return a configuration object. Theplatform handles storing of this configuration.

6.6.3. DRTaskExecutorMsgEvent

Predefined events, dispatched by the platform each time a task has executed. The platform uses theresult from getExecReport as the event message. If additional task events must be sent, aDRTaskExecutorMsgEvent may be created and dispatched within the execute method, usinggetEnv().dispatchEvent.

6.7. Desktop Window Plug-InsDTK provides the ability to integrate customer implemented user interface windows to be part of theConvergent Mediation Engine™ desktop.

6.7.1. DRDesktopPlugin

Desktop plug-ins are introduced by implementing the interface DRDesktopPlugin. The CodeServer will locate all such classes and add them to a new user menu in the desktop. The purpose ofDRDesktopPlugin is to act as a pure representation class of the actual Swing implementation. Itis therefore recommended to separate the plug-in class and the actual Swing implementation, since allDRDesktopPlugins are instantiated during startup and may allocate memory from the JVM.

Based on the result from the method getMenuName, the plug-in can be put into its own submenu.When a user selects the menu item, the startPlugin method is called where the returned JavaSwing internal frame is displayed.

Example 10.

For a desktop plug-in example, please refer to:

com.digitalroute.devkit.examples.udrlister.*

37


Diameter Agents 4.1


Version 1.20

1





2

Diameter Agents 4.1



2. The Diameter Base Protocol ......................................................................................... 52.1. Diameter Workflow semantics ............................................................................ 5

2.1.1. Special Error Handling ........................................................................... 53. Application Profile ...................................................................................................... 6

3.1. Diameter Application Tab .................................................................................. 63.2. Diameter AVPs Tab ........................................................................................... 6

3.2.1. The Diameter AVP Editor ........................................................................ 73.3. Diameter Commands Tab ................................................................................... 8

3.3.1. Diameter Command Editor ...................................................................... 94. Routing Profile ......................................................................................................... 11

4.1. Hosts Tab ...................................................................................................... 115. Diameter Stack Agent ................................................................................................ 12

5.1. Configuration ................................................................................................ 125.2. Introspection .................................................................................................. 135.3. Meta Information Model .................................................................................. 13


5.4. Node Message Events ...................................................................................... 136. Diameter Request Agent ............................................................................................. 14

6.1. Configuration ................................................................................................ 146.2. Introspection .................................................................................................. 146.3. Meta Information Model .................................................................................. 146.4. Node Message Events ...................................................................................... 14

7. Limitations .............................................................................................................. 157.1. General Remarks ............................................................................................ 157.2. Diameter Stack Agent ...................................................................................... 157.3. Diameter Request Agent .................................................................................. 157.4. UDR Mapping Limitations ............................................................................... 15

8. Diameter Example ..................................................................................................... 168.1. Diameter Application Profile ............................................................................ 168.2. Diameter Routing Profile ................................................................................. 178.3. Diameter Request Workflow ............................................................................. 18

8.3.1. TCP Agent .......................................................................................... 188.3.2. Analysis Agent .................................................................................... 188.3.3. Aggregation Agent ............................................................................... 198.3.4. Diameter Stack in Request Workflow ....................................................... 20

8.4. Diameter Stack Workflow ................................................................................ 208.4.1. Analysis Agent .................................................................................... 20

3

Diameter Agents 4.1

1. IntroductionThis document describes the Diameter agents being part of the extension real-time agents, availableon the Convergent Mediation Engine™ platform.



• Diameter Base Protocol (RFC 3588 http://www.faqs.org/rfcs/rfc3588.html)


4

Diameter Agents 4.1

2.The Diameter Base ProtocolThe Diameter Base Protocol is a much-improved successor to the Radius Protocol designed to providean extensive framework for any services which require authentication, authorization and accounting(AAA) support across many networks. The Diameter Base Protocol is, as opposed to the Radius Protocol,also capable of fulfilling new access control features while keeping the flexibility for further extension.

The Diameter Base Protocol doesn't contain much functionality by itself and should be viewed as astandard transport and management interface for AAA applications providing a well-defined function-ality subset. AAA applications generally consist of new command-code and AVP definitions that mapto the semantics of the application. Example applications include Diameter Credit-Control (RFC 4006.)

2.1. Diameter Workflow semanticsThe Convergent Mediation Engine™ Diameter implementation (Diameter Stack) manages the transport,decoding and encoding of Diameter messages and can be configured to support any Diameter applicationthrough the Diameter Application Profile user interface. The stack communicates with the workflowusing the special UDR type RequestCycleUDR.

When a request message arrives to the stack it will first decode and validate it into a pre-generatedUDR type configured through the Diameter Application Profile user interface. A RequestCycleUDRis then routed through the workflow with the Request field set to the incoming message. The workflowis responsible for populating the Answer field with an appropriate answer message (through theudrCreate APL function). The RequestCycleUDR must then be routed back to the Stack node totransmit the answer.

The commands Capability-Exchange-Request, Capability-Exchange-Answer, Disconnect-Peer-Request,Disconnect-Peer-Answer, Device-Watchdog-Request and Device-Watchdog-Answer from DiameterBase are handled internally by the Diameter stack and are never visible in the workflow.

To transmit requests from the workflow a Diameter Request agent must be added. Here the rules arereversed; a RequestCycleUDR with a populated Request field is routed into the Request agent. Atsome time in the future the same RequestCycleUDR will be routed back from the Request agent withthe Answer field set to a UDR containing the answer.

2.1.1. Special Error Handling

There are two special cases where the Convergent Mediation Engine™ Diameter stack produces specialerror answers for requests. In both cases an error answer is produced with special (Convergent MediationEngine™ internal) result codes.

• Timed out waiting

When a request was sent to a peer and no answer is received within a configurable timeout, an errorwith the result code 4999 will be returned.

• Connection to peer not open

When there is no connection with the peer that should receive the request, an error with the resultcode 4998 will be returned.

5

Diameter Agents 4.1

3. Application ProfileThe Application Profile captures a set of AVP and command-code definitions that will be recognizedat runtime by a Diameter stack.

The Diameter Application Profile Editor is accessed from the Configuration menu on the desktop.

3.1. Diameter Application TabThe identifiers in this tab define the advertised applications in the capabilities handshake. They areused whenever the Diameter stack initiates or responds to a new transport connection to negotiate thecompatible applications for the link. Use the Edit button to add or remove advertised applications.

Figure 1. Diameter Application Profile Editor, Diameter Application tab.

Here all supported authentication applications are listed.Supported Authentication Application IDs

Here all supported accounting application identifiersare listed.

Supported Accounting Application IDs

Here all supported vendors are listed.Supported Vendor IDs

Where applications for a specific vendor exists, theseare listed here.

Supported Vendor Specific Applications

3.2. Diameter AVPs TabAVPs carry the data payload in all Diameter messages. Convergent Mediation Engine™ staticallyknows about all the AVPs from Diameter Base (these definitions are immutable and always available).Other AVPs can be defined in three ways: through the UI, by importing ABNF definitions or by im-porting an existing Convergent Mediation Engine™ XML export file.

To import ABNF AVP definitions (as found in IETF RFC documents), select Import ABNF AVPSpecifications... from the Diameter menu. This will populate the columns in the Diameter AVPstab. To add or edit AVPs, select the Add or Edit button at the bottom of the configuration window.

6

Diameter Agents 4.1

Figure 2. Diameter Application Profile Editor, Diameter AVPs tab.

Name of the AVP.Name

Example: CC-Request-Number

The ID number representing the AVP. This number is unique and fixed for every AVP. Referto the AVP specification for details.

ID

Example: 415

The data type of the AVP. The type is a fixed value defined for that AVP. Refer to AVPspecification for details.

Type

Example: OctetString

A number representing the vendor.Vendor

Example: 1

3.2.1.The Diameter AVP Editor

This window is displayed when the user chooses to Add or Edit an AVP in the AVP table.

Figure 3. Diameter Application Profile Editor, Diameter AVP Editor.

7

Diameter Agents 4.1

AVP name: The name of the AVP.General

AVP id: The id number given to the AVP.

Vendor id: The Vendor id given to this AVP.

AVP type: The datatype for the value stored in this AVP. Please note that selectingGrouped or Enumeration will add configuration options to the Enumera-tion/group properties part.

Mandatory ('M') bit: The bit indicates whether support for the AVP is mandatory.See section 4.1 in the Diameter Base Protocol (RFC 3588). The default value is'Must', ie the bit must be set otherwise the message will be rejected.

Flag rules

Protection ('P') bit: The bit indicates need for encryption. See section 4.1 in theDiameter Base Protocol (RFC 3588). The default value is 'May', ie the bit may ormay not be set.

End-to-end security required: This is not implemented in Convergent MediationEngine™ 4.1.

This table is displayed only for Grouped or Enumerated AVP. It provides the pos-sibility to add, edit or remove AVPs or enumeration values.

Enumera-tion/groupproperties

When editing a grouped type, the AVP name should be written in the AVP columnon the <new> row. Existing rows can be deleted by using the keystrokeCtrl+Delete or moved up/down using the combination Ctrl+Up/Down.

3.3. Diameter Commands TabThe Diameter Commands to be used in the Application Profile may exist as a predefined set of customerrelated commands or defined base commands. To import definitions, select Import ABNF CommandSpecifications... from the Diameter menu. This will populate the columns in the Diameter Commandstab. To add or edit a command, select the Add or Edit button at the bottom of the configuration window.

Figure 4. Diameter Application Profile Editor, Diameter Commands tab.

Name of the command.Name

Example: Credit-Control-Request

8

Diameter Agents 4.1

ID number of the command.Id

Example: 272

The application identifier.Application Id

Example:

• 1 - NASREQ

• 2 - DIAMMIP

• 3 - Diameter Base Accounting

• 4 - Diameter Credit-Control

3.3.1. Diameter Command Editor

This window is displayed when the user selects to Add or Edit a Diameter Command in the table.

Figure 5. Diameter Application Profile Editor, Diameter Command Editor.

Command name: The name given to the command.General

Command id: The id number given to the Command

Application id: The diameter application that this command belongs to.

Request: If set, the message is a request. If cleared, the message is an answer.Flags

Proxiable: If set, the message MAY be proxied, relayed or redirected. If cleared, themessage MUST be locally processed.

Error: If set, the message contains a protocol error, and the message will not conformto the ABNF described for this command.

Retransmit: The retransmit can be set only in cases where no answer has been receivedfrom the server for a request and the request is sent again. This is accessible at runtimein the workflow through the variable Is_Retransmit (boolean).

9

Diameter Agents 4.1

This table displays all the AVPs that are part of this command. The table provides thepossibility to add, edit or remove AVP values. The AVP name should be written in the

AVP Layout

AVP column on the <new> row. Existing rows can be deleted by using the keystrokeCtrl+Delete or moved up/down using the combination Ctrl+Up/Down.

10

Diameter Agents 4.1

4. Routing ProfileThe routing profile determines the peer table and realm/application routing properties for a Diameterstack. Convergent Mediation Engine™ does not fully support the client role, but the client routingconfiguration is provided for future compatibility.

The Diameter Routing Profile Editor is accessed from the Configuration menu on the desktop.

4.1. Hosts Tab

Figure 6. Diameter Routing Profile Editor, Hosts tab.

A Diameter stack using the profile will try to maintain transport connections with all hosts defined inthis list. Connections (and handshakes) from hosts not in this list will be rejected with appropriateprotocol errors. Note that Convergent Mediation Engine™ will actively try to establish connectionsto any host in this list. Also note that Convergent Mediation Engine™ assumes that the DiameterIdentity and DNS hostname are the same.

A Diameter Identity (which must also be a DNS hostname).Hostname

Example: ggsn01.vendor.com

The port to connect to when initiating transport connections to the peer.Port

Example: 3868

If enabled, the Diameter Stack will require a secure connection from this host.Enable TLS

11

Diameter Agents 4.1

5. Diameter Stack Agent

5.1. ConfigurationThe Diameter Collection agent configuration window is displayed when the node in a workflow isdouble-clicked, or right-clicked, selecting Configuration....

Figure 7. Diameter Collection Agent configuration, Diameter Stack tab.

Select a predefined Application Profile. The Profile holds details aboutadvertised applications as well as supported AVPs and command codes.

Application Profile

Refer to Section 3, “Application Profile” for further information on howto set up an Application Profile.

Select a predefined Routing Profile. The Profile holds details about sup-ported hosts, listening ports, applications and realms.

Routing Profile

Refer to Section 4, “Routing Profile” for further information on how toset up a Routing Profile.

Fully qualified DNS hostname of this Diameter Node. This value is appliedto the Origin-Host AVP whenever a Diameter message is transmitted fromthis node (including handshakes).

Identity (host name)

If enabled, the Diameter Identity of the local node is resolved by lookingup the DNS hostname associated with the local IP address. If there is morethan one network interface, the agent will abort on startup.

Use DNS Host Name

The area of activity for this specific host. Determines the value of allOrigin-Realm AVPs in messages transmitted from this node.

Realm

The port which the Diameter agent will use to listen for incoming transportconnections.

Listening port

The maximum number of bytes allowed in a single Diameter message.Maximum Message Size

Will turn on debug. Useful for testing purposes.Enable debug events

The stack will not actively try to connect to the peers in the peer table, orreconnect if a connection is lost.

Do not open outgoingconnections (passivemode).

12

Diameter Agents 4.1

The path to a file containing the private certificate of the Diameter Stack(or the public part of the certificate if the stack resides on the Request

Keystore path

side). The path can also point to a trusted root certificate. The technologyused to implement the certificates is Java Keystore. The path is relativeto the Convergent Mediation Engine™ Execution Context where the stackruns.

The password used to access the keystore (will not be shown).Keystore password

The timeout (in seconds) before an unanswered request will be treated asan error.

Answer Timeout

5.2. IntrospectionThe agent emits and receives UDRs of type RequestCycleUDR.


5.3.1. Publishes

A number indicating the number of received messages.Incoming Messages

Type long, defined as MIM type global.

A number indicating the number of sent messages.Outgoing Messages


A number indicating the number of rejected messages.Rejected Messages


5.3.2. Accesses

The agent does not itself access any MIM resources.

5.4. Node Message EventsNone

13

Diameter Agents 4.1

6. Diameter Request Agent

6.1. ConfigurationThe Diameter Request agent configuration window is displayed when the node in a workflow is double-clicked, or right-clicked, selecting Configuration....

Figure 8. Diameter Processing agent configuration window, Diameter Request tab.

Selects the Diameter Stack requests will be sent from. Lists all DiameterStacks in the workflow.

Associated Diameter Stack

6.2. IntrospectionThe agent emits and receives UDRs of type RequestCycleUDR.

6.3. Meta Information ModelThe agent does not itself publish or access any MIM resources.

6.4. Node Message EventsNone

14

Diameter Agents 4.1

7. Limitations

7.1. General RemarksConvergent Mediation Engine™ provides the capability of a Diameter Server and Diameter Client,as defined in chapter 1.3 (Terminology) of the Diameter Base Protocol (RFC 3588). Convergent Me-diation Engine™ however does not provide the capabilities of a Diameter Agent defined in the samechapter.

SCTP is not supported.

The commands Abort-Session-Request and Abort-Session-Answer of Diameter Base aren't implemented.These commands have to be defined as part of the configuration work.

7.2. Diameter Stack AgentConvergent Mediation Engine™ can act as a Diameter Server for any application built on the DiameterBase Protocol, with the exceptions listed in Section 7.1, “General Remarks”.

7.3. Diameter Request AgentConvergent Mediation Engine™ can act as a Diameter Client with the exceptions listed in Section 7.1,“General Remarks” as well as the following exceptions:

• Failover / Failback

• Realm-based Routing

• Dynamic Peer Discovery

• Redirects

7.4. UDR Mapping LimitationsIf an AVP is defined to occur only once in a message, Convergent Mediation Engine™ will not generatethe DIAMETER_AVP_OCCURS_TOO_MANY_TIMES error when decoding a message that includesmore than one of these AVPs.

This behavior is non-standard but greatly improves decoding performance and has no impact whenboth client and server use the same message dictionaries.

15

Diameter Agents 4.1

8. Diameter ExampleThis chapter gives an example on how to set up a Diameter Request workflow as well as a DiameterStack workflow.

The example is based on the call scenario below where a client issues a Service-Authorization-Request in order to authorize a user for a certain service. The Diameter Stack sends back a Service-Authorization-Answer containing yes or no (1 or 0) depending on if the authorization wassuccessful or not

Figure 9. Diameter Example Call Scenario

8.1. Diameter Application ProfileRefer to Section 3, “Application Profile” for information on how to setup a profile. Please note thatthe Request workflow and the Stack workflow both use this profile. The following commands aredefined:

Service-Authorization-Request ::=< Diameter Header: 383, REQ, PXY> < Session-Id >{ Origin-Host }{ Origin-Realm }{ Destination-Host }{ Destination-Realm }{ Accounting-Record-Type } { Accounting-Record-Number }{ Auth-Application-Id }{ Service }[ User ]

Service-Authorization-Answer ::=< Diameter Header: 383, PXY>< Session-Id >{ Result-Code }{ Origin-Host }{ Origin-Realm }{ Accounting-Record-Type } { Accounting-Record-Number }{ Auth-Application-Id }

16

Diameter Agents 4.1

{ Authorized }

The commands use the following AVP:s

ServiceId ::= <AVP Header: 601 = Unsigned32> (M:MUST, P:MAY, Protected:NO)

ServiceGroup ::= <AVP Header: 602 = Unsigned32> (M:MUST, P:MAY, Protected:NO)

ServiceName ::= <AVP Header: 603 = OctetString> (M:MUST, P:MAY, Protected:NO)

Service ::= <AVP Header: 604> (M:MUST, P:MAY, Protected:NO) {ServiceId} [ServiceGroup] [ServiceName]

MSISDN ::= <AVP Header: 610 = OctetString> (M:MUST, P:MAY, Protected:NO)

IP ::= <AVP Header: 611 = OctetString> (M:MUST, P:MAY, Protected:NO)

User ::= <AVP Header: 612> (M:MUST, P:MAY, Protected:NO) [MSISDN] [IP]

Authorized ::= <AVP Header: 620 = Enumerated> (M:MUST, P:MAY, Protected:NO) %FALSE 0 %TRUE 1

8.2. Diameter Routing ProfileTwo peer tables are used. One is included from the stack in the Request workflow and the other onefrom the Stack in the Stack workflow. Each peer table points out the Destination Host for the respectiveDiameter Requests. Please note that only fully qualified host names can be used in the peer tables.

17

Diameter Agents 4.1

8.3. Diameter Request Workflow

Figure 10. Diameter Request Workflow Configuration

8.3.1.TCP Agent

This agent receives comma separated UDR:s on the format user,id where user is an IP Addressand id is a Service Id of a requested service. The ultra definition for the TCP UDR is as follows:

external TCPext { ascii user : terminated_by( "," ); ascii id : int(base10), terminated_by( 0xD );};

internal TCPint : extends_class( "com.digitalroute.wfc.tcpipcoll.TCPIPUDR" ) { string session_Id;};

in_map TCP_inMap : external( TCPext ), internal( TCPint ), target_internal( TCP_TI ) { automatic; };

8.3.2. Analysis Agent

This agent creates a Service_Authorization_Request and routes it to the Diameter Request Agent. TheDiameter Session Id is created uniquely using a synchronized function. In order for the workflow toknow to whom it shall return the answer the TCP Request is stored in an aggregation session. Thefollowing code is placed within this node:

import ultra.Diameter.example_profile;long sessionNo;initialize{ sessionNo=0;}

synchronized long sessionIncr(){

18

Diameter Agents 4.1

sessionNo=sessionNo+1; return sessionNo;}

consume { if (instanceOf(input,TCP_TI)){ debug("tcp udr"); // Initialize the aggregation session TCP_TI tcp_req = (TCP_TI)input; tcp_req.session_Id="session"+(string)sessionIncr(); udrRoute(tcp_req,"get_response");

// Create the Diameter request Service_Authorization_Request request = udrCreate(Service_Authorization_Request); request.Service = udrCreate(AVP.Service); request.User = udrCreate(AVP.User); request.Session_Id=tcp_req.session_Id;

request.Service.ServiceId=tcp_req.id; strToBA(request.User.IP,tcp_req.user);

RequestCycleUDR diam_req = udrCreate(RequestCycleUDR);

request.Destination_Host= "server.vendor.com"; request.Destination_Realm="vendor.com";

diam_req.Request=request;

udrRoute(diam_req,"Diameter_req"); debug("Request routed");

} else if (instanceOf(input,RequestCycleUDR)){ if (instanceOf(((RequestCycleUDR)input).Answer, Service_Authorization_Answer)){ Service_Authorization_Answer answ = (Service_Authorization_Answer)((RequestCycleUDR)input).Answer; udrRoute(answ,"get_response"); } }}

8.3.3. Aggregation Agent

This agent is used to store each TCP request in an aggregation session.The session is defined as follows:

import Diameter.tcp;session diam_Session{ TCP_TI orig_req; };

19

Diameter Agents 4.1

The incoming UDR can be either of type TCP_TI or of type Service_Authorization_Answer.The Session Id field of these UDR:s are used to identify the correct session. The following code isplaced in this agent:

consume { if (instanceOf(input,TCP_TI)){ session.orig_req=(TCP_TI)input; sessionTimeout(session,60); debug("session initialized"); return; } else if (instanceOf(input,Service_Authorization_Answer)){ Service_Authorization_Answer answ = (Service_Authorization_Answer) input; strToBA(session.orig_req.response,(string)answ.Authorized); udrRoute(session.orig_req,"tcp_response"); debug("session routed"); sessionRemove(session); }}

timeout{ strToBA(session.orig_req.response,(string)0); udrRoute(session.orig_req,"tcp_response"); debug("session timed out"); sessionRemove(session); }

8.3.4. Diameter Stack in Request Workflow

The Diameter Stack has to be included in the Request workflow since the Diameter Request Agent initself doesn't contain any functionality to carry out handshaking etc according to the Diameter Baseprotocol. The Analysis node connected with the Stack does nothing.

8.4. Diameter Stack Workflow

Figure 11. Diameter Stack Workflow Configuration

8.4.1. Analysis Agent

This node contains the code that is needed to read the Service_Authorization_Request. Normally itwould then make the authorization by populating a request to an external Charging/Rating System. Inthe code example below we will just create a positive Service Authorization Answer:

import ultra.Diameter.example_profile;initialize{

20

Diameter Agents 4.1

debug("wf started");}consume { //The Request is a subclass of RequestCycleUDR if (instanceOf(input.Request,Service_Authorization_Request)){ Service_Authorization_Request request = (Service_Authorization_Request)input.Request; //This is how you extract the AVP:s debug("Service Id: "+request.Service.ServiceId); debug("Service Name: "+baToStr(request.Service.ServiceName)); if(udrIsPresent(request.User)){ debug("IP: "+ baToStr(request.User.IP)); }

//This is how you create the answer Service_Authorization_Answer answer = udrCreate(Service_Authorization_Answer); //Set the mandatory values //OriginHost and OriginRealm are set by the server answer.Session_Id=request.Session_Id; answer.Result_Code = 2001; //Means ok answer.Accounting_Record_Type = request.Accounting_Record_Type; answer.Accounting_Record_Number = request.Accounting_Record_Number;

//Validate if the User has access to the requested Service // ... then ... //Set the configured AVP answer.Authorized = 1; // User authorized

//Attach the answer to the Diameter Session input.Answer = answer;

//Route the session back to the server udrRoute(input);

}}

21

Diameter Agents 4.1

Disk Agents 4.1


Version 1.26

1





2

Disk Agents 4.1



2. Disk Collection Agent ................................................................................................. 52.1. Configuration .................................................................................................. 5

2.1.1. Disk Tab .............................................................................................. 52.2. Transaction Behavior ........................................................................................ 7

2.2.1. Emits .................................................................................................. 72.2.2. Retrieves .............................................................................................. 7


2.4.1. Publishes ............................................................................................. 72.4.2. Accesses .............................................................................................. 8

2.5. Node Message Events ....................................................................................... 83. Disk Forwarding Agent ................................................................................................ 9

3.1. Configuration .................................................................................................. 93.1.1. Disk Tab .............................................................................................. 9




3.5. Node Message Events ...................................................................................... 11

3

Disk Agents 4.1

1. IntroductionThis document describes the Disk agents being parts of the standard agents available on the NokiaConvergent Mediation Engine™ platform.




4

Disk Agents 4.1

2. Disk Collection AgentThe Disk Collection agent collects files from a local file system and inserts them into a ConvergentMediation Engine™ workflow. Initially, the source directory is scanned for all files matching thecurrent filter. In addition, the Filename Sequence and Sort Order services may be utilized to furthermanage the matching of files, although they may not be used at the same time since it will causeworkflow abortion. All files found will be fed one after the other into the workflow.

When a file has been successfully processed by the workflow, the agent offers the possibility of moving,renaming, removing or ignoring the original file. The agent can also be configured to keep files for aset number of days. In addition, the agent offers the possibility of decompressing compressed (gzip)files after they have been collected. When all the files are successfully processed, the agent stops toawait the next activation, whether it is scheduled or manually initiated.

2.1. ConfigurationThe Disk Collection agent configuration window is displayed when the node in a workflow is double-clicked, or right-clicked, selecting Configuration.... Parts of the configuration may be done in theFilename Sequence service tab and the Sort Order tab, both described in the Convergent MediationEngine™ Core - User Guide.

2.1.1. Disk Tab

Figure 1. Disk Collection Agent configuration, Disk tab.

The Disk tab contains configurations related to the placement and handling of the source files to becollected by the agent.

Absolute pathname of the source directory on the local file system of the EC, where thesource files reside.

Directory

Name of the source files to be collected from the source directory. Regular expressionsaccording to Java syntax applies. For further information, please refer to


Filename

5

Disk Agents 4.1

Example 1.

To match all log filenames beginning with TTFILE, type: TTFILE.*

Compression type of the source files. Determines if the agent will decompress the filesbefore passing them on in the workflow.

Compres-sion

• No Compression - agent does not decompress the files. Default setting.

• Gzip - agent decompresses the files using gzip.

If enabled, the source files are moved to the automatically created subdirectoryDR_TMP_DIR, in the source directory, prior to collection. This option supports safe col-lection of source files, reusing the same name.

Move Be-fore Col-lecting

If the specified value is greater than zero, and if no file has been collected during thespecified number of hours, the following message is logged:

InactiveSource

The source has been idle for more than <n> hours, the last

Warning(hours)

inserted file is <file>.

If enabled, the source files will be moved from the source directory (or from the directoryDR_TMP_DIR, if using Move Before Collecting) to the directory specified in the Direct-ory field, after the collection

Move to

If the Prefix or Suffix fields are set, the file will be renamed as well.

It is possible to move collected files from one file system to another. However, thishas a negative impact on performance. Also, the workflow will not be transactionsafe, because of the nature of the copy, relative to the move, functionality.

If such functionality is desired, it is strongly recommended to route the Disk Col-lection agent directly to a Disk Forwarding agent, configuring the output node tostore the files in the desired directory.

If enabled, the source files will be renamed after the collection, remaining in the sourcedirector from which they were collected (or moved back from the directory DR_TMP_DIR,if using Move Before Collecting).

Rename

If enabled, the source files will be removed from the source directory (or from the directoryDR_TMP_DIR, if using Move Before Collecting), after the collection.

Remove

If enabled, the source files will remain in the source directory after collection.Ignore

Absolute pathname of the directory on the local file system of the EC into which the sourcefiles will be moved after collection.

Directory

This field is only enabled if Move to is selected.

Prefix and/or suffix that will be appended to the beginning respectively the end of thename of the source files, after the collection.

Prefix/Suf-fix

These fields are only enabled if Move to to or Rename is selected.

6

Disk Agents 4.1

If Rename is enabled, the source files will be renamed in the current directory(source or DR_TMP_DIR). Be sure not to assign a Prefix or Suffix, giving filesnew names, still matching the filename regular expression, or else the files will becollected over and over again.

Number of days to keep source files after the collection. In order to delete the source files,the workflow has to be executed (scheduled or manually) again, after the configurednumber of days.

Keep(days)

Note, a date tag is added to the filename, determining when the file may be removed. Thisfield is only enabled if Move to or Rename is selected.


2.2.1. Emits

Emitted right before the first byte of each collected file is fed into a workflow.Begin Batch

Emitted just after the last byte of each collected file has been fed into the system.End Batch

2.2.2. Retrieves

If a Cancel Batch message is received, the agent sends the batch to ECS.Cancel Batch

If the Cancel Batch behavior defined on workflow level is configured to abortthe workflow, the agent will never receive the last Cancel Batch message. Inthis situation ECS will not be involved, and the file will not be moved, but leftat its current place.

If a Hint End Batch message is received, the collector splits the batch at the end ofthe current block processed (32 kB), If the block end occurs within a UDR, the batchwill be split at the end of the preceding UDR.

Hint End Batch

After a batch split, the collector emits an End Batch message, followed by a BeginBatch message (provided that there is data in the subsequent block).

2.3. IntrospectionThe agent produces bytearray types.


2.4.1. Publishes

A parameter containing a timestamp, indicating when the file is stored inthe collection source directory.

File Modified Timestamp

7

Disk Agents 4.1

Type date, defined as MIM type header.

A parameter containing a timestamp, indicating when the file transfer starts.File Retrieval Timestamp


A parameter containing the file size, in bytes, of the source file.Source File Size

Type long, defined as MIM type header.

A parameter containing the name of the currently processed file, as definedat the source.

Source Filename


A parameter containing the number of files, available to this instance forcollection at startup. The value is constant throughout the execution of the

Source File Count

workflow, even if more files arrive during the execution. The new fileswill not be collected until the next execution.


A parameter containing the path to the currently processed file, as definedat the source.

Source Pathname


2.4.2. Accesses




Reported along with the name of the source file that has been collected and inserted into the workflow.

File cancelled: name

Reported along with the name of the current file, each time a Cancel Batch message is received. Thisassumes the workflow is not aborted; please refer to Section 2.2, “Transaction Behavior” for furtherinformation.

8

Disk Agents 4.1

3. Disk Forwarding AgentThe Disk Forwarding agent creates files on the local file system containing the received data. Filesare created when a Begin Batch message is received, and closed when an End Batch message is received.In addition, the agent offers the possibility to compress (gzip) the files, or to further process them, usingany command.

To ensure that downstream systems will not use the files until they are closed, they are stored in atemporary directory until the End Batch message is received. This behavior also applies to CancelBatch messages. If a Cancel Batch is received, file creation is cancelled.

3.1. ConfigurationThe Disk Forwarding agent configuration window is displayed when the node in a workflow is double-clicked, or right-clicked, selecting Configuration....

3.1.1. Disk Tab

Figure 2. Disk Forwarding agent configuration window, Disk tab.

Absolute pathname of the target directory on the local file system of the EC, wherethe forwarded files will be stored.

Directory

The files will be temporarily stored in the automatically created subdirectoryDR_TMP_DIR, in the target directory. When an End Batch message is received, thefiles are moved from the subdirectory to the target directory.

Compression type of the target files. Determines if the agent will compress the filesor not.

Compression

• No Compression - agent does not decompress the files. Default setting.


Note that no extra extension will be appended to the target filenames, even ifcompression is selected. The configuration of the filenames is managed in theFilename Template tab, only.

If a Command is supplied, it will be executed on each successfully closed temporaryfile, using the parameter values declared in Arguments.

Command

9

Disk Agents 4.1

At this point the temporary file is created and closed, but the final filenamehas not yet been created.

The entered Command has to exist on the Convergent Mediation Engine™execution environment, either including an absolute path, or to be found in thePATH for the execution environment.

This field is optional. Each entered parameter value has to be separated from thepreceding value with a space.

Arguments

The temporary filename is inserted as the second last parameter, and the final filenameis inserted as the last parameter, automatically. This means that if, for instance, noparameter is given in the field, the arguments will be as follows:

$1=<temporary_filename> $2=<final_filename>

If three parameters are given in the field Arguments, the arguments are set as:

$1=<parameter_value_#1>$2=<parameter_value_#2>$3=<parameter_value_#3>$4=<temporary_filename>$5=<final_filename>

If enabled, files will be produced although containing no data.ProduceEmpty Files



3.2.1. Emits

None.

3.2.2. Retrieves

When a Begin Batch message is received, the temporary directory DR_TMP_DIR isfirst created in the target directory, if not already created. Then, a target file is createdand opened in the temporary directory.

Begin Batch

When an End Batch message is received, the target file in DR_TMP_DIR is first closedand then the Command, if specified in After Treatment, is executed on. Finally, thefile is moved from the temporary directory to the target directory.

End Batch

If a Cancel Batch message is received, the target file is removed from the DR_TMP_DIRdirectory.

Cancel Batch

10

Disk Agents 4.1

3.3. IntrospectionThe agent consumes bytearray types.


3.4.1. Publishes

A parameter containing a timestamp, indicating when the target file is cre-ated in the temporary directory.

File Transfer Timestamp

Type date, defined as MIM type trailer.

A parameter containing the name of the target filename, as defined in File-name Template.

Target Filename


A parameter containing the path to the target filename, as defined in theDisk tab.

Target Pathname


3.4.2. Accesses

Various resources from the Filename Template configuration to construct the target filename.



Reported along with the name of the target file when it has been successfully stored in the target dir-ectory. If an After Treatment Command is specified, the message also indicate that it has been ex-ecuted.

11

Disk Agents 4.1

Duplicate Batch Agent 4.1


Version 1.19

1





2




2. Duplicate Batch Detection Agent ................................................................................... 52.1. Profile ............................................................................................................ 52.2. Configuration .................................................................................................. 62.3. Transaction Behavior ........................................................................................ 6

2.3.1. Emits .................................................................................................. 62.3.2. Retrieves .............................................................................................. 6



2.6. Node Message Events ....................................................................................... 73. Duplicate Batch Inspector ............................................................................................ 8

3


1. IntroductionThis document describes the Duplicate Batch Detection agent being part of the standard agents availableon the Nokia Convergent Mediation Engine™ platform.




4


2. Duplicate Batch Detection AgentThe Duplicate Batch Detection agent allows duplication control on passed data batches. Each databatch will be tested against already stored meta data from batches, to see if they are considered to beduplicates.

Meta data for data batches are kept for a configurable number of days. If the meta data of a batch hasbeen removed, a duplicate of this batch can no longer be detected.

If any duplicates are detected, a message is logged to the System Log, and the duplicate batch is can-celled, which may cause the workflow to abort. Please refer to Convergent Mediation Engine™ Core- User Guide, for further details.

To monitor the duplicate batches the Duplicate Batch Inspector may be used, please refer to Section 3,“Duplicate Batch Inspector”.

It is only appropriate to use the agent after nodes that may create duplicate batches. Normally this isa file based collection node. Several workflows may utilize the same Duplicate Batch Profile. In thiscase, their batches will be mutually compared.

2.1. ProfileA Duplicate Batch Detection node is configured in two steps. First a profile has to be defined, thenthe regular configuration of the node is done. The Duplicate Batch Profile Editor window is accessedfrom the Configuration menu on the desktop.

Figure 1. Duplicate Batch Profile Editor.

Number of days to keep batch information in the database.Cache Timeout (days)

If enabled, a check sum is created from the data batch, used for comparison.Use CRC

If enabled, the number of bytes in the data batch is used for comparison.Use Byte Count

If enabled, the selected MIM resources are used for the comparison. If thebatch is empty it is only stored in the database if Use MIM Value is enabled.

Use MIM Value

A list of user defined MIM names. They are assigned values in the nodeconfiguration window.

Named MIMs

5


If the Detection Method is modified after a Duplicate Batch Detection node has been executed,the already stored information will not match any records processed with the new profile version.

2.2. ConfigurationThe Duplicate Batch Detection agent configuration window is displayed when a Duplicate Batch De-tection node is double-clicked or right-clicked, selecting Configuration...

Figure 2. Duplicate Batch Detection agent configuration window .

A list of all defined Duplicate Batch profiles (see the Configuration menu).Profile

A list of user defined MIMs as defined in the profile.Named MIMs

A list of existing MIM values to be mapped against the user defined Named MIMs.MIM Resource

Selected MIM values to be used in duplicate detection message.Logged MIMs

The use and setting of private threads for an agent, enabling multi-threading within a workflow,is configured in the Thread Buffer tab. Please refer to Convergent Mediation Engine™ Core- User Guide, for further details.


2.3.1. Emits

Emitted if a duplicate is found.Cancel Batch

2.3.2. Retrieves

Removes all timed out detection data from the database cache.Begin Batch

Compares the incoming batch against the ones existing in the database. If a duplicateis found, a cancel mark is emitted and an error message is written in the System Log.

End Batch

If no duplicate is found, the data batch information for the current batch is stored in thedatabase.

6


2.4. IntrospectionThe agent produces and consumes bytearray types.


2.5.1. Publishes

The agent does not itself publish any MIM resources.

2.5.2. Accesses

Values from selected MIM resources in the Logged MIMs and Named MIMs lists.User selected

Read at Start Batch.


2.6. Node Message EventsConnected to DB

Reported when the agent has successfully connected to its duplicate detection database.

Duplicate batch detected

Reported when a duplicate batch has been identified.

7


3. Duplicate Batch InspectorThe Duplicate Batch Inspector is used for viewing the meta data cache used for duplicate checking.

The Duplicate Batch Inspector is accessed from the Inspection menu on the desktop. Initially, thewindow is empty. To populate it, search criteria needs to be specified in the Search Duplicate Batchesdialog. Select Search... from the Edit menu to access the dialog.

Figure 3. Search Duplicate Batches dialog.

Select the profile which corresponds to the data of interest.Profile

Option to search for data created during a certain time period.Creation Period

Select OK to view the matching data of the search.

Figure 4. Duplicate Batch Detection Inspector window.

Removes selected entry from the list. If no entry is selected, all entriesare deleted.

Edit menu Delete...

Displays the Search Duplicate Batches dialog, where search criteriamay be modified.

Edit menu Search...

Shows all MIM values for the selected duplicate batch.Edit menu Show MIM Val-ues

Matching entries are bundled into groups of 500. This list shows whichgroup, out of how many, is currently displayed. An operation targetingall matching entries, will have affect on all groups.

Show Batches

8


Note, there is a limit of 100 000 entries for a match. If the match exceedsthis limit, any bulk operation (deleting etc) must be repeated for eachmultiple of 100 000.

The index of the batch in the search results.ID

The transaction ID of the batch.Txn ID

The time when the transaction was created.Creation Time

The MIM data stored for the batch.MIM Values

9


Duplicate UDR Detection Agent 4.1


Version 1.24

1





2




2. Duplicate UDR Detection Agent .................................................................................... 52.1. Profile ............................................................................................................ 52.2. Configuration .................................................................................................. 6

2.2.1. Dup UDR Tab ....................................................................................... 72.3. Transaction Behavior ........................................................................................ 7

2.3.1. Emits .................................................................................................. 72.3.2. Retrieves .............................................................................................. 7



2.6. Node Message Events ....................................................................................... 83. Duplicate UDR Inspector ............................................................................................. 9

3


1. IntroductionThis document describes the Duplicate UDR Detection agent being part of the standard batch agents,available on the Nokia Convergent Mediation Engine™ platform.





4


2. Duplicate UDR Detection AgentThe Duplicate UDR Detection (DUD) agent provides duplication control on passed UDRs. Each UDRis compared against already stored UDRs to evaluate if it is a duplicate.

If a duplicate is found, a message is automatically logged in the System Log, and the UDR is markedas erroneous and routed on a user defined route - for instance to ECS. If routed to ECS, an automaticallygenerated ECS Error Code - DUPLICATE_UDR - is assigned to the UDR. That allows the ECS to besearched for duplicate UDRs.

Duplication comparison is not based on the contents of a complete UDR but on the contents of thefields selected by the user.

If the same file happens to be reprocessed, all UDRs will be considered being duplicates. If thefile contains a considerable number of UDRs, the process of inserting all of them in ECS maybe time consuming.

Having a Duplicate Batch Detection (DBD) node prior to the DUD node will only make theproblem worse. The DBD node will not detect that the batch is a duplicate until the end of thebatch. At that point all UDRs have already passed the DUD node and are inserted, as duplicates,into ECS. Since the DBD will flag for a duplicate batch, the batch is removed from the streamforcing the DUD node also to remove all UDRs from ECS.

2.1. ProfileA Duplicate UDR Detection node is configured in two steps. First a profile has to be defined, then theregular configurations of the node are made. The Duplicate UDR Profile Editor is accessed from theConfiguration menu on the desktop.

Figure 1. Duplicate UDR Profile Editor.

5


The execution context on which to store the duplicate repository. Since thisis a file based solution, it is necessary to lock the profile to a certain host.

Storage Host

An absolute path, on the selected storage host, in which to store the duplicaterepository.

Directory

The maximum age of records to keep in the repository. The age of a recordis calculated from the timestamp indexing field. If the indexing field is notspecified as a date field, this field is ignored. Default is 30 days.

Max Cache Age(days)

The maximum number of records to store in the duplicate repository. Defaultis 5000 records.

Max Cache Size(thousands)

The UDR type the node will process.Type

UDR field used as an index in the duplicate comparison. Fields of type longand date are valid for selection.

Indexing Field

For performance reasons, this field should preferably be either an increasingsequence number, or a timestamp with good locality. This field will alwaysbe implicitly evaluated.

If enabled (default), the indexing field will be treated as a timestamp insteadof a sequence number.

Date Field

The fields to use for the duplication evaluation, when deciding whether or notthe UDR is a duplicate.

Checked Fields

If the Checked Fields or Indexing Field are modified after a node isexecuted, the already stored information will be considered useless thenext time the workflow is activated. Hence, duplicates will never befound amongst the old information since other type of meta data hasreplaced them.

2.2. ConfigurationThe Duplicate UDR Detection configuration window is displayed when a Duplicate UDR Detectionnode is double-clicked or right-clicked, selecting Configuration....

6


2.2.1. Dup UDR Tab

Figure 2. Duplicate UDR Detection configuration window, Dup UDR tab.

Select from available profiles as defined in the Duplicate UDR Profile Editor.Profile

Indicates on which route to send detected duplicates.Duplicate Route

The list is not populated with output routes until the routes are created and thedialog is reopened.

A list of MIM values, used when creating the error information for the ECS (ifrouted to an ECS Forwarding node). To display it from ECS, double-click the

Batch Source In-formation

Error Code for the UDR (that is DUPLICATE_UDR for all duplicates, no matterfrom what workflow or profile they originate). Also, note that MIM values areselected from the original UDR, not the duplicate.

The use and settings of private threads for an agent, enabling multi-threading within a workflow, is configured in the Thread Buffer tab.Please refer to Convergent Mediation Engine™ Core - User Guide, forfurther details.


2.3.1. Emits

None.

2.3.2. Retrieves

None.

2.4. IntrospectionThe agent produces and consumes UDR types, selected from the UDR Type list.

7



2.5.1. Publishes

Number of detected duplicates in the current batch.Detected Duplicates

Type int, defined as MIM type trailer.

2.5.2. Accesses

Values to log in ECS.User selected


2.6. Node Message Events

File Repository Initialized

Reported when the agent has successfully opened its duplicate detection repository.

Number of number UDRs were duplicates(number were too old)

Reported after each processed batch. The last number denotes records that were too old to be comparedagainst, that is, they were older than the configured maximum age.

8


3. Duplicate UDR InspectorThe Duplicate UDR Inspector is displayed by selecting Duplicate UDR Inspector... from the Inspectionmenu on the desktop.

Initially, the inspector window is empty. To populate the window, select Search... from the Edit menuto display the Search Batches window.

Figure 3. The Duplicate UDR Inspector, Search Batches window.

Select the profile that corresponds to the data of interest.Profile

Option to search for batches processed during a certain time period.Processed Period

Option to search with respect to time span of the indexing field in batches. Thisoption is only available if the selected profile has a date indexing field.

Content Period

A regular expression search in the value of selected MIM resource.MIM Criteria

Disable Read Only if batches need to be deleted from the repository. Exclusiveaccess to the repository is required for deleting batches, meaning that if a currently

Lock Handling

running workflow is using the selected profile, the workflow needs to be stoppedto be able to get exclusive access.

Allows specification of the sort column and order.Sort Order

Once the search criteria has been specified, the Duplicate UDR Inspector window is populated withmatching batches.

9


Figure 4. The Duplicate UDR Inspector.

Commits changes made by Delete.File menu Save

Flags selected batches for deletion. Select Save to commit deletions.Edit menu Delete

Refresh search with previous search parameters.View menu Refresh

If a search results in a large number of batches, this enables switching betweendifferent pages in the results.

Show Batches

The index of the batch in the search results.ID

The transaction ID of the batch.Txn ID

The date the batch was processed.Processed Date

The MIM data stored for the batch. Double-click this field to view all MIMvalues.

MIM Values

The Duplicate UDR Detection agent stores batches in date segments. Thecolumns show the date range of the actual data that was duplication checked

Content Start / End

during transaction. If the transaction contains dates older than the Max CacheAge, configured in the Duplicate UDR Profile Editor, Outside range is dis-played.

If both Start and End show Outside range, all dates in the transaction wereolder than Max Cache Age. UDRs that are outside range are always routed asnon-duplicates since there is no duplicate data to compare them to.

These columns are only visible if Date Field is enabled in the Duplicate UDRProfile Editor.

The number of records processed for a given batch.Records

The number of duplicates found in the batch.Duplicates

10


Error Correction System 4.1


Version 1.30

1





2




2. ECS Inspection .......................................................................................................... 52.1. Searching the ECS ............................................................................................ 92.2. Error Codes ................................................................................................... 112.3. Reprocessing Groups ...................................................................................... 122.4. States ........................................................................................................... 132.5. Maintaining the ECS ....................................................................................... 13

2.5.1. Maintaining Persistent Data ................................................................... 133. ECS Statistics ........................................................................................................... 14

3.1. Unlimited Search ............................................................................................ 143.2. Search On Error Codes .................................................................................... 15

3.2.1. Searching the ECS Statistics ................................................................... 154. ECS Collection Agent ................................................................................................ 17

4.1. Configuration ................................................................................................ 174.2. Transaction Behavior ....................................................................................... 18

4.2.1. Emits ................................................................................................. 184.2.2. Retrieves ............................................................................................ 18


4.4.1. Publishes - UDR .................................................................................. 194.4.2. Publishes - Batch ................................................................................. 194.4.3. Accesses ............................................................................................. 19

4.5. Node Message Events ...................................................................................... 195. ECS Forwarding Agent .............................................................................................. 20


5.2.1. Emits ................................................................................................. 205.2.2. Retrieves ............................................................................................ 20



5.5. Node Message Events ...................................................................................... 216. Example - Sending UDRs to ECS ................................................................................ 22

6.1. ECS Forwarding Workflow ............................................................................... 226.1.1. ECS Inspector ..................................................................................... 226.1.2. Analysis Node ..................................................................................... 236.1.3. ECS Forwarding Node .......................................................................... 23

6.2. ECS Collection Workflow ................................................................................ 236.2.1. Workflow Preferences ........................................................................... 246.2.2. ECS Collection Node ............................................................................ 246.2.3. Analysis Node ..................................................................................... 24

7. Example - Sending Batches to ECS .............................................................................. 257.1. ECS Forwarding Workflow ............................................................................... 25

7.1.1. Workflow Preferences ........................................................................... 257.1.2. ECS Inspection .................................................................................... 267.1.3. Analysis Node ..................................................................................... 26

7.2. ECS Collection Workflow ................................................................................ 267.2.1. Workflow Preferences ........................................................................... 277.2.2. ECS Collection Node ............................................................................ 277.2.3. Analysis Node ..................................................................................... 27

3


1. IntroductionThis document describes the ECS System being part of the standard Convergent Mediation Engine™platform.




4


2. ECS InspectionThe Convergent Mediation Engine™ system provides the possibility to route UDRs from workflowsto an Error Correction System (ECS). The ECS is used when UDRs fail validation, and manual inter-vention is needed before they can be successfully processed.

It is also possible to send batches of raw data to the ECS. This requires the workflow to contain anagent that can issue a cancelBatch request.

The ECS Inspector allows the access group user to launch and maintain the data, that is delete UDRs,view and edit their content, and add them to RP (Reprocessing) Groups. The latter is a prerequisite forthe data to be collected by an ECS Collection node.

Besides just sending a UDR or batch to ECS, a workflow can be configured to associate user-definedinformation with the ECS data. For the UDRs this is Error Code and MIM information. For cancelledbatches, the Error UDR and Cancel Message may contain user defined information.

Only a reference and not the data itself is saved in the database. Physically, ECS data is savedin the directory defined by the parameter mz.ecs.path found in$MZ_HOME/etc/config.xml.

If the mz.ecs.path parameter is changed, the changes will take effect the next time data isinserted into ECS. Existing ECS data is left at its current location and must not be moved. Ifrequired to do so anyway, move the contents of the old mz.ecs.path directory to the new,and create a soft-link in the old directory pointing out the new location.

Take special precaution when changing, updating or renaming formats. If the updated formatdoes not contain all the fields of the historical format, which UDRs may already reside in theECS or Aggregation, data is lost. When a format is renamed, it will still contain all the fields.The data, however, cannot be collected.

To display the ECS Inspector, select ECS Inspector... from the Inspection menu on the desktop.

Initially, the window is empty and must be populated with data using the Search ECS dialog. Referto Section 2.1, “Searching the ECS” for details. Once populated, the window contains a table whereeach row represents one UDR or batch:

Figure 1. Error Correction System Inspector - UDRs.

5


Figure 2. Error Correction System Inspector - Batches.

The following menu items apply to both Batches and UDRs. Items not listed are considered to be ofgeneral type, refer to Convergent Mediation Engine™ Core - User Guide, for detailed information.

Displays the Search Error Correction System dialog where search criteriamay be defined to limit the entries in the list. Refer to Section 2.1,“Searching the ECS” for further information.

Edit menu Search...

Displays the ECS Error Code dialog where the Error Codes in the systemmay be configured. Refer to Section 2.2, “Error Codes” for further inform-ation.

Edit menu Error Codes...

Displays the ECS Reprocessing Groups dialog, where RP Groups aremanaged. Refer to Section 2.3, “Reprocessing Groups” for further inform-ation.

Edit menu ReprocessingGroups...

Defines selected or matched entries as New or Reprocessed (i.e. collectedby an ECS node). Already processed data can be reset to New to enablecollection.

Batch/UDR menu SetState...

Assigns selected or matched entries to an RP Group. Grouped entries canbe collected simultaneously by an ECS Collection node.

Batch/UDR menu Assignto RPG...

Note that assignment to RP Groups can be made automatically for eachnew entry. The data must be assigned an Error Code in the workflow, andthis Error Code must be mapped to the RP Group in the ECS Error Codedialog.

Removes selected or matched entries, provided that the RP State is set toReprocessed. The ECS does not have to be purged manually, there is a

Batch/UDR menu Delete

predefined cleanup task - ECS Maintenance. Please refer to Section 2.5,“Maintaining the ECS” for further information.

Matching entries are bundled in groups of 500. This list shows which group,out of how many, is currently displayed. An operation targeting allmatching entries, will have effect on all groups.

Show Batches/UDRs

6


Note that there is a limit of 100 000 entries for a match. If the match exceedsthis limit, any bulk operation (assigning to RP Groups, deleting etc) mustbe repeated for each multiple of 100 000.

UDR specific menu items:

Displays the contents of the selected UDRs (also displayed if the cell is double-clicked). From here, the contents of the UDR may be changed.

UDR menu ExploreUDR...

Refer to Convergent Mediation Engine™ Ultra Format Management - UserGuide, for further information.

Several UDRs (selected or matched) may be edited simultaneously with theBulk Editor. The editor displayed from ECS differs slightly from the editor

UDR menu BulkEdit...

displayed from the UDR Editor window. The ECS Bulk Editor has an AirEdit option, which makes it possible to preview the changes before approvingand saving.

Refer to Convergent Mediation Engine™ Ultra Format Management - UserGuide, for further information.

Batch specific menu items:

Opens up a Error UDR Viewer displaying the contents of the ErrorUDR (if any), also displayed if the cell is double-clicked.

Batch menu Explore ErrorUDR...

It is not possible to change the contents of an Error UDR.

Displays the selected batch as raw data, also displayed if the cell isdouble-clicked.

Batch menu View Batch...

7


The following columns apply to both Batch and UDR entries:

A sequence number, automatically assigned to an entry.ID

Date and time when the entry was inserted in the ECS.Date

Name of the workflow from which the data was sent.Workflow

Name of the node that sent the entry to ECS. For UDRs this will be the ECS Forwardingnode and for batches the ECS Collection node.

Node

The Error Code as defined in ECS.Error Code

Shows what group the entry is assigned to, if any. Assignments can be made bothmanually and automatically. For the latter case, an Error Code must be mapped to aReprocessing Group.

RP Group

Initially, an entry has the state New. When collected by an ECS Collection agent, thestate is changed to Reprocessed. It is possible to manually set the RP State. Note, onlyentries in state New may be collected.

RP State

MIM values to be associated with the entry may be configured. Double-clicking thisfield will display a new window, listing the MIM values. These values are set differentlyfor the two types of entries:

MIM Values

• Batch - From the workflow Preferences window.

• UDR - From the ECS Forwarding node.

UDR specific columns:

The UDR Type.UDR Type

Batch specific columns:

The name of the node that issued the cancelBatch request.Cancel Node

The message sent with the cancel request. In case of user defined request:Cancel Message

Example 1.

cancelBatch("undefined_number_prefixes.");

This UDR is populated either from the workflow Preferences window, or from anode utilizing APL. The UDR can contain any useful information which is needed

Error UDRType

in a workflow reprocessing a batch. The fields of the error UDR will automaticallyappear as MIM values in the reprocessing workflow.

8


Figure 3. Workflow Preferences window, Error tab.

Example 2.

myErrorUDR eUDR = udrCreate( myErrorUDR );eUDR.noOfUDRs = (long)mimGet( "IN", "Outbound UDRs" );udrAddError( eUDR, "nokSOURCE", "Switch not found." );cancelBatch( "Incorrect source.", eUDR );

The error UDR format is defined as any other format from the Ultra FormatEditor.

internal myErrorUDR{ long noOfUDRs;};

2.1. Searching the ECSThe Search Error Correction System dialog allows the user to locate erroneous UDRs and batchesin the ECS. The Search Error Correction System dialog is displayed when Search... is selected inthe Error Correction System Inspector. Select UDRs or Batches to choose the UDR or batch spe-cific options.

9


Figure 4. Search Error Correction System dialog - UDRs.

The following options apply to both Batches and UDRs:

The name of the workflow that sent the entry to ECS.Workflow

The name of the node that sent the entry to ECS.Node

The Error Code as defined in ECS.Error Code

A list displaying the Error Cases associated with the selected Error Code. Ifthe entry is too long to fit in the field, it will be truncated and the full stringcan be viewed in a tool-tip.

Error Case

An Error Case associates a string with a defined Error Code. Error Cases canonly be appended via APL code:

udrAddError( <UDR>, <Error Code>, <Error Case> );

Note that in case of a batch, the <UDR> parameter is the error UDR.

Enables selection of all Reprocessing Groups. Unassigned (UDR) will list allentries not associated with any RP Group.

Reprocessing Group

The status of the UDR. Can be New, or Reprocessed. Only entries in stateNew may be collected.

Reprocessing State

It is possible to configure a workflow to send descriptive MIM values withthe actual data (the Preferences window of the workflow). This could be usedto refine the search in ECS.

MIM

Refines the search in time when the data was entered into ECS.Period

10


UDR specific options:

The type of UDR.UDR Type

Batch specific options:

The name of the node that issued the cancelBatch call.Cancel Node Name

With the cancelBatch call, it is possible to send a message as argument -the Cancel Message.

Cancel Message

With a batch, it is possible to send one UDR containing crucial MIM information(or any desired information in case of populating the UDR from an APL node).

Error UDR Type

Only 100 000 entries at a time can be browsed. If the search results in more than 100 000 entries,bulk operations must be repeated for each multiple of 100 000.

The ECS Inspector caches the result when the user populates a list (for instance the Error Codes).This to avoid repopulation of workflow names, node names and error codes, since it is costlyin terms of performance. Therefore the Reset button must be pressed in order to repopulate thesearch window.

2.2. Error CodesError Codes allows errors to be classified in the Error Correction System. Error Codes can be tied toboth UDRs and Batches. A Reprocessing Group can also be assigned to an Error Code. When an entityis inserted in the Error Correction System it will thereby automatically be available for collection bythe ECS Collection agent.

There are two predefined codes within the system, AGGR_UNMATCHED_UDR and DUPLICATE_UDR,which are automatically set by the Aggregation and Duplicate UDR Detection agents when the corres-ponding error condition is detected. All other Error Codes are defined by the user.

To create an Error Code, select Error Codes... from the Edit menu. This will display the Error Codedialog. Selecting Add will enable assignment of new Error Codes.

Figure 5. ECS Error Code dialog.

A user may attach optional information to ECS from an Analysis or Aggregation node, as long as anError Code has been defined. To this code, any information may be appended, see the following example.Double-clicking the field will display a new window listing the optional string.

11


Example 3.

The Error Case can only be appended through APL code.

udrAddError( input, "CALL ID ERROR", "The callId: " + input.callId + ", Calling number: " + input.anum );

In this example the "CALL ID ERROR" is defined in the ECS Error Code dialog.

To clear the errors for a UDR the udrClearErrors function should be used.

2.3. Reprocessing GroupsIn order for ECS data to be reprocessable, it has to be assigned to a Reprocessing Group (and havestatus New). This group is then selected in the ECS Collection node of the reprocessing workflow.

To create an RP Group, select Reprocessing Groups... in the Edit menu of the Error CorrectionSystem Inspector.

Figure 6. ECS Reprocessing Group dialog.

The RP Groups are created by selecting unique group names. Select the Add button to display theAdd ECS Reprocessing Group dialog.

Figure 7. The Add ECS Reprocessing Group dialog.

The Error UDR Type is only applicable for Batch Data Type. If no Error UDR is to be used in re-processing this information is not required.

UDRs with several Error Codes mapped to different RP Groups cannot be automatically assignedto an RP Group. They must be assigned manually.

12


2.4. StatesEntries in the ECS may have different states which are assigned automatically or manually by the user.The following states are possible:

The UDR has not been reprocessed. In order for it to be collectible, it has to be assignedto an RP Group.

New

The UDR has been reprocessed, and cannot be reprocessed again unless its state ismanually changed back to New.

Reprocessed

Note, only entries set to Reprocessed can be removed.

2.5. Maintaining the ECSThe ECS Maintenance task removes outdated ECS data, provided that the state is Reprocessed. TheECS Maintenance task is accessed from the Task Manager in the Tools menu on the desktop.

From the Edit Task Configuration window, the number of days to keep data is set. It is also possibleto fully turn off the cleanup of UDRs, Batches, Statistics, or all of them. The Statistics may be reportedby email, if configured in the Report tab of the task.

Figure 8. Edit Task Configuration window - Cleanup tab.

If enabled, the number of days to keep UDRs in the ECS is considered (maximum 999days). If disabled, the UDRs will remain until manually cleaned out via the ECS Inspector.

UDRs

If 0 (zero) is entered, all UDRs with state Reprocessed will be removed, without consideringtheir dates.

If enabled, the number of days to keep Batches in the ECS is considered (maximum 999days). If disabled, the Batches will remain until manually cleaned out via the ECS Inspector.

Batches

If 0 (zero) is entered, all Batches with state Reprocessed will be removed, without consid-ering their dates.

If enabled, the number of days to keep Statistics in the ECS is considered (1 - 999 days).If disabled, the Statistics will remain in the system forever.

Statistics

2.5.1. Maintaining Persistent Data

The ECS is designed to store a fairly limited amount of erroneous UDRs and Batches. It is thereforeof importance that the data is extracted, reprocessed or deleted from ECS on a regular basis.

13


3. ECS StatisticsThe Error Correction System Statistics window is accessed from the Inspection menu on the desktop.The system calculates the number of new and reprocessed UDRs and Batches ending up in the ECSon a daily basis. The Error Correction System Statistics window allows inspection of these calculatedvalues, as well as more specific views as the system may be inspected down to Error Code level. Thedata may also be exported and printed.

The ECS Statistics data can only be gathered and calculated once a day.

Initially the Error Correction System Statistics window is empty. To populate the window, selectSearch... from the Edit menu to specify the search criteria.

3.1. Unlimited SearchIf no limitations are entered in the search dialog, the following window applies. Note, when selectingone row in the table, the four most frequent Error Codes for that date is shown where one is representingthe summary of all Error Codes with the name "Other".

Figure 9. ECS Statistics window - Search with no limitations.

Shows the Export dialog, allowing the statistics to be exported.File menu Export...

Shows the Print dialog, allowing the statistics to be printed.File menu Print...

Shows the Search ECS Statistics dialog. Refer to Section 3.2.1, “Searchingthe ECS Statistics” for further information.

Edit menu Search...

Holds the date and time when the values were calculated.Date

The amount of new UDRs or Batches that existed in ECS on the given date.New

The amount of reprocessed files that existed in ECS on the given date.Reprocessed

14


Enables to single out graphical statistics for either new or reprocessed UDRsor Batches separately.

Value Type

3.2. Search On Error CodesIf a search on specific Error Codes is performed, the following window applies:

Figure 10. ECS Statistics window - Search on Error Codes.

Shows the Export dialog, allowing the statistics to be exported.File menu Export...

Shows the Print dialog, allowing the statistics to be printed.File menu Print...

Shows the Search ECS Statistics dialog. Refer to Section 3.2.1, “Searchingthe ECS Statistics” for further information.

Edit menu Search...

Holds the date and time when the values were calculated.Date

The amount of new UDRs or Batches that reached the ECS on the given date.New

The amount of reprocessed files that reached the ECS on the given date.Reprocessed

This column is only visible when the search is made on Error Codes.Error Code

The last time the error occurred.Newest

The first time the error occurred.Oldest

Enables to single out graphical statistics for either new or reprocessed UDRsor Batches separately.

Value Type

3.2.1. Searching the ECS Statistics

Limitations to the search may be selected to single out the data of interest.

15


Figure 11. The Search ECS Statistics dialog.

Will determine if the search will be made on Batches or UDRs.Data Type

A list of available Error Codes, as defined in ECS. To search on several Error Codesthe Add button may be selected to append further fields.

Error Code

Refines the search in time when the data was entered into ECS.Period

Only 100 000 entries at a time can be browsed. If the search results in more than 100 000 entries,bulk operations must be repeated for each multiple of 100 000.

16


4. ECS Collection AgentThe ECS Collection agent fetches data sent to the internal Convergent Mediation Engine™ ErrorCorrection System (ECS) by workflows configured to do so. Data is sent to the ECS as UDRs orbatches. In the latter case, an error UDR may be associated with a batch containing any desired inform-ation. When collected by an ECS Collection node, this UDR's fields will be included as MIM valuesin the workflow.

Figure 12. A typical workflow collecting UDRs from ECS.

If batches are collected, the ECS Collection node produces bytearray data. A Decoder must normallybe used in the workflow. What UDRs/batches to collect is defined by the RP (Reprocessing) Groupname. Assigning data to RP Groups is done from the Error Correction System Inspector.

It is only possible to have one active ECS Collection workflow per RP Group at a time.

Collecting UDRs from ECS does not mean that they are physically deleted from ECS, only thattheir state is changed. The cleanup is managed by the predefined task ECS Maintenance, referto Section 2.5, “Maintaining the ECS” for further details. Manual deletion, directly from theError Correction System Inspector is also possible.

4.1. ConfigurationThe ECS Collection agent configuration window is displayed when an ECS Collection node is double-clicked or right-clicked, selecting Configuration...

Figure 13. ECS Collection configuration window.

In order for data to be collectible, it must belong to a predefined RP Group. Thegroups in this list are suffixed with their type - Batch or UDR.

ReprocessingGroup

RP Groups are defined in the Error Correction System Inspector. CollectedUDRs/batches will automatically be marked as Reprocessed. Reprocessed datacan be collected again, if the state is manually changed back to New in the ErrorCorrection System Inspector.

17


Enabled when a UDR reprocessing group has been selected. The value defineshow many UDRs will be collected before the ECS Collector finishes the current

Collection Size

batch and starts a new one. Valid range is 1,000 to 100,000 UDRs. A higher valuewill require more memory.

A group of UDRs can consist of several format types. Selecting Add will displaythe UDR Internal Format Browser. Select the type/types to collect. Any UDRsin the RP Group not matching the selected types will be ignored.

Routed Types

The default directory, used for storage of UDRs and Batches routed to the ECS, is$MZ_HOME/ecs.


4.2.1. Emits

Begin Batch • UDRs - The ECS Collection node will automatically create one batch for all UDRsmatching the collection definitions. Before routing the first UDR, beginBatch isemitted.

• Batches - Emitted prior to routing of a batch.

End Batch • UDRs - Emitted when all UDRs have been collected - or when a Hint End Batchrequest is received. The UDRs are marked as Reprocessed in ECS.

• Batches - Emitted after each batch has been processed. The batch is marked as Re-processed.

4.2.2. Retrieves

None.

If any node in the workflow emits a cancelBatch, the workflow will abortimmediately (regardless of workflow configuration).

Cancel Batch

4.3. IntrospectionThe agent produces data depending on the data type of the RP Group.

The selected types under Routed Types. If no types are selected the generic drudr will beproduced.

UDRs

Produces bytearray types.Batch

18



4.4.1. Publishes - UDR


4.4.2. Publishes - Batch

The name of the workflow that cancelled the batch.Originating Workflow

The name of the node that cancelled the batch.Originating Node

The ID of the batch as assigned in the Error Correction System Inspector(see the ID column).

Database ID

The original size of the cancelled batch.Source File Size

All field values in the error UDR associated with the RP Group.<error UDR fields>

4.4.3. Accesses



4.5. Node Message EventsReady with file: <filename>

This event is logged for batch collection only, and is reported at end batch stating the name of the filecurrently processed.

Ready with recovered file

Reported at when file is recovered in case of a crash.

Start collecting

This event is logged for UDR collection only, and is reported when the collection from ECS starts.

Commit started

This event is logged for UDR collection only, and is reported at end batch when starting to commitchanges to the database.

Commit done consumed <count> UDRs

This event is logged for UDR collection only, and is reported at end batch upon a successful commit.

19


5. ECS Forwarding AgentIn order to send UDRs to ECS, a workflow must contain an ECS Forwarding agent and the invalidUDRs have to be routed to it. It is recommended to use a preceding Analysis (or Aggregation) nodeto associate an Error Code with the UDR.

The ECS Forwarding agent is applicable for UDRs only. Batches are handled through the cancel-Batch functionality.

From the ECS Forwarding agent, it is possible to pass on MIM values to be associated with theUDR in the Error Correction System Inspector.

5.1. ConfigurationThe ECS Forwarding agent configuration window is displayed when an ECS Forwarding node isdouble-clicked or right-clicked, selecting Configuration...

Figure 14. ECS Forwarding agent configuration window.

MIM values to be associated with a UDR when sent to ECS.Logged MIMs


5.2.1. Emits

None.

5.2.2. Retrieves

None.

5.3. IntrospectionThe agent consumes selected UDR types.

20



5.4.1. Publishes

The name of the node.Node Name

The number of UDRs routed to the node.Inbound UDRs

5.4.2. Accesses



5.5. Node Message EventsNone.

21


6. Example - Sending UDRs to ECSThis example will show the necessary configurations for a workflow sending UDRs to ECS, and fora workflow configured to collect this data. The UDRs sent to ECS will have an Error Case, that is, astring associated with the Error Code.

Typically, a UDR may be sent to ECS as a result of a failing table lookup evaluation. To make surethe error was not temporary, that is, the tables simply needed to be updated first, these UDRs are re-cycled. A new workflow is created, collecting them from ECS and re-evaluating the same problem. Ifit still exists, the UDR is sent back to ECS.

6.1. ECS Forwarding WorkflowIn order to send a UDR to ECS, a workflow must contain an ECS Forwarding node. To perform atable lookup for all UDRs an Analysis node is used. If the lookup succeeded, the UDR is sent on theOK route to be saved on disk, while the failing UDRs are sent to the ECS Forwarding node.

Figure 15. A workflow sending UDRs to ECS.

UDRs may be sent to ECS without any Error Code or MIM values associated with it. However, thiswill make browsing in the Error Correction System Inspector more difficult, and no auto-assignmentto RP Groups is possible (valid for the Error Code).

6.1.1. ECS Inspector

Error Codes associated with RP Groups will make all new data with the specific code being automat-ically assigned to an RP Group. If this is ignored, the UDRs will have to be assigned manually.

Figure 16. The Add ECS Error Code window from which an RP Group may be selected.

22


6.1.2. Analysis Node

The Analysis node is used for validation and routing of the UDRs, and association to a valid (existing)Error Code. The following example appends an Error Code and an Error Case to the UDR beforesending it on to the ECS Forwarding node.

Example 4.

udrAddError( input, "AreaCode_ERROR", "Complete anumber: " + input.anum);udrRoute( input, "error" );

6.1.3. ECS Forwarding Node

In the ECS Forwarding node, the MIM values to associate with the UDR are appended. This is optional,but makes it easier to search for data and get additional information about the UDR from the ErrorCorrection System Inspector.

Figure 17. The ECS Forwarding node.

6.2. ECS Collection WorkflowIn the collection workflow the same evaluation is tried again. If it fails, the UDR is sent back to ECSwith the same configuration.

An existing RP Group, in combination with an RP State set to New, are the prerequisites for beingable to collect ECS data.

Figure 18. A workflow collecting and validating ECS data.

23


Since we want to redo the processing made in the forwarding workflow, we keep the configurationsof the Error Correction System Inspector and ECS Forwarding nodes the same as in the previousworkflow.

6.2.1. Workflow Preferences

The Error tab must not be configured to handle Cancel Batch behavior, since it will never be validfor ECS Collection workflows. No calls to cancelBatch are allowed from any node since it willcause the workflow to abort immediately.

6.2.2. ECS Collection Node

All UDRs conforming to the collection criteria will be selected and processed as a batch.


The Analysis node only needs to validate and route the UDRs. The Error Code and Error Case isalready associated with the UDR.

Example 5.

udrRoute( input, "error" );

Example 6. Reassigning to a Different RP Group

Suppose there is a workflow collecting and validating UDRs from ECS. If the validation fails,the UDRs will be sent back to ECS with an associated Error Code. If desired to associate thisUDR with a different RP Group, udrClearErrors must be called prior to udrAddError.The exception is if the new Error Code is associated with the same RP Group.

Case 1 - same RP Group

If the new Error Code belongs to the same RP Group:

• Using udrClearErrors will result in only the new Error Code (including the RP Group)being associated with the UDR in ECS.

• Leaving out udrClearErrors will result in both the old and the new Error Codes (includ-ing the RP Group) being associated with the UDR in ECS.

Case 2 - different RP Group

If the new Error Code belongs to a different RP Group:

• Using udrClearErrors will result in only the new Error Code and RP Group being as-sociated with the UDR in ECS.

• Leaving out udrClearErrors will result in no association to any RP Group, but bothError Codes, in ECS.

All UDRs collected at one activation of the workflow will be processed as one batch.

Any call to cancelBatch will cause the workflow to abort immediately.

24


7. Example - Sending Batches to ECSECS Forwarding agents are not used for batches. Instead, a batch is sent to ECS directly from a collectionnode when receiving a cancelBatch. Also, there is a possibility to associate an error UDR with abatch. This UDR in turn can be assigned error information with udrAddError.

7.1. ECS Forwarding WorkflowThe forwarding workflow contains an Analysis node which validates batches and sends them to ECSin case of failure.

Figure 19. The Analysis node can call the cancelBatch function.


MIM values to associate with the batch are mapped in the Workflow Preferences window. Also, thenumber of allowed cancelled batches is set from here. Note that Abort Immediately may not be set- no batch will be sent to ECS if the workflow aborts.

Figure 20. Workflow Preferences, Error tab.

25


The erroneous UDR is handled from the Analysis node, see previous section. APL code always overridesany desktop settings. Hence, the set Error Code will have no effect in this case.

7.1.2. ECS Inspection

Automatic assignment to RP Groups is done exactly the same way as for UDRs - via the Error Cor-rection System Inspector window (accessed from the Edit menu, selecting Reprocessing Groups...).Make sure to select the Error UDR Type. Then the UDR fields will be included as MIMs in the col-lection workflow

Figure 21. Selecting the Error UDR Type is optional.


The Error UDR may be mapped from the workflow Preferences window as well, but for this caseAPL code must be used. This since desired to insert other values than MIM values in the error UDRfields. Also, an Error Case will be assigned, not only an Error Code, which is the only possibility fromthe window. Refer to Section 2.2, “Error Codes” and Section 2.3, “Reprocessing Groups” for additionalinformation.

Example 7.

E.myErrorUDR eUDR = udrCreate( E.myErrorUDR );eUDR.FileSize = (long)mimGet( "IN", "Source File Size" );eUDR.TS = (date)mimGet( "IN", "File Modified Timestamp" );eUDR.message = "PROCESSED ONCE.";udrAddError( eUDR, "switch_ERROR", "Switch not found." );cancelBatch( "Incorrect source.", eUDR );

Note, to send error UDRs with the batch is optional. However, it is necessary if access to any applicationspecific information is wanted when reprocessing the batch. Error UDR fields will appear as MIMvalues in the reprocessing workflow. Also, the only possibility to associate an Error Code with thebatch is by appending an Error UDR.

7.2. ECS Collection WorkflowThe collection workflow needs a Decoder node, since batches are saved in their original format whensent to ECS.

26


Figure 22. A workflow collecting batches from ECS.


The Error tab may be configured to handle Cancel Batch behavior, however, it will never be validfor ECS batch collection workflows. Any call to cancelBatch will cause the workflow to abortimmediately upon activation.

7.2.2. ECS Collection Node

All batches conforming to the collection criteria will be selected. If a batch contains historic UDRs,that is UDRs belonging to old, not used format definitions, they will by default be converted automat-ically to the latest format. If this behavior is not desired, the automatic conversion may be disabledfrom the Ultra Format Converter (the Tools menu on the desktop). In this case the workflow willabort, logging an informative message in the System Log.


Calls to cancelBatch cannot be made; the workflow will abort immediately upon activation.

27


FTP Agents 4.1


Version 1.29

1





2

FTP Agents 4.1



2. FTP Collection Agent .................................................................................................. 52.1. Collection Strategy ........................................................................................... 52.2. Configuration .................................................................................................. 6

2.2.1. Connection Tab ..................................................................................... 62.2.2. Source Tab ........................................................................................... 72.2.3. Advanced Tab ....................................................................................... 9




2.6. Node Message Events ...................................................................................... 113. FTP Forwarding Agent ............................................................................................... 12

3.1. Configuration ................................................................................................ 123.1.1. Target Tab ........................................................................................... 123.1.2. Advanced Tab ...................................................................................... 14





3

FTP Agents 4.1

1. IntroductionThis document describes the FTP Collection and FTP Forwarding agents, being parts of the standardagents available on the Nokia Convergent Mediation Engine™ Platform.



• Standard FTP (RFC 959, http://www.ietf.org/rfc/rfc0959.txt)


4

FTP Agents 4.1

2. FTP Collection AgentThe FTP Collection agent collects files from a remote file system and inserts them into a ConvergentMediation Engine™ workflow, using the standard FTP (RFC 959) protocol. The agent supports FTPtowards Unix, Windows NT and VAX/VMS machines.

When activated, the collector establishes an FTP session towards the remote host. On failure, additionalhosts are tried if configured. On success, the source directory on the remote host is scanned for all filesmatching the current filter. In addition, the Filename Sequence service may be utilized to furthercontrol the matching files. All files found will be fed one after the other into the workflow.

The agent also offers the possibility of decompressing compressed (gzip) files after they have beencollected, before they are inserted into the workflow. When all the files are successfully processed,the agent stops to await the next activation, scheduled or manually initiated.

2.1. Collection Strategy

In most cases, the default strategy should be used, and no custom Collection Strategy should bedefined.

Select Collection Strategy... from the Configuration menu on the desktop.

Figure 1. Collection Strategies configuration window.

Select Add to enable one or several from the available Strategies, then save.

Figure 2. Collection Strategies configuration window, Add.

There are two Strategies available by default. There may exist other customer adapted Strategies,however, these are not covered by this user guide. The default strategies are:

• Default Source File Handling

This Strategy represents the Source File Handling section in the Source tab where the collectiontreatment of the file is chosen. Please refer to Section 2.2.2, “Source Tab” for details.

• Default File Information

5

FTP Agents 4.1

This Strategy represents the File Information section in the Source tab and where collection dir-ectory path and file name is specified using a single regular expression. Please refer to Section 2.2.2,“Source Tab” for details.

2.2. ConfigurationThe FTP Collection agent configuration window is displayed when the node in a workflow is double-clicked or right-clicked, selecting Configuration.... Part of the configuration may be done in the File-name Sequence or Filename Sort Order service tab described in the Convergent Mediation Engine™Core - User Guide.

2.2.1. Connection Tab

Figure 3. FTP Collection agent configuration window, Connection tab.

Primary host name or IP address of the remote host to be connected. If a connectioncannot be established to this host, the additional hosts specified in the Advancedtab, are tried.

Host

User Name for an account on the remote host, enabling the FTP session to login.User Name

Password related to the User Name.Password

Data transfer type to be used during file retrieval.Transfer Type

• Binary - agent uses binary transfer type. Default setting.

• ASCII - agent uses ASCII transfer type.

Type of file system on the remote host.File System Type

• Unix - remote host using Unix file system. Default setting.

• Windows NT - remote host using Windows NT file system.

• VAX/VMS - remote host using VAX/VMS file system.

6

FTP Agents 4.1

To be considered when selected VAX/VMS for File System Type. If a file is renamed aftercollection on a VAX/VMS system, the filename might become too long. In that case the followingrules will apply:

A VAX/VMS filename consists of <file name>.<extension>;<version>, where the maximumnumber of characters for each part is:

• <file name>: 39 characters

• <extension>: 39 characters

• <version>: 5 characters

If the new filename turns out to be longer than 39 characters, the agent will move part of thefilename to the extension part. If the total sum of the filename and extension part exceeds 78characters, the last characters are truncated from the extension.

An example:

A_VERY_LONG_FILENAME_WITH_MORE_THAN_39_CHARACTERS.DAT;5

will be converted to:

A_VERY_LONG_FILENAME_WITH_MORE_THAN_39_.CHARACTERSDAT;5

2.2.2. Source Tab

Figure 4. FTP Collection agent configuration window, Source tab.

The Source tab contains configurations related to the remote host, source directories and source files.The configuration available can be modified through the choice of a Collection Strategy. The belowtext describes the configuration options available when no custom strategy has been chosen.

7

FTP Agents 4.1

If a custom collection strategy is needed, select this button to choose from predefinedCollection Strategies. The Collection Strategy chosen controls which further configurationis available in the Source tab. If no strategy is chosen, the default strategy is used.

CollectionStrategy

The below describes the configuration available when using the default strategy.

For further information on setting up a Collection Strategy, please refer to Section 2.1,“Collection Strategy”.

Absolute pathname of the source directory on the remote host, where the source filesreside. The pathname might also be given relative to the home directory of the User Nameaccount.

Directory

Name of the source files on the remote host.Filename

Regular expressions according to Java syntax applies. For further information, please referto


Example 1.

To match all file names beginning with TTFILE, type: TTFILE.*

Compression type of the source files. Determines if the agent will decompress the filesbefore passing them on in the workflow.

Compres-sion

• No Compression - agent does not decompress the files.


If enabled, the source files will be moved to the automatically created subdirectoryDR_TMP_DIR in the source directory, before collection. This option supports safe collec-tion when source files reuse the same name.


If enabled, when the configured number of hours have passed without any file beingavailable for collection, a warning message (event) will appear in the System Log andEvent Area:

InactiveSourceWarning(hours)

The source has been idle for more than <n> hours, the last inserted file is <file>.

If enabled, the source files will be moved from the source directory (or from the directoryDR_TMP_DIR if using Move Before Collecting), to the directory specified in the Direct-ory field, after collection.

Move to

Note that the Directory has to be located in the same file system as the collectedfiles at the remote host. Also, absolute pathnames must be defined (relative path-names cannot be used).

If enabled, the source files will be renamed after the collection, and remain (or movedback from the directory DR_TMP_DIR if using Move Before Collecting) in the sourcedirectory from which they were collected.

Rename

If enabled, the source files will be removed from the source directory (or from the directoryDR_TMP_DIR, if using Move Before Collecting), after the collection.

Remove

8

FTP Agents 4.1

If enabled, the source files will remain in the source directory after the collection. Thisfield is not available if Move Before Collecting is enabled.

Ignore

Full pathname of the directory on the remote host into which the source files will be movedafter the collection. This field is only available if Move to is enabled.

Directory

Prefix and/or suffix that will be appended to the beginning and the end of the name of thesource files, respectively, after the collection. These fields are only available if Move toor Rename is enabled.

Prefix andSuffix

If Rename is enabled, the source files will be renamed in the current (source orDR_TMP_DIR) directory. Be sure not to assign a Prefix or Suffix, giving files newnames still matching the Filename regular expression. That will cause the files tobe collected over and over again.

Number of days to keep moved or renamed source files on the remote host after the col-lection. In order to delete the source files, the workflow has to be executed (scheduled ormanually) again, after the configured number of days.

Keep(days)

Note, a date tag is added to the filename, determining when the file may be removed. Thisfield is only available if Move to or Rename is enabled.

2.2.3. Advanced Tab

Figure 5. FTP Collection agent configuration window, Advanced tab.

The Advanced tab contains configurations related to more specific use of the FTP service, whichmight not be frequently utilized.

The value in this field defines which port number the FTP service will use onthe remote host.

Command Port

The value in this field defines the maximum time, in seconds, to wait for repliesfrom commands. 0 (zero) means to wait forever.

Timeout(sec)

Must be enabled if FTP passive mode is used for data connection.Passive Mode(PASV)

9

FTP Agents 4.1

In passive mode, the channel for data transfer between client and server isinitiated by the client instead of by the server. This is useful when a firewallis situated between the client and the server.

List of additional host names or IP addresses that may be used to access thesource directory, from which the source files are collected. These hosts are

Additional Hosts

tried, in sequence from top to bottom, if the agent fails to connect to the remotehost, set in the Source tab.

Use the Add, Edit, Remove, Move up and Move down buttons to configurethe host list.


2.3.1. Emits

Emitted right before the first byte of each collected file is fed into a workflow.Begin Batch

Emitted just after the last byte of each collected file has been fed into the system.End Batch

2.3.2. Retrieves



APL code where Hint End Batch is followed by a Cancel Batch will alwaysresult in workflow abort. Make sure to design the APL code to first evaluatethe Cancel Batch criteria to avoid this sort of behavior.

If a Hint End Batch message is received, the collector splits the batch at the end of thecurrent block processed (32 kB), provided that no UDR is split. If the block end occurswithin a UDR, the batch will be split at the end of the preceding UDR.

Hint EndBatch

After a batch split, the collector emits an End Batch Message, followed by a BeginBatch message (provided that there is data in the subsequent block).



2.5.1. Publishes

A parameter containing a time stamp, indicating when the file transfer starts.File RetrievalTimestamp

10

FTP Agents 4.1



Source Filename


A parameter containing the number of files that were available to this instancefor collection at startup. The value is static throughout the execution of the

Source File Count

workflow, even if more files arrive during the execution. The new files willnot be collected until the next execution.


A parameter containing the name of the host from which files are collected,as defined in the Source or Advanced tabs.

Source Host


A parameter containing the path name, as defined in the Source tab.Source Pathname


A parameter containing the login user name, as defined in the Source tab.Source Username


2.5.2. Accesses




Ready with file: filename

Reported, along with the name of the source file, when the file is collected and inserted into theworkflow.

File cancelled: filename

Reported, along with the name of the current file, when a Cancel Batch message is received. This as-sumes the workflow is not aborted; please refer to Section 2.3, “Transaction Behavior” for further in-formation.

Command trace

A printout of the control channel trace. This is only valid if FTP command trace in the Advancedtab is enabled.

11

FTP Agents 4.1

3. FTP Forwarding AgentThe FTP Forwarding agent distributes files to a remote host, using the standard FTP (RFC 959) protocol.Files are created when a Begin Batch message is received and closed when an End Batch message isreceived. In addition, the agent offers the possibility to compress (gzip) the files.

To ensure that downstream systems will not use the files until they are closed, they are maintained ina temporary directory on the remote host until the End Batch message is received. This behavior isalso used for Cancel Batch messages. If a Cancel Batch is received, file creation is cancelled.

3.1. ConfigurationThe FTP Forwarding agent configuration window is displayed when the node is double-clicked orright-clicked, selecting Configuration...

3.1.1.Target Tab

Figure 6. FTP Forwarding agent configuration window, Target tab.

Primary host name or IP address of the remote host to be connected. If a connectioncannot be established to this host, the additional hosts specified in the Advancedtab, are tried.

Host

User name for an account on the remote host, enabling the FTP session to login.User Name

Password related to the User Name.Password

Specifies the data transfer type to be used during file storage.Transfer Type

• Binary - Agent uses binary transfer type. Default setting.

• ASCII - Agent uses ASCII transfer type.

Type of file system on the remote host.File System Type

• Unix - Remote host using Unix file system. Default setting.

• Windows NT - Remote host using Windows NT file system.

12

FTP Agents 4.1

• VAX/VMS - Remote host using VAX/VMS file system.

To be considered when selected File system type is VAX/VMS. If a file is renamed after col-lection on a VAX/VMS system, the filename might become too long. In that case the followingrules will apply:

A VAX/VMS filename consists of <file name>.<extension>;<version>, where the maximumnumber of characters for each part is:

• <file name>: 39 characters

• <extension>: 39 characters

• <version>: 5 characters

If the new filename turns out to be longer than 39 characters, the agent will move part of thefilename to the extension part. If the total sum of the filename and extension part exceeds 78characters, the last characters are truncated from the extension.

An example:

A_VERY_LONG_FILENAME_WITH_MORE_THAN_39_CHARACTERS.DAT;5

will be converted to:

A_VERY_LONG_FILENAME_WITH_MORE_THAN_39_.CHARACTERSDAT;5

Absolute pathname of the target directory on the remote host, where the forwardedfiles will be placed. The pathname may also be given relative to the home directoryof the user's account.

Directory

The files will be temporarily stored in the automatically created subdirectoryDR_TMP_DIR in the target directory. When an End Batch message is received,the files are moved from the subdirectory to the target directory.


Compression

• No Compression - Agent does not compress the files.

• Gzip - agent compresses the files using gzip.

Note that no extra extension will be appended to the target filenames, evenif compression is selected.

If enabled, files are created although containing no data.Produce EmptyFiles

13

FTP Agents 4.1

3.1.2. Advanced Tab

Figure 7. FTP Forwarding agent configuration window, Advanced tab.

The value in this field defines which port number the FTP service will use onthe remote host.

Command Port

The value in this field defines the maximum time, in seconds, to wait for repliesfrom commands. 0 (zero) means to wait forever.

Timeout(sec)

Must be enabled if FTP passive mode is used for data connection.Passive Mode(PASV)

In passive mode, the channel for data transfer between client and server isinitiated by the client instead of by the server. This is useful when a firewallis situated between the client and the server.

List of additional host names or IP addresses that may be used to access thetarget directory for file storage. These hosts are tried, in sequence from top tobottom, if the agent fails to connect to the remote host set in the Target tab.

Additional Hosts


The names of the created files are determined by the settings in the Filename Template tab.The use and setting of private threads for an agent, enabling multi-threading within a workflow,is configured in the Thread Buffer tab. Please refer to Convergent Mediation Engine™ Core- User Guide, for further details.


3.2.1. Emits

None.

14

FTP Agents 4.1

3.2.2. Retrieves


Begin Batch

When an End Batch message is received, the target file in DR_TMP_DIR is closed and,finally, the file is moved from the temporary directory to the target directory.

End Batch


Cancel Batch



3.4.1. Publishes




A parameter containing the target filename, as defined in Filename Tem-plate.

Target Filename


A parameter containing the name of the target host, as defined in the Targetor Advanced tab of the node.

Target Hostname


A parameter containing the path to the target file, as defined in the FTP tabof the node.

Target Pathname


A parameter containing the login name of the user connecting to the remotehost, as defined in the FTP tab of the node.

Target Username


3.4.2. Accesses



15

FTP Agents 4.1


Ready with file: filename

Reported, along with the name of the target file, when the file is successfully written to the target dir-ectory.

Command trace

A printout of the control channel trace. This is only valid if FTP Command Trace in the Advancedtab is enabled.

16

FTP Agents 4.1

GTP' Agent 4.1


Version 1.7

1





2

GTP' Agent 4.1



2. GTP' Agent ................................................................................................................ 52.1. Configuration .................................................................................................. 6

2.1.1. Source Tab ........................................................................................... 62.1.2. Miscellaneous Tab ................................................................................. 72.1.3. Advanced Tab ....................................................................................... 82.1.4. Decoder Tab ......................................................................................... 9


2.3. Introspection .................................................................................................. 102.4. Meta Information Model .................................................................................. 102.5. Node Message Events ...................................................................................... 10

3. Limitations - GTP' Transported Over TCP ..................................................................... 11

3

GTP' Agent 4.1

1. IntroductionThis document describes the GTP' agent being part of the extension real-time agents, available on theNokia Convergent Mediation Engine™ platform.



• The GPRS Tunneling Protocol (GTP) across the Gn and Gp Interface [3GPP TS 29.060 V4.2.0]:http://www.3gpp.org/ftp/Specs/archive/29_series/29.060/29060-420.zip

• The call and event data for the Packet Switched (PS) domain [3GPP TS 32.015 V3.11.0]: ht-tp://www.3gpp.org/ftp/Specs/archive/32_series/32.015/32015-3b0.zip


4

GTP' Agent 4.1

2. GTP' AgentThe GTP' Agent collects messages and datagrams of charging protocol type GTP' from GSM nodes.This enables Convergent Mediation Engine™ to act as a Charging Gateway device, providing ChargingGateway Functionality within UMTS/GPRS networks.

Figure 1. Convergent Mediation Engine™ is able to provide UMTS/GPRS Charging GatewayFunctionality.

The GTP' agent awaits initialization from the GSN nodes of the types SGSN and GGSN. There aretwo available protocols; Transmission Control Protocol, TCP, and User Datagram Protocol, UDP. AGTP' Agent can only handle one protocol type at a time. Hence, there has to be at least two GTP' nodesin order to collect both protocol types. Note that they can both be part of the same workflow.

Also, the GTP' Agent offers the possibility of notifying the GSN nodes to route the incoming data toanother host, in case of failure. A possible alternative configuration is to set up an identical workflowexecuting on another Convergent Mediation Engine™ Execution Context.

The following scheme shows the message and data transfer between the GSN nodes and the GTP'agent when using UDP:

1. When started, the GTP' agent sends a Node Alive Request message to all configured GSNnodes.

2. The GTP' agent awaits a Node Alive Response.

3. If the Max Wait for a Response (sec), set in the Advanced tab is exceeded, the GTP' agent sendsthe same message again. This is repeated the number of times configured in the Max Numberof Request Attempts. A warning message is displayed in the System Log when the collectorruns out of attempts.

4. The GTP' agent receives a Node Alive Response from a GSN node, which then sends datato the GTP' agent. When safely collected, the transfer is acknowledged to the GSN node.

When the workflow is stopped, the message Redirection Request is automatically sent to allconfigured GSN nodes. The workflow will not stop immediately, but waits for a RedirectionResponse from each of the GSN nodes. If the configured Max Wait for a Response (sec) is exceeded,the workflow stops, regardless if Redirection Response from the GSN nodes has been received.

If using TCP, the behavior is different. Please refer to Section 3, “Limitations - GTP' TransportedOver TCP ” for further information.

5

GTP' Agent 4.1

2.1. ConfigurationThe GTP' agent configuration window is displayed when the node in a workflow is double-clicked,or right-clicked, selecting Configuration....

2.1.1. Source Tab

Figure 2. GTP' agent configuration window, Source tab.

The Source tab contains configurations related to the type of connection to be used by the agent.

Type of protocol to use; TCP (Transmission Control Protocol), or UDP (User Da-tagram Protocol, default). Please refer to Section 3, “Limitations - GTP' TransportedOver TCP ” for further information on TCP.

Protocol

The port on which to listen for incoming packages. This port must reside on thehost on which the used Execution Context is running. The default value is 3386.

Two workflows (within the same Execution Context) can subscribe on thesame Port and GSNs, if they use different Protocol settings.

Port

The IPv4 addresses of the GSN nodes providing the data.GSN IP Address

The server port number for each node.Server Port

6

GTP' Agent 4.1

2.1.2. Miscellaneous Tab

Figure 3. GTP' agent configuration window, Miscellaneous tab.

The Miscellaneous tab contains configurations related to format and storage of the data to be collectedby the agent.

GTP' formats containing fields of type IPv6 are not supported.

Must match the data record format in Data Record Packet IE whichis present conditionally if the Packet Transfer Command is Send Data RecordPacket or Send possibly duplicated Data Record Packet.

Format

Must match the data record format version in Data Record PacketIE which is present conditionally if the Packet Transfer Command is Send DataRecord Packet or Send possibly duplicated Data Record Packet.

Format Ver-sion

Absolute pathname of the target directory in the file system on the local host, wherethe intermediate data for the collection is stored.

Directory

The intermediate data includes a temporary file with the collected data that is automat-ically removed when committed, and files keeping track of sequence numbers andrestart values. Also, a directory, duplicates, is created in which possible duplicatesare saved. Note that this directory must be attended to manually.

The pathname might also be given relative to the home directory of the user account.

In case of using several Execution Contexts, make sure the file system containingthe GTP' information is mounted on all ECs.

7

GTP' Agent 4.1

2.1.3. Advanced Tab

Figure 4. GTP' agent configuration window, Advanced tab.

The maximum time the GTP' agent awaits a Node Alive Response.Max Wait for a Re-sponse (sec)

When this value in combination with Max Number of Request Attempts isexceeded, a message is entered in the System Log. For instance, if this valueis 20 and the latter value is 5, the warning message will be logged after 100seconds.

The value also indicates the maximum time the GTP' agent awaits a Redir-ection Response after it has sent a Redirection Request to thenodes it is configured to receive from. In this case, no retries are made, though;Max Number of Request Attempts is ignored. When the workflow is stopped,it will take this time before it actually goes down.

The maximum number of attempts to poll for a Node Alive Response.Max Number of Re-quest Attempts

This number indicates how many packages to keep in memory for sequencenumber checking.

Max OutstandingNumbers

Defines the maximum allowed gap between 2 sequence numbers.Max Drift BetweenTwo Numbers

When disabled, the agent will save the sequence number of the last collectedpackage when the workflow is stopped. When restarted, a package with thesubsequent number is expected.

Clear Checking

When enabled, the agent will ignore its expected sequence number when theworkflow is restarted after stop. The sequence number of the first incomingand collected package is then used to indicate the expected number of thesubsequent packet.

When enabled, possible duplicates are stored in the directory for persistentdata, given in the Miscellaneous tab. The packet remains there until a requestis received to release or cancel the packet.

Agent Handles Du-plicates

When selected, possible duplicates can be routed to the desired link.Route Duplicates to

The hostname or IP-address of a host running an alternate Charging Gatewaydevice, as a backup.

Alternate Node

8

GTP' Agent 4.1

In case of failure, Convergent Mediation Engine™ can notify the GSN nodesto route data to this alternate host, which could be another Convergent Medi-ation Engine™ Execution Context executing an identical workflow, for in-stance.

Handling the backup is beyond the scope of the GTP' agent. This mustbe handled from the GSN node. The GTP' agent is only responsible forsignaling a Redirection Request message to the GSN node whengoing down. If configuring primary and secondary workflows, makesure to configure the Clear Checking or a relevant Max Drift BetweenTwo Numbers. This since the primary collector will send a messageto the GSN node when it is up and running again. A standard GSN nodewill then redirect the packages back to the primary node.

2.1.4. Decoder Tab

Figure 5. GTP' agent configuration window, Decoder tab.

List holding defined, available decoders introduced via the Ultra Format Editor. Thedecoders are named according to the following syntax:

Decoder

<decoder> (<module>)

The option MZ Format Tagged UDRs indicates the expected UDRs are stored in theConvergent Mediation Engine™ specific format. If this option is selected the TaggedUDR Type list will be enabled.

List of available internal UDR formats stored in the Ultra and Code servers. The syntax:Tagged UDRType <internal> (<module>)

If the decoder is to reprocess UDRs of an internal format, the Decoder MZ formattagged UDRs has to be selected enabling this list. Once enabled, the internal formatmust be selected.

Full Decode • If selected, the UDR will be fully decoded before inserted into the workflow.This may have a negative impact on performance, since not all fields may be ac-cessed in the workflow, making decoding all fields in the UDR unnecessary.

• If this option is not selected (default), the amount of work needed for decoding isminimized, using a lazy method decoding sub fields.

9

GTP' Agent 4.1

This means that the actual decoding work may not be done until later in theworkflow, when the field values are accessed for the first time. Corrupt data (thatis, data for which decoding fails) may not be detected during the decoding stage,but may instead cause workflow abort at a later processing stage.


2.2.1. Emits

None.

2.2.2. Retrieves

None.





10

GTP' Agent 4.1

3. Limitations - GTP' Transported Over TCPNode Alive Request and Redirection Request are not transmitted to GSN nodes if usingthe TCP protocol.

11

GTP' Agent 4.1

HTTPD Agent 4.1


Version 1.4

1





2

HTTPD Agent 4.1



2. HTTPD Agent ............................................................................................................ 52.1. Configuration .................................................................................................. 5

2.1.1. HTTPD Tab .......................................................................................... 52.2. The HTTPD Format .......................................................................................... 62.3. Introspection ................................................................................................... 62.4. Meta Information Model .................................................................................... 62.5. Node Message Events ....................................................................................... 6

3. APL Functions ........................................................................................................... 73.1. HTTP Client Functions ..................................................................................... 7

3.1.1. httpGetURL ...................................................................................... 73.1.2. httpGetSecureURL .......................................................................... 73.1.3. httpPostURL .................................................................................... 83.1.4. httpPostSecureURL ........................................................................ 8

3.2. HTTP Helper Functions ..................................................................................... 93.2.1. httpGetLastErrorCode .................................................................. 93.2.2. httpGetLastErrorMessage ............................................................ 9

4. An Example ............................................................................................................. 104.1. The Format Definition ..................................................................................... 114.2. The Analysis Nodes ........................................................................................ 11

4.2.1. Query ................................................................................................ 114.2.2. Welcome ............................................................................................ 124.2.3. Login ................................................................................................. 124.2.4. Success .............................................................................................. 134.2.5. Failure ............................................................................................... 13

3

HTTPD Agent 4.1

1. IntroductionThis document describes the HTTPD agent being part of the extension agents, available on the NokiaConvergent Mediation Engine™ platform.

It also includes APL functions for connecting as a client to an external HTTP server.



• Hypertext Transfer Protocol version 1.1

(RFC 2616: http://www.ietf.org/rfc/rfc2616.txt)

1.2. Abbreviations and TermsAbbreviations and terms used in the Convergent Mediation Engine™ user documents, are describedin the Convergent Mediation Engine™ Core - User Guide.

4

HTTPD Agent 4.1

2. HTTPD AgentThe HTTPD agent (in combination with Analysis or Aggregation nodes) can act as a web server, re-ceiving requests and sending responses on a TCP/IP connection. The requests are turned into UDRs,using the standard Hypertext Transfer Protocol, and inserted into a Convergent Mediation Engine™workflow.

When a workflow acting as a web server is started, the HTTPD agent opens a port for listening andawaits a request. When the response has been sent, the socket is closed. The workflow remains activeuntil manually stopped. In addition, the agent offers the possibility to use an encrypted communicationsetup through SSL.

2.1. ConfigurationThe HTTPD agent configuration window is displayed when the node in a workflow is double-clickedor right-clicked selecting Configuration...

2.1.1. HTTPD Tab

Figure 1. HTTPD Collection Agent configuration window, HTTPD tab.

The number of receivers used for the communication.Receivers

If enabled, the communication channels will be encrypted.Use SSL

A keystore file containing the server certificate to be used with SSL. Enterthe full path to a keystore file on the local or mounted disk on the executionhost or the keystore file, available in any bundle, through the code server.

Keystore

The last alternative is to prefer if the disks on the execution host normallycannot be reached for updates.

The password for the keystore file.Password

5

HTTPD Agent 4.1

The local address that the server will bind to. If the field is left empty, theserver will bind to the default address.

Local Address

The port the server will listen to. Default port for non-encrypted communicationis 80 and for encrypted 443.

Port

The UDR Type, extended HttpdUDR, the collector will emit. Please refer toSection 2.2, “The HTTPD Format” for an example.

Content Type

Number of requests in queue waiting to be accepted. If the queue is full, newconnection attempts will be refused. Default value is 1.

Backlog

Number of seconds a client can be idle while sending the request, before theconnection is closed. If the timeout is set to 0 (zero) no timeout will occur.This is not recommended. Default value is 10.

Client Timeout (sec)

The number of seconds before the server closes a request and a 500 ServerError is sent back to the client. If the timeout is set to 0 (zero) no timeout willoccur. Default value is 0.

Server Timeout (sec)

List of encoding options to use for handling of responses.Character Encoding

2.2.The HTTPD FormatThe built-in HTTP format definition must be instantiated prior to usage of the HTTPD format. Addi-tional fields may be entered. This is useful mainly for transportation of variable values to subsequentnodes.

Extending the HTTP format:

internal MYHTTPD: extends_class ("com.digitalroute.wfc.http.HttpdUDR"){ // Additional fields (if required).};

2.3. IntrospectionThe agent consumes and produces UDR types extended with the built-in HTTP format.


The agent does not itself publish or access any MIM resources. For a listing of general ConvergentMediation Engine™ MIM resources, refer to Convergent Mediation Engine™ Core - UserGuide.


6

HTTPD Agent 4.1

3. APL Functions

3.1. HTTP Client FunctionsThe client functions are used to connect to an external HTTP server and to retrieve documents fromit. There is support for GET and POST methods, and both plain text and encrypted communication.Only basic authentication is supported.

3.1.1. httpGetURL

Retrieves a document from an HTTP server.

string httpGetURL ( string query, string host, int timeout, int port, //Optional string username, //Optional string password ) //OPtional

Parameters:

The requested document and any arguments.query

The name or IP address where the HTTP server is running.host

The number of seconds to wait for a response. If the value is set to 0 (zero), the defaulttimeout will be used. Default value is 15 seconds.

timeout

The port number to contact the HTTP server on. The default port, 80, will be used if notspecified.

port

The account in a restricted area of the HTTP server.username

The password to the username. Must be present if username is present.password

A document from the HTTP server. It will be null if any part of the communicationfails.

Returns

3.1.2. httpGetSecureURL

Retrieves a document from an HTTP server. The communication channel will be encrypted.

string httpGetSecureURL ( string query, string host, int timeout, int port, //Optional int port, //Optional string password ) //OPtional

Parameters:


7

HTTPD Agent 4.1


The number of seconds to wait for a response. If the value is set to 0 (zero), the defaulttimeout will be used. Default value is 15 seconds.

timeout

The port number to contact the HTTP server on. The default port, 443, will be used ifnot specified.

port




Returns

3.1.3. httpPostURL

Retrieves a document from an HTTP server. The request will also contain a document.

string httpPostURL ( string query, string contentType, string content, string host, int timeout, int port, //Optional int username, //Optional string password ) //OPtional

Parameters:


The MIME type of the attached document.contentType

The attached document.content


The number of seconds to wait for a response. If the value is set to 0 (zero), the de-fault timeout will be used. Default value is 15 seconds.

timeout

The port number to contact the HTTP server on. The default port, 80, will be usedif not specified.

port




Returns

3.1.4. httpPostSecureURL

Retrieves a document from an HTTP server. The request will also contain a document. The commu-nication channel will be encrypted.

string httpPostSecureURL ( string query, string contentType, string content, string host, int timeout,

8

HTTPD Agent 4.1

int port, //Optional int username, //Optional string password ) //OPtional

Parameters:


The MIME type of the attached document.contentType

The attached document.content


The number of seconds to wait for a response. If the value is set to 0 (zero), the de-fault timeout will be used. Default value is 15 seconds.

timeout

The port number to contact the HTTP server on. The default port, 443, will be usedif not specified.

port




Returns

3.2. HTTP Helper Functions

3.2.1. httpGetLastErrorCode

Gets the last error code set.

int httpGetLastErrorCode()

The HTTP status code. It will be -1 if no error occurred.Returns

3.2.2. httpGetLastErrorMessage

Gets the last error message set.

string httpGetLastErrorMessage()

The HTTP status message. It will be null if no error occurred.Returns

9

HTTPD Agent 4.1

4. An ExampleTo give an example of how a workflow acting as an HTTP server may be designed, suppose there isan HTML login page requiring login name and password. The login parameters are checked againsta user database, opening a new HTML page if authentication succeeded. If not, a new login attemptis requested.

Figure 2. The login page and the resulting message page, in case of failed authentication.

An example of how such a workflow may be designed is shown in the following. Each of the Analysisnodes will perform a validation, resulting in either the HTTP UDR being sent back with an updatedresponse field to the collector, or it is sent unchanged to a subsequent node.

1. The Query node checks the correct page is requested. If the page does not exist, an error messageis sent back to the HTTPD node on the not_found route.

2. If the page exists, the HTTP UDR is routed on to the Welcome or Login nodes, depending onif the login page was addressed.

3. The Login node checks if the user exists in the database. If authentication fails, the HTTP UDRis routed to the Failure node which displays a new page stating the username/password is in-correct.

4. If authentication succeeds, a new page is opened.

Figure 3. A workflow acting as an HTTP Server.

10

HTTPD Agent 4.1

The data sent to the HTTP node in the example (the content field) is an HTML page. SinceUFDL cannot handle the HTML protocol, the field is defined as a string. If, for instance, XMLwas used instead (which may be handled with UFDL), this would require an Analysis node,which turned the field into a bytearray and then decoded it (use the APL functions strToBAand udrDecode).

4.1.The Format DefinitionCreate an instance of the built-in HTTP format definition. Additional fields may be entered. This isuseful mainly for transport of variable values to subsequent nodes. In this case, re-usage of the usernameas parsed from the login page is desired.

internal MYHTTPD: extends_class ("com.digitalroute.wfc.http.HttpdUDR"){ string username;};

4.2.The Analysis NodesThe Analysis nodes handle all validation and response handling.

4.2.1. Query

The query node checks if the addressed page exists. If it does not, an error message is inserted into theresponse field and the UDR is returned to the collector.

consume{ if (input.query == "/") {

// Show welcome page.

udrRoute(input, "index"); return; }

if (input.query == "/login") {

// Try to log in.

udrRoute(input, "form"); return; }

// The request is not found here.

input.response = "404 Not found."; input.responseType = "text/plain"; udrRoute(input, "not_found");

11

HTTPD Agent 4.1

}

4.2.2. Welcome

Populates the response field with an HTML page, and send the UDR back to the collector.

final string welcome = "<html > <title >Welcome</title > <body bgcolor=#3b7d73 > <font face=Verdana,Arial > <h2 >HTTPD Doc Example</h2 > Login<p ><form action=/login method=post > <table ><tr ><td align=right > Username</td ><td ><input name=username ></td ></tr > <tr ><td align=right >Password</td > <td ><input name=passwd type=password ></td ></tr > <tr ><td > </td > <td ><input name=Login type=submit ></td ></tr> </table></form ></body ></html >";

consume{ input.responseType="text/html"; input.response = welcome; udrRoute(input);}

4.2.3. Login

Verifies that the user is authenticated, by performing a lookup against a database table.

table tmp_tab;

initialize { tmp_tab = tableCreate("select user_Name, user_PW from users");}

consume{ if (input.requestMethod != "POST") { input.response = "Wrong method."; udrRoute(input, "No"); return; }

// Find username.

string p = input.content;

int i = strIndexOf(p, "username="); p = strSubstring(p, i + 9, strLength(p)); i = strIndexOf(p, "&"); string u = strSubstring(p, 0, i); p = strSubstring(p, i + 1, strLength(p));

12

HTTPD Agent 4.1

i = strIndexOf(p, "passwd="); p = strSubstring(p, i + 7, strLength(p)); i = strIndexOf(p, "&"); p = strSubstring(p, 0, i);

// Verify the login.

table rowFound_u = tableLookup(tmp_tab, "user_Name", "=", u); int hit_u = tableRowCount(rowFound_u);

if (hit_u == 1) { if (p == (string)tableGet(rowFound_u,0,1)) { input.username = (string)tableGet(rowFound_u,0,0); udrRoute( input, "Yes" ); } else { input.response = "Wrong password."; udrRoute(input, "No"); } } else { input.response = "Wrong username."; udrRoute(input, "No");; } }

4.2.4. Success

The response field is populated with an HTML page.

final string success1 = " <html > <title >Welcome </title > <body bgcolor=#3b7d73 > <font face=Verdana,Arial > <h2 >HTTPD Doc Example </h2 >Welcome ";

final string success2 = "! <p > <a href=/ >Back to start page </a > </body > </html >";

consume{ input.responseType = "text/html"; input.response = success1 + input.username + success2; udrRoute(input);}

4.2.5. Failure

The response field is populated with an HTML page.

final string failure1 = "<html><title>Welcome</title> <body bgcolor=#3b7d73><font face=Verdana,Arial> <h2>HTTPD Doc Example</h2>Failed to log in; ";

final string failure2 = "<p><a href=/>Back to start page</a> </body></html>";

consume

13

HTTPD Agent 4.1

{ input.responseType = "text/html"; input.response = failure1 + input.response + failure2; udrRoute(input);}

14

HTTPD Agent 4.1

Installation Instructions 4.1


Revision 1.58

1





2




2. Core Installation ......................................................................................................... 52.1. Preparation ..................................................................................................... 52.2. Generation of Oracle Database Definition Files ...................................................... 92.3. Oracle Database Creation ................................................................................... 92.4. Convergent Mediation Engine™ with Cloudscape Installation ................................. 102.5. Convergent Mediation Engine™ Software Setup .................................................. 11

3. Client Installation ...................................................................................................... 124. Installation of Execution Context only ........................................................................... 16

4.1. Preparation .................................................................................................... 164.1.1. Preparing your system ........................................................................... 164.1.2. Choosing an Execution Context type ........................................................ 164.1.3. Configuring a Standard Execution Context ................................................ 164.1.4. Configuring a Standalone Execution Context ............................................. 174.1.5. Configuring the Web Interface for an Execution Context .............................. 17

4.2. Convergent Mediation Engine™ Software Setup .................................................. 185. Removing and Adding Packages .................................................................................. 19

5.1. Listing Packages ............................................................................................ 195.2. Removing Packages ........................................................................................ 195.3. Adding or Updating Packages ........................................................................... 19

6. Java Configuration .................................................................................................... 216.1. Out of Memory .............................................................................................. 216.2. Kernel Parameters for HP/UX ........................................................................... 21

7. Communications Through Firewalls ............................................................................. 227.1. Convergent Mediation Engine™ Servers Located Behind a Firewall ......................... 227.2. Firewall Setup ................................................................................................ 23

8. System Requirements ................................................................................................ 248.1. Platform and Execution Context ........................................................................ 248.2. Desktop ........................................................................................................ 24

9. Oracle Configuration ................................................................................................. 25

3


1. IntroductionThis document describes the routines, needed to install the Nokia Convergent Mediation Engine™Core. If not stated differently elsewhere, this guide uses Oracle as the Convergent Mediation Engine™standard database server.



• Database structure

• UNIX


4


2. Core InstallationThis chapter covers how to install Convergent Mediation Engine™ with the Oracle database. For moredetails on what steps that are necessary to install Convergent Mediation Engine™ with the Cloudscapedatabase please see Section 2.4, “Convergent Mediation Engine™ with Cloudscape Installation”.

The core installation for Oracle consists of the following four steps:

1. Preparing where the software is copied and values are supplied for predefined parameters thatmay vary between installations.

2. Generation of database definition files to be used during the database creation.

3. Creation of the database instance.

4. Setup of the software for the Convergent Mediation Engine™ instance.

2.1. PreparationInitially, the UNIX user to execute the installation and later on be running the Convergent MediationEngine™ instance must be created, preferably mzadmin which is referenced throughout this document.

The next step is to install the necessary third party products which are Oracle and Java, please referto Section 8, “System Requirements” for further information. Then, set up the following environmentfor the user mzadmin. The example values will be used later on in the installation.

Example value: /opt/mzMZ_HOME

This environment variable specifies where the Convergent Mediation Engine™software is to be installed. The directory must exist and be empty prior to the install-ation.

Example value: /opt/jdk/jdk1.5.0_06JAVA_HOME

This environment variable specifies where the JDK is installed.

Example value: /opt/oracle/product/10.1.0ORACLE_HOME

This environment variable specifies where Oracle is installed.

Example value: MEDORACLE_SID

This environment variable specifies the Oracle SID to be used for this ConvergentMediation Engine™ instance.

The search path must contain the following directories:$JAVA_HOME/bin:$ORACLE_HOME/bin:$MZ_HOME/bin

PATH

This environment variable specifies the search path.

Continue by copying all files from the distribution CD into a temporary installation directory. Thenextract the installation parameters file, using the following command session. It is assumed that thetemporary installation directory is the working directory before the command session is executed.

$ cd ./CD $ java -jar core*.mzp extract

5


Extract additional packages, if any. The additional installation parameters are then merged into thealready created install.xml file. Example:

$java -jar webinterface*.mzp extract

Update the installation parameters in the file install.xml to suit the local installation. The followinglisting contains a description of all the installation parameters within the file, including example values.

Example value: platform,ec

This variable specifies which parts of Core to be installed.Possible values are any combination of platform, desktop,ec or ecsa.

install.types

Example value: dr

This variable specifies the password for the ConvergentMediation Engine™ administrative user mzadmin. The

install.admin.password

value must always be dr for a new installation, that iswhen a database instance is to be created. If this is anupdate of an existing installation and the password hasbeen changed for the mzadmin user, then this variablemust be assigned the new password.

Example value: /opt/mz

This variable specifies the target directory for the Con-vergent Mediation Engine™ software. The same value

mz.home

must be assigned to the environment variable MZ_HOME,previously described.

Example value: 192.168.0.190

This variable specifies the IP-address of the host wherethe Convergent Mediation Engine™ platform will berunning.

pico.rcp.codeserver.host

Example value: 6790

This variable specifies the corresponding port for theConvergent Mediation Engine™ platform, to be used bydesktops and Execution Contexts.

pico.rcp.codeserver.port

Example value: MED01

This variable specifies the name of the Convergent Me-diation Engine™ instance to be created.

mz.name

Example value: medadmin

This variable specifies the user name of the database,that the Convergent Mediation Engine™ platform will

mz.jdbc.user

be using during runtime. The user name must not beidentical to the user name specified in install.data-base.owner. Please refer to the note at the end of thesection.

Example value: xxx

This variable specifies the password for the jdbc user,previously described.

mz.jdbc.password

6


Example value: medadmin

This variable specifies the sending identity to be used forevent generated e-mails.

mz.notifier.mailfrom

Example value: SMTP.server

This variable specifies the mail server to be used for eventgenerated e-mails.

mz.mailserver

Example value: EC1, EC2

This list specifies the Execution Context(s) to to be in-stalled. Only applicable if install.types includesec.

mz.eclist

Example value: ECSA1

This list specifies the Standalone ECs to be installed.This kind of EC is for real-time workflows only. The

mz.ecsalist

main advantage of this type of EC is that it can continueto execute detached from the platform. Only applicableif install.types includes ecsa.

Example value: ${mz.home}/tmp

This variable specifies the directory where stand-aloneECs can store their message backlog.

ecsa.backlog.dir

Example value: true

This variable specifies if the web interface for ECs shouldbe active.

ec.httpd.enabled

Example value: 9090

This variable specifies the TCP port for the web interfaceof ECs.

ec.httpd.port

Example value: dr

This variable specifies the password to be used when thecorresponding web interface is accessed.

ec.httpd.password

Example value: dbowner

This variable specifies the Oracle user name for thedatabase owner. The user will own all the data definitions

install.database.owner

in the Oracle instance to be created. The user name mustnot be identical to the user name specified in mz.jd-bc.user. Please refer to the note at the end of the sec-tion.

Example value: xxx

This variable specifies the password for the databaseowner, previously described.

install.database.pass-word

Example value: host

This variable specifies IP-address or host name for thedatabase instance. This can only be specified when install-ation Convergent Mediation Engine™ with Oracle.

install.database.host

7


Example value: medium

This variable specifies the size of the Oracle databaseinstance. Possible values are small, medium or large.

install.database.size

• small is aimed for test and demo installations.

• medium is aimed for small and medium sized oper-ators.

• large is aimed for large operators.

Example value: 1521

This variable specifies the Oracle database port.

install.ora.port

Example value: /opt/oracle/product/10.1.0

This variable specifies the Oracle home directory. Thesame value must be assigned to the environment variableORACLE_HOME previously described.

install.ora.home

Example value: MED

This variable specifies the Oracle SID for the instanceto be created. The same value must be assigned to the

install.ora.sid

environment variable ORACLE_SID, previously de-scribed.

Example value: /opt/dbfiles

This variable specifies a directory path to be used inother Oracle directory definitions. The variable is used

install.ora.data

for convenience, when the same directory path is repeatedfor any of the following definitions.

Example value: ${install.ora.data}/tables

This variable specifies the directory where the Oracletablespace data files are to be created.

install.ora.data.tab

Example value: ${install.ora.data}/indexes

This variable specifies the directory where the Oracleindex data files are to be created.

install.ora.data.idx

Example value: ${install.ora.data}/temp

This variable specifies the directory where the Oracletemporary tablespace data files are to be created.

install.ora.data.temp

Example value: ${install.ora.data}/logfiles

This variable specifies the directory where the Oraclelog files are to be created.

install.ora.data.log

Example value: ${install.ora.data}/rollback

This variable specifies the directory where the Oraclerollback data files are to be created.

install.ora.data.roll

Example value: ${install.ora.data}/control

This variable specifies the directory where the Oraclecontrol data files are to be created.

install.ora.data.ctrl

8


Default value: 9000

This variable specifies the port for the platform web-in-terface. When installing a 2nd Convergent Mediation

mz.wi.port

Engine™ instance, this port value must be replaced inorder to not clash with the first instance.

This parameter is only available when the web-interface package is extracted.

If new directories were specified for the Oracle data, these directories must be created as the oracleuser.

The installation will fail if the same name is configured for both the database owner and theruntime (jdbc) user. If it is necessary to have the same name for both users, then theoracle_user.sql file must be manually updated prior to creation of the database.

2.2. Generation of Oracle Database Definition FilesThe generation of the data definition files for Oracle is simply to ask each package to create the corres-ponding data definition files. The following command will generate data definitions for all packagesin a subdirectory of the working directory. It is assumed that the working directory is not changedsince the previous command session.

$ ./create_db_def.sh

The database definition files are created in the subdirectory ./database/oracle. If this step isto be repeated, simply remove the ./database directory. The following command is used to createa tar file which contains all the database definition files.

$ tar cvf ./database.tar ./database

2.3. Oracle Database Creation

If using Oracle 9, two lines must be removed from the database creation fileoracle_create.sql. The lines reside after the ORACLE9 tag.

The database creation is performed as the oracle UNIX user, or as another UNIX user with the sameprivileges. The database may be installed on any machine within the network. The oracle UNIXuser is assumed in this installation.

Make sure that all previously defined directories, specified in the install.xml file, exist and arewritable for the oracle user. Log in as oracle and then untar the previously created tar file,named database.tar. The following command is then used to untar the file.

9


$ tar xvf ./database.tar

Make sure that a listener has been configured and started for the database instance. Then create theOracle instance by executing the following command session. It is assumed that the working directoryis not changed since the previous command session.

$ cd database/oracle $ ./oracle_create_instance.sh

When the instance creation script has been executed, there will be three log files in the /tmp directory.The mz.log file contains Oracle responses from the database creation, the mzuser.log file containsOracle responses from the user creation, and the mz_db.log file contains Oracle responses from thetable creations. The rest of this section can be ignored if no error occurred.

If the database creation fails, a cleanup is needed prior to the next try. That begins with trying to ter-minate the bad instance by executing the following commands.

$ export ORACLE_SID=MED

Start sqlplus as follows, and issue the command shutdown abort to make sure that the badinstance is stopped. The sqlplus program is terminated by issuing the quit command.

$ sqlplus "/ as sysdba"

The final step of the cleanup process, is to remove any files created by Oracle in any of the previouslydefined directories for Oracle data. If there is a need to change any installation parameter, defined inthe install.xml file, the installation must be restarted from the generation step, please refer toSection 2.2, “Generation of Oracle Database Definition Files”.

2.4. Convergent Mediation Engine™ with CloudscapeInstallation

Installing Convergent Mediation Engine™ with the Cloudscape database is a much easier task thaninstalling with Oracle. The installation follows the same steps as those described above but setting upthe database is managed by Convergent Mediation Engine™ itself. Since the database is running inthe same process as the platform, the platform will require more memory than when running towardsOracle. A result of having the database in the same instance is that the database does not need to bemonitored in a cluster environment.

To install, first of all make sure the preparation steps above are performed. See chapter Section 2.1,“Preparation” for details. All environment variables and properties in the install.xml should beedited in the same way except that all Oracle specific entities should be ignored.

Once you are done editing follow the instructions in Section 2.5, “Convergent Mediation Engine™Software Setup”.

The database files belonging to Cloudscape will be placed in a directory called mzdb under$MZ_HOME.

10


2.5. Convergent Mediation Engine™ Software SetupThe final tasks of the installation must be executed as the mzadmin UNIX user. Log in as mzadminand set the working directory to the temporary installation directory, as previously described. Thefollowing command session will setup the directory structure and install all the software.

$ cd ./CD $ java -jar core*.mzp install

The following command will install all other packages, except core.

$ ./install_system.sh

Restart everything to complete the installation.

$ mzsh shutdown platform $ mzsh startup platform ec1

11


3. Client InstallationThe client installation will install the Convergent Mediation Engine™ desktop and the commandlinetool mzsh.

The first step is to install the necessary third party product, which is Java. Please refer to Section 8,“System Requirements”.

If the required Java version is not available, the following workaround is suggested:

• start a command shell

• change directory to where the desktop installer is located

• invoke the 5.0 java binary:

<path to java-bin>\java -jar desktop-installer-standard_*.jar

The next step is to start the desktop installation by double-clicking the filedesktop-installer-standard_*.jar on the distribution CD on the target PC or runningthe file through

java -jar desktop-installer-standard_*.jar

The file may be copied to the PC before starting it, if preferred.

Figure 1. Introduction dialog.

The first dialog briefly welcomes to the installation of Convergent Mediation Engine™ . Select Nextto continue with the desktop installation or select Quit to cancel the desktop installation.

During the desktop installation, most of the needed values will have suitable defaults, and if these areacceptable the Next button may be pressed to move to the next dialog. Pressing Next without selectinganything in a dialog will automatically select the first entry (in a list or of a set of radio buttons).

12


Figure 2. Select installation path.

The second dialog is for specifying the installation directory. If you want to have a different directorythan default, it has to exist prior to this selection.

Figure 3. Platform dialog.

The next step is to enter the host and port for the platform. The host can be entered either as an IP-ad-dress or as a host name. The port number must be the same as the platform was configured for duringinstallation. After this step, the installation starts and it only takes a few seconds to install the necessarycomponents.

13


Figure 4. Installation progress dialog.

Now, the final task is to choose where the shortcut(s) should reside. Shortcuts are available to mostplatforms while the application menu is available on Windows.

Figure 5. Installation shortcut dialog available on Windows.

When the installation is finished, the following dialog will appear. Clicking the Done button will ter-minate the installation process.

14


Figure 6. Installation Complete dialog.

15


4. Installation of Execution Context onlyThe installation process consists of two steps. It starts with preparation, where the software is copiedand values are supplied for predefined parameters that may vary between installations. The final stepis to setup the software for the Execution Context.

4.1. Preparation

4.1.1. Preparing your system

Initially, create the Unix user to execute the installation and later on be running the execution context,preferably mzadmin which is referenced throughout this document. The next step is to install thenecessary third party product which is Java, please refer to Section 8, “System Requirements”.

Set up the following environment for the mzadmin user. The example values will be used later on inthe installation parameters file install.xml.

Example value: /opt/mzMZ_HOME

This environment variable specifies where the Convergent Mediation Engine™ softwareis installed. The directory must exist and be empty before the installation can start.

Example value: /opt/jdk/jdk1.5.0_06JAVA_HOME

This environment variable specifies where the JDK is installed.

The search path must contain the following directories:$JAVA_HOME/bin:$MZ_HOME/bin

PATH

This environment variable specifies the search path.

Continue by copying the core_4.1.*.mzp and mz.license files from the distribution CD tothe target machine. Then extract the installation parameters file, using the following command session.

$ java -jar core*.mzp extract

4.1.2. Choosing an Execution Context type

Two different types of Execution Contexts are available in Convergent Mediation Engine™ ; Standardand Standalone. A Standalone Execution Context is an execution context that can operate uninterruptedeven when the Control Zone is unavailable. To to able to guarantee this, workflows running on aStandalone EC cannot depend on any external resources. As a result, only real-time workflows, andonly agent profiles configured to run locally on the execution context, can be used in workflows ex-ecuting on Standalone ECs.

Only real-time workflows, validated as being standalone, can execute on a Standalone ExecutionContext.

4.1.3. Configuring a Standard Execution Context

Update the installation parameters in the install.xml file to suit the local installation. The followingtable contains a description of the installation parameters that will influence an Execution Context in-stallation, including with example values.

16


Example value: ec

This variable specifies that we are about to install anExecution Context only.

install.types

Example value: dr

This variable must be assigned the password for the mz-admin as defined in the Convergent Mediation Engine™platform.

install.admin.password

Example value: /opt/mz

This variable specifies the target directory for the Con-vergent Mediation Engine™ software. The same value

mz.home

must be assigned to the environment variable MZ_HOMEpreviously described.

Example value: 192.168.0.190

This variable specifies the IP-address of the host wherethe Convergent Mediation Engine™ platform will berunning.

pico.rcp.codeserver.host

Example value: 6790

This variable specifies the corresponding port for theConvergent Mediation Engine™ platform to be used bythis Execution Context.

pico.rcp.codeserver.port

Example value: EC4

This list specifies the Execution Context(s) to to be in-stalled.

mz.eclist

4.1.4. Configuring a Standalone Execution Context

To install a Standalone Execution Context, the following changes should be made to the install.xmlconfigured in the previous chapter.

Example value: ecsa

This variable specifies that we are about to install aStandalone Execution Context only.

install.types

Example value: ECSA4

This list specifies the Standalone Execution Context(s)to to be installed. For a Standalone EC installation, thisparameter replaces mz.eclist.

mz.ecsalist

4.1.5. Configuring the Web Interface for an Execution Context

The Execution Context embeds a web server, which allows the user to inspect and perform maintenancedirectly on the Execution Context. The following properties in the install.xml file configures theweb interface.

Example value: true

This variable specifies whether the web interface shouldbe enabled or not. To allow for finer grained EC mainten-

ec.httpd.enabled

17


ance, Nokia recommends that the web interface is leftenabled.

Example value: 9090

This value specifies on which port the web interfaceshould be bound. The port must not be in use by anyother process on the target system.

ec.httpd.port

Example value: dr

This value specifies the password for accessing the webinterface.

ec.httpd.password

If multiple execution contexts are installed in an install session, the$MZ_HOME/etc/executioncontext.xml file must be manually updated. To ensure thateach execution context binds its web-interface to an available port, the property ec.ht-tpd.port must be changed for each execution context.

4.2. Convergent Mediation Engine™ Software SetupBefore the setup step is started, it is assumed that the working directory is not changed since the previouscommand session. The following command will install the Convergent Mediation Engine™ software.

$ java -jar core*.mzp install

Start the mzsh as follows and login as mzadmin. If not prompted for username/password, then thereis a connection problem with the platform. Check the assigned values for pico.rcp.codeserv-er.port and pico.rcp.codeserver.host, and make sure the platform is up and running.

When mzadmin has logged in, the new Execution Context machine (IP-address) will be visible inthe Pico Manager on the desktop. Edit the machine entry and add the new Execution Context, EC4 inthis case. When the pico entry has been saved, start the new Execution Context. The Execution Contextis started by issuing the command startup EC4 at the mzsh prompt.

$ mzsh startup EC4

If a separately installed Execution Context will contain an aggregation storage server, an interworkflow storage handler or a duplicate UDR storage, the$MZ_HOME/etc/executioncontext.xml file must be updated. The propertypico.bootstrapclass must be extended with the following boot strap class(es):

For Aggregation: com.digitalroute.wfc.aggregation.storage.Aggregation-StorageImpl

For Duplicate UDR: com.digitalroute.wfc.dupudr2.RDupUDR2StorageImpl

For Inter Workflow: com.digitalroute.wfc.interwf.storagehandler.Batch-StorageImpl

18


5. Removing and Adding Packages

5.1. Listing PackagesThe following mzsh command session shows how to list a specific package and how to list all packages.

MZ>> plist -name XFER_X25

Overview of XFER_X25: User: mzadmin Version: 4.1.2.2 Date: 2005-10-12 16:14:04

MZ>> plist ...

5.2. Removing PackagesThe following mzsh command shows how to delete a specific package.

MZ>> premove XFER_X25

Removing XFER_X25, 4.1.2.2...done.

After a package has been removed, the platform and execution contexts must be stopped and restarted.If a removed package contained a bootstrap class, a cache error message will be displayed. The corres-ponding bootstrap class must be removed from platform.xml and/or executioncontext.xmlbefore the unit could be restarted.

5.3. Adding or Updating PackagesDuring upgrade or addition of new packages, the mz.license file must be available. The licensefile is accessed through the mz.home property. If new packages are added, the new license file mustbe copied to the $MZ_HOME/etc directory before the installation.

The addition/upgrade process consists of three steps. The first step is to create database definition files.If no database definitions are required for a package, the create command will simply respond:Nothing to create for <PACKAGE>. The second step is ignored if a package does not requireany database definitions to be created. The final step is to install the new package.

The create command will create all database definitions needed for a package or simply respondthat there is nothing to create for this package. The following command is an example of a createcommand.

$ java -Dmz.home=$MZ_HOME -jar archiving_4.1.1.1.mzp create

Unpacking database/oracle/packages/archiving/oracle_tables.sql...done

19


Unpacking database/oracle/packages/archiving/oracle_grants.sql...done Unpacking database/oracle/packages/archiving/oracle_synonyms.sql...done

Then, log in as database administrator and execute the corresponding sql files. finally the packagemay be installed. The following is an example of an install command.

$ java -Dmz.home=$MZ_HOME -jar archiving_4.1.1.1.mzp install ... Inserting Archiving, 4.1.1.1...done.

20


6. Java ConfigurationDescribes how to set JDK arguments in the configuration files for the programs.

6.1. Out of MemoryThere are a few parameters for the JVM that might need to be tweaked a little for a given server install-ation. Depending of the amount of available primary memory and the amount of disk swap space, itmight be necessary to tell the JVM how much memory it is allowed to allocate. The Unix process,running the Execution Context JVM, can fork itself depending on configurations in individual agents.The Disk Forwarding agent, for instance, can be configured to run an external binary after every for-warded file. The JVM performs a native fork-call to do this and the forked JVM process will initiallyhave the same memory footprint as the parent process. If there is not enough primary memory and/orswap space available, the Execution Context will abort with the following exception:

java.io.IOException: Not enough space at java.lang.UNIXProcess.forkAndExec(Native Method)

If this happens, the maximum heap size for the JVM must be lowered or more memory must be addedto the machine. Lowering memory is done with the -Xmx parameter. This parameter is specified forall programs in their respective configuration file.

The following row is an example of how to specify this in a parameter file.

<jdkarg value="-Xmx128M"/>

Unfortunately, it is difficult to advise a good value of this parameter. The parameter specifies themaximum heap size, meaning that the JVM probably will not reach this limit for a while, dependingon how the JVM manages its heap. That in turn means that the forking will work for a while and whenthe heap size in the JVM has grown big enough, the fork will fail in case there is no free memory pagesavailable in the machine.

The only possible recommendation is to lower the maximum heap size value or to add more systemresources (memory or swap disk). If the physical host runs more than one Execution Context, then thememory allocation of these Execution Contexts must be taken into account as well.

The more memory that is given to a JVM, the better it will perform. However, make sure that the heapnever pages. The sum of all maximum heaps must fit in physical memory. Make sure to adapt thesevalues to better fit the memory in the installed machine. Increasing the heap size for the ExecutionContext can make a big difference to performance.

6.2. Kernel Parameters for HP/UXWhen running Convergent Mediation Engine™ on an HP system there are two kernel parameters tobe updated; max_thread_proc and nkthreads. These parameters manages the allowed amountof threads per process and in total for the system. Since Java itself, as well as running many workflows,require many threads, the unable to create new native thread or out of memoryerror will be received unless these parameters are updated. Please see HP website for more informationhttp://www.hp.com/products1/unix/java/infolibrary/prog_guide/config-uration.html.

21


7. Communications Through FirewallsConvergent Mediation Engine™ is a dynamic and scalable system that can run on one or multiplehosts. A common case is when the Convergent Mediation Engine™ servers and Execution Context(s)run on host(s) behind a firewall, and clients connect from hosts outside the firewall. No ConvergentMediation Engine™ configuration changes need to be made to communicate through firewalls.

7.1. Convergent Mediation Engine™ Servers LocatedBehind a Firewall

In Convergent Mediation Engine™ , it is always the client that establishes a connection to its server.Once a connection has been established, it is used as a two-way communication channel; the servernever needs to open an outgoing connection. Typically, the platform has the server role in a ConvergentMediation Engine™ system, while mzsh, desktops, execution contexts and web-browsers act as clients.

It is also possible to inspect execution contexts through a web-interface, in which case the executioncontext also acts as a server. The same connection rules applies to this situation; the connection is es-tablished from the web-browser to the execution context.

The port used to communicate with the platform is specified by the pico.rcp.codeserver.hostparameter in the $MZ_HOME/etc/common.xml file, and is 6790 in a standard installation.

The port used to communicate with the platform web-interface is specified by the mz.wi.portparameter in the $MZ_HOME/etc/platform.xml file, and is 9000 in a standard installation.

The port used to communicate with an execution context web-interface is specified by the ec.ht-tpd.port parameter in the $MZ_HOME/etc/executioncontext.xml file, and is 9090 in astandard installation.

Figure 7. Firewall setup.

22


7.2. Firewall SetupListed below are the actions that should be taken to allow the respective client-server pairs to commu-nicate in the Convergent Mediation Engine™ system. Standard installation ports are assumed, replacewith your custom values if you have changed the default ports.

To allow mzsh, desktops and execution contexts to communicate with the Convergent Mediation En-gine™ platform, incoming data to, and outgoing data from the platform's port 6790 must be allowed.

To allow the platform web-interface to be accessed from outside the firewall, incoming data to, andoutgoing data from the platform's port 9000 must be allowed.

To allow the execution context web-interface to be accessed from outside the firewall, incoming datato, and outgoing data from the execution context's port 9090 must be allowed.

23


8. System RequirementsThe Convergent Mediation Engine™ platform host needs to have the Java Platform Standard EditionDevelopment Kit installed prior to the installation. The Java 2 Runtime Environment is sufficient fordesktops and Execution Contexts. This JDK or JRE may be downloaded from ht-tp://java.sun.com for Solaris and Linux and from http://www.hp.com for HP-UX.

8.1. Platform and Execution Context• Solaris 9 and 10, RedHat Advanced Server 4.0 and HP-UX 11

• Oracle 9.2.0.1 and 10.1.0.2

• Java 2 Development Kit or Runtime EnvironmentFor Linux and Solaris there is Java Platform Standard Edition Development Kit 5.0 from ht-tp://java.sun.com. For HP/UX there is HP-UX Development Kit for the Java platform(product T1456AA) from http://www.hp.com/products1/unix/java. The JDK isonly required for platform hosts while the JRE is sufficient for Execution Contexts and desktops.

• OS patchesAlso make sure all necessary patches to run java are installed in the operating system. For Solaris,these can be found on http://sunsolve.sun.com/pub-cgi/show.pl?tar-get=patches/J2SE. For HP-UX, these can be found on ht-tp://www.hp.com/products1/unix/java/patches.

8.2. DesktopWindows XP, Solaris 9, Solaris 10, RedHat Advanced Server 4.0 and HP-UX 11. Java PlatformStandard Edition (JRE) 5.0 is required.

24


9. Oracle ConfigurationFor Convergent Mediation Engine™ to function properly, the following Oracle file needs configuration:

• $ORACLE_HOME/network/admin/tnsnames.ora

• $ORACLE_HOME/network/admin/listener.ora

Example 1.

The following shows an entry on the host server-host:

MZ = (DESCRIPTION = (ADDRESS_LIST = (ADDRESS =(PROTOCOL=TCP)(HOST=server-host)(PORT=1521)) ) (CONNECT_DATA = (SERVICE_NAME = mz) )

$ORACLE_HOME/network/admin/listener.oraLISTENER = (DESCRIPTION_LIST = (DESCRIPTION = (ADDRESS_LIST = (ADDRESS =(PROTOCOL=IPC)(KEY=EXTPROC)) ) (ADDRESS_LIST = (ADDRESS =(PROTOCOL=TCP)(HOST=server-host)(PORT=1521)) ) ) (DESCRIPTION = (PROTOCOL_STACK = (PRESENTATION = GIOP) (SESSION = RAW) ) (ADDRESS =(PROTOCOL=TCP)(HOST=server-host)(PORT=2481)) ) )

SID_LIST_LISTENER = (SID_LIST = (SID_DESC = (SID_NAME = PLSExtProc) (ORACLE_HOME = /opt/oracle ) (PROGRAM = extproc) ) (SID_DESC = (GLOBAL_DBNAME = mz) (ORACLE_HOME = /opt/oracle) (SID_NAME = MZ) ))

25


Inter Workflow Agents 4.1


Version 1.20

1





2




2. General ..................................................................................................................... 53. Inter Workflow Profile ................................................................................................. 6

3.1. Configuration .................................................................................................. 64. Inter Workflow Forwarding Agent .................................................................................. 8

4.1. Configuration - Batch Workflow ......................................................................... 84.2. Configuration - Realtime Workflow ..................................................................... 84.3. Transaction Behavior - Batch Workflow ................................................................ 9

4.3.1. Emits .................................................................................................. 94.3.2. Retrieves .............................................................................................. 9



4.6. Node Message Events ...................................................................................... 105. Inter Workflow Collection Agent .................................................................................. 11


5.2.1. Emits ................................................................................................. 125.2.2. Retrieves ............................................................................................ 12




3


1. IntroductionThis document describes the Inter Workflow agents being part of the standard agents available on theConvergent Mediation Engine™ platform. The agents are used for temporary storage of stream data,merging stream data and as gateway between Realtime and Batch workflows. The Inter WorkflowForwarding agent can be part of both Batch and Realtime workflows, while the collection agent isonly used in Batch workflows.




4


2. GeneralThe Inter Workflow agents allow files to be distributed between workflows within the same ConvergentMediation Engine™ system. It is especially useful when transferring data from Realtime workflowsto Batch workflows.

The Inter Workflow agents use an Inter Workflow storage server to manage the actual data storage.The storage server can either run on an Execution Context or on the Platform. The storage server andbase directory to use is configured in an Inter Workflow profile.

Figure 1. The Inter Workflow application distributes files from one workflow to another.

Several forwarding workflows may be configured to distribute batches to the same profile, but onlyone collection workflow at a time can be activated to collect from it.

5


3. Inter Workflow ProfileThe Inter Workflow Profile allows the user to configure the storage server that the Inter Workflowforwarding and collection agents use for communication.

It is safe to build up a lot of data in the storage server directory. When the initial set of directories hasbeen populated with a predefined number of files, new directories are automatically created to avoidproblems with file system performance.

3.1. ConfigurationThe Inter Workflow Profile Editor is displayed when Inter Workflow Profile... is selected fromthe Configuration menu on the desktop.

Figure 2. Inter Workflow Profile

A list containing all registered storage handlers. Select the target for file storage.Storage Host

• Add a parameter to the pico.bootstrapclass in the$MZ_HOME/etc/executioncontext.xml file for the Execution Context(in case the handler resides on an Execution Context).

• Add a parameter to the pico.bootstrapclass in the$MZ_HOME/etc/platform.xml file (in case the handler resides on thePlatform).

Note that a default storage handler is set by the installation.

Absolute pathname to the directory on the Storage handler where the temporary fileswill be placed.

Root Directory

An optional parameter stating the limit of the space consumed by the files stored inthe Root Directory. If the limit is reached, any Inter Workflow Forwarding nodeusing this profile will abort.

Max Bytes

An optional parameter stating the maximum number of batches stored in the RootDirectory. If the limit is reached, any Inter Workflow Forwarding node using thisprofile will abort.

Max Batches

6


A list of user defined MIM names. Note that these variables do not have any valuesassigned. They are populated with existing MIM values from the Inter Workflow

Named MIMs

Forwarding agent. This way, MIMs from the forwarding workflow can be passedon to the collecting workflow.

7


4. Inter Workflow Forwarding AgentThe Inter Workflow Forwarding agent is responsible for sending data to the storage server. The agentcan be part of both Batch and Realtime workflows. In the latter case, the user has to define batchclosing criteria for the data to be saved in the storage directory. This is based on UDR count, bytecount, or elapsed time.

4.1. Configuration - Batch WorkflowThe Inter Workflow Forwarding agent configuration window is displayed when the node is double-clicked or right-clicked, selecting Configuration...

Figure 3. Workflow Forwarding agent configuration window - Batch workflow.

The name of the profile as defined in the Inter Workflow Profile Editor(select Inter Workflow Profile... from the Configuration menu on thedesktop).

Profile

The user defined MIM names according to the definitions in the selectedProfile.

Named MIM

Selected, existing MIM values of the workflow that the Named MIMs aremapped to. This way, MIM values from this workflow are passed on to thecollection workflow.

MIM Resource

If enabled, empty files will be created even if no UDRs are forwarded froma batch.

Produce Empty Batches


4.2. Configuration - Realtime WorkflowThe Inter Workflow Forwarding agent configuration window is displayed when a node is double-clicked or right-clicked, selecting Configuration...

8


Figure 4. Workflow Forwarding agent configuration window - Realtime workflow.

The name of the profile as defined in the Inter Workflow Profile Editor (selectInter Workflow Profile... from the Configuration menu on the desktop.

Profile

The user defined MIM names according to the definitions in the selected Profile.Named MIM

Selected, existing MIM values of the workflow that the Named MIMs are mappedto. This way, MIM values from this workflow are passed on to the collectionworkflow.

MIM Resource

The size, in bytes, of the files saved in the storage directory.Volume (bytes)

The size, based on number of UDRs, of the files saved in the storage directory.Volume (UDRs)

The size, based on time interval, of the files saved in the storage directory.Timer (sec)

Since there are no natural batch boundaries within a Realtime workflow, Volume and/or Timercriteria has to be set for when the file, outputting data, will be closed and a new opened. If sev-eral file closing criteria have been selected, all will apply, using a logical OR.

If the workflow is deactivated before any of the file closing criteria has been fulfilled, the UDRscurrently stored in memory will be flushed regardless. Hence, the size of the last file cannot bepredicted. In case of a crash, nor the contents of this last batch can be predicted. The errorhandling is taken care of by the Inter Workflow Collection node. If the file is corrupt, it will bethrown away and an informative message is logged in the System Log. The collector will auto-matically continue with the next batch in order.

4.3.Transaction Behavior - Batch Workflow

4.3.1. Emits

None.

4.3.2. Retrieves

Creates and opens a target file in a temporary directory.Begin Batch

Moves the file from the temporary directory to the target directory.End Batch

Deletes the current file from the temporary directory.Cancel Batch

9




4.5.1. Publishes


4.5.2. Accesses

Various resources from the workflow and all its nodes. This to configure the mapping to the NamedMIMs (that is, what MIMs to pass on to the collection workflow).


4.6. Node Message EventsInserted batch: filename

Reported when a file has been closed in the target directory (and hence made ready for collection).The message is only used in batch workflows.

10


5. Inter Workflow Collection AgentThe Inter Workflow Collection agent collects data submitted by the Inter Workflow Forwarding agentbeing part of a Batch or Realtime workflow. The data is collected from a storage server where it hasbeen placed by the Forwarding agent. The agent also offers the possibility to merge the collected fileson a byte or number of files basis. The smallest entity is one complete source file.

5.1. ConfigurationThe Inter Workflow Collection agent configuration window is displayed when the node is double-clicked or right-clicked, selecting Configuration...

Figure 5. Inter Workflow Collection agent configuration.

The name and most recent version of the profile as defined in the InterWorkflow Profile Editor (select Inter Workflow Profile... from theConfiguration menu on the desktop).

Profile

If enabled, each incoming batch will generate one outgoing batch.No Merge

Decides if the incoming batches will be merged into larger entities. If en-abled, incoming files will be merged, using a logical OR, as soon as any of

Merge Batches Basedon Criteria

the merge criteria defined in the Merge Definition are met. Note that it isnot possible to import MIM values for all batches in a merge; header MIMsfor the first batch will be selectable, and trailer and batch MIMs for the lastbatch.

If enabled, all incoming batches are inserted into the workflow before theagent closes the transaction. This option is used with periodic schedulingto retrieve all available data at each activation.

Merge All AvailableBatches

The size of the batches produced by the collection node. The incoming filesare never split. For instance, if 300 is entered and the source files are 200bytes each, the produced batches will be 400 bytes.

Number of Bytes

The number of incoming batches to merge.Number of Batches

Indicates for how long (in seconds) the node will wait after the first incomingbatch. When this time has expired, an outgoing batch will be produced, evenif the Number of Bytes/Batches criteria has not been fulfilled.

Age of Oldest Batch(sec)

11



5.2.1. Emits

Emitted right before the first byte of the collected file(s) are fed into a workflow.Begin Batch

Emitted just after the last byte of last collected file has been fed into the system.End Batch

5.2.2. Retrieves

If a Cancel Batch message is received, the agent sends the batch to ECS. Note that thebatch will be closed and moved immediately, no matter the criteria defined in the MergeDefinition.

Cancel Batch




5.4.1. Publishes

The number of incoming batches to merge in the current outgoing batch. Ifno merge is selected this MIM is static and set to 1 (one).

Number Of SourceBatches

Type int, defined as MIM type header.

The size of the batch produced by the node.Outgoing Batch Size

Type long, defined as MIM type header.

Any as named in the Inter Workflow Profile Editor and mapped in thecorresponding Inter Workflow Forwarding node. These MIMs originate fromthe forwarding workflow.

<any>

Type string, defined as MIM type header. All imported MIMs are automat-ically converted to strings, no matter the original data type.

5.4.2. Accesses



12


5.5. Node Message EventsReady with batch: name

Reported when a batch is finished and then sent on to the subsequent node.

Merging batch: name

Reported during batch merging.

Last merged batch: name

Reported upon activation, and right before merging of a new batch is started.

Batch cancelled: name

Reported when a batch has been cancelled and routed to ECS.

13


LDAP Package 4.1


Version 1.3

1





2

LDAP Package 4.1



2. LDAP Functions ......................................................................................................... 52.1. ldapCreate ................................................................................................. 52.2. ldapSearch ................................................................................................. 5

3

LDAP Package 4.1

1. IntroductionThis document describes the LDAP APL functions being an extension to the Nokia Convergent Medi-ation Engine™ platform.




• Lightweight Directory Access Protocol (LDAP)


4

LDAP Package 4.1

2. LDAP FunctionsThe LDAP functions allow a user to connect to an LDAP, using anonymous or simple authentication,and perform searches based on names, filters and explicitly define what attributes to be returned. TheLDAP functions use connection pooling for best performance.

2.1. ldapCreate

Creates a connection towards an LDAP server, using either anonymous or simple authentication. Thisfunction is usually invoked in the initialize block.

any ldapCreate ( string host, int port, string name, string principal, //Optional string credentials ) //Optional

Parameters:

The host name of the LDAP server.host

The port number of the LDAP server.port

Name which identifies the context in which the search will be performed. The valuecan span multiple naming systems and must be fully qualified and identify entries

name

from the root of the LDAP server. If specified as null this means that the searchwill be performed from the LDAP root.

Optional argument that defines the user to connect as. If omitted, the function willconnect to the LDAP server using anonymous authentication. This argument requiresthat the credentials argument is supplied.

principal

Optional argument that defines the user password.credentials

An identifier used when invoking the search function.Returns

Example 1.

any ctx = ldapCreate("10.0.0.1", 389, "o=users");...any ctx = ldapCreate("10.0.0.1", 389, "o=users", "cn=Administrator, o=users", "secret");

2.2. ldapSearch

Performs a search in a LDAP server based on a number of arguments. The search function requiresthe ldapCreate function to be invoked and that the result is given as an argument.

list<ldapResult> ldapSearch

5

LDAP Package 4.1

( any identifier, string name, string filter, //Optional list<string> retAttrs ) //Optional

Parameters:

The connection identifier returned from ldapCreate.identifier

The name of the object to be searched for. This is matched relative to the contextspecified with the name argument in ldapCreate. If no object matches the name,an error will be thrown stating there is no such name.

name

Optional value, specifying the attributes of the requested LDAP objects. This valueconforms to RFC 2254. The specified filter will match objects, relative to the nameparameter.

filter

Optional list of strings that defines what attributes to be returned.retAttrs

A list of ldapResult UDRs, where every entry contains a name, an attribute anda list of values. The name value is the name relative to the target context of the search.

Returns

If the target context matches the search filter, then the returned name will be an emptystring. The attribute is the attribute name and the values are all values for the givenattribute.

6

LDAP Package 4.1

Example 2.

list<ldapResult> result = ldapSearch(ctx, "ou=Members");...

// limit returned attributes list<string> returnedAttributes = listCreate(string);listAdd(returnedAttributes, "homePhone");listAdd(returnedAttributes, "emailAddress");

list<ldapResult> result = ldapSearch(ctx, "ou=Members", "cn=Kent Andersson", returnedAttributes);...

// Print out the contents from a ldapSearchint resultIdx = 0;int resultSize = listSize(result);while (resultCount < resultSize) { ldapSearch entry = listGet(result, resultCount); debug("Name: " + entry.name); debug("Attribute: " + entry.attribute);

int valueCount = 0; int valueSize = listSize(entry.values); while (valueCount < valueSize) { debug("Value: " + listGet(entry.values, valueCount)); valueCount = valueCount + 1; } resultCount = resultCount + 1;}

// debug can also be invoked directly on every entrydebug(listGet(result, resultCount));

7

LDAP Package 4.1

Netflow Agent 4.1


Version 1.5

1





2

Netflow Agent 4.1



2. NetFlow Agent ........................................................................................................... 52.1. Configuration .................................................................................................. 5

2.1.1. Connection Tab ..................................................................................... 62.1.2. UDR Type Tab ...................................................................................... 72.1.3. Interface Mapping Tab ............................................................................ 7



2.4. Node Message Events ....................................................................................... 8

3

Netflow Agent 4.1

1. IntroductionThis document describes the NetFlow agent being part of the extension agents, available on the NokiaConvergent Mediation Engine™ platform.



• The Cisco NetFlow export formats


4

Netflow Agent 4.1

2. NetFlow AgentThe NetFlow agent gathers traffic data from one or many Cisco routers. NetFlow data contains inform-ation, such as source and destination IP address, down- and uploaded bytes.

Each router can potentially be identified through several IP addresses (interfaces) and if so, it maysend UDP packets on any of these interfaces to the agent. The agent offers a possibility of mappingall these IP addresses into one that enables detection of the fact that they originated from the samerouter.

Figure 1. Example of NetFlow network.

When activated, the agent will connect to the configured port and start listening for incoming packetsfrom the routers. Each received packet will be unpacked into one or several flow records. Based oninformation in the flow record, the agent will create and populate one of the standard NetFlow UDRtypes available in Convergent Mediation Engine™ and forward the UDR into the workflow. If theagent fails in unpacking or reading the packet or a flow record, it will silently be removed from thestream.

Since Cisco routers do not offer the possibility of requesting historic data, the agent will lose all datathat is delivered from the router while the agent is not executing.

The real-time job queue may fill up, in which case a warning will be raised in the System Logstating the job queue is full. This means records arriving to a full queue will be thrown away.A message in the System Log will state when the queue status is back to normal.

2.1. ConfigurationThe NetFlow agent configuration window is displayed when the node in a workflow is double-clickedor right-clicked, selecting Configuration...

5

Netflow Agent 4.1


Figure 2. NetFlow agent configuration window, Connection tab.

The port number where the NetFlow agent will listen for packets from the routers.Port

Since the routers will be configured to communicate with a specific hoston this port, it is of importance that the workflow that contains the NetFlownode is configured to execute on that specific host and not on a randomhost.

Two NetFlow nodes may not be configured to listen on the same port onthe same host.

If enabled, the agent only accepts packets from hosts that have been specified inthe Interface Mapping tab. Data from other hosts will be discarded.

Only from Pre-defined Hosts

If disabled, all arriving data will be accepted. This may be useful when havingmany routers, most of them sending from only one interface (IP-address) each,but some of them using the settings in the Interface Mapping tab. Hence, whendisabling this option, the one-interface-routers do not have to be added to the in-terface mapping list.

Determines if a warning will be raised in the System Log when the sequencenumber gap between two sequential PDUs from the same router is equal to orlarger than specified in Minimum Gap.

Warn on SequenceGap

The minimum sequence number gap between two flow records that will cause awarning to be raised in the System Log.

Minimum Gap

6

Netflow Agent 4.1

2.1.2. UDR Type Tab

Figure 3. NetFlow agent configuration window, UDR Type tab.

Expected UDR Types to be delivered to this agent. If other types arrive, the NetFlownode will abort.

UDR Type

2.1.3. Interface Mapping Tab

Figure 4. NetFlow agent configuration window, Interface Mapping tab.

Maps several interface IP addresses to one main IP address. Each router that uses more than one interfaceIP address when sending data to the agent, must be registered here. One of the IP addresses, supportedby the router, must be registered as the Main IP Address. The others are configured in the IP Addresslist.

If a packet arrives from an IP address configured in the IP Address list, it will be mapped to the cor-responding Main IP Address. This way it will look as if all packets originate from the same IP address.

One address from each router that supports multiple interfaces must be added tothis list. When an existing row is selected, the contents in the IP Address tablewill reflect the slave IP addresses for the selected Main IP Addresses.

Main IP Address

Additional IP addresses that will be mapped to their corresponding main IP addressby the agent.

IP Addresses

7

Netflow Agent 4.1

2.2. IntrospectionDepending on the incoming flow records, the agent may produce one of the following UDR types.Their names reflect the NetFlow versions:

• V1UDR (netflow)

• V5UDR (netflow)

• V7UDR (netflow)

• V8ASMatrixUDR (netflow)

• V8DestinationPrefixMatrixUDR (netflow)

• V8PrefixMatrixUDR (netflow)

• V8ProtocolPortMatrixUDR (netflow)

• V8SourcePrefixMatrixUDR (netflow)


2.3.1. Publishes

..Number of incoming messages

2.3.2. Accesses

None.



8

Netflow Agent 4.1

Radius Agent 4.1


Version 1.10

1





2

Radius Agent 4.1



2. Radius Agent ............................................................................................................. 52.1. Configuration .................................................................................................. 5

2.1.1. NAS Tab .............................................................................................. 52.1.2. Miscellaneous Tab ................................................................................. 6

2.2. The Radius Format ........................................................................................... 72.3. Introspection ................................................................................................... 72.4. Meta Information Model .................................................................................... 7


2.5. Node Message Events ....................................................................................... 83. An Example ............................................................................................................... 9

3.1. The Workflow Setup ......................................................................................... 93.2. The Ultra Format Definition ............................................................................... 9

3.2.1. Shortened Ultra Format Definition .......................................................... 103.3. The Analysis Node .......................................................................................... 11

3

Radius Agent 4.1

1. IntroductionThis document describes the Radius Collection agent being part of the extension agents, available onthe Nokia Convergent Mediation Engine™ platform.



• RADIUS (RFC 2865, http://www.ietf.org/rfc/rfc2865.txt)

• RADIUS Accounting (RFC 2866, http://www.ietf.org/rfc/rfc2866.txt)


4

Radius Agent 4.1

2. Radius AgentThe Radius agent receives accounting data (UDP package) from one or several Network AccessServers (NAS). Radius accounting data contains information about who has logged in, when it occurred,the duration of the session etc. Other than collecting such data, the Radius agent may act as an extensionto the NAS, creating accounting data itself.

For instance, when receiving a package containing a login request, it may reply with an accept or rejectpackage. The reply logic is handled through APL code (an Analysis or Aggregation node).

Figure 1. A typical Radius workflow.

Note the absence of a Decoder. For Realtime workflows, field decoding is handled via APL commands.The Radius format is included when a Radius bundle is committed into the system. The formatcontains record identification information on the first level (code, identifier, length, authenticator andattributes) to be used by the Radius agent. Hence, the agent is responsible for recognizing the type ofdata, while the Analysis node does the actual decoding of the contents (the attributes). A UFDL formatneeds to be defined for this purpose.

When activated, the agent will bind to the configured port and wait for incoming UDP packages fromNASes. Each received UDP will be converted to a UDR and forwarded into the workflow. If fieldsare missing in a UDP, the agent will still create a UDR, filling in all found fields. If the data in theUDP is corrupt, or if data arrives from a host not present in the configuration window of the node, amessage will be sent to the System Log and the data will be discarded.

Since NASes do not offer the possibility of requesting historic data, the agent will lose all data that isdelivered from the NAS while the agent is not executing.

2.1. ConfigurationThe Radius agent configuration window is displayed when the node in a workflow is double-clickedor right-clicked, selecting Configuration...

2.1.1. NAS Tab

Figure 2. Radius agent configuration window, NAS tab.

5

Radius Agent 4.1

In the NAS tab, all NASes that the agent will collect information from are specified.

The IP address that the NAS, sending packages, is located on.IP Address

Key used for authentication of the received package. This key must be identical to theone defined in the NAS.

Secret Key

2.1.2. Miscellaneous Tab

Figure 3. Radius agent configuration window, Miscellaneous tab.

The port number where the Radius agent will listen for packages from the NAS(es).Port

Since the NASes will be configured to communicate with a specific hoston this port, it is of importance that the workflow containing the Radiusnode is configured to execute on the associated EC for that host and noton a random one.

Two Radius nodes may not be configured to listen on the same port, on thesame host.

If set to a value larger than 0 (zero), duplicate check is activated. The buffer savedfor comparison is the packages collected during the set time frame.

PDU Lifetime(millisec)

If enabled, the check for MD5 signatures is excluded.Skip MD5 Calcu-lation

This is necessary if the Radius client does not send MD5 signatures along withthe packages, in which case they would be discarded by the Radius agent.

When Skip MD5 Calculation is enabled, the authenticator field in all re-sponse messages will be 0 (zero).

Checking for duplicate packages can be made based on:Duplicate Check

• Radius Standard - the identifier within the package (byte number 2).

• CRC32 - check sum for the complete package.

When a duplicate is detected, it is silently thrown away (no message is logged)and the Radius agent responds as if a normal package was received.

6

Radius Agent 4.1

2.2.The Radius FormatEach Radius UDR will be packed containing both the request and response UDRs for each case, alongside the general NAS information, such as port and IP address.

Included with the Convergent Mediation Engine™ Radius bundle is a general Radius format, con-taining all possible record types valid for Radius. The Radius agent will use this format for recognizingthe type of data. The actual decoding of the contents (requestMessage), and the encoding of thereply (responseMessage) must be handled through a user defined format.

Provided that duplicate checking is enabled, and CRC32 is selected as method,this field will contain the calculated check sum for the package. For all othercases it will be set to NULL.

hashKey

The IP address of the NAS.remoteIP

The port used for communication with the NAS.remotePort

The field containing a request UDR. Depending on the settings on the NAS,it can be any of the available standards, for instance, Access-Request or Ac-counting-Request.

requestMessage

An Ultra Format Definition must be designed to handle decoding of this field.

The field containing a response UDR. Depending on the settings on the NAS,it can be any of the available standards, for instance, Access-Accept or Access-Reject.

responseMessage

An Ultra Format Definition must be designed to handle decoding of this field.

2.3. IntrospectionNone.


2.4.1. Publishes

..Number of messages

..Number of duplicate messages/type

..Number of rejected messages/type

2.4.2. Accesses

None.


7

Radius Agent 4.1


Access request with invalid passwd/secret from IP

Indicates an invalid password was entered or is missing for the connection.

Accounting request with invalid signature from IP

Indicates the calculated sum, based on secret specified for the node, is not equal to the secret in theincoming package.

Incoming invalid code xx from host name

Indicates the incoming request is not ACCESS_REQUEST or ACCOUNTING_REQUEST.

8

Radius Agent 4.1

3. An ExampleA Radius agent can act as an extension to a NAS. To illustrate such a scenario an example is introduced,where an Analysis node is used to validate the contents of the received UDP package, and a reply issent back (also in form of a UDP package) depending on the outcome. Valid UDRs are routed to thesubsequent node, while invalid UDRs are deleted. Schematically, the workflow will perform the fol-lowing:

1. Decode the data into a UDR. Discard and continue with the next package upon failure.

2. Validate the UDR. If it is of type Access_Request_Int, a lookup against a subscriber tablemust be performed to make sure the user is authorized (that is, exists in the table). All other UDRtypes must be deleted.

3. If the user was found in the table, send the UDR to the next node and a reply UDR of type Ac-cess_Accept_Int back to the Radius node. If the user was not found, delete the UDR andsend a reply UDR of type Access_Reject_Int to the Radius node. Both reply UDRs musthave the Identifier field updated first.

To keep the example as simple as possible, the valid records are not processed. Usually, no replyis sent back until the UDRs are fully validated and manipulated. The example aims to focus onthe Convergent Mediation Engine™ specific issues, such as decoding, validation and replyhandling.

3.1.The Workflow SetupA Radius node must always be connected to an Analysis or Aggregation node, since it is dependenton receiving replies which are issued through APL commands.

Figure 4. The Analysis node handles all reply logic.

The radius node will forward all received packages. The actual discarding and validation of the datais handled in the Analysis node.

3.2.The Ultra Format DefinitionTo simplify the example, only one type of request UDRs is accepted, and one of two types of replyUDRs is sent back. Hence, the requestMessage and responseMessage fields of the RadiusUDR will be populated with any of these UDR types:

requestMessage • Access_Request UDR - A login request from a supposed user whichmust be authenticated against a subscriber database.

responseMessage • Access_Accept UDR - Sent back in case the authentication succeeded.

9

Radius Agent 4.1

• Access_Reject UDR - Sent back in case the authentication failed.

The full Ultra Format Definition for the example is not shown, since it is beyond the scope of thismanual to handle package contents or UFDL syntax.

3.2.1. Shortened Ultra Format Definition

The format definition is here stored in the Default folder with the name extendedRadius.

external Access_Request_Ext sequential : identified_by(Code == 1), dynamic_size(RecLength) { int Code: static_size(1); int Identifier: static_size(1); int RecLength: static_size(2), encode_value(udr_size); bytearray Authenticator: static_size(16); switched_set( Type ) { int Type: static_size(1); int Length: static_size(1), encode_value(case_size); case(1) { ascii User_Name: dynamic_size( Length - 2 ); }; case(2) { ascii User_Password: dynamic_size( Length - 2 ), terminated_by(0); }; // ... // Further field definitions. // ... };};

external Access_Accept_Ext sequential : identified_by(Code == 2), dynamic_size(RecLength) { // ... // Further field definitions. // ...};

external Access_Reject_Ext sequential : identified_by(Code == 3), dynamic_size(RecLength) { // ... // Further field definitions. // ...};

internal Vendor_Specific_Int { int Type; int Length; int VendorID; int SubAttrID; int VendorLength;

10

Radius Agent 4.1

int InfoCode; string Data;};

in_map Access_Request_Map : external(Access_Request_Ext), target_internal(Access_Request_Int) { automatic { Vendor_Specific_Ext: internal( Vendor_Specific_Int ), target_internal( Vendor_Specific_TI1 ); };};

in_map Access_Accept_InMap: external(Access_Accept_Ext), target_internal( Access_Accept_Int ) { automatic { Vendor_Specific_Ext: internal( Vendor_Specific_Int ); };};

out_map Access_Accept_Map : external(Access_Accept_Ext), internal(Access_Accept_Int) { automatic;};

in_map Access_Reject_InMap: external(Access_Reject_Ext), target_internal( Access_Reject_Int ) { automatic { Vendor_Specific_Ext: internal( Vendor_Specific_Int ); };};

out_map Access_Reject_Map : external(Access_Reject_Ext), internal(Access_Reject_Int) { automatic;};

decoder Request_Dec: in_map(Access_Request_Map);

encoder Response_Enc: out_map(Access_Accept_Map), out_map(Access_Reject_Map);

3.3.The Analysis NodeAll decoding, validation and manipulation is performed from the Analysis node. The code logic is asfollows:

1. Each time the workflow is activated, a subscriber table is read into memory. To keep the examplesimple, the table contents are assumed to be static. For a real implementation, it is recommendedto re-read the table on a regular basis.

2. Decode the UDP package. Consider only UDRs of type Default.extendedRadius.Ac-cess_Request_Int.

11

Radius Agent 4.1

3. Perform a lookup against the subscriber table, and create a reply of type Default.extendedRa-dius.Access_Accept_Int or Default.extendedRadius.Access_Reject_Int,depending on if the subscriber was found in the table.

4. Route the reply back to the Radius node.

table tmp_tab;

initialize { tmp_tab = tableCreate("select SUBSCRIBER from VALID_SUBSCRIBERS");}

consume { list<drudr> reqList = listCreate(drudr); radius.Radius r = (radius.Radius) input; string err = udrDecode("Radius.Request_Dec", reqList, r.requestMessage, true);

if ( (err != null) || (listSize(reqList) != 1) ) { debug("Decoding error: " + err); return; }

drudr elem = (drudr) listGet(reqList, 0);

if (instanceOf(elem, Default.extendedRadius.Access_Request_Int)) { Default.extendedRadius.Access_Request_Int req = (Default.extendedRadius.Access_Request_Int) elem; table rowFound = tableLookup( tmp_tab, "SUBSCRIBER", "=", req.User_Name );

if (tableRowCount(rowFound) > 0) { Default.extendedRadius.Access_Accept_Int resp = udrCreate(Default.extendedRadius.Access_Accept_Int); resp.Identifier = req.Identifier; r.responseMessage = udrEncode("Default.extendedRadius.Response_Enc", resp); udrRoute( r ); } else { Default.extendedRadius.Access_Reject_Int resp = udrCreate(Default.extendedRadius.Access_Reject_Int); resp.Identifier = req.Identifier; r.responseMessage = udrEncode("Default.extendedRadius.Response_Enc", resp); udrRoute( r, "Response" ); }

} else { debug("Invalid request type"); }}

12

Radius Agent 4.1

SFTP Agent 4.1


Version 1.17

1





2

SFTP Agent 4.1



2. Installation Note ......................................................................................................... 53. SFTP Agents .............................................................................................................. 6

3.1. Supported Ciphers ............................................................................................ 63.2. Authentication ................................................................................................. 63.3. Server Keys ..................................................................................................... 7

4. SFTP Collection Agent ................................................................................................ 84.1. Collection Strategy ........................................................................................... 84.2. Configuration .................................................................................................. 9

4.2.1. Connection Tab ..................................................................................... 94.2.2. Source Tab .......................................................................................... 104.2.3. Advanced Tab ...................................................................................... 12




4.6. Node Message Events ...................................................................................... 145. SFTP Forwarding Agent ............................................................................................. 15

5.1. Configuration ................................................................................................ 155.1.1. Target Tab ........................................................................................... 155.1.2. Advanced Tab ...................................................................................... 17





3

SFTP Agent 4.1

1. IntroductionThis document describes the SFTP agent being part of the extension agents, available on the NokiaConvergent Mediation Engine™ platform.



• The SSH2 and SFTP protocols


4

SFTP Agent 4.1

2. Installation NoteThe SFTP agent uses a file with known host keys to validate the server identity during connectionsetup. The location and naming of this file is managed through the mz.ssh.known_hosts_fileproperty. This property must be specified in executioncontext.xml to manage where the fileis saved. The default value is ${mz.home}/etc/ssh/known_hosts.

The SSH implementation uses the standard JCE cryptography support, which means that there aresome limitation on key sizes for the standard Java distribution. This is usually not a problem, however,there may be some cases where the unlimited strength cryptography policy is needed. For instance, ifhost RSA keys, larger than 2048 bits, are used (depending on the SSH server configuration).

If this is an issue, the Java Runtime Environment (JRE) that runs the execution context must have twojar-files updated. These are; US_export_policy.jar and local_policy.jar. The originaljar-files are installed in the directory lib/security in the JRE installation. Due to license restrictions,the jar-files are not distributed with Convergent Mediation Engine™ , however, they may be downloadedfrom Sun.

Turn to "Other Downloads" on the http://java.sun.com/j2se/1.5.0/download.jsppage to get hold of the "Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction PolicyFiles", which holds the jce_policy-1_5_0.zip file. Download this file, unpack it and replacejar-files in the JRE installation.

5

SFTP Agent 4.1

3. SFTP AgentsBoth the SFTP Collection agent and the SFTP Forwarding agent share a number of common attributes,described below.

3.1. Supported CiphersCurrently, the SFTP agent only supports two cipher algorithms. It will first try to use the blowfishcipher, and if the server does not support this, the 3DES cipher will be used instead (all SSH2 serversare required to support 3DES).

3.2. AuthenticationThe SFTP agents support authentication through both user name/password and private keys. Privatekeys can optionally be protected by a passphrase, and most commonly used private key files may beimported into Convergent Mediation Engine™ .

Typical command line syntax (most systems):

ssh-keygen -t <keyType> -f <directoryPath>

The type of key to be generated. Both RSA and DSA key types are supported.keyType

Where to save the generated keys.directoryPath

6

SFTP Agent 4.1

Example 1.

The keys may be created using the following command line:

> ssh-keygen -t rsa -f /tmp/keystore Enter passphrase: xxxxxx Enter same passphrase again: xxxxxx

Then the following is stated:

Your identification key has been saved in /tmp/keystore Your public key has been saved in /tmp/keystore.pub

When the keys are created they may be imported to the SFTP node:

Finally, append /tmp/keystore.pub to $HOME/.ssh/authorized_keys.

3.3. Server KeysThe SSH protocol uses host verification to guard against attacks where an attacker manages to reroutethe TCP connection from the correct server to another machine. Since the password is sent directlyover the encrypted connection, it is critical for security that an incorrect public key is not accepted bythe client.

The agent uses a file with the known hosts and keys. It will accept the key supplied by the server ifeither of the following is fulfilled:

1. The host is previously unknown. In this case the public key will be registered in the file.

2. The host is known and the public key matches the old data.

3. The host is known but has a new key and the user has configured to accept the new key. Pleasesee the Advanced tab for more information.

If the host key changes for some reason, the file will have to be removed (or edited) in order for thenew key to be accepted.

7

SFTP Agent 4.1

4. SFTP Collection AgentThe SFTP Collection agent collects files from a remote host and inserts them into a Convergent Medi-ation Engine™ workflow, using the SFTP protocol over SSH2.

Upon activation, the agent establishes an SSH2 connection and an SFTP session towards the remotehost. If this fails, additional hosts are tried, if configured. On success, the source directory on the remotehost is scanned for all files matching the current filter. In addition, the Filename Sequence servicemay be utilized for further control of the matching files. All files found, will be fed one after the otherinto the workflow.

When a file has been successfully processed by the workflow, the agent offers the possibility of moving,renaming, removing or ignoring the original file. The agent can also automatically delete moved orrenamed files after a configurable number of days. In addition, the agent offers the possibility of un-compressing (gzip) files after they have been collected, before they are inserted into the workflow.

When all the files have been successfully processed the agent stops, awaiting the next activation,scheduled or manually initiated.

4.1. Collection Strategy

In most cases, the default strategy should be used, and no custom Collection Strategy should bedefined.

Select Collection Strategy... from the Configuration menu on the desktop.

Figure 1. Collection Strategies configuration window.

Select Add to enable one or several from the available Strategies, then save.

Figure 2. Collection Strategies configuration window, Add.

There are two Strategies available by default. There may exist other customer adapted Strategies,however, these are not covered by this user guide. The default strategies are:

• Default Source File Handling

This Strategy represents the Source File Handling section in the Source tab where the collectiontreatment of the file is chosen. Please refer to Section 4.2.2, “Source Tab” for details.

8

SFTP Agent 4.1

• Default File Information

This Strategy represents the File Information section in the Source tab and where collection dir-ectory path and filename is specified using a single regular expression. Please refer to Section 4.2.2,“Source Tab” for details.

4.2. ConfigurationThe SFTP Collection agent configuration window is displayed when the node in a workflow is double-clicked or right-clicked, selecting Configuration.... Part of the configuration may be done in the File-name Sequence or Filename Sort Order service tabs described in the Convergent Mediation Engine™Core - User Guide.


Figure 3. SFTP Collection Agent configuration window, Connection tab.

The Connection tab contains configuration settings related to the remote host and authentication.

Primary host name or IP-address of the remote host to be connected. If a connec-tion cannot be established to this host, the additional hosts, specified in the Ad-vanced tab, are tried.

Host

Type of file system on the remote host. This information is used to construct theremote filenames.

File System Type



Choice of authentication mechanism. Both password and private key authentic-ation is supported.

Authenticate With

Username for an account on the remote host, enabling the SFTP session to login.Username

Password related to the specified Username. This option only applies whenpassword authentication is enabled.

Password

The Select... button will display a window where the private key may be inserted.If the private key is protected by a passphrase, the passphrase must be provided

Private Key

9

SFTP Agent 4.1

as well. This option only applies when private key authentication is enabled.Please see Section 3.2, “Authentication” for more information.

4.2.2. Source Tab

Figure 4. SFTP Collection Agent configuration window, Source tab.

The Source tab contains configurations related to the remote host, source directories and source files.The configuration available can be modified through the choice of a Collection Strategy. The belowtext describes the configuration options available when no custom strategy has been chosen.

If a custom collection strategy is needed, select this button to choose from predefinedCollection Strategies. The Collection Strategy chosen controls which further configurationis available in the Source tab. If no strategy is chosen, the default strategy is used.

CollectionStrategy

The below describes the configuration available when using the default strategy.

For further information on setting up a Collection Strategy, please refer to Section 4.1,“Collection Strategy”.

Absolute pathname of the source directory on the remote host, where the source filesreside. The pathname might also be given relative to the home directory of the UserName account.

Directory

Name of the source files on the remote host. Regular expressions according to Javasyntax applies. For further information, please refer to


Filename

Example 2.

To match all log filenames beginning with TTFILES, type: TTFILES.*

10

SFTP Agent 4.1

Compression type of the source files. Determines whether the agent will decompress thefiles before passing them on in the workflow or not.

Compres-sion

• No Compression - the agent agent will not decompress the files.

• Gzip - the agent decompresses the files using gzip.

If enabled, the source files will be moved to the automatically created subdirectoryDR_TMP_DIR in the source directory, prior to collection. This option supports safecollection of a source file reusing the same name.


If enabled, when the configured number of hours have passed without any file beingavailable for collection, a warning message (event) will appear in the System Log andEvent Area:

InactiveSourceWarning(hours)

The source has been idle for more than <n> hours, the lastinserted file is <file>.

If enabled, the source files will be moved from the source directory (or from the directoryDR_TMP_DIR, if using Move Before Collecting) after collection, to the directory spe-cified in the Directory field. If Prefix or Suffix are set, the file will be renamed as well.

Move to

If enabled, the source files will be renamed after the collection, remaining (or movedback from the directory DR_TMP_DIR, if using Move Before Collecting) in the sourcedirectory from which they were collected.

Rename

If enabled, the source files will be removed from the source directory (or from the direct-ory DR_TMP_DIR, if using Move Before Collecting), after the collection.

Remove

If enabled, the source files will remain in the source directory after the collection. Thisoption is not available if Move Before Collecting is enabled.

Ignore

Absolute pathname of the directory on the remote host into which the source files willbe moved after the collection. This field is only available if Move to is enabled.

(After Col-lection)Directory

The Directory has to be located in the same file system as the collected files atthe remote host. Also, absolute pathnames must be defined. Relative pathnamescannot be used.

Prefix and/or suffix that will be appended to the beginning and/or the end, respectively,of the source files after the collection. This field is only available if Move to or Renameis enabled.

Prefix andSuffix

If Rename is enabled, the source files will be renamed in the current directory(source or DR_TMP_DIR). Be sure not to assign a Prefix or Suffix, giving filesnew names still matching the Filename Regular Expression. That would causethe files to be collected over and over again.

Number of days to keep moved or renamed source files on the remote host after thecollection. In order to delete the source files, the workflow has to be executed (scheduledor manually) again, after the configured number of days.

Keep(days)

Note, a date tag is added to the filename, determining when the file may be removed.This field is only available if Move to or Rename is selected.

11

SFTP Agent 4.1

4.2.3. Advanced Tab

Figure 5. SFTP Collection Agent configuration window, Advanced tab.

The Advanced tab contains configurations related to more specific use of the SFTP service, whichmight not be frequently utilized.

The port number the SFTP service will use on the remote host.Port

The maximum time, in seconds, to wait for replies from commands. 0 (zero)means to wait forever.

Timeout (sec)

If selected, the agent overwrites the existing host key when the host is repres-ented with a new key. The default behaviour is to abort when the key mis-matches.

Selecting this option causes a security risk since the agent will acceptnew keys regardless if they possibly belong to another machine.

Accept New HostKeys

List of additional host names or IP-addresses that may be used to establish aconnection. These hosts are tried, in sequence from top to bottom, if the agentsfail to connect to the remote host set in their Source tabs.

Additional Hosts



4.3.1. Emits

Will be emitted right before the first byte of each collected file is fed into a workflow.Begin Batch

12

SFTP Agent 4.1

Will be emitted just after the last byte of each collected file has been fed into the system.End Batch

4.3.2. Retrieves


If the Cancel Batch behavior defined on workflow level is configured to abortthe workflow, the agent will never receive the last Cancel Batch message. Inthis situation, ECS will not be involved, and the file will not be moved, but leftat its current place.

APL code where Hint End Batch is followed by a Cancel Batch will alwaysresult in workflow abort. Make sure to design the APL code to first evaluate theCancel Batch criteria to avoid this sort of behavior.

If a Hint End Batch message is received, the collector splits the batch at the end of thecurrent processed block (as received from the server), provided that no UDR is split.

Hint EndBatch

If the block end occurs within a UDR, the batch will be split at the end of the precedingUDR.

After a batch split, the collector emits an End Batch Message, followed by a BeginBatch message (provided that there is more data in the subsequent block).



4.5.1. Publishes

A parameter containing a timestamp, indicating when the file transfer starts.File RetrievalTimestamp


A parameter containing the number of files that were available to this instancefor collection at startup. The value is static throughout the execution of the

Source File Count

workflow, even if more files arrive during the execution. The new files willnot be collected until the next execution.



Source Filename


A parameter containing the name of the host from which files are collected,as defined in the Source tab.

Source Host


A parameter containing the path from where the currently processed file wascollected, as defined in the Source tab.

Source Pathname

13

SFTP Agent 4.1


A parameter containing the login user name to the host from which the filewas collected, as defined in the Source tab.

Source Username


4.5.2. Accesses



4.6. Node Message EventsReady with file: filename

Reported along with the name of the source file that has been collected and inserted into the workflow.

File cancelled: filename

Reported along with the name of the current file, each time a Cancel Batch message is received. Thisassumes the workflow is not aborted; please refer to Section 4.3, “Transaction Behavior” for furtherinformation.

Removing already inserted file: filename

Reported along with the name of the source file, each time a file with the same name as the last collectedfile is detected. This can only happen during transaction recovery cleanup if the workflow abortedduring the last activation.

14

SFTP Agent 4.1

5. SFTP Forwarding AgentThe SFTP Forwarding agent forwards files to a remote host using the SFTP protocol over SSH2. Uponactivation, the agent establishes an SSH2 connection and an SFTP session towards the remote host.On failure, additional hosts are tried, if configured.

To ensure that downstream systems will not use the files until they are closed, they are maintained ina temporary directory on the remote host until the End Batch message is received. This behavior isalso used for Cancel Batch messages. If a Cancel Batch is received, file creation is cancelled.

5.1. ConfigurationThe SFTP Forwarding agent configuration window is displayed when the node in a workflow is double-clicked or right-clicked, selecting Configuration.... Part of the configuration may be done in the File-name Template service tab described in the Convergent Mediation Engine™ Core - User Guide.

5.1.1.Target Tab

Figure 6. SFTP Forwarding Agent configuration window, Target tab.

The Target tab contains configuration settings related to the remote host, target directories and targetfiles.

Primary host name or IP-address of the remote host to be connected. If a connec-tion cannot be established to this host, the additional hosts, specified in the Ad-vanced tab, are tried.

Host

Type of file system on the remote host. This information is used to construct theremote filenames.

File System Type



15

SFTP Agent 4.1

Choice of authentication mechanism. Both password and private key authentica-tion is supported.

Authenticate With

User name for an account on the remote host, enabling the SFTP session to login.Username

Password related to the specified Username. This option only applies whenpassword authentication is enabled.

Password

The Select... button will display a window where the private key may be inserted.If the private key is protected by a passphrase, the passphrase must be provided

Private Key

as well. This option only applies when private key authentication is enabled.Please see Section 3.2, “Authentication” for more information.

Absolute pathname of the target directory on the remote host, where the forwardedfiles will be placed. The pathname may also be given relative to the home direct-ory of the user's account.

Directory

The files will be temporarily stored in the automatically created subdirectoryDR_TMP_DIR in the target directory. When an End Batch message is received,the files are moved from the subdirectory to the target directory.

Compression type of the destination files. Determines whether the agent willcompress the output files as it writes them.

Compression

• No Compression - the agent agent will not compress the files.


Note that no extra extension will be appended to the target filenames, evenif compression is selected.

If enabled, the agent will create empty output files for empty batches rather thanomitting those batches.

Produce EmptyFiles

16

SFTP Agent 4.1

5.1.2. Advanced Tab

Figure 7. SFTP Forwarding Agent configuration window, Advanced tab.

The Advanced tabs contain configurations related to more specific use of the SFTP service, whichmight not be frequently utilized.

The port number the SFTP service will use on the remote host.Port

The maximum time, in seconds, to wait for replies from commands. 0 (zero)means to wait forever.

Timeout(sec)

If selected, the agent overwrites the existing host key when the host is repres-ented with a new key. The default behaviour is to abort when the key mis-matches.

Selecting this option causes a security risk since the agent will acceptnew keys regardless if they possibly belong to another machine.

Accept New HostKeys

List of additional host names or IP-addresses that may be used to establish aconnection. These hosts are tried, in sequence from top to bottom, if the agentsfail to connect to the remote host set in their Source tabs.

Additional Hosts



5.2.1. Emits

None.

17

SFTP Agent 4.1

5.2.2. Retrieves


Begin Batch

When an End Batch message is received, the target file in DR_TMP_DIR is closed and,finally, the file is moved from the temporary directory to the target directory.

End Batch


Cancel Batch



5.4.1. Publishes




A parameter containing the target filename, as defined in Filename Tem-plate.

Target Filename


A parameter containing the name of the target host, as defined in the Targetor Advanced tab of the node.

Target Hostname


A parameter containing the path to the target file, as defined in the FTP tabof the node.

Target Pathname


A parameter containing the login name of the user connecting to the remotehost, as defined in the FTP tab of the node.

Target Username


5.4.2. Accesses



18

SFTP Agent 4.1

5.5. Node Message EventsReady with file: filename

Reported, along with the name of the target file, when the file is successfully written to the target dir-ectory.

Sending file

Reported when the file transfer to the remote host starts.

Sending and gzip-compressing file

Reported when the file compression and transfer to the remote host starts.

19

SFTP Agent 4.1

SNMP Agent 4.1


Version 1.5

1





2

SNMP Agent 4.1



2. SNMP Agent ............................................................................................................. 52.1. Configuration .................................................................................................. 5

2.1.1. Connection Tab ..................................................................................... 52.1.2. Hosts Tab ............................................................................................. 6

2.2. Introspection ................................................................................................... 62.3. Meta Information Model .................................................................................... 62.4. Node Message Events ....................................................................................... 7

3

SNMP Agent 4.1

1. IntroductionThis document describes the SNMP agent being part of the extension agents, available on the NokiaConvergent Mediation Engine™ platform.



• SNMP MIB-2 (RFC 1213, http://www.ietf.org/rfc/rfc1213.txt, RFC 1573, ht-tp://www.ietf.org/rfc/rfc1573.txt)


4

SNMP Agent 4.1

2. SNMP AgentThe SNMP agent gathers interface information from MIB-2 agents, so called NMAs. The agent actsas a Network Management Station, NMS. When activated, the SNMP agent periodically polls allconfigured NMAs and requests information about all installed interfaces. For each interface data entry,a UDR will be created and inserted into the workflow, containing information about the interface.

The SNMP agent will automatically detect interfaces added to an NMA at the next polling interval,and its data will be available in a new UDR.

Figure 1. Example of an SNMP network.

Since MIB-2 agents does not offer the possibility of requesting historic data, the SNMP agent willlose all data that is available at the NMA while the NMS is not executing.

2.1. ConfigurationThe SNMP agent configuration window is displayed when the node in a workflow is double-clickedor right-clicked, selecting Configuration...


Figure 2. The SNMP agent configuration window, Connection tab.

The type of MIB-2 to be used, according to RFC 1213 or RFC 1573.MIB Type

5

SNMP Agent 4.1

The number of seconds to wait between each attempt to gather interface datafrom all MIB-2 agents. Default is 35 seconds.

Polling Interval (sec)

Optional authentication string, required by the MIB-2 agents.Community

Number of times a request is sent if no response is received. Once the limithas been reached, an entry is sent to the System Log. Also, the MIB-2 agent

Retries

is disabled for 15 minutes. That is, the SNMP agent will not request interfaceinformation from the MIB-2 agent during the next 15 minutes. Default numberof retries is 4.

Number of hundredths of a second between each retry attempt, defined inRetries. Default timeout is 35.

Timeout (1/100s)

Port on the MIB-2 agents to use for the communication. All MIB-2 agentsspecified in the Hosts tab have to use this port. Default SNMP port is 161.

Port

2.1.2. Hosts Tab

Figure 3. The SNMP agent configuration window, Hosts tab.

In the Hosts tab all MIB-2 agents, that the SNMP Collection agent will collect information from, arespecified.

The host names and/or IP-addresses of the MIB-2 agents to be checked at each polling interval.Hosts

The items in the list may be moved up or down, using the arrow buttons, to change thepresentation order of the entries.

2.2. IntrospectionThe agent produces one of the two predefined UDR Types:

• MIB_2_RFC_1213 (snmp)

• MIB_2_RFC_1537 (snmp)


6

SNMP Agent 4.1


2.4. Node Message EventsSuspension cancelled for host: host name

Reported when polling a host after it has been suspended.

Host: host name is suspended

Reported when polling a host which does not respond. A nonresponding host will then not be polledfor another 15 minutes.

7

SNMP Agent 4.1

TCP/IP Agent 4.1


Version 1.8

1





2

TCP/IP Agent 4.1



2. TCP/IP Collection Agent .............................................................................................. 52.1. Configuration .................................................................................................. 5

2.1.1. TCP/IP Tab ........................................................................................... 52.1.2. Decoder Tab ......................................................................................... 6

2.2. The TCPIP Format ........................................................................................... 72.3. Introspection ................................................................................................... 72.4. Meta Information Model .................................................................................... 72.5. Node Message Events ....................................................................................... 8

3. An Example ............................................................................................................... 93.1. Single TCP/IP Connection ................................................................................. 9

3.1.1. The TCP/IP Collection Node .................................................................... 93.1.2. The Analysis Node ............................................................................... 10

3.2. Multiple TCP/IP Connections ........................................................................... 103.2.1. The TCP/IP Collection Node .................................................................. 103.2.2. The Format Definition ........................................................................... 113.2.3. The Analysis Node ............................................................................... 11

3

TCP/IP Agent 4.1

1. IntroductionThis document describes the TCP/IP Collection agent, being part of the extension real-time agents,available on the Nokia Convergent Mediation Engine™ platform.



• TCP/IP


4

TCP/IP Agent 4.1

2.TCP/IP Collection AgentThe TCP/IP Collection agent allows data to be collected and inserted into a workflow, using thestandard TCP/IP protocol. Several connections at a time are allowed. It is also possible to send responsesback to the source in form of a bytearray or in the case with several connections as a UDR containinga response field. All response handling is done through APL commands.

Figure 1. A TCP/IP workflow may be configured to send responses to the source.

Upon activation, the collector binds to the defined port and awaits connections to be accepted. Notethe absence of a Decoder in the workflows. The collector has built-in decoding functionality, supportingany format as defined in the Ultra Format Editor.

2.1. ConfigurationThe TCP/IP Collection agent configuration window is displayed when the node in a workflow isdouble-clicked or right-clicked, selecting Configuration...

2.1.1.TCP/IP Tab

Figure 2. TCP/IP Collection agent configuration window, TCP/IP tab.

The IP address or hostname to which the TCP collector will bind. If left emptyTCP collector binds to all IP addresses available on the system.

Host

The port number from which the data is received. Make sure the port is not usedby other applications.

Port

The port can also be changed while the agent is running. Double click theagent in the Workflow Editor in monitor mode and modify it. To triggerthe agent to use the new port the workflow must be saved. See the Coreuser guide for more information on updating agent configurations whilethe workflow is running.

5

TCP/IP Agent 4.1

If enabled, several TCP/IP connections are allowed simultaneously. If disabled,only one at a time is allowed.

Allow MultipleConnections

If Allow Multiple Connections is enabled, the maximum number of simultaneousconnections is specified as a number between 2 and 999.

Number of Con-nections Allowed

If enabled, the collector will be able to send a response back to the source. Theresponse will be queued if the response is not read by the TCP client. When the

Send Response

queue is full the collector stop reading data and will close the socket after a timeoutwith an entry written to the system log. If Allow Multiple Connections is enabled,the collector expects a UDR extended with the default TCPIP UDR as reply. Ifdisabled, it expects a bytearray.

Drag and release in the opposite direction in the workflow to create a re-sponse route between the nodes. The TCP/IP node must be connected to anode utilizing APL. This since responses are created with APL commands.

Figure 3.

Please refer to Section 3, “An Example” for a description on differences betweensingle and multiple connection setups.

2.1.2. Decoder Tab

Figure 4. TCP/IP Collection agent configuration window, Decoder tab.

List holding available decoders introduced via the Ultra Format Editor. The decodersare named according to the following syntax:

Decoder

<decoder> (<module>)

The option MZ format tagged UDRs indicate that the expected UDRs are stored inthe Convergent Mediation Engine™ specific format. If this option is selected, theTagged UDR Type list will be enabled.

List of available internal UDR formats stored in the Ultra and Code servers. The formatsare named according to the following syntax:

Tagged UDRType

<internal> (<module>)

If the decoder is to reprocess UDRs of an internal format, the Decoder MZ formattagged UDRs has to be selected to enable this list. Once enabled, the internal formatmay be selected.

6

TCP/IP Agent 4.1

If enabled, the UDR will be fully decoded before inserted into the workflow. This mayhave a negative impact on performance since all fields may not be accessed in theworkflow, making decoding of all fields unnecessary.

Full Decode

If disabled (default), the amount of work needed for decoding is minimized, using a"lazy" method decoding sub fields. This means the actual decoding work may not bedone until later in the workflow, when the field values are accessed for the first time.Corrupt data (that is, data for which decoding fails) may not be detected during thedecoding stage and could cause the UDR to be discarded at a later processing stage.

2.2.The TCPIP FormatIn case Allow Connections and Send Response are selected, UDRs are expected as reply back to thecollector from the APL node. Extend the internal format to contain the built-in TCPIP format.

external my_ext sequential { // field definitions};

internal TCP_Int : extends_class( "com.digitalroute.wfc.tcpipcoll.TCPIPUDR" ) {};

in_map TCP_InMap : external( my_ext ), internal( TCP_Int ), target_internal( my_TCP_TI ) { automatic;};

decoder myDecoder : in_map( TCP_InMap );

2.3. IntrospectionThe agent produces UDRs in accordance with the Decoder tab. If Send Response is enabled, the agentconsumes bytearray types for single connections and TCPIPUDR for multiple connections.



7

TCP/IP Agent 4.1


8

TCP/IP Agent 4.1

3. An ExampleA workflow containing a TCP/IP Collection agent can be set up to send responses back to the sourcefrom which the incoming data was received. This requires an APL node (Analysis or Aggregation) tobe part of the workflow.

Figure 5. A TCP/IP workflow can be configured to send responses to the source.

To illustrate how such a workflow is defined, an example is given where an incoming UDR is validated,resulting in either the field anum or a sequence number being sent back as a reply message to thesource. Depending on if one or several TCP/IP connections are allowed, the format of the reply messagesent from the Analysis node differs:

A bytearray is sent back as reply.Single Connection

A UDR, extended with the built-in TCPIP UDR format. The reply messagemust be inserted in the response field (a bytearray).

Multiple Connections

To keep the example as simple as possible, the valid records are not processed. Usually, no replyis sent back until the UDRs are fully validated and processed. The example aims to focus onthe response handling only.

3.1. Single TCP/IP ConnectionDisabling the Allow Connections option in the TCP/IP Collection node, will allow only one TCP/IPsession at a time. If another attempt to create a connection is made while a connection already exists,the new connection will be rejected.

3.1.1.The TCP/IP Collection Node

In order to be able to send reply messages, Send Response must be enabled in the configuration windowof the node. Drop an Analysis node in the workflow and connect it to the TCP/IP node. Drag and releasein the opposite direction to create a response route in the workflow.

Also, an Ultra format for decoding of incoming data must be defined. Note, no format has to be definedfor the response - it will be sent as a bytearray from the Analysis node.

Figure 6. TCP/IP node configuration.

9

TCP/IP Agent 4.1

3.1.2.The Analysis Node

The Analysis node handles both the validation of the incoming records, as well as sending the response.If the field duration is less than or equal to zero, the UDR is discarded and the field anum, in formof a bytearray, is sent back as response. All other UDRs are routed to the next node in turn, andinstead a sequence number is sent as response.

Note the use of the synchronized keyword. Updating a global variable within a real-time workflowmust be done within a synchronized function. This to assure consistency between threads. Bydefault, a real-time workflow utilizes several threads.

int seqNum;

synchronized int createSeqNum() { seqNum = seqNum + 1; return seqNum;}

consume { bytearray reply;

if ( input.duration <= 0 ) { strToBA( reply, input.anum ); udrRoute( reply, "response" ); } else { strToBA( reply, (string)createSeqNum() ); udrRoute( reply, "response" ); udrRoute( input, "UDRs" ); }

}

3.2. Multiple TCP/IP ConnectionsEnabling the Allow Multiple Connections option in the TCP/IP Collection node, will allow severalsimultaneous TCP/IP sessions at a time. If an attempt to open a new connection is made when themaximum number of connections are already open, the new connection will be refused.

3.2.1.The TCP/IP Collection Node

In order to be able to send reply messages, Send Response must be enabled in the configuration windowof the node. An additional connection point will appear on the node, to which an Analysis node is tobe linked. Also, an Ultra format for the decoding of the incoming data must be defined. This formatmust contain the built-in TCPIP format (refer to the following section).

10

TCP/IP Agent 4.1

Figure 7. TCP/IP node configuration.

3.2.2.The Format Definition

The incoming external format must be extended with the TCPIP UDR format.

external asciiSEQ_ext sequential { ascii callId: terminated_by(":"); int seqNum: terminated_by(","); ascii anum: terminated_by(","); ascii bnum: terminated_by(","); ascii causeForOutput: terminated_by(","); int duration: terminated_by(0xA);};

internal TCP_Int : extends_class( "com.digitalroute.wfc.tcpipcoll.TCPIPUDR" ) {};

in_map TCP_InMap : external( asciiSEQ_ext ), internal( TCP_Int ), target_internal( ascii_TCP_TI ) { automatic;};

out_map ascii_TCP_outMap : external( asciiSEQ_ext ), internal( ascii_TCP_TI ) { automatic;};

decoder TCPData : in_map( TCP_InMap );

encoder TCPData : out_map( ascii_TCP_outMap );

3.2.3.The Analysis Node

The Analysis node handles both the validation of the incoming records, as well as sending the response.If the field duration is less than or equal to zero, the UDR is discarded and the field anum is inserted

11

TCP/IP Agent 4.1

into the response field, and the complete UDR is sent back as response. All other UDRs are routed tothe next node in turn and a sequence number is inserted into the response field before any routing.

Note the use of the synchronized keyword. Updating a global variable within a real-time workflowmust be done within a synchronized function. This to assure consistency between threads. Bydefault, a real-time workflow utilizes several threads.

int seqNum;

synchronized int createSeqNum() { seqNum = seqNum + 1; return seqNum;}

consume { bytearray reply;

if ( input.duration <= 0 ) { strToBA( reply, input.anum ); input.response = reply; udrRoute( input, "response" ); } else { strToBA( reply, (string)createSeqNum() ); input.response = reply; udrRoute( input, "response" ); udrRoute( input, "UDRs" ); }

}

12

TCP/IP Agent 4.1

Ultra Format Management 4.1


Version 1.35

1





2




2. Format Management Overview ...................................................................................... 72.1. Format Storage ................................................................................................ 72.2. Decoding of a UDR .......................................................................................... 8

3. Ultra Format Editor ..................................................................................................... 94. Decoder Agent .......................................................................................................... 11


4.2.1. Emits ................................................................................................. 124.2.2. Retrieves ............................................................................................ 12

4.3. Introspection .................................................................................................. 124.4. Meta Information Model .................................................................................. 124.5. Node Message Events ...................................................................................... 12

5. Encoder Agent .......................................................................................................... 135.1. Configuration - Batch Workflow ........................................................................ 135.2. Configuration - Realtime Workflow ................................................................... 135.3. Transaction Behavior - Batch workflow .............................................................. 14

5.3.1. Emits ................................................................................................. 145.3.2. Retrieves ............................................................................................ 14

5.4. Introspection .................................................................................................. 145.5. Meta Information Model .................................................................................. 145.6. Node Message Events ...................................................................................... 145.7. Agent Services - Batch Workflow ...................................................................... 14

6. UDR File Editor ....................................................................................................... 166.1. Editing a UDR ............................................................................................... 18

6.1.1. Advanced Edit - List and Update UDR Fields ............................................ 186.2. Editing a Bulk of UDRs ................................................................................... 19

6.2.1. Add Matcher ....................................................................................... 206.2.2. Add Assignment .................................................................................. 21

7. UDR Views .............................................................................................................. 238. Ultra Format Converter .............................................................................................. 24

8.1. Conversion Rules ........................................................................................... 248.2. Disabling Automatic Conversion ....................................................................... 24

9. Introduction to the Ultra Format Definition Language ...................................................... 269.1. Building Blocks ............................................................................................. 26

9.1.1. External ............................................................................................. 289.1.2. Internal and In-Map .............................................................................. 299.1.3. Decoder ............................................................................................. 309.1.4. Out-map ............................................................................................. 319.1.5. Encoder ............................................................................................. 329.1.6. Event Types ........................................................................................ 32

9.2. The Ultra Module ........................................................................................... 329.2.1. The import Statement (APL Code) ....................................................... 339.2.2. The import Statement (UFDL Code) ..................................................... 33

10. External - Sequential Formats .................................................................................... 3610.1. Record Declaration ....................................................................................... 36

10.1.1. terminated_by ............................................................................. 3710.1.2. dynamic_size ............................................................................... 3710.1.3. static_size ................................................................................. 3710.1.4. identified_by ............................................................................. 38

10.2. Field Declarations ......................................................................................... 3810.2.1. Primitive Field Types .......................................................................... 3810.2.2. Primitive Field Options ........................................................................ 39

3


10.2.3. Field Size Specification Options ............................................................ 3910.2.4. Field Options for Optional Fields .......................................................... 4010.2.5. Other Field Options ............................................................................ 4010.2.6. udr_size and remaining_size .................................................... 4010.2.7. Bit Blocks ......................................................................................... 4110.2.8. Constructed Types .............................................................................. 4110.2.9. set Construct ................................................................................... 4210.2.10. switched_set Construct ............................................................... 4210.2.11. Encoding Specifications and Expressions .............................................. 44

10.3. Expressions ................................................................................................. 4611. External - Ericsson IOG/IN Records ........................................................................... 47

11.1. Record Declaration ....................................................................................... 4711.2. Field Declaration .......................................................................................... 47

12. External - ASN.1/BER Formats .................................................................................. 4812.1. Notes on ASN.1 ............................................................................................ 48

12.1.1. Inter Module References ...................................................................... 4812.1.2. ASN.1 Primitive Type Mapping ............................................................. 48

12.2. Ultra Extensions ........................................................................................... 4912.2.1. Direct BCD Support ............................................................................ 4912.2.2. Date Support ..................................................................................... 4912.2.3. Using Sequential Record Types ............................................................. 4912.2.4. bigint Support ................................................................................ 50

12.3. ASN Language Limitations ............................................................................. 5112.4. BER Limitations ........................................................................................... 51

13. External - XML Records ........................................................................................... 5313.1. XML Primitive Type Mapping ......................................................................... 5313.2. IPDR Compliance ......................................................................................... 5413.3. Limitations .................................................................................................. 54

14. Internal Formats ...................................................................................................... 5514.1. Class Specifications ...................................................................................... 5614.2. Event types .................................................................................................. 57

15. In-Maps ................................................................................................................. 5815.1. Examples of Non-Automatic Maps ................................................................... 5915.2. Automatic Maps ........................................................................................... 60

15.2.1. target_internal ......................................................................... 6115.2.2. Automatic Mapping Specifications ........................................................ 6115.2.3. use_external_names Automatic Option .......................................... 6415.2.4. Automatic Block Internal Specification ................................................... 64

15.3. discard_output Specification ................................................................... 6516. Out-Maps .............................................................................................................. 66

16.1. Optional Fields ............................................................................................. 6617. Decoders ............................................................................................................... 69

17.1. Simple Decoders ........................................................................................... 6917.2. Constructed Decoders .................................................................................... 70

18. Encoders ................................................................................................................ 7119. A Constructed Decoder Example ................................................................................ 72

19.1. Headers and Trailers ...................................................................................... 7219.2. internal ................................................................................................. 7219.3. external ................................................................................................. 7319.4. in_map ..................................................................................................... 7419.5. decoder ................................................................................................... 7419.6. out_map ................................................................................................... 7519.7. encoder ................................................................................................... 75

20. A Sequential Format Example .................................................................................... 7620.1. external ................................................................................................. 76

20.1.1. Alternative Syntax .............................................................................. 7920.2. internal ................................................................................................. 8020.3. in_map ..................................................................................................... 80

4


20.4. decoder ................................................................................................... 8121. An ASN.1 Format Example ....................................................................................... 82

21.1. Constructed Internal ...................................................................................... 8221.1.1. external ....................................................................................... 8221.1.2. in_map and out_map ...................................................................... 8321.1.3. decoder and encoder .................................................................... 83

21.2. Extending the target_internal ................................................................ 8321.3. Creating a New UDR ..................................................................................... 86

21.3.1. external ....................................................................................... 8621.3.2. internal ....................................................................................... 8721.3.3. in_map and out_map ...................................................................... 8721.3.4. decoder and encoder .................................................................... 87

22. An XML Format Example ......................................................................................... 8922.1. external ................................................................................................. 8922.2. in_map ..................................................................................................... 9022.3. decoder ................................................................................................... 91

5


1. IntroductionThis document describes the Ultra Format Management utilities, a standard part of the ConvergentMediation Engine™ platform. This includes a detailed description of the Ultra Format DefinitionLanguage and how to create format definitions in the language syntax.





6


2. Format Management OverviewConvergent Mediation Engine™ is capable of handling various UDR formats provided that theirstructure is known. Vendor UDR formats are often complex and hard to work with in agents that willview or manipulate data. Therefore, Convergent Mediation Engine™ converts the external UDRformats to an internal representation, and vice versa when UDRs are leaving the system.

In order to make this conversion, both internal and external format definitions have to be defined.When both internal and external definitions exist, there has to be a mapping description that maps eachexternal field of interest to an internal field.

Ultra does not require a one-to-one relationship between external and internal field names. Normally,only fields of interest or those that are needed when leaving the system are declared in the internalformat definition. Often the internal definition holds additional fields that initially are empty but arepopulated on the way through a workflow. Convergent Mediation Engine™ does not place any con-straints on how these fields are used.

In this example the external field E1C is not needed so it is not mapped at all. Internally wemight manipulate IFA and IFB. We will somehow give IFE a value, probably depending onvalues defined in IFC and IFD during the UDRs way through the workflow. At exit, we mapthe two original fields into E2A and E2B plus the new (IFE) into E2C.

E1A

E1B

E1C

IFA

IFB

IFC

IFD

IFE

E2A

E2B

E2C

Convergent Mediation Engine™ introduces a language called Ultra Format Definition Language(UFDL) that is used when describing UDR formats and mappings.

Apart from describing external formats, internal formats and mappings, the UFDL requires separatedecoder and encoder specifications. A decoder or encoder refers to one or several mappings that areused for the actual translation.

2.1. Format StorageIn order to operate on UDRs, there has to be a compiled version (Java class) of the internal formatdefinition available in the Code or Ultra server. Such Java classes are introduced in the servers in twodifferent ways:

1. Format definitions entered in the Ultra Format Editor are compiled and inserted in the Ultraserver when saved.

2. Precompiled format definitions (for instance; Radius, NetFlow, and SNMP protocols) are insertedin the Code server during package commit.

7


A format can be recompiled while workflows using it are running. In this case, the changes to theformat will take effect the next time the workflow is activated.

All Java classes generated by the Ultra server are kept in the system. This is to make UDRs, for instancestored in ECS, using the old format version reprocessable. Refer to Section 8, “Ultra Format Converter”

Saving a format in the Ultra Format Editor may take some time, since the system may haveto revalidate and regenerate many dependent configurations.

If an Ultra format is moved to another folder or is renamed, the configurations using the formatwill become invalid.

2.2. Decoding of a UDRThe Ultra engine is designed to decode only the necessary parts of a UDR, to keep workflow perform-ance as high as possible. By default (that is, provided that Full Decode is not selected), the decoderwill only decode enough to calculate the size of the UDRs. The UDR contents will be decoded as thefield values are accessed. Hence, if there is a decoding error, this may not be discovered by the decoderbut by the node accessing a specific field.

Full decoding of complicated nested UDR structures can be time consuming, but in most casesthe performance cost of this option is small.

8


3. Ultra Format EditorThe Ultra Format Editor allows users to introduce format definitions in the system to be used bynodes in workflows. Definitions are described using the Ultra Format Definition Language (UFDL).

A saved format definition will have its name displayed in the title. Besides selecting Decoder, Encoder,and UDR Type definitions, formats can be imported from APL code, as well as directly from otherUltra definitions:

import ultra.folder_name.module_name; // From APL codeimport folder_name.module_name; // From UFDL code

The format editor is displayed when Ultra Format... is selected from the Configuration menu on thedesktop.

Figure 1. Ultra Format Editor window.

The editor will try compiling the entered code, checking for grammatical andsyntactical errors. If errors are detected, a message is displayed informing aboutthe nature and location of the problem.

Edit menu Validate

The text is color coded according to the following definitions:Code Definition

Brown - Strings.

Blue (dark) - Functions.

Blue (light) - Constants.

Green - Types.

Orange - Own (user defined) types.

Purple - Key words.

Red - Comments.

9


User interface items not listed are considered to be of general type. Refer to Convergent MediationEngine™ Core - User Guide, for further information.

10


4. Decoder AgentThe Decoder agent converts raw data to internal UDRs, based on information from a UFDL decoderdefinition or MZ format tagged UDRs (the Convergent Mediation Engine™ specific general format).

4.1. ConfigurationThe Decoder configuration window is displayed when a Decoder node is double-clicked, or right-clicked selecting Configuration....

Figure 2. Decoder configuration window, Decoder tab.

List of available decoders introduced via the Ultra Format Editor, as well as thedefault built-in decoder for the Convergent Mediation Engine™ internal format (MZformat tagged UDRs).

Decoder

If the decoder for the MZ format tagged UDRs format is chosen, the Tagged UDRtype list is enabled.

List of the available internal UDR formats, stored in the Ultra and Code servers.Tagged UDRtype

Options to control how to react upon decoding errors.On Error

• Cancel Batch - The entire batch is cancelled. This is the standard behavior.

• Route Raw Data - Route the remaining, undecodable, data as raw data. This optionis useful if you want to implement special error handling for batches that are par-tially processed.

If enabled, the UDR will be fully decoded before output from the decoder node. Thisaction may have a negative impact on performance, since not all fields may be accessed

Full Decode

in the workflow, making decoding all fields in the UDR unnecessary. If it is importantthat all decoding errors are detected, this option must be enabled.

If this option is disabled (default), the amount of work needed for decoding is minim-ized using "lazy" decoding of field contents. This means that the actual decoding workmay not be done until later in the workflow, when the field values are accessed forthe first time. Corrupt data (that is, data for which decoding fails) may not be detected

11


during the decoding stage, but may instead cause a workflow to abort at a later pro-cessing stage.

The use and settings of private threads for an agent, enabling multi-threading within a workflow,are configured in the Thread Buffer tab. Please refer to Convergent Mediation Engine™ Core- User Guide, for further details.


4.2.1. Emits

Emitted on failure to decode the received data.Cancel Batch

4.2.2. Retrieves

Unless this has been generated by a Hint End Batch message, the decoder checks that allthe data in the batch was decoded. When using a constructed or blocked decoder, the de-coder does additional validation of the structural integrity of the batch.

End Batch

4.3. IntrospectionThe agent produces one or more UDR types depending on configuration and consumes bytearraytype.

4.4. Meta Information ModelThe agent does not itself publish or access any MIM resources. For a listing of general ConvergentMediation Engine™ MIM resources, refer to Convergent Mediation Engine™ Core - User Guide.


Splitting batch

Emitted when the Hint End Batch event occurs.

12


5. Encoder AgentThe Encoder agent converts internal UDRs to raw data. It can be used both in Batch and Realtimeworkflows. The main difference between these two modes of operation is that within a Batch workflowthe Encoder agent is capable of adding headers and trailers.

5.1. Configuration - Batch WorkflowThe Encoder configuration window is displayed when an Encoder node is double-clicked or right-clicked, selecting Configuration....

Figure 3. Encoder configuration window, Encoder tab.

If enabled, the agent will not encode the incoming data. It expects a raw bytearray as the input type and will pass it through untouched. This mode is usedwhen only a header and/or a trailer is added to a data batch.

Suppress Encoding

List of available encoders introduced via the Ultra Format Editor, as well asthe default built-in encoder for the Convergent Mediation Engine™ internalformat (MZ format tagged UDRs).

Encoder

The Header and Trailer tabs are described in Section 5.7, “Agent Services - Batch Workflow”.The use and setting of private threads for an agent, enabling multi-threading within a workflow,is configured in the Thread Buffer tab. Please refer to Convergent Mediation Engine™ Core- User Guide, for further details.

5.2. Configuration - Realtime WorkflowThe Encoder configuration window is displayed when an Encoder node is double-clicked or right-clicked, selecting Configuration....

Figure 4. Encoder configuration window - Realtime workflow, Encoder tab.

List of available encoders introduced via the Ultra Format Editor, as well as the defaultbuilt-in encoder for the Convergent Mediation Engine™ internal format (MZ format taggedUDRs).

Encoder

13


5.3.Transaction Behavior - Batch workflow

5.3.1. Emits

None.

5.3.2. Retrieves

Possible headers defined in the Header tab are created and dispatched on all outgoingroutes before the first UDR is encoded.

Begin Batch

Possible trailers defined in the Trailer tab are created and dispatched on all outgoingroutes after the last UDR has been encoded.

End Batch

5.4. IntrospectionThe agent produces bytearray type and consumes the UDR types corresponding to the selected Encoder.If Suppress Encoding is enabled bytearray type is consumed.


None.Publishes

None.Accesses


5.7. Agent Services - Batch WorkflowThe agent utilizes two specialized services allowing the user to add header and trailer information intoeach data batch.

They both offer the possibility of using MIM values, constants and user defined values in the headeror trailer. When selecting MIM resources, note that MIM values used in the data batch header aregathered when a new batch begins, while MIM values used in the data batch trailer are gathered whena batch ends. Thus, the numbers of outbound bytes or UDRs for any node will always be zero if theyare referenced in data batch headers.

The windows for both header and trailer configuration are identical.

Figure 5. Encoder configuration window - Header tab.

14


Indicates if the header/trailer will be added to the batch even if the batch doesnot contain any data (UDRs or byte arrays).

Suppress On No Data

Select Add to populate the columns with items to be added to the header ortrailer of the file. They will be added in the order they are specified.

Value

Figure 6. Add Header/Trailer Content dialog.

If enabled, a MIM value will be part of the header. Size and Padding must be enteredas well.

MIM Defined

Note that for data batch headers, the MIM values are gathered at beginBatch.

If enabled, a user defined constant must be entered. If Size is empty or less than thenumber of characters in the constant, Size is set to the number of characters in the

User Defined

constant. If Size is greater than the length of the constant, Padding must be enteredas well.

If enabled, a string is added according to the value entered for Size, filled with Paddingcharacters.

Pad Only

Size must always be entered to give the item a fixed length. It can only be omitted ifUser Defined is selected, in which case it will be calculated automatically.

Size

Character to pad any remaining space with. Either a user defined character can beentered, or one of the four predefined/special characters can be selected (Carriagereturn, Line feed, Space, Tabulator).

Padding

Left or right alignment within the allocated field size.Alignment

Enabled when a MIM of type date is selected. A Date Format Chooser dialog isopened, where a date format may be entered.

Date Format

15


6. UDR File EditorThe UDR File Editor allows the user to view and edit the contents of UDR files.

The UDR files may be of any format as long as it has an associated decoder. The Open dialog enablesdecoder selection. Correspondingly, the Save dialog includes a choice of which encoder to use.

Since UDRs often contain a large number of fields, it is possible to create an UDR View to selectfields to be viewed directly as columns in the UDR list. (select UDR View... in the Configurationmenu). Each UDR can also be opened in a separate window where all fields are available.

The UDR File Editor will by default only support input files up to size 3MB. If the file sizeexceeds this limit, the UDR File Editor will only read up to 3MB of the file and will then stop,showing only the read records.

To support decoding of files larger than 3MB, a property needs to be set in the desktop.xmlfor the corresponding desktop; mz.gui.udreditor.limit. Setting the property to yesmeans the 3MB limit applies. Setting the property to no will bypass the limitation mechanism,which may potentially cause out of memory errors in the desktop.

The UDR File Editor is displayed when UDR File Editor... is selected from the Tools menu on thedesktop.

Figure 7. UDR File Editor window.

Items not listed are considered to be of general type, refer to Convergent Mediation Engine™ Core -User Guide for further information.

16


Displays a dialog where the local file system can be browsed for files to open.The Decoder to use when reading the file is also configured here.

File menu Open

Full Decode is enabled by default, which will ensure that any decoding errorsare detected when the file is read. If this option is disabled, decoding errorsmay not be detected until the UDR fields are accessed.

Saves the selected UDRs in an ASCII file, which may be opened in any texteditor. All UDRs are displayed as rows, where the fields are separated by anyof the following:

File menu Export

• Comma

• Semicolon

• Tab

Non-existing fields are marked [null]. Note that the first line will alwayshold the field names.

Prints a summary table based on the current UDR selection (the Table option).It is also possible to get a detailed printout (the Details option). In the latter

File menu Print

case the UDRs will be presented the way they appear when the Edit button ispressed.

Opens a new dialog from which a group of selected UDRs can be edited simul-taneously. Refer to Section 6.2, “Editing a Bulk of UDRs” for further informa-tion.

Edit menu BulkEdit...

A list from which a UDR view may be selected. A UDR view is a user definedfilter which displays additional columns containing UDR field data. If (None)

UDR View

is selected, the list will consist of two columns only: Pos and Type. However,if a file is opened a third column Orig Pos becomes visible as well. Refer toSection 7, “UDR Views” for further information.

Shows the original position of the UDR. New indicates that the UDR did notexist prior to the last Save.

Orig Pos

The current position of the UDR. The order applies upon saving.Pos

The UDR types.Type

17


All columns following the Type column, will list the field names of the UDRtype according to the UDR view selection. If there is no field with the definedname, No such field is stated.

Field Name

6.1. Editing a UDRDouble-clicking a UDR will result in a new window, containing the complete UDR which allows thecontained data to be edited.

Figure 8. Edit UDR window.

The name of the UDR fields. A black field (MANDATORY) can be edited,however, Present cannot be disabled. A red field (OPTIONAL) can be edited

Name

as well as set to Present. Note that removing a field containing sub-fields andthen creating it again will not recreate the sub-fields. The same applies to elementsin a list field.

The special field OriginalData holds the raw UDR input data.

Holds the field values. To edit a value, select it and type in the new value.Value

Shows whether the fields are present in the UDR View or not.Present

Only display present fields (as specified by the Present flag)Show Only Present

6.1.1. Advanced Edit - List and Update UDR Fields

It is also possible to add and remove elements from fields of list type, as well as to create UDRs forfields of UDR type. Place the pointer on the list/UDR field (the Value column) and right-click to selectAdvanced Edit....

18


Figure 9. Edit UDR window - editing byte array fields.

Sets the field value to NULL.Null

Opens up a new dialog, where the field value may be changed. To append values,click on the subsequent byte and type in the value. In this case (byte array) onlyhexadecimal digits are allowed.

Advanced Edit...

6.2. Editing a Bulk of UDRsBesides editing UDRs one at a time, it is also possible to edit a group of UDRs at once. Select theUDRs of interest, or none (this way all UDRs within the file will be considered) and select Bulk Edit...from the Edit menu.

Figure 10. The Bulk Editor.

Runs the configured queries.Edit menu Activate

19


Displays a new dialog, where the resulting APL code may be examined.Select Manual APL Edit to enable manual change of the APL code.

View menu Show APL...

Displays the result of the bulk edit. This dialog also pops up automat-ically when Activate has finished. The UDR Status column gives theeffect of the edit operation for each UDR.

View menu Show Edit Res-ult...

A list from which a UDR type can be selected.UDR Type

Selecting this feature will disable both the Matchers and Assignments,and allow the user to write custom APL code in the Bulk Editor - APLView window.

Manual APL Edit

When editing UDRs in manual mode, there will be no traceabilityon what UDRs were changed. All UDRs in the batch will beupdated, although their values may not be changed.

Defines the UDRs to be targeted for update. Refer to Section 6.2.1,“Add Matcher” for further information on adding Matchers.

Matchers

Indicates if All or Any will apply to the conditions in the Matcherslist.

Join Style

Specifies the fields to examine when defining target UDRs.Field

The expression to apply when matching.Matcher Configuration

Defines the changes to be made to the targeted UDRs. Refer to Sec-tion 6.2.2, “Add Assignment” for further information on adding Assign-ments.

Assignments

Fields to edit.Field

The new value of the field.Assignment Configuration

6.2.1. Add Matcher

A Matcher is added by selecting Add in the Matchers section. The following dialog is displayed:

Figure 11. Add Field Matching window.

The new value of the field.UDR Field

20


Defines the check criterion for the field. The contents of the list vary with field type.Supported types and some of their possible matchers:

Matcher Type

• Alphanumeric (string, char) - Contains, Begins With, Matches RegExp etc.

• Date - Equals, Before, After etc.

• Numeric - Not Equals, Greater Than, Less Than etc.

Some field types can be made up of several parts. For instance, date types can containseparate values for year, month, and day. Regular numeric/alphanumeric fields willhave Value stated.

Parameter

Indicates the possibility to enter variable values for the validation field via the useof APL code in the Value column. This feature makes it possible to refer to other

Is APL

fields within the same UDR (as well as call to APL functions) during processing. Inthe following figure, the P155_PackageLength field is of type int. If Is APLis disabled, the Matcher Configuration is interpreted as an int instead of a function.

The value of the field. By default, a constant is entered. If Is APL is selected, APLsyntax can be used to create a variable definition.

Value

6.2.2. Add Assignment

An Assignment is added by selecting Add in the Assignments section. The following dialog is dis-played:

Figure 12. Add Field Assignment window.

List of available fields. Depending on the field type, the Assignment Type listwill update to display valid choices.

UDR Field

Defines the changes to be made to the field. The contents of the list vary with fieldtype. Supported types and some of their possible assignments:

Assignment Type

• Alphanumeric (string, char) - Assign, Append, Replace, etc.

• Date - Assign Current Date, Increase/Decrease Year etc.

• Numeric - Assign, Increase etc.

Some field types can be made up of several parts. For instance, date types cancontain separate values for year, month, and day. Regular numeric/alphanumericfields will have Value stated.

Parameter

Indicates the possibility to enter variable values for the validation field via the useof APL code in the Value column. This feature makes it possible to refer to otherfields within the same UDR (as well as call to APL functions) during processing.

Is APL

21


The value of the field. By default, a constant is entered. If Is APL is selected, APLsyntax can be used to create a variable definition.

Value

22


7. UDR ViewsFrom the UDR View Editor, it is possible to define filters to control how UDR data is listed andpresented in the UDR File Editor main window and the Aggregation Session Inspector.

Figure 13. UDR File Editor with no UDR View applied.

Applying a view will add the fields of the view as columns. A UDR View does not have to be applicableon all UDR types within a displayed file. If a field does not exist for a particular UDR, No such fieldis displayed.

Figure 14. UDR File Editor with a UDR View.

To display the UDR View Editor, select UDR View... from the Configuration menu on the desktop.

Figure 15. UDR View Editor window.

A list from which a UDR type is selected.UDR Type

Displays the selected fields that make up the view.UDR Fields

23


8. Ultra Format ConverterWhen an Ultra Format Definition is updated, the Convergent Mediation Engine™ system by defaultautomatically updates persistent data when it is accessed, to reflect the changes. The automatic conver-sion can be disabled completely, or enabled only for a selection of formats. It is also possible to setdefault values for added fields. The access group is permitted to launch and maintain the data in theUltra Format Converter.

The following list of Convergent Mediation Engine™ applications shows what agents can handlepersistent data, and how they handle historic (outdated) UDRs when the automatic conversion is enabled(or the specific format is part of the selection in the Ultra Format Converter).

Collected UDRs are updated.Database Collection Agent

Processed UDRs are updated.Decoder (using MZ Tagged)

All UDRs belonging to the same format definition - historic and updated- will be considered valid for comparison.

Aggregation Sessions

In case historic UDRs exist within old sessions, or the session formatitself has been updated, the session and its content is updated the nexttime it is accessed.

UDRs will be updated whenever they are read from ECS by the ECSCollector or accessed through the Error Correction System Inspector.

ECS

Attempts to collect, process or update historic UDR data when the automatic conversion is dis-abled for a specific format will cause workflow abort (or an exception when done outsideworkflow processing).

8.1. Conversion RulesFields removed from the format definition are deleted and cannot be retrieved. Added fields will bydefault be set to 0 (zero) (numeric types), false (boolean types), or null (all other types).Changing the type of existing fields may cause the field contents to be deleted. For instance, changingfrom string to int will set the field to 0 (zero), provided that no valid default value is defined.

Only numerical types (int, long, etc) are directly interchangeable. If a numeric value exceeds thenumber of bytes allowed in the new type, it will be truncated.

8.2. Disabling Automatic ConversionIt is possible to disable the automatic conversion to new format versions, and instead define defaultvalues for UDR fields added in the new version. This is done from the Ultra Format Converter...which is accessed from the Tools menu on the Convergent Mediation Engine™ desktop:

24


Figure 16. Ultra Format Converter window.

Indicates the UDR types for which the conversion is enabled. There are two options:Convert

• Selected Only - Only the formats listed in this window will be considered.

• All - All formats are considered, regardless of the types listed.

A list of the formats to be included as valid for conversion (provided that SelectedOnly is selected as the Convert option).

UDR Types

Selecting a UDR Type will activate this option.Default Values

In case fields have been added to the format, or the type of an existing field ischanged, default values can be defined. In the latter case this is especially useful,since the old field values are reset if the types are not interchangeable.

Note that it is not possible to assign values to old, already existing fields (itcan be configured, but will have no effect upon update.)

25


9. Introduction to the Ultra Format DefinitionLanguage

Ultra Format Definition Language (UFDL) is the language used to configure the Convergent MediationEngine™ format handling subsystem. It describes the physical structure of incoming and outgoing(external) data, internal (working) formats, as well as decoding and encoding rules.

The agents responsible for the translations are the Decoder and the Encoder. The Decoder turns externalraw data (byte arrays) into internal data (UDRs). The Encoder works the opposite way, convertinginternal formats into raw output data for the Forwarding agents.

Figure 17. Decoders and Encoders translate data to and from a format to be processed byConvergent Mediation Engine™ .

UFDL syntax is entered in the Ultra Format Editor found in the Configuration menu on the Conver-gent Mediation Engine™ desktop.

9.1. Building BlocksUFDL uses the following building blocks:

An external format describes the physical layout of the raw data. It can be declaredin multiple ways such as through an asn_block block (for BER encoded formats)or an Convergent Mediation Engine™ external block.

External formats

An internal format describes the structure of the internal, working UDR type. Theseformats are used in workflow activities (such as APL processing) and are declared

Internal formats

through an internal block or generated from an automatic in-map (through atarget_internal specification).

A mapping from an external type to an internal type. This is the basic componentused to describe a decoder, and it is declared through the in_map specification.

In-map

A mapping from an internal type to an external type. This is the basic componentused to describe an encoder, and it is declared through the out_map specification.

Out-map

A decoder uses one or more in-maps to specify a set of decoding rules to be usedwhen decoding input data. Blocking specifications as well as other file structurespecifications can also be added.

Decoder

An encoder uses one or more out-maps to specify a set of encoding rules to be usedwhen decoding input data. Blocking specifications can also be added.

Encoder

As an example, consider incoming call detail records containing information on the type of call, whohas called, and which number was dialed. That means the record contains three fields. We need to addtwo fields to this record and new termination marks (;) for each field in the record in order to conformto the expected output format.

26


For both incoming and outgoing external formats, each record (the last field in the record) is terminatedby a newline character.

Collected data stream (bytearray) Forwarded data stream (bytearray)

Format definition Format definition

1 22336688 0012121212 1 ; 22336688 ; 0012121212 ; 2 ; 2

type Anum Bnum type Anum Bnum field1 field2

external recB {int type : terminated_by(";");ascii Anum : terminated_by(";");ascii Bnum : terminated_by(";");int field1 : terminated_by(";");int field2 : terminated_by(0xA);

};

external recA {int type : static_size(1);ascii Anum : static_size(8);ascii Bnum : terminated_by(0xA);

};

Figure 18. An example of two external format definitions.

As can be seen in Figure 18, “An example of two external format definitions. ”, an external specificationis used to describe the raw data formats.

The UDR format used by the Convergent Mediation Engine™ processing agents is generated throughthe target_internal specification. This format is automatically generated based on the incomingexternal data and includes any fields in the internal specification of the in-map.

In this case, the generated internal format will consist of the fields from the external incoming formatwith two additional fields. Hence, we introduce an internal containing the new fields only.

Figure 19. Relation between declared and generated internal formats.

Finally, the in_map and out_map definitions are used when creating decoder and encoder definitionsto be used in Convergent Mediation Engine™ .

27


UFDL code of the example is described as follows:

external recA { int type : static_size(1); ascii Anum : static_size(8); ascii Bnum : terminated_by(0xA);};

internal recA_int { int field1; int field2;};

external recB { int type : terminated_by(";"); ascii Anum : terminated_by(";"); ascii Bnum : terminated_by(";"); int field1 : terminated_by(";"); int field2 : terminated_by(0xA);};

in_map A_map : external(recA), internal(recA_int), target_internal(recA_TI) { automatic; };

decoder decode_A : in_map(A_map);

out_map B_map : internal(recA_TI), external(recB) { automatic; };

encoder encode_B : out_map(B_map);

9.1.1. External

Network elements delivering data to Convergent Mediation Engine™ produce records in variousphysical formats. An external format describes the physical structure of this incoming data, as well asthe data delivered by the Convergent Mediation Engine™ system. It can be any type of ASCII,ASN.1/BER, XML, or binary data. Typically, one external block per record type is defined. Fileheaders and trailers are also considered being records.

The external formats are described through a number of different methods with different syntax. Theexternal formats, currently supported by the Convergent Mediation Engine™ Platform, are:

• Sequential formats: ASCII or binary based on static or dynamic sizes for both records and fields(refer to Section 10, “External - Sequential Formats”). These are the generic external specificationsdefined via the external block.

• Ericsson IOG/IN formats: (refer to Section 11, “External - Ericsson IOG/IN Records”). This specialformat type is described through the inw variant of the external block.

28


• ASN.1/BER formats: A subset of ASN.1 is supported (refer to Section 12, “External - ASN.1/BERFormats”).

• XML formats: A subset of the standard XML schema syntax is supported and can be used to handledifferent XML formats (refer to Section 13, “External - XML Records”).

Records of different format types may exist in the same record. An example of this could be an ASN.1record that contains a field whose format is a sequential record (specified through an externalblock).

Figure 20, “A definition of a sequential external record.” illustrates the syntax for a sequential recordtype.

Figure 20. A definition of a sequential external record.

9.1.2. Internal and In-Map

There are two ways to define an internal UDR format in UFDL. Either by using the automatic generationthrough an in_map specification, or through an internal declaration block. For more complexformats it is often impractical to manually write an internal format when all that is desired is a directmapping from the fields in the external format. In these cases the target_internal specificationin the in_map is used. An additional internal can be specified in the in_map if additional pro-cessing fields are needed.

The in_map manages the mapping of external data into the internal UDR structure. An externalmust be defined, and one or both of the internal and target_internal options must be usedto specify the internal format to map to.

For detailed information on how to specify an internal format, refer to Section 14, “InternalFormats”, and for information about in_map specifications, refer to Section 15, “In-Maps”.

internal AXE_DAM

{str ing ANumberAreaCode;str ing ANumberClass : opt ional ;str ing BR_IncomingPQR;str ing BR_NumberClass;int LenANumber ;int LenBRNumber ;int LenBSNumber ;str ing PreProcRecordClass;

} ;

Figure 21. An internal type declaration.

An internal declared through target_internal will be automatically generated based on the fieldsof the external and internal formats.

The target_internal is the format used by the processing agents in Convergent Mediation En-gine™ . It can be a direct representation of the external incoming format, or a combination of the ex-ternal and additional fields from an internal definition. An internal definition is needed in case any

29


changes to the default mapping from the external format are required,or if several externals are mappedto the same UDR.

To understand the structure of the generated target_internal format, it is necessary to understandhow the automatic mapping of an in_map is performed. When automatically mapping an externalformat (that is, using the automatic keyword), all external fields are mapped to an internal fieldunless the external format specifies a different automatic mapping behavior (for example through theexternal_only keyword).

For each external field the automatic in-map generation will perform the following steps:

1. If there is an explicit mapping specified for the field, use that mapping (explicit field mappingspecifications overrides automatic mapping).

2. If there is a field in the internal format with the same name as in the external format field,the external field is mapped to the internal field.

3. Otherwise, a new field is created in the generated internal format with the same name as the ex-ternal field, and a type chosen according to the external format (the default mapping type).

If the automatic mapping algorithm needs to add new fields, a new internal type is first created. Thistype has the name specified in the target_internal and is a subtype of the internal type (ifspecified). Decoders using this in_map will then produce records of that internal type.

The default mapping type of an external field depends on the external format specification, and if theexternal field is of a constructed type (a list or record type), the automatic mapping logic will be appliedrecursively to any sub-formats.

Besides automatic mappings, it is possible to specify explicit type mappings, where the correspondingexternal and internal fields are directly specified.

Figure 22. A resulting target_internal.

In Figure 22, “A resulting target_internal.” a combination of explicit (the e: and i: specifications)and automatic mapping is used to create the resulting target_internal CDR_TI.

9.1.3. Decoder

A Decoder defines what external records to accept, and also describes any additional file structure in-formation, such as whether any file blocking is used in the input data.

30


A decoder definition consists of in_maps or other decoders. The most basic decoder definitiontakes a single in-map as argument. In the following example, myDecoder will appear in the config-uration window of a Decoder agent:

Example 1.

The example creates the decoder myDecoder, taking one in-map. The incoming data is blockedin 2048 bytes, with the block padding 0x00.

decoder myDecoder : in_map( my_in_map ),block_size(2048), terminated_by(0x00);

A decoder definition may also consist of several in-maps, that is, the data stream received may containseveral record type definitions. Another possible definition is combining several decoders, a so calledconstructed decoder. The latter case introduces some validation logic to the incoming stream of data.Consider Figure 23, “ Two ways of defining a decoder. ”.

decoder TTFi le1 {decoder header;decoder udr *;decoder trailer;

};

decoder TTFi le2 : in_map( header_m ),in_map( udr_m ),in_map( trailer_m );

Figure 23. Two ways of defining a decoder.

The first case, TTFile1, will require a batch to start with a header, end with a trailer, and containany number of UDRs in between. The second decoder definition (TTFile2) ignores the order andoccurrence of arriving records.

For further information about decoder specifications, refer to Section 17, “Decoders”

9.1.4. Out-map

An out-map defines how the internal data will be mapped to the outgoing format. It takes an internaland an external definition as argument, and maps fields either automatically or through explicit fieldmappings.

out_map Access_Reject_Map : external( Access_Reject_Ext ),internal( Access_Reject_Int ) {

automatic;};

Figure 24. The minimum required to define an out-map.

Figure 24, “The minimum required to define an out-map.” creates an external Access_Re-ject_Ext using the internal Access_Reject_Int. The automatic keyword will resultin all fields with the same name being mapped, while field names present in the internal but notthe external are ignored.

For further details on how to specify an out_map refer to Section 16, “Out-Maps”.

31


9.1.5. Encoder

Encoders define the rules to use when mapping internal UDRs to external data structures (raw data).

An encoder must be defined in order for any out_map to be usable, that is, selectable from thewindow. The most basic definition takes one out_map as its only argument. In the following example,myEncoder will appear in the configuration window of an Encoder agent:

Example 2.

encoder myEncoder : out_map( my_out_map );

Similarly to a decoder, an encoder takes one or several out-maps to define the structure of the outgoingdata. For detailed information on encoder specifications refer to Section 18, “Encoders”.

9.1.6. Event Types

Event format types are special internal types that can be sent to the Convergent Mediation Engine™Event Server. Events have additional internal information that is used during event processing. Forperformance reasons and to avoid confusion, event types should be used only as “message UDRs”dispatched to the Event Server. In other words, do not use event UDRs to transport ordinary UDRdata.

Event types are declared in the same way as internal types, except that the keyword event isused instead of internal. Also refer to Section 14, “Internal Formats”

The following code will result in the events in Figure 25, “ Event format showing added fields. ”:

event AMA_Event {

int anum; int bnum; ...}

Figure 25. Event format showing added fields.

For more information on Event handling, please refer to the Convergent Mediation Engine™ Core -User Guide.

9.2.The Ultra ModuleA saved format will be referred to by the qualified name under which it was saved (its module name).In APL or UFDL it can be referenced by the module name followed by a dot and the name of the in-

32


ternal definition. Alternatively import statements can be used to avoid the necessity of always specifyingthe module name.

9.2.1.The import Statement (APL Code)

In Analysis and Aggregation nodes, module names are not needed if the format is selected in the UDRTypes list in the configuration window of the node. Also, the APL Editor allows to select the UDRtypes from a list. Select UDR Assistance... when right-clicking in the Code Area to display the UDRInternal Format Browser.

If none of these lists are used, the full format name must be referred to, or the import keyword mustbe used. Note that the module name must be prefixed with ultra. to distinguish format imports fromAPL code imports.

Example 3. Referring to UDR Types in the APL Editor

The module in the example is named Default.CDR, and contains the target_internalCDR_TI which is to be used in a function created in the APL Editor.

Case 1 - import statement

import ultra.Default.CDR;

CDR_TI newUDR() { CDR_TI myUDR = udrCreate( CDR_TI ); myUDR.field1 = 22; myUDR.field2 = 55; return myUDR;}

Case 2 - no import statement

Default.CDR.CDR_TI newUDR() { Default.CDR.CDR_TI myUDR = udrCreate( Default.CDR.CDR_TI ); myUDR.field1 = 22; myUDR.field2 = 55; return myUDR;}

9.2.2.The import Statement (UFDL Code)

It is possible to split different blocks of a format between several format definitions. For these cases,the import statement may be used to avoid referring to the module name. Note that the ultra.prefix is not needed (or allowed) in the UFDL import statement.

Using import in UFDL Code

Suppose there is a format definition with the module name Default.AMA containing an internalmyInt, and a new format definition is needed to include the internal in the in_map. This can be ac-complished in two ways:

Case 1 - import

33


import Default.AMA;

in_map exMap : external( myCDR ), internal( myInt ), target_internal( myCDR_TI ) { automatic;};

Case 2 - no import

in_map exMap : external( myCDR ), internal( Default.AMA.myInt ), target_internal( myCDR_TI ) { automatic;};

Referring to ASN.1 structures from outside the format definition will require the ASN.1 module (ifdefined) to be part of the reference.

9.2.2.1. Name Lookup Rules

Consider the following ASN.1 block example. Depending on from where a format name is referenced,the name lookup will work differently.

Reference from within the asn_block specification:

• If there is a name in the asn_block specification, choose that name.

• If there is a name in the same format definition, choose that name.

• Check the import statements.

Reference from within the same module but outside the asn_block:

• If there is a name in the same format definition (outside any asn_block specifications), choosethat name.

• If there is a name in the asn_block specification, choose that name.

• Check the import statements.

Example 4. Shortened ASN.1 Block

asn_block {

exchangeRec DEFINITIONS IMPLICIT TAGS ::= BEGIN

main_udr ::= SEQUENCE

// Field specifications };

34


References to existing ASN.1 formats in UFDL requires the ASN.1 module to be part of thereference. Suppose the previous external specification, saved under the name Default.myAS-Nformat, is to be included in a new in_map:

Case 1 - import

import Default.myASNformat.exchangeRec;

in_map exMap : external( main_udr ), internal( myInt ), target_internal( myCDR_TI ) { automatic;};

Case 2 - no import

in_map exMap : external( Default.myASNformat. exchangeRec.main_udr ), internal( myInt ), target_internal( myCDR_TI ) { automatic;};

35


10. External - Sequential FormatsA sequential external format specification consists of two parts; the record declaration and the fielddeclarations. The record declaration can contain information on UDR size and how the record type isidentified. The field declarations may be grouped into larger structures such as bit_block and set.

Syntax - external format declarations

external <format_name> [: <record declaration>] { <content declarations>};

The comma separated elements of the record declaration list may be (in any order):

• terminated_by( <terminator> )

• dynamic_size( <expression> )

• static_size( <constant expression> )

• identified_by( <conditional expression> )

The content declarations can be:

• field declarations

• bit_block declarations

• set declarations

• switched_set declarations

10.1. Record DeclarationTo be able to split the incoming data stream into individual records the record size must be calculated.The record size calculation is managed by the record options terminated_by, static_sizeand dynamic_size. The different ways to calculate the size are tried in the following order:

1. If this record is a sub-record (a field of another record) and the parent external field size is specified,that size is used. How field sizes are calculated depends on the external type of the parent recordtype. This is a typical case if you have an ASN.1 type containing a sequential subtype in whichcase the BER size specification is used.

2. If the static_size option is specified, that size is used.

3. If the dynamic_size option is specified, that value is calculated and used.

4. If the terminated_by option is specified, the input stream is scanned until the terminator isfound (this implies that no field can contain this terminator since the first occurrence will be seenas the end of the UDR).

5. If everything else fails, an attempt is made to calculate the size as the total size of all the contentdeclarations.

36


10.1.1. terminated_by

terminated_by identifies the byte which a record is terminated by. It can be declared numerically(decimal or hexadecimal), or as a plain ASCII char. In the latter case, the literal must be enclosed inquotes. Note that only one byte is allowed.

Example 5.

The terminated_by allows both numeric and ASCII values:

external myExternal : terminated_by(0xA) { // <contents declaration>};

external myExternal : terminated_by(';') { // <contents declaration>};

10.1.2. dynamic_size

Binary encodings that vary in size often contain a field with information on their specific length. Forthese cases, the dynamic_size declaration may be used, specifying how to calculate the record sizefrom the values of one or more fields.

Example 6.

external myExternal : dynamic_size(recordLength) { int(little_endian) RecordLength : static_size(2); // <continued contents declaration>};

10.1.3. static_size

Records of fixed size are declared using static_size.

Example 7.

external myExternal : static_size(<recordLength>) { // <contents declaration>};

37


10.1.4. identified_by

If present, the identified_by option specifies the condition under which a record of this type ispresent in the stream. This is used when the input data contains different record types, in which casethe decoder uses the identified_by expression to check if the data matches the record type.

The condition is also evaluated when deciding whether to use a specific out map during encoding.

Example 8.

external myUDRType1 : identified_by( udrType == 1 ) { int udrType : static_size(1); // <continued contents declaration>};

10.2. Field DeclarationsThe fields of a sequential record are declared as follows:

Syntax - record field declarations

<field_type> <field_name> : <field options> ;

10.2.1. Primitive Field Types

The following primitive types are supported:

ASCII encoded string. This type may also be used for other types of string encod-ings with the char_encoding option. The default encoding used is ISO8859-1.

ascii

Special type used to encode a BER encoded size specification. This decodes tothe BER length specification plus the length of the length specification itself.

asn_length

This type is available to make it possible to decode some special cases of BERencoded data without using ASN.1 format specifications. It is also possible touse the special option content_only to get only the BER length specification.

Array of digits encoded in BCD. Nibble order can be specified as bcd(msn_fd)or bcd(lsn_fd). The msn_fd means that the most significant nibble is the

bcd

first digit, while lsn_fd uses the least significant nibble as first digit. Themsn_fd is default.

Byte array.bytearray

Convenience type that is equivalent to ascii: char_encod-ing("Cp1047"). Since this type is just a special case of ascii all optionsavailable for ascii fields are also available for ebcdic fields.

ebcdic

Binary encoded float value. This type supports IEEE754 standard 32-bit and 64-bit data encodings. The only difference between float and double is that the fieldis automatically mapped to the corresponding internal type.

float types(float, double)

Binary coded integer value. The first byte is the most significant (that is, big en-dian order). You can use the field options unsigned and signed to specify if

integer types(byte, short,

38


int, long, bi-gint)

the data will be decoded as unsigned (which is the default) or two-complementsigned. The byte order can also be explicitly specified by using either, for instance,int(little_endian) or int(big_endian) as the type name.

The only difference between the types is that the field is automatically mappedto the corresponding internal type when using an automatic in_map.

Note that since the integer types are handled internally as fixed-length signed in-tegers (except for bigint), there can be overflows in both decoding and encoding.When this occurs the integer values are truncated.

10.2.2. Primitive Field Options

Specifies that the field is encoded with the encoding named <encod-ing_name>, using the standard Java encoding support.

char_encoding(<en-coding_name>)

Specifies that the encoded value of the field always is <expr>, regardlessof decoded or set value during processing. This is used for encoding only.

encode_value(<ex-pr>)

Only applicable for ascii fields. Informs the decoder that the value isactually a float value specified as a string and that the automaticmapping of the field is the internal type float or double.

float, double

Only applicable for ascii and bcd fields. Informs the decoder that thevalue is actually an integer of decimal or hexadecimal base and that the

int(base10),int(base16)

automatic mapping of the field is of the internal type int. The other integertypes (byte, short, long, bigint) can be used instead of int, inwhich case automatic mapping is done to the specified type.

The least significant bit of the field is <int_constant>. Value rangeis 0 to 7 (both inclusive). Only applicable for size 1 integer type fields.This option is typically used together with the lsb option.

lsb(<int_con-stant>)

The most significant bit of the field is <int_constant>. Value rangeis 0 to 7 (both inclusive). Only applicable for size 1 integer type fields.This option is typically used together with the lsb option.

msb(<int_con-stant>)

Specifies the number of BCD digits for a bcd declared field. Note that thissize does not cover field size calculation, so dynamic_size must typ-ically also be specified.

native_size(<ex-pr>)

10.2.3. Field Size Specification Options

Used to specify a static size of a field (in bytes).static_size(<constant>)

Used to specify a dynamic size of a field (in bytes).dynamic_size(<expr>)

Used to specify a dynamic field terminated by a specific con-stant.

terminated_by(<constant>)

Used to specify padding character.padded_with(<constant>)

Specifies that the field is right aligned.align(right)

Specifies that the field is left aligned, default.align(left)

When decoding a field, the size calculation is done in two steps. First the occupying size is calculated.This is the size the field needs in the record. Then the core size and offset is calculated, which is thepart of the field that will actually be decoded into the internal field.

The occupying size is calculated as follows:

1. If static_size is specified, then this is used.

2. Otherwise, if dynamic_size is specified, then this is used.

39


3. Otherwise, if terminated_by is specified, then this is used. The field size includes the termin-ation character, but will never take more than the total remaining size in the UDR (The reasonthis is not a decoding error is to support the trailing_optional field option).

4. Otherwise (if the field type supports it) the field size will be deduced directly from the type. Thisis supported by constructed types (sub-records) and the asn_length primitive type.

The core field data is always the full occupying size for constructed fields (record fields). For primitivefields the size is specified as follows:

1. For a BCD field, and native_size is specified, then this along with the alignment specificationis used.

2. If terminated_by was used to find the occupying size, then this terminator char (or nibblefor BCD) is removed.

3. Any padding is removed (while considering the alignment specification). The padding is: pad-ded_with if specified, else terminated_by if specified and the occupying size was notcalculated using the terminator (this case is present for historical reasons and padded_withshould be used instead), else if the field is an ASCII field, space is used as default padding.

10.2.4. Field Options for Optional Fields

The following field options are used to specify when a field is present.

The field is present if the <condition> evaluates to true.present if(<condi-tion>)

The field is present unless the end of the UDR data has been reached.This is a convenience option that is equivalent to present_if(remaining_size > 0 ).

trailing_optional

10.2.5. Other Field Options

The field will not be automatically created in the target_internal whenperforming automatic mapping. This is useful for fields containing “decoding

external_only

logic” and provide no useful information after decoding. Typical examples couldbe recordLength and recordType fields.

10.2.6. udr_size and remaining_size

Fields may need to use the size of the containing record in expressions. This is done by using theudr_size keyword.

Example 9.

external SimpleSequential { int recordType : static_size(1); ascii secretData : dynamic_size(udr_size-1);};

In the previous example, the size of SimpleSequential is unknown at declaration time. However,when a size is provided (specified in a parent record type), the secretData field will occupy thisentire space minus one byte (which is used by the recordType field in this example).

40


Note that if the size is not supplied by a parent record, the record size calculation rules will result inan undefined size since the udr_size value is not available before the size has been calculated. Thiswould cause a decoding error.

The other special value that depends on the record size is remaining_size, which is the size re-maining until the end of the record. The previous example could have been written using remain-ing_size instead of udr_size.

Example 10.

external SimpleSequential { int recordType : static_size(1); ascii secretData : dynamic_size(remaining_size);};

10.2.7. Bit Blocks

Bit blocks are used when multiple fields share the space of a single byte. When declaring fields in bitblocks the most and least significant bit of the field should be specified using msb and lsb, as previ-ously described.

Syntax - bit blocks

bit_block : static_size(1) [, present if(<cond>) ] { <bit_block fields>};

A current limitation of bit blocks is that they can only span one byte, that is, the field must have astatic_size of one. Bit blocks are typically used whenever there are a number of flags in a singlebyte.

Example 11.

bit_block : static_size(1) { int(little_endian) LACLength : msb(7), lsb(4); int(little_endian) OwnerIDLength : msb(3), lsb(0);};

10.2.8. Constructed Types

A sequential field can be of a type that is an instance of another external format.

41


Example 12.

external MyParentFormat { int field1 : static_size(4); MyEnclosedFormat field1;};

Here MyEnclosedFormat can be any external format.

10.2.9. set Construct

The set construct is used for decoding formats that contain optional blocks of additional data. Thesyntax is obtained in the following example:

Example 13.

external MyFormat: dynamic_size(recordSize) { int recordSize: static_size(4); set : dynamic_size( remaining_size ) { MyPackage1 package1: optional; MyPackage2 package2: optional; list<MyPackage3> package3; };};

All the formats, MyPackage1-3, must have been declared with the identified_by option. Theoptional packages may appear in any order in the input file and it is validated that they do not appearmore than once. Currently all fields in a set construct must be declared as optional.

If the field type in the set is a list type, the set may contain multiple records of the list element type.Also note that the list type fields are not optional. Instead, when no matching records are found, thelist is empty.

If a size is not specified on the set level, Ultra cannot validate that all the data in the UDR has beendecoded. It is recommended that the user specifies the size unless the set size is unknown in advance(for instance if the record is terminated by a terminator package, or the set size calculation is neededfor the record size calculation). The dynamic_size(remaining_size) specification used inthe previous example is often correct.

10.2.10. switched_set Construct

The switched_set construct can be used in many cases instead of the set construct. It has advant-ages (in performance and ease of use) primarily when the separate sub-packages are simple. The syntaxis more complex compared to that of the basic set construct:

Syntax - switched_set construct

switched_set( <switch field> )

42


[: <size specification> ] { <prefix fields> <switch cases> [<default case>]};

The size specification is allowed to contain the normal size options. The other parts of the declarationare the prefix fields which are fields, decoded for each package in the set. The prefix fields must allhave static sizes. The switch field must be one of the prefix fields, each case of the structure:

Syntax - switch case

case( <case value> ) [: include_prefix] { <case fields>};

The case fields are normal field specifications with the additional possibility of declaring list fieldsfor the case where a package can be present multiple times. If include_prefix is specified, thenthe case body will be decoded including the prefix fields. The default case is written as:

Syntax - default case

default [: include_prefix] { <case fields>};

The decoding of a switched_set is performed according to the following steps:

1. Decode the prefix fields.

2. Decode the case matching the value of the switch field. If no case matches, then decode the defaultcase. If there is no default case, then end switched_set decoding.

3. Repeat steps 1-2 until the switched_set size (or the end of the UDR) has been reached.

An example of a format with a switched_set:

43


Example 14.

external SwitchedSetExample: terminated_by(0xA) { // Size is remaining_size -1 (minus the terminator linefeed) switched_set( packageId ): dynamic_size( remaining_size - 1 ) { ascii packageId: int(base10), static_size(1); ascii packageLength: static_size(1), int(base10), encode_value( case_size - 2 ); case(1) { list<ascii> list1: dynamic_size( packageLength ); }; case(2): include_prefix { ascii packageId_3: int(base10), static_size(1), encode_value(3), external_only; ascii packageLength_3: int(base10), static_size(1), encode_value(case_size - 2), external_only; ascii body_3: dynamic_size( packageLength_3 ); }; default: include_prefix { list<ascii> defaultContent: dynamic_size( packageLength + 2 ); }; };};

10.2.11. Encoding Specifications and Expressions

To support encoding to binary formats, it is often necessary to explicitly specify which value to beencoded in the external fields. Normally the value is taken from the corresponding internal field, butthere are cases when this is not desirable. For instance, if there is no mapped internal field (becausethe external_only option has been used), or the value must be calculated from information aboutthe encoding (for instance, udr_size). This is done through the encode_value option (refer toprevious section), and there are several special constructs that may be used in the value expression.

1. udr_size - evaluates to the encoded size of the UDR. That means this is not necessarily thesame value as during decoding.

2. field_size(fieldName) - evaluates to the encoded size of the named field.

3. field_present(fieldName) - evaluates to true if the named field is present in the en-coding. It is always true for non-optional external fields.

4. case_size - this is only usable within switched_set blocks and evaluates to the encodedsize of the current case (including prefix fields).

If the size expressions are used, the field encoding has to be postponed until the size is known. To beable to do this, Ultra requires that any such fields are static_size.

An example of these concepts:

44


Example 15.

external Ext: dynamic_size( udrSize ) { ascii udrSize: int(base10), static_size(3), align(right), padded_with("0"), encode_value( udr_size ); ascii fieldSize: int(base10), static_size(3), align(right), padded_with("0"), encode_value( field_present( strField ) ? field_size( strField ):0 ); ascii strField: dynamic_size( fieldSize ), present if( fieldSize > 0 );};

When processing an encode_value instruction, Ultra automatically decides how to convert thevalue depending on the result type of the expression. When deciding this, Ultra starts with the defaultinternal type of the external field. If we call this type defaultType end the expression type encode-Type, the encoding rules are:

1. If the defaultType is assignable from encodeType, use the default mapping.

2. If the defaultType is string or bytearray and the encodeType is numeric, encodeas a simple ascii value (one byte).

3. If the defaultType is bytearray and the encodeType is string, do standard encoding(ISO-8859-1) of the string.

4. If the encodeType is string and the external base type is ascii (for example usingint(base10)), use direct string encoding.

If none of these rules are applicable, the format will not compile

To understand what this means, consider the following field definitions

Example 16.

ascii strField1: static_size(1), encode_value("10"); ascii strField2: static_size(1), encode_value(10); ascii intField1: static_size(1), int(base10), encode_value("10"); ascii intField2: static_size(1), int(base10), encode_value(10);

What should the encoded result be for these fields?

Both defaultType and encodeType are string. So the normal encoding willbe used to get the result "1" (since we have static_size(1), the result "10" istruncated to one byte).

strField1

defaultType is string and encodeType is byte (numeric). This means thesecond rule is applied, so that the result is ascii 10 (newline).

strField2

45


defaultType is int and encodeType is string. There is no mapping betweenthese types. However, since the external base type is ascii, the string is mapped outas for strField1, and the result is "1".

intField1

defaultType is int and encodeType is byte. Since encodeType is directlyassignable to defaultType, it is mapped out as normal, and the output is again "1".

intField2

10.3. ExpressionsUFDL may use APL in order to introduce expressions to validate the incoming data. Basically all APLfunctions are available, except for plug-ins.

Example 17.

external myRecord : identified_by( !strStartsWith( myField, "0201" ) ) { ascii myField : terminated_by(0xA);};

46


11. External - Ericsson IOG/IN Records

11.1. Record DeclarationAn IN external format is declared as follows:

Syntax - IN external

external <format_name> inw { <field declarations>};

No size declarations are necessary for an IN record since this is given by the enclosing sequential record.In fact, an INW record can only be used as the final field in a sequential record.

11.2. Field DeclarationAn IN field is declared as follows:

Syntax - IN field

bcd tag_id(<int_constant>) <field_name> <: options>;

The available tag_id values are given by the Ericsson IN documentation.

The available options are:

Indicates that the field has a number frame. Two fields will be generatedif this is specified, one with the specified field name and one suffixedwith _NumFrame.

num_frame

Indicates that the field has a GNCI. Two fields will be generated if thisis specified, one with the specified field name and one suffixed with_GNCI.

gnci

As gnci, but the GNCI is only present if the expression is evaluatedto true.

gnci on (<expr>)

Identical to sequential definitions.terminated_by(<ex-pr>)

Identical to sequential definitions.int(base10)

Identical to sequential definitions.int(base16)

47


12. External - ASN.1/BER FormatsUltra provides support for parsing a subset of ASN.1 definitions, which can be used to decode fromand encode to the corresponding BER encoded data. ASN.1 parsing is requested in UFDL via theasn_block construct.

Syntax - ASN.1 blocks

asn_block { -- ASN.1 definitions here};

All ASN.1 constructed types that are declared as either of SEQUENCE, SET or CHOICE, will betreated as an external format declaration. The name of the resulting external format will be the nameof the ASN.1 definition. Any ASN.1 module name will be added to the name space (that is, the totalname space for the ASN.1 definition is <folder>.<configuration name>.<ASN.1 modulename>).

All occurrences of the dash character (-) in identifiers will be converted to underscores since dashesare illegal in Ultra type naming.

12.1. Notes on ASN.1

12.1.1. Inter Module References

It is currently not possible to refer to non-constructed types or list types (that is, SEQUENCE OF andSET OF) that are declared in other modules. These must be contained within the asn_block wherethey are referenced. Any referenced constructed ASN.1 types must be specified in an ASN.1 IMPORTstatement to be available. For instance,

MyType ::= SET(field1 TAC, field2 MobileOriginatedCall)

If TAC and MobileOriginated are declared in another asn_block in the same module:

TAC ::= OCTET STRING;MobileOriginatedCall ::= SET ( .... )

Then the following applies:

1. The TAC declaration must be duplicated in the asn_block of MyType.

2. MobileOriginatedCall must be imported within the asn_block containing MyType.

12.1.2. ASN.1 Primitive Type Mapping

ASN.1 types are automatically mapped by Ultra as follows (this applies when there are automaticstatements in the in-maps):

Ultra Type mapped to:ASN.1 Type:

booleanBOOLEAN

48


stringbcd

stringAll ASN.1 string types except OCTET STRING

bytearrayOCTET STRING

intENUMERATED

intINTEGER

floatREAL

bitsetBIT STRING

bytearray (which will have the value null)NULL

12.2. Ultra ExtensionsWithin a UFDL asn_block it is possible to use some constructs that are not part of the ASN.1standard. These extensions are added to provide better automatic decoding support for some formats.

12.2.1. Direct BCD Support

A bcd type has been introduced. Very often ASN.1 formats encoded in BER uses OCTET STRINGto describe BCD data, which leads to complicated processing. By replacing these entries with the bcdtype, Ultra will automatically convert such entries.

Syntax - bcd type declaration

bcd(lsn_fd)bcd(msn_fd)bcd(lsn_fd) terminated_by(<expr>)bcd(msn_fd) terminated_by(<expr>)

12.2.2. Date Support

Many ASN.1 formats declare date and time information as OCTET STRING. The date type converterhas been introduced to manage an automatic conversion to date instances. For example, the followingis possible:

Syntax - date declaration

Time ::= OCTET STRING date({HH,mm,ss}) (SIZE(3))Date ::= OCTET STRING date({yy,MM,dd})(SIZE(3)) Date ::= OCTET STRING date({cc,yy,MM,dd} , {yy,MM,dd}) DateTime ::= OCTET STRING date({cc,yy,MM,dd,HH,mm,ss})

Note that these are applicable on OCTET STRINGS only.

12.2.3. Using Sequential Record Types

Some ASN.1 definitions contain data with an OCTET STRING declaration that contains additionalstructures. In order to manage this, it is possible to split such declarations into sequential record types.

It is also possible to use sequential formats to describe constructed ASN.1 types. In this case the tagmust be declared as constructed (a Convergent Mediation Engine™ specific keyword) to allowUltra to correctly encode the type.

49


Example 18.

As an example of a simple type, a definition

AddressString ::= OCTET STRING (SIZE(1..20))

within an asn_block (module GSM), can be brought outside the asn_block and be redefinedas:

external GSM.AddressString { bit_block : static_size(1) { int npi : msb(3), lsb(0); int ton : msb(7), lsb(4); }; bcd(msn_fd) msisdn : dynamic_size(udr_size-1), terminated_by(0xF);};

Example 19.

As an example of a complex type, consider a sequence where a field fieldB is to be decodeddifferently depending on the value of another field fieldA. The ASN.1 definition

ComplexType ::= [APPLICATION 1] SEQUENCE { fieldA INTEGER, fieldB OCTET STRING}

Could then be replaced with the sequential format

external ComplexType_Seq { int tagA: static_size(1); int lengthA: static_size(1); int fieldA: dynamic_size(lengthA);

// Definitions of SubType1 and SubType2 not included // in this example. SubType1 fieldB1: present_if( fieldA == 1 ); SubType2 fieldB2: present_if( fieldA == 2 );};

and the extended ASN.1

ComplexType ::= [APPLICATION 1 constructed] ComplexType_Seq

12.2.4. bigint Support

Since INTEGER types are automatically mapped to int, which is a 32-bit integer type, INTEGERslonger than 4 bytes will cause decoding errors. This can be avoided by using the bigint type in placeof INTEGER. The only difference between bigint and INTEGER is that bigint is automaticallymapped to the Convergent Mediation Engine™ bigint type, which can support INTEGERs of anysize.

50


12.3. ASN Language LimitationsThe ASN.1 compiler handles most of the type notation of ASN.1. However, it does not support anypart of the value notation or the macro notation. The following elements of the type notation are notsupported:

• COMPONENTS OF

• WITH COMPONENT

• WITH COMPONENTS

• ABSENT / PRESENT

• ANY, ANY DEFINED BY

• ObjectDescriptor

• DEFAULT

• DEFINITIONS

• EXPLICIT, EXPLICIT TAGS

• INCLUDES

• MACRO

• PRIVATE

• UTCTime

• EXTERNAL

• GeneralizedTime

• OPERATIONS

12.4. BER Limitations• Indefinite length encoding is not supported for the following types: BIT STRING, OCTET

STRING, and all the character string types.

• Explicit tags are not supported. All tags are implicit by the default (with the exception of tags onCHOICE types which are always assumed to be explicit according to the ASN.1 standard). If anyattempt is made to specify explicit tagging, this will result in a compilation error.

• The ObjectIdentifier type is not supported.

• The EXTERNAL type is not supported.

• The GeneralizedTime and UTCTime types are not supported.

• The ANY type is not supported.

• All string fields are encoded/decoded according to ISO8859-1.

• For SEQUENCE and SET types, no validation is performed to ensure that the mandatory fields areactually present.

51


• Not all character types are supported. Currently supported are: GraphicalString,IA5String, VisibleString, and NumericString.

52


13. External - XML RecordsUltra provides support to compile a subset of XML schema definitions and to decode the correspondingXML data. XML parsing may be requested in UFDL via the xml_schema construct.

Syntax - xml schema

xml_schema { <xml schema definitions here>};

The XML schema definition support includes:

• simpleType

• simpleTypes with restriction declarations

• complexType

• complexTypes with sequence declarations

• complexTypes with extension declarations

• global element declarations and references

• simpleContent elements with extension declarations

All complex types defined in the XML schema will be interpreted as UDR types. The elements andattributes within the complex types will be interpreted as UDR fields. The field types will vary dependingon the XML specifications:

Element

1. minOccurs = 0;

The field is optional.

2. maxOccurs

> 1; The field is of list type.

Attribute

1. Optional depending on the used specification.

13.1. XML Primitive Type MappingXML types are mapped automatically by Ultra as follows:

Ultra Type mapped to:XML Type:

int, except for long and unsignedLong which are mapped to bi-gint.

<numeric>

floatfloat

doubledouble

53


string<all other types>

13.2. IPDR ComplianceConvergent Mediation Engine™ is fully IPDR compliant. UFDL also supports IPDR compact decod-ing/encoding. This is done by specifying the option ipdr_compact to the map definition.

13.3. LimitationsThe following XML schema constructs are not supported in UFDL.

• list type

• union

• complexType defined as an extension of a simpleType

• anyType

• choice

• group

• all

• attributeGroup

• nillable declarations

• include

• redefine

• substitutionGroup

54


14. Internal FormatsConvergent Mediation Engine™ uses internal formats to represent data entities that it can process. Allprocessing agents (for instance, Analysis and Aggregation) work with these internal formats.

An internal format is declared as follows:

Syntax - internal format

internal <name> [: <class specifications>] { <field_type> <field_name> [:optional] ; ...};

The field types may be any of the following:

Any type.any

Big integer.bigint

Boolean.boolean

Byte array.bytearray

Integer type (8-bit signed).byte

Integer type (16-bit unsigned).char

Integer type (16-bit signed).short

Integer type (32-bit signed).int

Integer type (64-bit signed).long

Float type (32-bit).float

Float type (64-bit).double

Date type, having the capability to hold date parts, time parts or both.date

A set of bits.bitset

An IP address.ipaddress

An instance of any other internal (all internal are drudr instances).drudr

String.string

In addition, field_type can be any other internal or list type:

Example 20.

internal I1 { I2 f1;};internal I2 { list<int> f1;};

List types are declared as follows:

55


list< ElementType >

where ElementType may be any of the previous, including an internal format identifier, or anotherlist type.

Example 21.

internal I1 { list<list<I2> > f1;};

It is also possible to specify a field as optional:

Example 22.

internal I1 { drudr f1: optional;};

Internal formats can also be automatically generated from in_map definitions. See target_intern-al specification in the in_map following description.

14.1. Class SpecificationsAll internal formats will be compiled into Java classes. It is possible to specify additional interfacesfor the class to implement:

Example 23.

internal I1 : implements("Interface1"), implements("Interface2") { ...};

However, this requires that Interface1 and Interface2 only declare methods that are latergenerated by Ultra when it creates the Java class. For more information on methods and types for UDRtype methods, refer to the Convergent Mediation Engine™ Development Toolkit - User Guide.

It is also possible to use alternative base UDR definitions for the generated Ultra classes by the ex-tends_class option. This is used by some Convergent Mediation Engine™ agents (for instance,the HTTP agent) for better processing support.

56


14.2. Event typesIt is possible to declare user defined event types in Ultra by using the event keyword instead of in-ternal. Such an event is a special type of internal format with added event processing support.

57


15. In-MapsThe in_map construct is used to map external formats to internal formats during decoding. The gen-eral syntax for an in-map declaration is as follows:

Syntax - in_map

in_map <map_name> : <in_map options> { <explicit map specifications> <automatic> : <automatic options> { <automatic mapping specifications> }; <sub-external specifications>};

The in-map options are:

Specifies the external format to map from. This is a mandatoryparameter.

external(<extern-al_name>)

The internal format name to map to. This is a mandatory parameter,unless target_internal is specified.

internal(<intern-al_name>)

The target internal format name created when automatically gener-ating a map. This parameter is only valid for automatic map-pings.

target_internal(<tar-get_internal_name>)

Specifies that the in_map will produce no output when used in adecoder. This can be useful for uninteresting "filler records" thatare not needed for processing.

discard_output

Specifies that a decoder using this in_map will not route out thetop level UDR. Instead, the content of the named fields (whichmust contain records or lists of records) are routed.

emit_field(field1,field2, ...)

Specially named options can also be supplied in the in_map depending on the type of the externalformat. Currently the only supported format dependent option is ipdr_compact for XML basedexternals (refer to the previous list of options).

Describes how the external fields are mapped to the internal fields. Thesespecifications are optional.

i:<internal field> and

<explicit mapspecifications>

e:<external field> [ using in_map <sub_map> ] ;

Specifies that all external fields that are not explicitly mapped in <expli-cit_map_specifications> will be implicitly mapped according to

<automatic>

the external formats implicit type conversions. It is also possible to controlthe behavior of the automatic mapping through options or a specificationblock (refer to following section for details).

This is used to handle the special case where the mapped external is theparent of other externals that must be considered for decoding. This is cur-

<sub-externalspecifications>

rently only supported for XML schema based externals where it is used tosupport, for instance, IPDR decoding.

58


15.1. Examples of Non-Automatic MapsNon-automatic maps provide an explicit mapping between the external and internal format. Suchmappings work on fields with primitive types and/or fields that are instances of other formats. In thelatter case, the maps can be nested indefinitely. The following two examples demonstrate these concepts.

Example - All primitives

In the following example a decoder, based on the in_map, will do the following:

1. Read an instance of the external record ExtFormat.

2. Associate ef1 with if1, that is, any reference to if1 will force decoding of the information ofef1 and in the process convert it to an integer.

3. Associate ef2 with if2, that is, any reference to if2 will force decoding of the information ofef2 and in the process convert it to a string.

Example 24.

external ExtFormat { ascii ef1 : static_size(4), padded_with(" "); int ef2 : static_size(2);};

internal IntFormat { int if1; ascii if2;};

in_map InMapFormat: external(ExtFormat), internal(IntFormat) { e:ef1 and i:if1; e:ef2 and i:if2;};

Example - Externals in externals

In the following example a decoder, based on the in_map, will do the following:

1. Read an instance of the external record ExtFormat2.

2. Associate ef1 with if1, that is, any reference to if1 will force decoding of the information ofef1. The conversion between ExtFormat1 and IntFormat1 is, in this case, dictated by thein-map named InMapFormat1 as declared in InMapFormat2.

59


Example 25.

external ExtFormat1 { ascii ef1 : static_size(4);};

external ExtFormat2 { ExtFormat1 ef1;};

internal IntFormat1 { int if1;};

internal IntFormat2 { IntFormat1 if1;};

in_map InMapFormat1: external(ExtFormat1), internal(IntFormat1) { e:ef1 and i:if1;};

in_map InMapFormat2: external(ExtFormat2), internal(IntFormat2) { e:ef1 and i:if1 using in_map InMapFormat1;};

As seen in the example, explicit map specifications for complex formats can become very large. Evendescribing all the internals to map data to, can be a huge task. For this reason automatic mapping isnormally used.

15.2. Automatic MapsAutomatic mapping is used when all the fields of the external format are to be included in the internalformat. This does not preclude explicit mapping of selected fields or types. Automatic mapping complywith the following steps for each field in the external format:

1. Check if the in_map contains an explicit mapping for the external field. If so, use that explicitmapping.

2. If the preceding step fails, check if the internal format (if specified) contains a field namethat is identical to the external field name. If so, create a mapping between the two field names,with appropriate type conversions.

3. If the preceding step fails, and the external field is not marked as external_only, create anew field in the target_internal with the same field name and the appropriate type. Thencreate a mapping from the external field to this new internal field.

The type of the created internal field is controlled by the external field type.

Ultra Type mapped toExternal Type

Maps to the corresponding primitive internal type. This depends on the ex-ternal format specification.

Primitive external

60


Unless a using in_map specification is applicable, a new in_map isautomatically created to handle the mapping between the external sub-record

External format (sub-re-cord)

and the internal field. The automatic map has by default no specified in-ternal or target_internal name, but this can be modified withautomatic mapping specifications.

Maps to an internal list type, where the list element type is deduced fromthe external list element type according to these rules (recursively).

List

15.2.1. target_internal

When using automatic, a new internal format may be automatically generated if any new internalfields are needed for the automatic mapping. The name of this new internal format is given by thetarget_internal specification of an in_map declaration. If no target_internal namespecification is given, the format will still be generated if needed, but it will have no usable name (thatis, the format cannot be directly referenced in APL).

If a target_internal name has been given, this format will be created even if no new fields arecreated.

If the in_map has a specified internal format, the generated format is a subtype of this format (hence,inherits all its fields).

15.2.2. Automatic Mapping Specifications

It is possible to modify the behavior of the automatic mapping algorithm when generating the sub-maps. The most direct way to do this is to specify how each external type will be mapped. This canbe done either by explicitly specifying the sub-map or just specifying the internal and/or tar-get_internal name.

Syntax

The two forms of mapping specifications are:

<external type>: <in_map (target_)internal options> ;

and

<external type>: using in_map <map name>;

61


Example 26.

internal IntBase { int recordSequenceNumber;};asn_block { ExtDataFile ::= CHOICE( mc MobileCall, dp DataPacket) MobileCall ::= SET ( ... ) DataPacket ::= SET ( ... )};

in_map InMapDataFile : external(ExtDataFile), target_internal(IntDataFile) { automatic;};

In this case, two new internal types will be automatically generated for MobileCall andDataPacket. However, these types may require to be named in a certain way, or that they inheritthe fields of some declared base internal format.

This can be accomplished by adding some automatic mapping specifications:

Example 27.

in_map InMapDataFile :external(ExtDataFile),target_internal(IntDataFile) { automatic { MobileCall: internal(BaseInt), target_internal(TIMobileCall); DataPacket: internal(BaseInt), target_internal(TIDataPacket); };};

This could also be done by specifying a using in_map declaration. For instance, the previous ex-ample is equivalent to the more complicated one:

62


Example 28.

in_map MobileCallMap: internal(BaseInt), target_internal(TIMobileCall) { automatic;};

in_map DataPacketMap: internal(BaseInt), target_internal(TIDataPacket) { automatic;};

in_map InMapDataFile :external(ExtDataFile),target_internal(IntDataFile) { automatic { MobileCall: using in_map MobileCallMap; DataPacket: using in_map DataPacketMap; };};

The type map specifications are inherited by every automatically generated sub-map:

Example 29.

asn_block {... MobileCall ::= SET (location Location;)...};external Location { int longitude; int latitude;};

in_map InMapDataFile : external(ExtDataFile), target_internal(IntDataFile) { automatic { MobileCall: internal(BaseInt), target_internal(TIMobileCall); DataPacket: internal(BaseInt), target_internal(TIDataPacket); Location: internal(BaseInt), target_internal(TILocation); };};

In this example, even though Location is not a part of the direct mapping of ExtDataFile itself,the type map information will still be considered when creating TIMobileCall.

63


15.2.3. use_external_names Automatic Option

When declaring a target_internal name for a large number of sub-formats, one alternative isto use the use_external_names option on the automatic block. This will cause all generatedmaps to get a target_internal name that is identical to the name of the external format.

Example 30.

in_map AMARecord_Map : external(AMARecord), target_internal(AMARecord) { automatic { RecordOwnerType : target_internal(RecordOwnerType); RecordOwnerDN : target_internal(RecordOwnerDN); Package_100 : target_internal(Package_100); Package_101 : target_internal(Package_101); Package_102 : target_internal(Package_102); };};

The use_external_names specification will ease the syntax by removing the need for all thetarget_internal specifications.

Syntax - use_external_names

use_external_names(<external 1>, <external 2>, ...)

The previous example includes all of the specified external definitions within the file. Hence, thein_map could be rewritten as:

Example 31.

in_map AMARecord_Map : external(AMARecord), target_internal(AMARecord) { automatic : use_external_names;};

It is possible to combine this with automatic mapping specifications. In this case the explicit specific-ations will override the use_external_names behavior.

15.2.4. Automatic Block Internal Specification

It is also possible to append internal type specifications to the constructed in_maps:

64


Example 32.

in_map AMARecord_Map : external(AMARecord), target_internal(AMARecord) { automatic : use_external_names, internal(myInternal);};

If automatic mapping specifications are in use, these will override the internal specifications on theautomatic level (even if they only specify target_internal).

15.3. discard_output SpecificationSometimes it is necessary to describe parts of the external format as an in_map even though you arenot interested in processing the decoded data. That is, when there are more complex types of filler dataand this information is to be discarded.

Example 33.

in_map FillerInMap: external(FillerRecord), internal(EmptyInternal), discard_output { automatic;};

decoder DataInMap: external(DataRecord), target_internal(DataTarget) { automatic;};

decoder TotalDecoder: in_map(FillerInMap), in_map(DataInMap);

This example will produce DataTarget UDRs and discard any filler (assuming that the referredformats have been properly defined).

65


16. Out-MapsOut-maps are used in encoders to map an internal format to an external format.

Example 34.

internal IFormat1 { int f1; ascii f2;};

external EFormat1 { ascii f1 : static_size(2), terminated_by(" "); ascii f2 : static_size(10), terminated_by(" ");};

out_map OutMap : internal(IFormat1), external(EFormat1) { automatic;};

The automatic mapping for out-maps only attempts to bind every field name of the external format tothat of the internal format. No additional formats or fields are created as in the case of in-maps.

Like in-maps, out-maps support explicit mappings.

16.1. Optional FieldsThere are some additional considerations for the case where optional external fields are encoded.

An internal field can be defined as optional, enabling to encode the corresponding external field as notpresent. Note that internal fields with value null may still be considered present and is encoded as,for instance, an empty string. To override this, use the APL command udrUnsetPresent prior toencoding.

Consider the following example, where an optional field within an incoming ASN.1 record will beencoded (the definition of myInt varies).

66


Example 35. Encoding Optional Fields

asn_block { main_udr ::= SEQUENCE { fieldA [APPLICATION 5] INTEGER OPTIONAL };};

in_map inM : external( main_udr ), internal( myInt ) { automatic;};

out_map outM : internal( myInt ), external( main_udr ) { automatic;};

decoder myDEC : in_map( inM );

encoder myENC : out_map( outM );

Case 1 - Optional internal field:

If fieldA is defined as optional in the internal format definition, it will be present in the encodedrecord if it was present in the internal record, even if the value is null. It will not be presentif it was not present in the internal record.

internal myInt { int fieldA : optional;};

Case 2 - Mandatory internal field:

If fieldA is defined as mandatory in the internal format definition, it will always be presentin the encoded record.

internal myInt { int fieldA ;};

Case 3 - No mapped internal field:

If fieldA does not exist in the internal format definition, it will never be present in the encodedrecord.

internal myInt {

67


};

68


17. DecodersA decoder specifies how data is arriving from a source. There are two basic types of decoders:

• Simple decoders - Decode one or more external formats.

• Constructed decoders - Coordinate the decoding between multiple simple or constructed decoders.

17.1. Simple DecodersA simple decoder is declared as follows:

Syntax - simple decoder

decoder <name> : <decoder options>;

The decoder options are:

Specifies what in-maps to use. At least one is required.in_map(<map name>)

Specifies that this is a blocked format with the specified blocksize.

block_size(<size>)

Specifies the block filler used. This option has no effect ifblock_size has not been specified.

terminated_by(<terminat-or>)

The decoder may contain one or several in-maps, depending on whether it manages a single or multiple(mixed) record type. For multiple maps, the corresponding external records except the last one mustsupport identification. The decoder will try each in_map in the specified order. The first one forwhich the identification criteria are met will be used.

How the record identification is specified depends on the actual external record type. For example,sequential external records must use the identified_by option while BER encoded records supportidentification by the standard tagging scheme.

Simple decoders may be given blocking information through the block_size and terminated_byconstructs. The block_size parameter contains the size of a block and the terminated_byparameter specifies the start of padding character.

Example 36.

decoder SimpleDecoder : in_map(Map1), in_map(Map2), block_size(2048), terminated_by(0x00);

This decoder starts by reading the next byte and evaluates if it equals to 0x00. Should it be 0x00, itwill jump to the next even boundary of 2048 and repeat this procedure. If it is not 0x00, it will checkthe identification for both of the externals specified in Map1 and Map2 (in that order).

69


17.2. Constructed DecodersA constructed decoder is defined in terms of a number of other simple or constructed decoders. Theyare used to specify the decoders to be used in sequence, for example to specify separate decodersmanaging header, trailer, and record information. For instance, three external record types (Rheader,Rudr, Rtrailer), with corresponding in_maps (Mheader, Mudr, and Mtrailer) and decoders(Dheader, Dudr, and Dtrailer).

The following constructed decoder specifies the header record to be decoded first. Then any numberof UDR records are decoded, followed by a single trailer record.

Example 37.

decoder TTFile { decoder Dheader; decoder Dudr *; decoder Dtrailer;};

As seen, the constructed decoder does not (and cannot) have any decoder options.

The asterisk after Dudr indicates that zero or more such entries can occur before a terminating trailerrecord. For such a sub-decoder, the constructed decoder will switch to the next decoder when the sub-decoder cannot handle the input (as deduced by the identification criteria of the in_maps in the sub-decoder). In this case this means that the Dudr decoder must support identification logic, or theDtrailer decoder will never be reached (and the decoder will abort with an "Unexpected EOF" error).

This example could also have been supported with a simple decoder:

Example 38.

decoder TTFile : in_map(Mheader), in_map(Mudr), in_map(Mtrailer);

The difference is that the order of header, UDR, and trailer would not be enforced for the simple decoder.The simple decoder will also not work if, for instance, the header does not support identification.

70


18. EncodersAn encoder specifies how data is to be encoded. The syntax is:

Syntax - encoders

encoder <encoder_name> : <encoder options>;

The encoder options are:

Specifies what out-maps to use. At least one is required.out_map(<map name>)

Specifies that this is a blocked format with a certain block size.block_size(<size>)

Specifies the block filler used. This option has no effect ifblock_size has not been specified.

terminated_by(<terminat-or>)

When encoding a record, the encoder tries each out-map in the order specified. If the out-map canencode the record, then this out-map is used, otherwise the next out-map is tried.

An out-map can encode the data if:

1. The record type matches the internal type specified in the out-map.

2. All format specific requirements are met. At the moment this evaluation is performed only forsequential data with an identified_by condition. Only data where the mapped fields meetthe identification rule will be accepted. This is to support mapping to different external recordtypes from the same internal type.

There is no such thing as a constructed encoder. If the records are required to be forwarded ina specific order, an APL node handling the output logic must precede the Encoder.

71


19. A Constructed Decoder ExampleFor some cases, a so called constructed Decoder is useful. The main advantage is that it introducessome validation logic, making it possible to check the order of the arriving records. For instance,suppose the incoming files contain one header and one trailer which must be present at file start andend, in order for the file to be accepted. In between, data records may or may not be present. The datarecords can be of two types. Note that headers and trailers are considered being records as well, sothere will actually be four record types in this format definition.

The source file for which a decoder is exemplified in this appendix:

Figure 26. An example of the source files discussed in this section.

19.1. Headers and TrailersSince headers and trailers are treated as records, the same syntax as for external records apply. Notethat APL syntax can be used within UFDL code.

external FileHeader { ascii header : terminated_by(0xA);};

The first line is always a header. Note that no identified_by is needed, since the decoder willnot check the input stream to see if the next record is a header or not. This compared to whether toread a TypeA or TypeB record, when an identification test is called for.

external FileTrailer : identified_by( strStartsWith( trailer, "Date") ){ ascii trailer : terminated_by(0xA);};

The identified_by for the trailer is not crucial, however, it provides additional validation duringdecoding, since it evaluates that the trailer really starts with "Date".

19.2. internal

Suppose it is desired to output one record type as a replacement for the incoming types A and B. Thenthe simplest way is to create a mutual internal.

72


internal MyInternal {

// These are common fields to TypeA and TypeB

string RecordType; string SequenceNumber; string A_number; string B_number;

// These may or may not be present depending on // record type

string CallingCountry: optional; string LocalAreaCode: optional; string SequenceNumber: optional; string Duration: optional; string CauseForOutput: optional; string CalledCountry: optional;};

Both TypeA and TypeB records are mapped to MyInternal. The common fields will always be set.The others are defined as optional, hence, their presence depends on the record type. Note, the Record-Type in the internal type is needed for encoding, since the encoder needs to check the record type todecide whether to encode as TypeA or TypeB.

19.3. external

The definition of TypeA and TypeB is fairly straight forward. Note, no encode_value for Record-Type is set, since this is evaluated from the internal UDR during encoding (refer to previous section).

external TypeA : identified_by( RecordType == "A" ), terminated_by(0xA) { ascii RecordType : static_size(2), terminated_by(":"); ascii A_number : terminated_by(":"); ascii B_number : terminated_by(":"); ascii SequenceNumber : terminated_by(":"); ascii Duration : terminated_by(0xA);};

external TypeB : identified_by( RecordType == "B" ), terminated_by(0xA) { ascii RecordType : static_size(2), terminated_by(","); ascii CallingCountry : terminated_by(","); ascii SequenceNumber : terminated_by(","); ascii LocalAreaCode : terminated_by(","); ascii A_number : terminated_by(","); ascii B_number : terminated_by(","); ascii CauseForOutput : terminated_by(","); ascii CalledCountry : terminated_by(0xA);

73


};

19.4. in_map

Both record types A and B are mapped to the same internal. This approach is useful to simplify APLsyntax within processing (a lot of if-statements used to determine the record type, may be eliminated),or in case one resulting output type is produced.

TypeA and TypeB are both mapped to MyInternal (see previous section).

in_map TypeA_in : external( TypeA ), internal( MyInternal ) { automatic;};

in_map TypeB_in : external( TypeB ), internal( MyInternal ) { automatic;};

The headers are not wanted in processing, therefore discard_output is set. However, the tar-get_internal is still useful since it enables to produce headers for encoding.

in_map Header_in : external( FileHeader ), target_internal( Header ), discard_output { automatic;};

// The trailer gets a special record type.

in_map Trailer_in : external( FileTrailer ), target_internal( Trailer ) { automatic;};

19.5. decoder

The following constructed decoder definition will expect all batches to start with a header, end witha trailer and have zero/one/several A and B records in between. If not, the decoder will abort.

// The sub-decoders.

decoder Records : in_map( TypeA_in ), in_map( TypeB_in );decoder Header : in_map( Header_in );decoder Trailer : in_map( Trailer_in );

// The total (file) decoder. // '*' means zero/one/several records are expected between// one header and one trailer for each file collected.

74


decoder Total { decoder Header; decoder Records *; decoder Trailer;};

19.6. out_map

Suppose we want to encode back to the original format.

out_map TypeA_out: external(TypeA), internal( MyInternal ) { automatic;};

out_map TypeB_out: external(TypeB), internal( MyInternal ) { automatic;};

out_map Trailer_out: external(FileTrailer), internal(Trailer) { automatic;};

out_map Header_out: external(FileHeader), internal(Header) { automatic;};

The out-maps and encoder are simple. Note that TypeA and TypeB both are encoded from MyInternal.Which type to use depends on the value of the RecordType field.

19.7. encoder

A constructed encoder cannot be created. Hence, the following encoder definition will not care for theorder of arriving records, nor that all types must be present in the output file.

encoder Total: out_map( TypeA_out ), out_map( TypeB_out ), out_map( Header_out ), out_map( Trailer_out );

75


20. A Sequential Format ExampleThe example illustrated in this appendix is a format definition for decoding a shortened version ofEWSD AMA (Automatic Message Accounting).

All incoming records are of the same record type, but their contents vary. The first four fields arepresent in all records, the fifth field varies depending on the contents of the field RecordOwnerType-Present. The last three fields are optional.

Figure 27. A schematic representation of the example discussed in this section.

20.1. external

Since the last three optional fields (including RecordOwnerType and RecordOwnerDN) consistof several fields, they will each be defined as its own type. That is, an external sequential format.

Note the FillerRecord_0x00_EXT construct. This is the padding which may be present betweenrecords. Hence, it is defined as a record type identified by the decoder, but not routed on to the sub-sequent node (refer to the in_map definitions).

If the format is only used for decoding (which is the normal case for switch output formats), the encodinginstructions (encode_value) in the following code is skipped.

external AMARecord_EXT: identified_by(RecordIdentifier == 0x84), dynamic_size(RecordLength)

{ int(little_endian) RecordIdentifier :static_size(1), external_only, encode_value(0x84); int(little_endian) RecordLength :static_size(2), external_only, encode_value(udr_size);

This byte contains several flags, from which only one is of interest. Therefore, if RecordOwnerType-Present is set, then the RecordOwnerType is present or else the RecordOwnerDN data is. Seethe presence specifications in the following code.

bit_block :static_size(1) { int(little_endian) RecordOwnerTypePresent: msb(7),lsb(7),

76


external_only, encode_value( (field_present(RecordOwnerType)?1:0)); };

Since three bytes of unwanted data is present, this data is specified as external_only to stop thefield from getting automatically generated in the target internal. No encoding is specified (0 paddingis used).

byte ignoredFields: static_size(3), external_only;

Either RecordOwnerType or RecordOwnerDN is present. For encoding it is important to notethat exactly one of these fields is to be present in the output data.

int(little_endian) RecordOwnerType: static_size(1), present if(RecordOwnerTypePresent == 1); RecordOwnerDN_EXT RecordOwnerDN: present if(RecordOwnerTypePresent == 0);

The rest of the record consists of optional packages with additional information. The full AMA formatcontains a lot of other packages (and additional information in the header), however, in this exampleonly three packages are included. Any unrecognized package means that the decoding will fail, sincethe size of the set is specified (all the remaining data must be handled by the set decoding).

set: dynamic_size(remaining_size) { Package_100_EXT DateTimeDuration : optional; Package_101_EXT PartnerDirectoryNumber : optional; Package_102_EXT ServiceInfo : optional; };};

external RecordOwnerDN_EXT { bit_block : static_size(1) { int(little_endian) LACLength : msb(7), lsb(4); int(little_endian) OwnerIDLength : msb(3), lsb(0), external_only, encode_value( strLength( LACAndDN )); };

The following is a typical construction for BCD data with a nibble length specification. Note that bothnibble size (native_size) and field size (dynamic_size) must be specified.

bcd LACAndDN : dynamic_size((OwnerIDLength+1)>>1),

// Alternative syntax: // dynamic_size((OwnerIDLength+1)/2)

native_size(OwnerIDLength);};

77


//Package with PackageNumber 100 (0x64):external Package_100_EXT: identified_by(PackageNumber == 0x64){ int(little_endian) PackageNumber : static_size(1), external_only, encode_value( 0x64 ); int(little_endian) Year : static_size(1); int(little_endian) Month : static_size(1); int(little_endian) Day : static_size(1); int(little_endian) Hour : static_size(1); int(little_endian) Minutes : static_size(1); int(little_endian) Seconds : static_size(1); int(little_endian) Flags : static_size(1); int(little_endian) Duration : static_size(3);};

// Package with PackageNumber 101 (0x65). external Package_101_EXT: identified_by(PackageNumber == 0x65) { int(little_endian) PackageNumber: static_size(1), external_only, encode_value( 0x65 ); int(little_endian) NumberOfDigits: static_size(1), external_only, encode_value( strLength( Digits ));

// Again the typical BCD decoding specification

bcd Digits: dynamic_size((NumberOfDigits+1)/2), native_size(NumberOfDigits);};

// Package with PackageNumber 102 (0x66). external Package_102_EXT: identified_by(PackageNumber == 0x66){ int(little_endian) PackageNumber : static_size(1), external_only, encode_value( 0x66 ); int(little_endian) ServiceIndicator : static_size(1); int(little_endian) AdditionalInformation : static_size(1); int(little_endian) Flags : static_size(1);};

Note the identified_by expression, which must be specified for any format used in a set construct.

// Filler needed to be able to recognize on the input stream:external FillerRecord_0x00_EXT: identified_by(RecordIdentifier == 0x00), static_size(32){ int(little_endian) RecordIdentifier : static_size(1), external_only, encode_value(0x00);

// Note that no other fields are specified. // The UDR size (32) will be consumed and discarded (see in_map).

};

78


20.1.1. Alternative Syntax

As an alternative to the set construct used in the external definition for AMARecord_ext,switched_set could be used. This will have an impact on the syntax for the Package_*_EXTtypes. Note that only the syntax differing from the original example is shown. The main reason forusing switched_set instead of set, is when performance must be increased.

external AMARecord_EXT:

// ...// All preceding fields according to the original specification.// Only the set construct is replaced with switched_set.// ...

switched_set(PackageNumber): dynamic_size(remaining_size) { case( 0x64 ) { Package_100_EXT DateTimeDuration; }; case( 0x65 ) { int(little_endian) NumberOfDigits_101: static_size(1), external_only, encode_value( strLength( Digits_101 ));

// Again the typical BCD decoding specification.

bcd Digits_101: dynamic_size((NumberOfDigits_101+1)/2), native_size(NumberOfDigits_101); }; case( 0x66 ) { int(little_endian) ServiceIndicator_102 : static_size(1); int(little_endian) AdditionalInformation_102 : static_size(1); int(little_endian) Flags_102 : static_size(1); }; };};

external RecordOwnerDN_EXT { bit_block : static_size(1) { int(little_endian) LACLength : msb(7), lsb(4); int(little_endian) OwnerIDLength : msb(3), lsb(0), external_only, encode_value( strLength( LACAndDN )); };

The following is a typical construction for BCD data with a nibble length specification. Note that bothnibble size (native_size) and field size (dynamic_size) must be specified.

bcd LACAndDN : dynamic_size((OwnerIDLength+1)>>1), native_size(OwnerIDLength);};

// Package with PackageNumber 100 (0x64). external Package_100_EXT

79


{ int(little_endian) Year : static_size(1); int(little_endian) Month : static_size(1); int(little_endian) Day : static_size(1); int(little_endian) Hour : static_size(1); int(little_endian) Minutes : static_size(1); int(little_endian) Seconds : static_size(1); int(little_endian) Flags : static_size(1); int(little_endian) Duration : static_size(3);};

Note that no identified_by or PackageNumber is present, since this is handled in the containingswitched_set.

20.2. internal

No internal is used. For this case, the target_internal is sufficient; all field names and fieldtypes are in order and there is only one type of record present in the input and no additional fields arerequired.

20.3. in_map

The padding in the records is recognized by the decoder, but is not actually mapped in to the systemdue to the use of the discard_output flag.

The AMARecord_Map contains sub-automatic specifications (the target_internal specificationswithin the automatic block), which will give five additional internal formats (besides the AM-ARecord). This is useful when it is desired to route them as individual records.

in_map FillerRecord_0x00_Map : external(FillerRecord_0x00_EXT), target_internal(FillerRecord_0x00), discard_output { automatic; };

in_map AMARecord_Map : external(AMARecord_EXT), target_internal( AMARecord ) { automatic { RecordOwnerType_EXT : target_internal( RecordOwnerType ); RecordOwnerDN_EXT : target_internal( RecordOwnerDN ); Package_100_EXT : target_internal( Package_100 ); Package_101_EXT : target_internal( Package_101 ); Package_102_EXT : target_internal( Package_102 ); };};

80


20.4. decoder

The padding pseudo-records and data records can arrive in any order, therefore there is no need todefine a constructed decoder.

decoder AMAFile : in_map( AMARecord_Map ), in_map( FillerRecord_0x00_Map );

81


21. An ASN.1 Format ExampleThis appendix shows how incoming ASN.1 records can be encoded into a sequential external format.

An ASN.1 format definition can be pasted directly into an Ultra asn_block. Nested structures willbe saved as they are in the target_internal type, but the nested fields cannot easily be encodedto a different format without the use of an APL node (Analysis or Aggregation). In this example, theincoming ASN.1 record is encoded to a sequential record, mapping all the field values to correspondingfields in the sequential format.

This sort of mapping - nested structures to plain, sequential structures - can be accomplished in threeways:

• By creating a constructed external format definition, that is, where the external record definitionconsists of sub-externals.

• By extending the target_internal with first level temporary fields which will hold the valuesof the nested fields to be encoded.

• By creating a new UDR which is populated with values from the incoming nested UDR.

21.1. Constructed InternalIt is possible to encode to a constructed sequential external, that is, an external sequential definitioncontaining other externals to represent the nested fields. The disadvantage with this approach is thatit is not possible to mix fields of different levels in the produced output record.

21.1.1. external

Name the fields in the outgoing external exactly as in the incoming ASN.1 structure. This allows theuse of automatic mapping.

asn_block {

exchangeRec DEFINITIONS IMPLICIT TAGS ::=BEGIN

main_udr ::= SEQUENCE{ duration [ APPLICATION 1 ] INTEGER OPTIONAL, calledNumber [ APPLICATION 2 ] INTEGER OPTIONAL, callingNumber subUDR1}

subUDR1 ::= [ APPLICATION 3 ] SEQUENCE { category [ APPLICATION 4 ] INTEGER OPTIONAL, adressString subUDR2 }

subUDR2 ::= [ APPLICATION 5 ] SEQUENCE { number [ APPLICATION 6 ] INTEGER OPTIONAL, ton [ APPLICATION 7 ] INTEGER OPTIONAL, npi [ APPLICATION 8 ] INTEGER OPTIONAL

82


}

END};

//-------------------------------------------------------

external out { ascii duration : static_size(2); ascii calledNumber : static_size(8); subUDR1 callingNumber : static_size(8);};

external subUDR1 { ascii category : static_size(2); subUDR2 adressString : static_size(6);};

external subUDR2 { ascii number : static_size(2); ascii ton : static_size(2); ascii npi : static_size(2);};

21.1.2. in_map and out_map

Automatic mapping considers sub-UDRs as well.

in_map inM : external( main_udr ), target_internal( myTI ) { automatic;};

out_map outM : internal( myTI ), external( out ) { automatic;};

21.1.3. decoder and encoder

decoder myDec : in_map( inM );

encoder myEnc : out_map( outM );

21.2. Extending the target_internal

Create an internal which will hold the nested fields to be mapped to the sequential format. Definea target_internal holding both the asn_block and the internal. The values of the nested

83


fields of the target_internal are copied to the fields added with the internal format specification,using APL code.

Figure 28. APL code is needed to extract the nested values.

Format Definition (shortened):

asn_block {: main_udr:};

Note that all the fields in the following are declared as optional. This is to be able to differentiatebetween the two cases of absent value and a value of zero (default for int type). Thus, if no value isentered it will be encoded as empty in output format, as opposed to 0 (zero).

internal exchangeRec_Int { int duration_i : optional; string calledNumber_i : optional; int category_i : optional; string number_i : optional; string ton_i : optional; string npi_i : optional;};

Note that since no automatic mapping specifications are given, no named internal types for SubUDR1and SubUDR2 are received. This is not a problem as long as there is no need for referencing the typesdirectly (for instance in APL).

in_map exchangeRecord_MAP_IN: external(MainUdr), internal(exchangeRec_Int), target_internal(exchangeRec_TI){ e:duration and i:duration_i; e:calledNumber and i:calledNumber_i; automatic;

84


};

A structure of sub-UDRs with the same field names as in the internal to map from is created (that is,the target_internal which has the same structure as the ASN.1 external). This will onlyproduce a line-based comma separated output file.

external ConstructedOut: terminated_by("\n") { ascii duration : terminated_by(","), int(base10); ascii calledNumber : terminated_by(","); SubOut1 callingNumber;};

external SubOut1: terminated_by("\n") { ascii category : terminated_by(","), int(base10); SubOut2 adressString;};

external SubOut2: terminated_by("\n") { ascii number : terminated_by(","); ascii ton : terminated_by(","); ascii npi : terminated_by("\n");};

out_map Constructed_Map: external(ConstructedOut), internal(exchangeRec_TI) { e:duration and i:duration_i; e:calledNumber and i:calledNumber_i; automatic;};

encoder ConstructedEnc: out_map(Constructed_Map);

APL Code:

consume {

if ( udrIsPresent( input.callingNumber.adressString ) ) { input.category_i = input.callingNumber.category; input.number_i = input.callingNumber.adressString.number; input.ton_i = input.callingNumber.adressString.ton; input.npi_i = input.callingNumber.adressString.npi; }

udrRoute( input );}

85


21.3. Creating a New UDRCreate an internal for the sequential format, but do not add it to the target_internal of theincoming ASN.1 structure. Instead, for each incoming UDR, create a new UDR and copy the fieldvalues from the ASN.1 UDR. The ASN.1 UDR can then be discarded, routing the new internal further.

Figure 29. Internal presentation of the input ASN.1 UDR.

Figure 30. Internal presentation of the output sequential UDR.

APL Code Definition:

exRec.exchangeRec_Int outUDR = udrCreate( exRec.exchangeRec_Int );

consume { outUDR.duration_i = input.duration; outUDR.calledNumber_i = input.calledNumber;

if ( udrIsPresent( input.callingNumber.adressString ) ) { outUDR.category_i = input.callingNumber.category; outUDR.number_i = input.callingNumber.adressString.number; outUDR.ton_i = input.callingNumber.adressString.ton; outUDR.npi_i = input.callingNumber.adressString.npi; }

udrRoute( outUDR );}

21.3.1. external

The ASN.1 definition can be copied directly into an Ultra asn_block definition. An externalfor the sequential outgoing UDRs is created.

Format Definition (shortened):

asn_block {:

86


main_udr:};

// The same field names as in the internal format are used// to be able to use automatic mapping.

external exchangeRecSEQ { ascii duration_i : terminated_by(","), int(base10); ascii calledNumber_i : terminated_by(","); ascii category_i : terminated_by(","), int(base10); ascii number_i : terminated_by(","); ascii ton_i : terminated_by(","); ascii npi_i : terminated_by("\n");};

21.3.2. internal

An internal, containing fields matching the external field names, is created in order to hold thevalues to be encoded.

internal exchangeRec_Int { int duration_i : optional; string calledNumber_i : optional; int category_i : optional; string number_i : optional; string ton_i : optional; string npi_i : optional;};

21.3.3. in_map and out_map

The incoming external is turned into a target_internal, without adding any fields. The in-ternal is mapped to the sequential external format.

in_map exchangeRecord_MAP_IN_II: external(main_udr), target_internal(exRec_TI){ automatic;};

out_map exchangeRecord_MAP_OUT: external(exchangeRecSEQ), internal(exchangeRec_Int) { automatic;};

21.3.4. decoder and encoder

decoder exchangeRec: in_map(exchangeRecord_MAP_IN);

87


encoder exchangeRecSEQ: out_map(exchangeRecord_MAP_OUT);

88


22. An XML Format ExampleThe format definition in this appendix decodes collected XML data files.

An XML schema definition can be pasted into a Convergent Mediation Engine™ xml_schema block.Incoming XML UDRs are often terminated by one or several whitespace characters. This is handledby introducing another temporary record type - the whitespace, which is recognized but not ignoredwhen mapping.

Suppose there is an input XML data file according to the following image, for which a format definitionwill be created:

Figure 31. An XML data file.

22.1. external

Paste the XML schema definition as it is, into a UFDL xml_schema block.

xml_schema {

<?xml version="1.0" encoding="ISO-8859-1"?><schema xmlns = "http://www.w3.org/2001/XMLSchema">



<element name="MerchantRequest"> <complexType> <sequence> <element ref="Transaction" use="required"/> </sequence> <attribute name="version" type="string" /> <attribute name="operator" type="string" /> </complexType></element>

<element name="Transaction"> <complexType> <sequence> <element name="UserID" type="string"/>

89


<element ref="Category"/> <element name="Reason" type="string"/> </sequence> <attribute name="timestamp" type="string" use="required" /> </complexType></element>



<element name="Category"> <complexType> <simpleContent> <extension base="string"> <attribute name="id" type="string" /> <attribute name="text" type="string" /> </extension> </simpleContent> </complexType></element>

</schema>};

One problem with XML decoding is that a pure XML format does not include any trailing whitespacein the input data. This can often be present (and it is valid XML), so it is often useful to add anotherin_map to the decoder that discards any whitespace between XML data blocks.

external WhiteSpace : identified_by( value == 0x20 || value == 0xA || value == 0xD ) { int value: static_size(1);};

22.2. in_map

in_map MerchantRequest_map: external(MerchantRequest), target_internal(MerchantRequest) { automatic;};

The discard_output needs to be specified in order to discard the whitespace.

in_map WS_map: external(WhiteSpace), target_internal(WS_int), discard_output { automatic;};

90


22.3. decoder

The WS_map is set first, to remove as much unnecessary whitespace as possible from the processeddata.

decoder MIA_decoder: in_map(WS_map), in_map(MerchantRequest_map);

91


Web Interface 4.1


Version 1.15

1





2

Web Interface 4.1



2. Overview .................................................................................................................. 52.1. Startup ........................................................................................................... 5

3. Functionality ............................................................................................................. 63.1. Logout ........................................................................................................... 63.2. Pico Viewer ..................................................................................................... 6

3.2.1. Pico Viewer Details ................................................................................ 63.3. System Log Inspector ........................................................................................ 7

3.3.1. System Log Details ................................................................................ 83.4. Task Manager .................................................................................................. 93.5. Workflow Manager ......................................................................................... 103.6. User Settings ................................................................................................. 11

3

Web Interface 4.1

1. IntroductionThis document describes the Convergent Mediation Engine™ Web Interface, used to access selectedparts of the Nokia Convergent Mediation Engine™ system using a standard web browser.



1.2. Abbreviations and TermsAbbreviations and terms used in the Convergent Mediation Engine™ user documents are describedin the Convergent Mediation Engine™ Core - User Guide.

4

Web Interface 4.1

2. OverviewThe Web Interface tool enables access to selected parts of the Convergent Mediation Engine™ system,using a standard web browser only. Without any additional software installed on the client, it is possibleto perform the following actions.

• Pico monitoring

• System Log examination and filtering

• Task monitoring, activation and deactivation

• Workflow monitoring, activation and deactivation

The Web Interface resembles the user interface in terms of layout and interaction. HTML forms havevery limited functionality, hence, there are differences in terms of component outlooks. Also, theHTTP protocol only offers the request/response technique. Thus, individual areas of a displayed pagecannot be updated in real-time. HTML offers a meta-tag that defines the complete page to be updatedat given intervals.

Each page of the Web Interface holds the Convergent Mediation Engine™ logo, name and IP addressof the connected Convergent Mediation Engine™ system, logged on user and the main menu.

Any existing user will be able to log into the Web Interface. However, what tools the user willhave access to is previously defined in the Access Controller in the Convergent MediationEngine™ User Interface. For further information, please refer to Convergent Mediation Engine™Core - User Guide.

2.1. StartupTo access the Web Interface, open a standard web browser. Type in the following into the URL fieldof the browser: http://<hostname>:<port>/mz/main

If encryption is enabled, the URL is https://<hostname>:<port>/mz/main

Please refer to Convergent Mediation Engine™ Core - User Guide, for further information onhow to enable encryption.

Represents the IP-address or the name of the host on which Convergent Mediation En-gine™ is running. For instance;

<hostname>

• 10.0.1.101

• medhost

The <hostname> is set to the default IP-address of the server. In order to link thewebinterface to another address, edit the mz.wi.serveraddress property inplatform.xml:<property name="mz.wi.serveraddress" value="10.0.1.99"/>

Represents the port on the host through which the connnection is established. The defaultport is 9000. In case another port is used, the platform.xml file needs modification,

<port>

followed by a restart of Convergent Mediation Engine™ in order for the new port con-figuration to take effect.

5

Web Interface 4.1

3. FunctionalityThis section describes the tools available through the Web Interface. The main menu is available atall times to ease navigation between the tools. It is placed to the left on each page.

3.1. LogoutSelecting this button will log out the user and display the Login page.

3.2. Pico ViewerThe Pico Viewer page displays a list of all pico clients currently available in the system. This pagecan be automatically updated at a user defined interval, see Section 3.6, “User Settings”.

Figure 1. The Pico Viewer.

Holds the Convergent Mediation Engine™ pico clients: Code Server, Desktop, ExecutionContext etc.

Name

To view additional information, select the pico viewer of interest by clicking it. This willlaunch a Pico Viewer Details page, see Section 3.2.1, “Pico Viewer Details”.

Holds the time the pico client was last activated.Start Time

Used, available and maximum memory on the hosting JVM (Java Virtual Machine) inthat order. A JVM may host several pico clients.

Memory

The time the pico client needed to respond to a connected request from the server.Response

Manually updates the page.Refresh

3.2.1. Pico Viewer Details

To view additional details, select the execution context of interest by clicking it. This will launch thePico Viewer Details page. The page holds information on the OS and Java versions on which the picoclient is running.

6

Web Interface 4.1

3.3. System Log InspectorInitially the Search System Log page is displayed, where the user may enter criteria needed to browsefor certain messages and errors encountered in the Convergent Mediation Engine™ system.

If a search previously has been carried out, a new search is performed. The conditions specified in themost recent search are repeated and the result is shown. To change the search criteria, select Searchat the top of the page.

Figure 2. The Search System Log page.

Which part of system that reported the entry. At least one must be enabled.Log Area

Type of severity. At least one type must be enabled.Severity Type

Start date of the period from which entries will be viewed. Year, month and day isselected.

Period - From

Finishing date of the period from which entries will be viewed. Year, month and dayis selected.

Period - To

Contains options to filter out data on specific workflows.Workflow

Enables to report actions performed by a specific user.User

Text field where a string is entered. All messages containing this string will be extrac-ted.

Message

When the search criteria is specified and OK is selected, the System Log Inspector is displayed:

7

Web Interface 4.1

Figure 3. The System Log Inspector page - matched entries.

The severity of the message, could be any of the following:Sev

• I (Information) - An informative message which is logged, for instance, when a userlogs in or a workflow is activated.

• W (Warning) - A warning message is also informative, but is considered being slightlymore serious than an information message. A warning message is logged, for instance,when a workflow sends data to ECS.

• E (Error) - An error is logged when any part of the system fails, for instance, whena workflow aborts.

• D (Disaster) - Is usually never used, other than possibly for user defined agents.

Icon to show in what log area the message occurred.Area

The log message and the date when it last occured.Message

To receive additional information on a matched entry, select it by clicking it. This willdisplay the System Log Details page, see Section 3.3.1, “System Log Details”.

The name of the workflow from which the message originates.Workflow

3.3.1. System Log Details

Shows information on what date the message was logged, the IP address and the pico name from wherethe message originated and finally the messages produced.

Shows the Search System Log page, where the search criteria may be changed.Search

Shows the stack trace details. This information is advised to be included whencontacting Nokia Support in error message errands. This option is only availablefor messages of Error type.

Show Stack Trace

8

Web Interface 4.1

3.4.Task ManagerFrom the Task Manager page the state of all configured tasks within Convergent Mediation Engine™may be inspected. The user may also activate a task manually.

To receive additional information on a task select it by clicking its status. This will display the ExecutionReport page, see Section 3.3.1, “System Log Details”.

Figure 4. The Task Manager page.

The name of the task. There are a set of predefined tasks which are included by de-fault. They may not be edited, deleted or deactivated.

Task Name

• Configuration Cleaner - Manages the clean up of historic configurations.

• System Backup - Performs backups of all configurations in the system. However,it only makes a backup of the current configuration. The task also manages theclean up of backed up files.

• System Log Cleaner - Manages the clean up of the System Log.

• Statistics Cleaner - Calculates daily statistics and cleans out old statistics entriesaccording to user defined number of days to keep value.

The date and time of the last execution.Last Executed

Shows the status of the task. It is one of the following:Status

• OK - The task has terminated successfully.

• Running - The task is currently executing.

• Aborted - Returned if the task cannot execute at all, or if it did exit with statusother than 0 (zero).

The criteria controlling the next execution of the task. It is one of the following:Next Execution

• <date> - If the task is scheduled to run at a regular basis, the next planned dateand time is displayed.

• By trigger only - Displayed if the task is only activated by a trigger.

• <date>, and by trigger - A combination of date and trigger scheduling can beused. For these cases, both criteria are displayed.

9

Web Interface 4.1

• None - Displayed for tasks with no scheduling criteria. These types can only beactivated manually (select Activate).

Activates selected tasks. Even if the task is scheduled to commence later, it will runimmediately when this button is pressed.

Activation

3.5. Workflow ManagerThis page allows workflows to be monitored and started/stopped. Clicking an aborted state will displayabortion details.

Figure 5. The Workflow Manager page.

Name of workflows available for execution.Workflow

State of the workflow. If the workflow has a scheduled state, the next plannedexecution time is displayed.

State

Name of the host that the workflow is running on, and the numerical ID of theexecution context (appended after ':'). This information is only displayed whenthe workflow is active.

Execution Context

Will display a query on whether to activate the workflow. If write access is notpermitted for the workflow, an error message is produced when trying to activate.

Activation

In cases the workflow is scheduled to execute later, it will still run immediatelywhen this button is pressed.

10

Web Interface 4.1

3.6. User SettingsCertain Web Interface options may be managed through the User Settings page. The settings are storedin the current users profile to remain between the web sessions.

Figure 6. The User Settings page.

To perform a regular auto-refresh, simply fill in the Refresh Rate (sec) box to specify the interval.The service is available to the Pico Viewer, Task Manager and Workflow Manager. Setting thevalue to 0 (zero) will disable the auto-refresh functionality.

The size of the matched entries list in the System Log Inspector, may be changed. The Page Size boxdetermines the maximum number of entries to be shown on each page.

11

Web Interface 4.1

Education

Nokia Convergent Mediation Engine