User Guide Series - sparxsystems.com · Due to the decoupling of the Test Domain from the codebase, it is a simple choice of ... variables in its object or local scope. They are output

TestpointsHow to use Testpoints? In Sparx Systems

Enterprise Architect, apply constraints on modelobject behavior to applications; one TestDomain verifies many applications, withtolerance to code change, or no change to

source code if behavioral rules alter.

Enterprise Architect

User Guide Series

Author: Sparx SystemsDate: 7/08/2019

Version: 1.0

CREATED WITH

Table of Contents

Testpoints 4Test Domain Diagram 11Test Cut 14Test Set 16Test Suite 17

The Testpoints Window 18Testpoints Toolbar 22Testpoint Editor 25Testpoint Constraints 30

User Guide - Testpoints 7 August, 2019

Testpoints

Testpoints present a scheme by which constraints and rulesgoverning the behavior of objects can be taken from themodel and applied to one or more applications. Theadvantages that schemes such as this offer are tolerance tocode changes - adding and subtracting lines from a functionhas no effect on the constraints that govern it. Anotheradvantage is that changes to the behavioral rules do notrequire a corresponding change to any source code; meaningnothing has to be re-compiled!

Also, the ability to verify multiple applications using asingle test domain is a simple rather than onerous matter.The Test Domain is a both a logical and relational model;constraints in the Class model can be partitioned with TestCuts. These can be aggregated simply into Test Sets andTest Suites using connectors. Due to the decoupling of theTest Domain from the codebase, it is a simple choice ofbuttons to run a program normally, or run it for a specificTest Domain. This system also delivers practical benefits inthat no instrumentation is required at all. Test results aredisplayed in the report window during the run, in real-time,as the program runs. These results can be retained, andreviewed at any time in the 'Test Details' dialog or usingEnterprise Architect's documentation features.

(c) Sparx Systems 2019 Page 4 of 36


Features

Feature Details

TestpointComposition

Testpoint composition is performed usingthe Testpoint Window. The TestpointWindow is context-sensitive and displaysthe Test Domain for the selected elementin either the Browser window or diagram.Selecting a single Class will display theClass structure. A 'pencil' icon isdisplayed against Classes and methodsthat have existing constraints.When you select a Test Cut, Set or SuiteTest, the Testpoint window displays theentire Domain structure, including all theClasses that make up the domain. Note:



You can navigate the domain hierarchyusing the 'Navigation' pane on the right.Testpoints are composed as expressions,using the variable names of the Classmembers. The Intelli-sense shortcutCtrl+Space is available within the editorto help you find these. Expressions thatevaluate to True are taken to mean a pass.Returning False is taken to mean a fail.

You can add or edit an existing Invariantby double-clicking the Class.You can add or edit an existing pre- orpost-condition similarly bydouble-clicking the method.Double-clicking a Testpoint willautomatically display the source code if itis available.Line conditions are best added fromwithin the code editor using its shortcutmenus.This image is of a pre-condition in the



Test domain.

TestpointTraceStatements

Each Testpoint can have its own Tracestatement. The Trace statement is adynamic message that can referencevariables in its object or local scope.They are output during the evaluation of atest. They can be configured to be outputevery time a constraint is evaluated, ormore usually when a test has failed. Tracestatements can be directed to the'Testpoints' tab of the System OutputWindow, or to an external file. You canconfigure this in any Analyzer Script.

Test DomainComposition

The Test Domain diagram is a dynamicmedium where Testpoints are assembledto test Use Cases. Use Cases in a Test



Domain diagram are provided in threedifferent stereotypes: Test Cut, Test Setand Test Suite. Management of thedomain is as easy as modeling on anydiagram. The toolbox and shortcut menusprovide access to any Test DomainArtifacts. In brief, Testpoints frommultiple Classes are aggregated into TestSets. Test Sets are then linked to formTest Suites. Both Test Cuts and Test Setsare re-usable assets. Linking the sameTest Set to the one or more Test Suites isa matter of drawing connectors.

«testset»

Sell Stock

«testcut»Sell Account

«testcut»Sell Broker

«testcut»Sell Stock

Account StockBroker

Test Domainand the ClassModel

Rarely would a Use Case involve all themethods of a single Class. Most likely itis realized using a variety of methodsfrom collaborating Classes. We call this



subset of methods a cut, and the Test CutArtifact is the tool we use to make thesecuts. The Testpoint Window will adaptdepending on the context, to be that for aTest Domain or Class element. Thisimage shows the Testpoint window whena Test Cut has been selected. Note thecheckboxes, which are only visible for aTest Cut. They denote the methods (TestCut) which are contributing to a Test Set.In this example the Test domain wasgenerated by the Execution Analyzer,which did the method identification workfor us.

TestpointEvaluation

The Testpoint window is used to evaluateTest domains. The window has a toolbarfor starting or attaching to the targetapplication. The domain to test is alwaysreflected by the element that has context,so if you select a Class the window willshow only the Class structure andTestpoints of that Class. If you select aTest Suite, the window will display the



entire domain hierarchy and all theTestpoints included in it. Clicking on theRun button will load the Testpointdomain in the Execution Analyzer, whichwill then evaluate, collect and update thereport window as Use Cases pass or faileach test. The exact details of eachconstraint type and the when and how ofthat constraint's capture are:

A Class Invariant is evaluated by the·

Analyzer whenever any method calledon an object of this Class type iscompleted; the invariant serves to testthat the state of a complying object isboth known and permittedPre-conditions are evaluated·

immediately before an operation iscalledPost-conditions are evaluated (at the·

same time as a Class invariant) whenthe method is completedLine-conditions are evaluated if and·

when their specific line of code comesinto scope during program execution



Test Domain Diagram

The Test Domain diagram is the medium where youassemble and group test cases for a particular domain. Anexample of a Test domain might be 'Customer'. The breadthand depth of the domains you assemble is up to you. Youmight have separate domains for 'Add Customer' and 'DeleteCustomer', depending entirely on how you consider best tobalance the domain hierarchy. The Diagram Toolbox andShortcut menu provide a number of Artifacts to help modelthe domain. Because the medium is dynamic, allowing youto revisit and build on relationships between Test domains,the system is a great model for delivering reusable assets toan organization that are low overhead and integrate withboth the UML view of the world, and the SoftwareEngineering nuts and bolts of daily life.

Facilities

Facility Details

Test DomainGeneration

If you think the process of composing aTest Domain is complex, it can be, buthelp is at hand! The Execution Analyzercan produce a Test Domain diagram foryou. It cannot write the Tests for you, butit can do some of the leg work. It canidentify the Classes and pick out only



those methods that participated in a UseCase. And this is not guesswork. TheAnalyzer Test Domain is obtained from arunning program. This image shows theTest Domain generated by the ExecutionAnalyzer from recording an ExampleModel program.

«testset»

Buy Stock

«testcut»Buy Account

«testcut»Buy Exchange

«testcut»Buy Sector

«testcut»Buy Broker

«testcut»Buy Stock

And this is the recording itself (as aSequence diagram) from which the TestDomain was generated.

Broker: Broker Exchange.Exchange Exchange.Stock: Stock Exchange.Account:Account

Exchange.Sector

Sequence diagram generated in Enterprise Architect using recording marker in a Use Case

1.1 FindAccount(1): Account*

1.7 GetSector(): ESector

1.0 Buy(1, 2, 100): bool

1.5 GetBalance(): INT

1.10 Move(100)

1.4 GetStockCost(): FLOAT

1.3 GetStockVolume(): UINT1.2 FindStock(2): Stock*

1.8 UpdateSector(secEquities, 100)

1.9 GetSector(secEquities): Sector*

1.6 Credit(2, 100, -100)

Test DomainComposition

The first task on a Test Domain diagramis to create the Use Cases (Test Sets).



These define this particular domain'sresponsibility. The Diagram Toolbox andshortcut menu provide Artifacts to helpyou achieve this. The first of theseelements is the Test Cut, which is used inthe next step; identifying those methods(from the Class model) that you considerto be participants in the Use Case. TheTest Cut Artifact is useful because itallows us to partition a Class, selectingonly those methods that are relevant. TestCuts can be run individually or linked toone or more Test Sets. Test Sets in turncan be linked to one or more Test Suites.In any case, any element of the Testdomain tree can be run individually or asa whole.

«testsuite»

Trade Stock

(from Buy)

«testset»

Buy Stock

(from Sell)

«testset»

Sell Stock Test Set generated from recording history of the Exchange::Sell Use Case

Test Set generated from recording history of Exchange:Buy Use Case



Test Cut

Description

A Test Cut element is a stereotyped Object element, usedinternal to Enterprise Architect for defining test sets usingthe Testpoint code testing facilities.

A task, such as 'Print', might involve operations on differentClasses. In order to create a 'Print' test, you would want toinclude only the 'Print' operations of these Classes andexclude any other operations.

A Test Cut enables you to capture only the operations thatrepresent the behavior (in this case, 'Print') defined for asingle Class. You might then place the Test Cut from eachof several Classes into a single task as a Test Set.

When you drag a Test Cut element onto a Test Domaindiagram, you create a Dependency relationship with therequired Class element. As a result, when you select the TestCut element on the Testpoints Window, the operations ofthe Class are listed in the window, each with a checkbox.You then select the checkbox against each Class operationto include in the Test Cut.



Toolbox icon



Test Set

Description

A Test Set element is a stereotyped Use Case element usedto aggregate one or more groups of methods (Test Cuts),which perhaps span multiple Classes, into a single task. TestSets can also be aggregated into Test Suites.

You link the Test Cut elements to the Test Set usingDependency connectors.

Toolbox icon



Test Suite

Description

A Test Suite element is a stereotyped Use Case element,used to aggregate one or more groups of tasks (Test Sets).

You link the Test Set elements to the Test Suite usingDependency connectors.

Toolbox icon



The Testpoints Window

The Testpoints Window is the hub where Test Domainconstraints are composed. It is also the control that lets youverify a particular Test Domain on a program. The programmight be already running or it can be launched using thecontrol's Toolbar. Here you will also be able to see theresults of your tests, as they happen. This control iscontext-sensitive, responding to the selection of elements inthe Browser window or on a diagram. Depending on theselection, tests can be carried out on a single class, a UseCase (Test Set) or a collection of Use Cases (A Test Suite).

Access

Ribbon Execute > Tools > Tester > ShowTestpoint Window

Testpoint Window Columns

Column Usage

Tests Displays the name of the selected



Testpoint object and the hierarchy ofobjects beneath it.The selected object can be a:

Class·

Operation·

Test Cut·

Test Set or·

Test Suite·

Id For an Operation, this column shows aTestpoint marker icon ( ) when theAnalyzer has successfully bound thisoperation in the target application. If noicon appears in this column during a run,it indicates that the model and code basemight not be synchronized; perhaps thesignature of the function has changed, orthe operation is a new method you areworking on that exists in the source codebut not yet in your model.For a Testpoint, this column shows agenerated id number. This id number isused in trace output to indicate whichconstraint is being referenced.

Constraints A pencil icon ( ) in this column indicatesthat one or more constraints are definedfor this Class or Operation.



Status During a test run, indicates these possiblestatuses:

( ) Failed - Constraint has evaluated as·

false one or more times.( ) Invalid Statement - Constraint·

failed to parse due to invalid syntax.( ) Variable not found - A referenced·

variable name was not found at thelocation where the constraint wasevaluated.

No icon is shown if a constraint hasPassed.

Evals During a test run, indicates the number oftimes the Execution Analyzer hasevaluated this constraint.

Passes During a test run, indicates the number oftimes the test passed.

Fails During a test run, indicates the number oftimes the test failed.

Last Run By Displays the username of the last personto run this test. (Values are derived fromthe Project Author definitions in the'People' dialog - 'Configure > ReferenceData > Model Types > People > Project



Authors'.)

Last RunDate

Displays the date and time this test waslast evaluated.

Last RunResult

Displays the result of the last test run.

ParentCollectionsPane

Lists any parent collections that includethe selected object as part of their design.Double-click this collection to make it theselected object in the left pane.The Parent Collections pane can behidden by clicking the Show / HideParent Collections pane button on theTestpoints Window Toolbar.



Testpoints Toolbar

The Testpoints Window Toolbar provides options to executeconfigured tests on the currently selected Testpoint object,stop a test run currently in progress, filter the displayeditems, and save the results of a completed test run.

Access

Ribbon Execute > Tools > Tester > ShowTestpoint Window

Testpoints toolbar options

ToolbarButton

Action

Field showing the name of the currentlyselected Testpoint object.

Execute the test run.

Stop the test run currently in progress.



Toggle between showing all items andshowing only those items that haveconstraints defined.

Toggle between showing all items andshowing only operations that have beenmarked for inclusion in this Test Cut; thisbutton is only enabled when a Test Cutobject is selected.When a Test Cut is selected, each of theoperations of its associated Class aredisplayed with a checkbox; you use thischeckbox to mark the operations thatapply to this Test Cut.

Click on the drop arrow next to this iconto display the 'Test Run Options' menu,providing these options:

'Prefix Trace output With Function·

Call' - Prefix all trace output lines withthe executing function name'Enable Standard Breakpoints during·

Testing' - When not checked, the testrun ignores any breakpoints in thecurrent breakpoint set, and anyattempts to set breakpoints during therun are ignored'View Trace output' - Display the·



'Testpoints' tab of the System Outputwindow

Click on this icon after completion of atest run to save the results to Test item onthe current object. Saved tests can beviewed using the Testing Workspace.A prompt displays to select the Test Class- Unit, Integration, System, Inspection,Acceptance or Scenario. Select theappropriate Test Class and click on theOK button.

Display the Testpoint Management Helptopic.

Show or hide the Parent Collections pane.



Testpoint Editor

The Testpoint Editor is used to compose constraints forClasses and Operations. The types of constraints permittedare dependent on the selected object. For Classes, the typewill always be Invariant. For operations, the type can beeither Pre-Condition, Post-Condition or Line-Condition.

Invariants are evaluated by the Analyzer when any methodcalled on an object of the selected Class type completes.Pre-conditions are evaluated at the beginning of each call tothe specified operation. Post-conditions are evaluated uponcompletion of each call to the specified operation.Line-conditions are evaluated each time the specified line ofcode is executed.

Access



Ribbon 1 Execute > Tools > Tester > ShowTestpoint Window2 In Testpoint Window, double-click on aClass or Operation to display editor

Constraint Group fields

Field Usage

Type The type of constraint for the selectedClass or Operation:

Invariant - Evaluated after any method·

called on the specified Class hascompletedPre-Condition - Evaluated at the·

beginning of each call to a specificOperationPost-Condition - Evaluated after·

completion of each call to a specificOperationLine-Condition - Evaluated upon·

execution of a specific line of codewithin an Operation



Offset For Line-Conditions only, the Linenumber within the specified operationupon which to evaluate the constraint.An offset value is automatically set if theTestpoint was created using the CodeEditor context menu.

Condition The constraint to be evaluated when thisTestpoint is triggered. A status of pass orfail will be recorded depending uponwhether this constraint conditionevaluates as true or false.

Action onFail

Click on the drop-down arrow and selectfrom the three options:

'Continue' - ignore failure of this·

constraint and continue execution'Break execution' - halt execution and·

display the Stack trace'Disable on fail' - do not execute the·

constraint again after failing once

EvaluateWhen

(Optional) An additional constraint whichmust be met before the main TestpointCondition is evaluated, providing greatercontrol over test coverage.



Trace Group fields

Option Action

Level Specifies when the trace statement (ifdefined) will be output. Available optionsare:

'Fail Only' - Output trace statement·

only when this Testpoint condition fails'Always' - Output trace statement every·

time this Testpoint is evaluated

Statement (Optional) A message to be output whenthis Testpoint is evaluated.Variables currently in scope can beincluded in a trace statement output byprefixing the variable name with a $token for string variables, or a @ tokenfor primitive types such as int or long.Output from a Trace Statement can bedirected either to the 'Testpoints' tab ofthe System Output Window, or to anexternal file, as configured by theAnalyzer Script for the parent Package.





Testpoint Constraints

A Constraint is typically composed using local and membervariables in expressions, separated by operators to defineone or more specific criteria that must be met. A constraintmust evaluate as true to be considered as Passed. If aconstraint evaluates as false, it is considered as Failed.

Any variables referenced within the constraint must be inscope at the position where the Testpoint or Breakpoint isevaluated.

General/Arithmetic Operators

Operator Description

+ AddExample: a + b > 0

- SubtractExample: a - b > 0

/ DivideExample: a / b == 2

* MultiplyExample: a * b == c



% ModulusExample: a % 2 == 1

() Parentheses - Used to define precedencein complex expressions.Example: ((a / b) * c) <= 100

[ ] Square Brackets - Used for accessingArrays.Example: Names[0].Surname == "Smith"

. Dot operator - Used to access membervariables of a Class.Example: Station.Name == "Flinders"

-> Alternative notation for the Dot operator.Example: Station->Name == "Flinders"

Comparison Operators


= Equal ToExample: a = b



== Equal ToExample: a == b

!= Not Equal ToExample: a != b

<> Not Equal ToExample: a <> b

> Greater ThanExample: a > b

>= Greater Than or Equal ToExample: a >= b

< Less ThanExample: a < b

<= Less Than or Equal ToExample: a <= b

Logical Operators




AND Logical ANDExample: (a >= 1) AND (a <= 10)

OR Logical ORExample: (a == 1) OR (b == 1)

Bitwise Operators


& Bitwise ANDExample: (1 & 1) = 1(1 & 0) = 0

| Bitwise ORExample: (1 | 1) = 1(1 | 0) = 1

^ Bitwise XOR (exclusive OR)Example: (1 ^ 1) = 0(1 ^ 0) = 1



Additional Examples

Example Description

((m_nValue&0xFFFF0000)== 0)

Use a Bitwise AND operator (&) with ahexadecimal value as the right operand totest that no bits are set in high order bytesof the variable.

((m_nValue&0x0000FFFF)== 0)

Use a Bitwise AND operator (&) with ahexadecimal value as the right operand totest that no bits are set in low order bytesof the variable.

m_value[0][1] = 2

Accessing a multi-dimensional array

a AND (bOR c)

Combining AND and OR operators,using parentheses to ensure precedence.In this example, variable 'a' must be true,and either 'b' or 'c' must be true.

Notes



String comparisons are case-sensitive·




Documents

User Guide Series - sparxsystems.com · Due to the decoupling of the Test Domain from the codebase, it is a simple choice of ... variables in its object or local scope. They are output