IDS NXT

IDS Imaging Development Systems GmbHDimbacher Str. 6-8

D-74182 Obersulm, GermanyT: +49 7134 96196-0

E: info@ids-imaging.comW: http://www.ids-nxt.com

Vision Apps – CNN manager

November 2019 2

IDS NXT: Vision Apps – CNN manager

Contents

1 Preface ............................................................................................................................................. 3

2 Vision app "CNN manager" ............................................................................................................... 4

2.1 Overview ...................................................................................................................................... 4

2.2 Actions ....................................................................................................................................... 5

2.3 Settings ...................................................................................................................................... 5

2.4 Selecting ROI .............................................................................................................................. 5

2.5 Files ........................................................................................................................................... 6

2.6 Result display .............................................................................................................................. 6

3 Interface description vision app "CNN manager" .............................................................................. 7

3.1 /cnnmanager ............................................................................................................................... 7

3.2 /cnnmanager/rois/evaluationarea .................................................................................................... 7

3.3 /cnnmanager/images/default ......................................................................................................... 8

3.4 /cnnmanager/resultsources/last ..................................................................................................... 9

3.5 /cnnmanager/configurables .......................................................................................................... 10

3.6 /cnnmanager/actions/deletecurrentpackage .................................................................................. 11

3.7 /cnnmanager/files/cnnfile/data ..................................................................................................... 12

3.8 Error codes ................................................................................................................................ 12

Index ................................................................................................................................................. 14

November 2019 3

IDS NXT: Vision Apps – CNN manager

1 Preface

Introduction

IDS Imaging Development Systems GmbH has taken every possible care in preparing this manual. We howeverassume no liability for the content, completeness or quality of the information contained therein. The content ofthis manual is regularly updated and adapted to reflect the current status of the software. We furthermore donot guarantee that this product will function without errors, even if the stated specifications are adhered to.

Under no circumstances can we guarantee that a particular objective can be achieved with the purchase of thisproduct.

Insofar as permitted under statutory regulations, we assume no liability for direct damage, indirect damage ordamages suffered by third parties resulting from the purchase of this product. In no event shall any liabilityexceed the purchase price of the product.

Please note that the content of this manual is neither part of any previous or existing agreement, promise,representation or legal relationship, nor an alteration or amendment thereof. All obligations of IDS ImagingDevelopment Systems GmbH result from the respective contract of sale, which also includes the complete andexclusively applicable warranty regulations. These contractual warranty regulations are neither extended norlimited by the information contained in this manual. Should you require further information on this product, orencounter specific problems that are not discussed in sufficient detail in the manual, please contact your localdealer or system installer.

Trademarks

The IDS logo is a registered trademark of IDS Imaging Development Systems GmbH,registered for U.S. (Reg.No. 4,513,138) and other countries.

IDS NXT and uEye are registered trademarks of IDS Imaging Development SystemsGmbH. Microsoft and Windows are trademarks or registered trademarks of Microsoft Corporation. All otherproducts or company names mentioned in this manual are used solely for purposes of identification ordescription and may be trademarks or registered trademarks of the respective owners.

Copyright

© IDS Imaging Development Systems GmbH. All rights reserved. This manual may not be reproduced,transmitted or translated to another language, either as a whole or in parts, without the prior written permissionof IDS Imaging Development Systems GmbH.

Status: November 2019

Contact

Visit our web site http://www.ids-nxt.com where you will find all the latest information about our software andhardware products. Please contact your local IDS distributors for first level support in your language. For a listof IDS distributors worldwide please go to our website http://www.ids-nxt.com.

Address IDS Imaging Development Systems GmbHDimbacher Str. 6-8D-74182 Obersulm, Germany

T +49 7134 96196-0

E info@ids-imaging.com

W http://www.ids-nxt.com

November 2019 4

IDS NXT: Vision Apps – CNN manager

2 Vision app "CNN manager"The CNN manager vision app allows you to upload trained neural networks ("CNN package") for IDS NXTcameras and execute them on the camera. The vision app provides the results of the image analysis for othervision apps.

Symbols and hints

This symbol indicates hints with useful information for better understanding and using featuresand functions.

This symbol indicates important warnings for product safety to prevent damage.

2.1 Overview

Fig. 1: CNN manager

1. Selection of the vision app

2. Information about the vision app

3. Visio app actions

4. Vision app settings

5. Selection of the image detail (ROI - region of interest)

6. Upload CNN file in vision app

7. Result of image analysis

8. Selection of the image view (live image, image detail)

At the first start of the vision app the included CNN packages will be installed. Therefore, itmay take a few seconds until the vision app is ready for use.

November 2019 5

IDS NXT: Vision Apps – CNN manager

2.2 Actions

Fig. 2: Delete CNN package

Deletes the current active CNN package on the device. When deleting a CNN package, the image acquisitionis temporarily stopped. After deleting, the first CNN package from the CNN package list is activated.

At least one CNN package must always be present. If there is only 1 CNN package on thedevice, it cannot be deleted.

2.3 Settings

Fig. 3: Select CNN package

· Switches between different CNN packages.

· Displays the information about the loaded CNN package.

· Selection of the CNN within the CNN package.

2.4 Selecting ROI

Fig. 4: Set ROI

Here, you define the image detail (ROI - region of interest):

· Sets the complete image as ROI.

· Sets a ROI. This ROI can be moved, enlarged or reduced with the mouse.

November 2019 6

IDS NXT: Vision Apps – CNN manager

2.5 Files

Fig. 5: Upload CNN package

Here, you upload a CNN file that you have previously trained in IDS NXT lighthouse.

The IDS NXT Cockpit displays an error message if the CNN package is large and the uploadtakes a long time. The upload is still performed, however. After uploading, the CNN package isinstalled, which causes the vision app to take a few seconds to run (as it did the first time thevision app was started).

2.6 Result display

Fig. 6: Result display

· Switching between the entire image and an image detail

· Inference: Shows the best 4 hits (top 4).

· Probability Shows the probability of the best 4 hits. The probability can be between 0 ... 1.

· Inference time in milliseconds

November 2019 7

IDS NXT: Vision Apps – CNN manager

3 Interface description vision app "CNN manager"The description of the REST interface here is limited to the parameters/values that are specific for the CNNmanager. A more detailed description of the IDS NXT REST interface can be found in the manual "IDS NXT -REST interface".

REST queries of IDS NXT vision apps are only possible if the corresponding vision app is activated, as they aregenerated dynamically during runtime.

Depending on the action and parameters, the used HTTP methods are color-coded and assigned to thepermission of IDS NXT user profiles (see manual "IDS NXT - Setup").

3.1 /cnnmanager

/vapps/cnnmanager

· GET

· OPTIONS

Transfers information to the vision app "CNN manager".

Response format

The content of the HTTP response is transferred in JSON format (application/json).

Return values

· Activated (Boolean): Activation status of the vision app

· Brief (String): Short description of the vision app

· Description (String): Long description of the vision app

· Manufacturer (String): Manufacturer of the vision app

· Name (String): Name of the vision app

· Status (String): Runtime status of the vision app "NotRunning", "Starting", "Running", "Crashed","FailedToStart", "TimedOut", "ReadError", "WriteError", "UnknownError", "Incompatible"

· Title (String): Vision app title

· Type (String): Vision app type

· Version (String): Version of the vision app

Error response

See Error codes

Sample code

curl -i -X GET -u admin:ids "http://<ip>/vapps/cnnmanager"

3.2 /cnnmanager/rois/evaluationarea

/vapps/cnnmanager/rois/evaluationarea

· GET

November 2019 8

IDS NXT: Vision Apps – CNN manager

· OPTIONS

Transfers the current ROI settings.

Response format

The content of the HTTP response is transferred in JSON format (application/json).

Return values

· Brief (String): Short description of the ROI

· Description (String): Description of the ROI

· Height (Integer): ROI height

· OffsetX (Integer): The X coordinate of the upper left corner of the ROI.

· OffsetY (Integer): The Y coordinate of the upper left corner of the ROI.

· Title (String): ROI title

· Width (Integer): ROI width

Error response

See Error codes

Sample code for GET

curl -X GET http://<ip>/vapps/cnnmanager/rois/evaluationarea --user admin:ids

Sample output

{ "Brief": "", "Description": "", "Height": 295, "OffsetX": 595, "OffsetY": 306, "Title": "", "Width": 379}

Sample code for PATCH

Setting the ROI

curl -X PATCH http://<ip>/vapps/cnnmanager/rois/evaluationarea --useradmin:ids -i --data 'Width=1280&Height=1024&OffsetX=0&OffsetY=0'

3.3 /cnnmanager/images/default

/vapps/cnnmanager/images/default

· GET

· OPTIONS

Transfers the current ROI as JPEG.

Response format

The Accept Token is used to select the format in which the result image is transferred.

· image/jpeg: Transfers a JPEG image.

November 2019 9

IDS NXT: Vision Apps – CNN manager

Parameters

The result image can be further specified using additional parameters.

· Quality (Integer): JPEG quality of the result image in percent if the content type “image/jpeg” is set: 0…100

Return values

· message-body: Image data

Error response

See Error codes

Sample code

curl -H Accept:image/jpeg -X GET http://<ip>/vapps/cnnmanager/images/default?quality=100 --user admin:ids --output roi.jpg

3.4 /cnnmanager/resultsources/last

/vapps/cnnmanager/resultsources/last

· GET

· OPTIONS

Returns the result of the last image analysis.

Response format

The content of the HTTP response is transferred in JSON format (application/json).

Return values

· inference (String): Shows the best 4 hits (top 4).

· inference_propability (Double): Shows the probability of the best 4 hits. The probability can be between0...1.

· inferencetime (Integer): Inference time in milliseconds

Error response

See Error codes

Sample code

curl -X GET http://<ip>/vapps/cnnmanager/resultsources/last --user admin:ids

Sample output

{ "inference": { "Top1": "snowplow", "Top2": "ski", "Top3": "hand_blower", "Top4": "space_shuttle" }, "inference_propability": { "Top1": "0.16", "Top2": "0.11", "Top3": "0.06", "Top4": "0.03", }, "inferencetime": {

November 2019 10

IDS NXT: Vision Apps – CNN manager

"Content": "102" }}

3.5 /cnnmanager/configurables

/vapps/cnnmanager/configurables

· GET

· OPTIONS

· PATCH

Shows the active CNN package.

Response format

The content of the HTTP response is transferred in JSON format (application/json).

Return values

· packages (String): CNN package

Error response

See Error codes

Sample code for GET

curl -X GET http://<ip>/vapps/cnnmanager/configurables --user admin:ids

Sample output for GET

{ "packages": "ImageNet",}

Sample code for PATCH

curl -X PATCH http://<ip>/vapps/cnnmanager/configurables -i --user admin:ids--data 'packages=<New package from OPTIONS-->Range>'

Sample code for OPTIONS

curl -X OPTIONS http://<ip>/vapps/cnnmanager/configurables -i --user admin:ids

Sample output for OPTIONS

{ "GET": { "application/json": { "Description": "Read VApp Configurables.", "Values": { "packages": { "Range": [ "Blumen160d2a5", "Blumen192d2a1", "DoodleTector", "ImageNet", "MultiArchTest", "imagenet_1.0.2" ], "Title": "CNN Packages",

November 2019 11

IDS NXT: Vision Apps – CNN manager

"Type": "Enum", "ZIndex": "0" }, "activecnn": { "Range": [ "imagenet 1.0.2", "inception_v3_1000", "resnet50", "squeezenet_1000" ], "Title": "Active CNN", "Type": "Enum", "ZIndex": "0" } } } }, "Objects": [ ], "PATCH": { "application/x-www-form-urlencoded": { "Description": "Modify VApp Configurables.", "Values": { "packages": { "Range": [ "Blumen160d2a5", "Blumen192d2a1", "DoodleTector", "ImageNet", "MultiArchTest", "imagenet_1.0.2" ], "Title": "CNN Packages", "Type": "Enum", "ZIndex": "0" }, "activecnn": { "Range": [ "imagenet 1.0.2", "inception_v3_1000", "resnet50", "squeezenet_1000" ], "Title": "Active CNN", "Type": "Enum", "ZIndex": "0" } } } }}

3.6 /cnnmanager/actions/deletecurrentpackage

/vapps/cnnmanager/actions/deletecurrentpackage

· GET

November 2019 12

IDS NXT: Vision Apps – CNN manager

· OPTIONS

· POST

Deletes the current active CNN package on the device. After deleting, the first CNN package from the CNNpackage list is activated.

At least one CNN package must always be present. If there is only one CNN package on thedevice, it cannot be deleted.

Response format

The content of the HTTP response is transferred in JSON format (application/json).

Return values

· HTTP code 200

Error response

See Error codes

Sample code

curl -X POST http://<ip>/vapps/cnnmanager/actions/deletecurrentpackage -i --user admin:ids

3.7 /cnnmanager/files/cnnfile/data

/vapps/cnnmanager/files/cnnfile/data

· GET

· OPTIONS

· PUT

Adds a new CNN.

Response format

The content of the HTTP response is transferred in JSON format (application/json).

Return values

· HTTP code 200

Error response

See Error codes

Sample code

curl http://<ip>/vapps/cnnmanager/files/cnnfile/data -i --user admin:ids -T /<path>/<file name>.cnn -H "Content-Type: application/octet-stream"

3.8 Error codes

For the REST queries, HTTP response codes are used as return values. The following HTTP codes are possiblereturn values of the REST queries, in general any HTTP code would be possible:

HTTP code Description

200 OK

November 2019 13

IDS NXT: Vision Apps – CNN manager

HTTP code Description

404 The requested object could not be found (e. g. wrong path)

405 The query must be made using other HTTP methods, e. g. OPTIONS instead of GET.

500 Internal device error

503 Result not available (e.g. no code read yet)

November 2019 14

IDS NXT: Vision Apps – CNN manager

- C -

CNN manager

GUI 4

CNN package

delete 5, 11

display 10

information 5

upload 6

- E -

Error code 12

- I -

Image

analysis 6

detail 5

ROI 5

Image analysis

last 9

result 9

- N -

Neural network

select 5

switch 5

- R -

REST

query 7

Result

display 6

query 12

ROI 5

JPEG 8

position 7

size 7

- V -

Vision app

CNN manager 4, 7