Working with Smallworld 5!

Mark Field

Good afternoon. Thank you for staying towards the end of the conference.

1

Overview

• How to develop and debug in Smallworld 5

– Most slides in presentation, mostly for your offline review

• How to manage Smallworld 5 installation

• Simple Java/Magik interface tutorial

• Open for discussion on how you manage Smallworld 5

This presentation will go over some tips, tricks and tools for developing and

debugging your code under the Smallworld 5 platform.

The majority of the slides will be around developing and debugging, and plan on

quickly going over each one. I recommend reviewing them off line for details because

I want to present the other topics.

I will present some options of managing your Smallworld 5 installation, a magik/java

interface, and then open the floor to see how you are developing Smallworld 5

functionality.

2

Quick Survey!

• How many are on Smallworld 5 now?

– What versions?

– Are you keeping up with releases?

• How many are in process of upgrading now?

– From what release?

• How many are planning to upgrade this year?

• No Current plans?

Before I start I would like to do a quick survey. .

3

Difficulties with Developing Applications

• No out of the box debugger

– 4.x had very useful single method debugger in Development Tools Application

• Traceback/stack trace does not report arguments

– But does give true exemplar, filename and line number!

• No GE CODE – No Enhanced API Documentation!

– Can’t review how GE implemented functionality

– Can’t review what properties are set on certain GUIs to implement similar behavior

As you know Smallworld 5 is running within the Java Virtual Machine. I believe this is

a great technological step forward and can open Smallworld up to many more

applications and interfaces.

However great change always seem to cause a great large learning curve. Some say

more than previous major Smallworld release from 3 to 4 or 2 to 3. Personally I think

it not as bad as previous 3 to 4. The application with plugins and dataabus are the

same in 5. Even magik_session is very similar to magik_image. It is just different.

Here are some big changes with Smallworld 5’s environment. The first two are

primarily do to the fact Java limitations.

There is no out of the box debugger as there was in 4.x. There is an unreleased pre-

alpha debugger that some at GE are using and may be available upon request.

The Java trace stack does not supply arguments in the output. Compared to the 4.x

traceback which would provide the exemplar that called the method theSsmallworld

5 traceback provides the actual exemplar, the source file, and line numbers. Which is

very good information and helps you go directly to the source.

The biggest change is no source code is delivered from GE. Not even the application

code to review how things communicate with databus and how GUIs may be created.

4

GE has made improvements to the documentation on line, this documentation still

lacks details on the usage of all the gui components. More tutorials and examples

would be very helpful to have available.

4

Current 5.x Developers Tools• Developer tools application module available after reinit of sw_core product

• Some limitations due to unavailable Java functionality

• Available Tools– Object Inspector – similar to 4.3• databus_monitor_dialog.open({app.plugin(:plotting).dialogs.an_element().plugin(:layout_manager)})

– GUI Explorer – new in SW 5.1

– Icon Viewer & Remote CLI also available

• Missing Tools (from 4.3)– Debugger

– Monitor Clicks

– Magik Coverage Analyzer

– Thread Monitor

– System Monitor

– XPRF - Profiler

There are develop tools with Smallworld 5. The sw_dev_tools product may not be

available in the module manager dialog by default. You will need to reinitialize the

sw_core product for the product and application module to be available.

Because Smallworld is built upon Java, there are limitations to what can be exposed

versus the magik virtual machine.

There are really only 2 development tools available. The Object Inspector and the

new GUI Explorer. Other tools like icon viewer and remote command line interface

are still available. The Object Inspector is the same as the 4.x. The GUI Explorer

exposes the SWIFT components that are currently opened. This allows you to easily

identify the exact GUI component that you are interested in. This may point to some

GUI settings that are not thoroughly documented.

The missing tools are listed here.

5

GUI Explorer

The GUI Explorer allows you to see information about the GUI and the components.

Here is an example. You can expand the gui component tree and select a component.

It highlights the selection in the viewer pane and then in the right pane it shows the

properties. The properties are a way of learning settings that you may need to

duplicate functionality that you see in a core GUI.

6

Where to Start Debugging?

• From last year’s “Improving your experience with Support Central” presentation

– Develop Smallworld Sessions that can isolate code

• Core

• 3rd Party

• Custom

– If problem exist in 3rd Party/Core

• Reproduce in demo database: Cambridge, Design Manager (UPS), PNI, etc.

• Punt to GE/Vendor support

• Determine if Method is GE, 3rd Party, or Custom

– Use exemplar.method(:meth_name|()|).source_file

– F3-j in Emacs Class Browser

So where do you start?

First determine where the problem is originating. Is it Core, 3rd Party, or Custom

code?

At the 2017 Grid Software Solutions Americas User Conference, I presented ways to

isolate your builds in “Improving your experience with Support Central”

If you isolate it to GE or vendor, than punt the problem to the appropriate support.

You will need to develop processes or create simplified code to reproduce the

problem.

You can use source_file method to identify the location of the method that is causing

the problem. From the response, you will be able to identify where the code came

from.

7

Ensure Clean Builds

• Thoroughly review your log files

• FCSI’s Method Information

– Records global being defined during build

• Left over debug code for example: p(my_list) dp(thing)

• Mistakenly used globals for example: gpm, appl

– Records methods being overwritten

• Leverages “Infraction Checker”

• Verify purpose

• Verify change – may need to request method from GE Support

– CSV file for easy review in spreadsheet program

Ensure that your code builds are clean. Review your logs carefully look for undefined

globals. Ensure that there isn’t debug coded loaded that may not be initialized when

the method is called.

I have developed tools that identify the undefined globals and methods that have

been overwriten and produces a easy to review CSV file.

8

Global/Method information Output

Here is an example of the our CSV output.

At the top you can see the globals defined in this build. It shows the global name and

the file it was created in

Then you can see the redefined methods. It gives the exemplars and the method

name. You can see which file has the overwrite and where it was originally defined.

9

FCSI Method Manipulation Methods

• Available on GE Smallworld.org WIKI (http://sw-gis.wikidot.com/local--files/swv-debugging/fcsi_core.zip)

– fcsi_show_args()

• fcsi_method_manip.fcsi_show_args(xml_parse_acp,:|parse_file()|)

• Shows arguments for the given exemplar/method

– fcsi_copy_method()

• Used in above methods

• Allows you to rename(temporarily or permanent) existing methods that you can call from an overwritten method

• Preserves core code!

• fcsi_create_traceback_snapshot()

– Creates a 4.x like traceback_snapshot that stores the arguments

So you have determined that custom code is causing the problem. But the error is

occurring in code that you don’t have access to.

You can use fcs_show_args() method to report the arguments when the method is

called. You can run this method on the exemplar and method of that you do not have

to help identify the problem.

The show_args method uses the copy method that does in memory copies of the

methods and does a show with all the arguments before calling the original copied

method.

I have developed a wrapper around this that actually creates a traceback snapshot list

similar to 4.x. You can use the traceback snapshot list records the arguments and

objects in an ordered collection that you can do further investigation with. This is

limited release code by FCSI. It is free, but since it is beta code it is released by

request only. This will give you easy access to the object that is having problems. You

can investigate the object more thoroughly.

10

Review Method Changes• Smallworld 5.1 Online Documentation->Smallworld product

history (5.1x)->Method Changes

– https://smallworld.gedigitalenergy.com/docs/sw51/en/swDocs5.htm#../S

ubsystems/Core/Content/ReleaseInfo/ProductHistory51.htm

• Need to review this list with your list of overwrites

– Need to request method from Support to integrate changes

– Need to review to see if functionality can done be before/after core code

• Use fcsi_copy_method() to add your functionality

• Need to think outside the box!

– Use dynamics to send data to subsequent calls

Around the 5.1.4 release, GE started publishing the method changes for each release.

Review this list with your custom code. If you have overwritten this functionality,

review what was changed. If the change can be isolated before or after the core code,

look at leveraging the fcsi_copy_method() functionality to copy the core method into

another method name, write your specialized code in the original method name and

call the new copied method. This will allow you to make changes without the need to

compare code in the future. You still need to review any changes to arguments and or

responses from the core code, but that is easier than comparing lines of code.

If your change is integral to the core code (within an if statement, or adding some

additional arguments) you can request the singular method source from GE support.

11

Copy Method Examples• Permanent Code Example

– fcsi_method_manip.fcsi_copy_method(sw_action,:|perform_safely()|)

_method sw_action.perform_safely(control,selector,_gather args)

_dynamic !fcsi_control! << control

_return self.sw! perform_safely(control,selector,_scatter args)

_endmethod

• Example debugging use

– fcsi_method_manip.fcsi_copy_method(sw_action,:|perform_safely()|)

_method sw_action.perform_safely(control,selector,_gather args)

_global a_ctrl << control

_return self.sw! perform_safely(control,selector,_scatter args)

_endmethod

Here are some examples of using the copy method approach.

The first is something you might do as a permanent solution to set a dynamic prior to

calling the core code.

The second is how you can set up a debug global to grab the item of interest and then

calling the core code.

12

Rudimentarily Trace Stack

• FCSI has developed a rudimentarily trace stack similar to 4.x (with

developer tools loaded)

• Trace information when method is called, not specifically in

traceback order

• traceback_snapshot.fcsi_set_up_methods_from_traceback()

– Uses the output of traceback to call

fcsi_method_manip.fcsi_create_traceback_snapshot() on each method

The trace stack functionality developed by FCSI allows you to get a handle on the

objects and the arguments for the method calls of a traceback.

To use this, you store the traceback displayed to you in a text file. You will run the

fcsi_set_up_methods_from_traceback() method with the text file. It will call the

fcsi_create_traceback_snapshot() method on each method in the trace stack.

After you run your code you will be able to see the ordered list of trace information in

traceback_snapshot. fcsi_stack. You will be able to inspect the objects and

arguments. This list is generated in order of the actual method calls. So you may see

extra trace information in the list, but it does closely line up with the traceback calls.

You will be able to locate any unexpected unset arguments which is in many cases is

the root problem. Otherwise study the trace information carefully. You should verify

the actual traceback is how you expected the code to be called. Look for any

unexpected arguments.

13

Other Tools: MDT®

• MDT

– Eclipse based development environment

– Limited information on local & global variables

– Does show arguments!

We will now look at other tools that are available. The first being Magik Development

Tool (MDT) which is an Eclipse based IDE for Smallworld Magik. It is maintained by

Adips (www.adips.com).

The following page shows the debugger.

14

Example of MDT Debugger

15

Java VisualVM

• VisualVM (https://visualvm.github.io/)

– Profiler for Java

– Some GE personnel use this

In 4.3 XPRF is used to identify the performance issues with processes. In Smallworld

5, GE personnel are using Java VisualVM to do some performance analysis of Java and

Magik code.

16

Java VisualVM

This slide shows an example view of the data in VisualVM. The data provided shows

the Java form of the Magik names.

17

Node.js Debugger

Under development at GE is a Node.js debugging interface. Here is an example of it.

You will note it is a webclient, so there is some configuration that is required. This is

one main reason it has not been released. From what I understand you can request

this debugger under a pre-alpha release.

18

Other Potential GE Tools?

• Developers Toolkit?

– What is this versus known Development Tools application????

I understand there is a developers toolkit. I have not seen or know what it includes.

You can ask your GE support representative about this.

19

Radical Ideas to Help Debug

• Java is Open Source

– Maybe creating a “debug” Java release that reports arguments in trace

• Some discussion on this in Java Community

– Is there Other things that can be “hacked” to provide additional

information?

– This modified Java would only be loaded in development environments

One radical idea to help debug is to look at developing a special Java that includes the

ability to report arguments in the tracestack. There may be other things that can be

added to this special debug version of Java.

20

Managing Smallworld 5 Installations• More frequent releases

– Full releases

– Look at updating your custom code at same time

• Locked JAR files during use

– Smallworld Release and your customized compiled code

– Everyone (and process) needs to be out

– Attempt to have symbolic link for JARs

• Rename libs dir to libs_<date or release>

• mklink /d libs libs_<date or release>

– For custom code ween of using compiled JARs before re building

Smallworld 5.1.X versions are being released almost every quarter. These are full

replacement releases. This means you will need to reduce any code overwrites to a

bare minimum so the review is not as large.

Smallworld 5 lock the JAR files under the libs directories in all products with loaded

modules. This means you cannot replace JARs from a release or build new JARs with

anyone or any process still using a Smallworld session.

You can potentially create release “libs” directories and create symbolic links. You can

remove the link and create new ones before any session is started.

If you have customized code, you can remove the libs directory when the server is

rebooted and not compile new JARs until testing is completed. The image will load

slower without the JARs but you will be able to replace code easier until final release.

21

Managing Smallworld 5 Installations

• Possibly leverage virtual drives to limit down time

– Prepare and build on new virtual drive

– During release shut down all sessions

– Replace old virtual drive with new

– Create using Disk Management->Action->Create VHD

Another possibility is using virtual drives to mount your installations. You will be able

to unmount the drive when you need to change to a new release. Note any existing

session will crash. This should be done with caution.

22

Magik/Smallworld Interface

• Sample of calling Java from Magik

• Sample of calling Magik from Java

23

Audience Input!

• How do you debug in 5?

• What can you suggest to make debugging more efficient?

• More questions???

I would like to open the floor to how you are developing under Smallworld 5.

24

Questions? / Contact

• Mark Field

•

• FCSI Booth 20

• Mobile: 407-376-6105

• Office: 407-275-9351

• @mbfbsae90

• https://www.linkedin.com/in/mbfbsae90

• https://www.linkedin.com/company/field-csi/

• https://sw-gis.slack.com

25