MOB342 - Writing Mobilink Synchronization Scripts Using Java and.NET Reg Domaratzki Senior Software Developer iAnywhere Solutions

MOB342 - Writing Mobilink Synchronization Scripts Using Java and .NET

Reg Domaratzki

Senior Software Developer

iAnywhere Solutions

Outline

Introduction Describing the Sample Implementation using SQL Historical Implementation using Java Historical Implementation using .Net Understanding Direct Row Handling in version 10 Direct Row Handling using Java Direct Row Handing using .Net

Introduction

Since version 8.0.2 of MobiLink, it has been possible to write your synchronization scripts in three different languages • SQL• Java• .Net

This talk will discuss how to implement the exact same synchronization logic using all three languages

We will also discuss the new feature in version 10.0.0 of MobiLink called Direct Row Handling and show how the same logic is implemented using this new feature as well

Introduction

If you are reading a hard copy of this presentation, it may seem a little bare

Much of this talk will be demonstrations on setting up the synchronizations, and discussions of the assocaited code

Check the notes on each slides for the actual code that performs the synchronization, as it’s often difficult to fit into a PowerPoint slide without breaking it up over multiple slides, making it difficult to understand• During the actual presentation, I’ll be switching between the

slides, Sybase Central, and my text editor to discuss the code

Outline


Describing the Sample

The sample I’ll be using throughout this presentation will contain only three tables• The entire contents of the Admin table will exist at all the

remote sites. Because the data is shared by all the remotes, we need to implement conflict resolution on this table.

• The Parent table contains a column called “user_id”. This column will be used to partition the data in the Parent table based on the MobiLink user that is currently synchronizing.

• The Child table has a foreign key relationship back to the Parent table. It will also be partitioned by MobiLink user, but no “user_id” column exists on this table. A join must be made to the Parent table to determine the rows to be downloaded on the Child table


TODO : Add ERP diagram


Primary keys on all the tables are defined using the BIGINT data type with a default value of GLOBAL AUTOINCREMENT• This will ensure that primary keys are unique across the

entire system

The database will also contain a stored procedure calls “rep_data” that can be used to add data into each of the three tables with a single call to a stored procedure


At the remote databases, we need to define several objects to define what data from the remote will be synchronized to MobiLink server• A publication is created that contains all the columns from all

three tables• A unique synchronization user will be defined at each remote

database• A synchronization subscription will be created at each

remote database to identify the location of the MobiLink server, as well as the script version to use during synchronization

Outline


Implementation Using SQLSchema Changes at the Consolidated

Without schema modifications to the table on the consolidated, there is no way to filter out rows that have been previously download

Each table on the consolidated will have an additional column called “last_modified” with a data type of timestamp

This column will track the last time the row was modified by using the special default “TIMESTAMP”

Implementation Using SQLWrite Upload Scripts

MobiLink will need to know how to handle the inserts, updates and deletes that are being synchronized from the remote database

This is done by writing three events for each table• upload_insert• upload_update• upload_delete

Implementation Using SQLWrite the download_cursor events

The download_cursor for each table will be using timestamp synchronization, so each download_cursor will include “where last_modified >= ?” to filter out previously downloaded rows

The Parent table will reference the user_id column and the MobiLink user passed into the script to filter out previously downloaded rows and partition the data by MobiLink user

The Child table will need to execute a join back to the Parent table to partition rows by MobiLink user

Implementation Using SQLWrite the download_delete_cursor

In order to download deletes to the remote database, we need create a shadow table to track the rows that are removed from the consolidated database

Since each table uses BIGINT as the primary key in the table, the sample uses a single table to track deletes on all the tables

A delete trigger is also needed on each table to capture all the deletes that occur

Implementation Using SQL Write the download_delete_cursor

Once the gear is in place to capture deletes in a shadow table, we need to write the download_delete_cursor events to send down the primary key values of rows that were deleted on each table

Note that although timestamp based synchronization is used to ensure that the same deletes are not sent multiple times, no care has been taken to partition the deletes being downloaded on the Parent and Child tables by MobiLink user

Implementation Using SQLDefine a handle_error event

The sample chooses to define primary key violations and unique index violations as errors that will NOT cause the synchronization to stop, but to continue• An action code of 1000 is returned in these cases

All other errors will cause the synchronization to abort by returning an action code of 3000.

Implementation Using SQLImplement conflict resolution on the Admin table

There are two ways to implement conflict resolution using the MobiLink

The first involves first writing an upload_fetch event that is used to get the current contents of the row being uploaded so that it can be compared to old row value that has been passed up in the upload stream• If MobiLink determines that the old row value in the upload

stream does not match the existing row on the consolidated database, then three additional events are fired – upload_old_row_insert– upload_new_row_insert– resolve_conflict


The upload_old_row_insert and upload_new_row_insert events are used to stored the old and new row images from the remote

The resolve_conflict event will then fire to actually resolve the conflict

Note that you do not need to write a resolve_conflict event, as you can delay the actual conflict resolution for a later time, such as the end_upload table level or connection level event


The second method of conflict resolution can be implemented using only the upload_update event

The upload_update event is actually passed both the old and new row images in the upload stream

A stored procedure can be written that handles the conflict in the upload_update event without the need for the three events mentioned in the previous slides

The sample implements the first method of conflict resolution


The upload_fetch event takes the primary key as parameters and fetches the current value of the row from the consolidated database

If a conflict occurs, then the old and new images of the row will need to be stored somewhere so that the conflict resolution code can have access to the values

The sample uses a GLOBAL TEMPORARY TABLE for these values for several reasons• Each connection to the database only sees it’s own values in

the table• A COMMIT will clear all the rows from the table


The upload_old_row_insert and upload_new_row_insert events do nothing other than insert the row images into the global temporary table that we defined

The sample delays the actual conflict resolution to the end_upload event on the Admin table, so that all the conflicts can be handled at once

The code in the AdminConflict stored procedure is NOT to be used in a real production system• It randomly chooses one of the three row images as the

“winner” of the conflict simply to illustrate that you have a choice between the three images

Outline


Historical Implementation using Java

A Java Virtual Machine can run in the MobiLink server when you add “-sl java” to the MobiLink start line

This allows you to write your synchronization logic uses Java instead of SQL

SQL Anywhere does not ship with a Java compiler, so your classes must be compiled outside of SQL Anywhere• When compiling your scripts, you need to add the

%SQLANY10%\java\mlscript.jar file to your CLASSPATH


The important thing to note about the implementation is that the actual data values in the upload and download streams are not available to a majority of the synchronization events• The upload_insert, upload_update, upload_delete,

download_cursor, and download_delete_cursor events, when implemented in Java, need to return a SQL statement which is then executed against the consolidated database through the ODBC connection


The benefit of the historical implementation lies in the ability to define other synchronization events, such as prepare_for_download, authenticate_user and upload_end in the Java language, which is much more flexible than SQL

You also gain access to the MobiLink Java API giving you access to events that occur in the MobiLink server


A common workaround to the fact that the data values are not accessible in the upload events is to upload all your data into holding tables and then execute the Java code in your end_upload connection event and process the data that was inserted into the holding tables

A similar approach can be used for the download by placing the data you want to download using Java code in the begin_download event into holding tables, and then telling your download_cursor events to select from the holding tables


To define a Java synchronization event, you use the ml_add_java_table_script and ml_add_java_connection_script stored procedures, and specify the name of the Java Class

call ml_add_java_table_script( 'v1', 'Admin', 'upload_insert', 'Example.uploadInsert‘ );

The class must be found in the classpath that is specified on the MobiLink start line

mlsrv10 –c “dsn=tw” –sl java(-cp c:\mlclass)

Historical Implementation using JavaMobiLink Java API Overview

ServerContext Interface• public java.sql.Connection makeConnection( )

– Creates and returns a new Connection to the consolidated database

• public void addErrorListener( LogListener ll )– Adds a LogListener Interface allowing you to capture error

messages written to the MobiLink output Log

• public void addWarningListener( LogListener ll )– Adds a LogListener Interface allowing you to capture Warning

messages written to the MobiLink output Log

• public void addShutdownListener( ShutdownListener sl )– Add a ShutdownListener Interface allowing you to capture when

the MobiLink Server is being shut down


DBConnectionContext interface • This interface is passed to the constructor of classes

containing scripts • public java.sql.Connection getConnection( )

– Returns the existing connection java.sql.Connection as a JDBC connection

– The connection is the same connection that MobiLink uses to execute SQL scripts

– This connection must not be committed, closed or altered in any way that would affect the MobiLink server use of this connection


SynchronizationException class • Thrown to indicate that there is an error condition that makes

the completion of the current synchronization impossible• Throwing this exception will force the MobiLink server to

rollback

Historical Implementation using JavaCreating your Java Class

The java class that you will compile must import a few classes that will need to be found at compile and run time• ianywhere.ml.script.*• java.sql.*• java.util.* (if you want to print to the MobiLink output log)

The sample defines a few global variables for the class, as well a constructor that accepts takes a DBConnectionContext interface to initialize the global variables

Historical Implementation using JavaCreating your Java Class

Once you’ve created your .java file, you compile it using javac.exe from Sun’s Java JDK

javac -classpath %SQLANY10%\java\mlscript.jar Example.java

If you place the compiled Example.class file in the same directory where you start the MobiLink server, the following MobiLink start line will ensure that the class is found when needed

mlsrv10 –c “dsn=tw” –sl java( -cp . )

Historical Implementation using JavaWrite Upload and Download Scripts

Remember that the upload_insert, upload_update, and upload_delete java synchronization events need to return a SQL statement that will be executed against the consolidated database using the existing ODBC connection

A simple method that returns a String is needed for each of the upload and download scripts that we defined when using the SQL implementation from the last section

Historical Implementation using JavaWrite Upload and Download Scripts

Once the Java Class contains the needed method to return the upload and download scripts, you need to define the methods that are going to be used by calling the ml_add_java_table_script stored procedure

Historical Implementation using JavaDefining Conflict Resolution

The upload_fetch, upload_old_row_insert and upload_new_row_insert events are again simply returning SQL Statements that will be executed against the consolidated database

The calls to ml_add_java_table_script also need to be called to define the methods that will return the SQL

Historical Implementation using JavaDefining Conflict Resolution

The end_upload event that was previously written as a stored procedure will be re-written to make use of the Connection object that we created in the constructor of the class

It will implement the same functionality as the stored procedure, but is written in Java

Historical Implementation using JavaConflict Resolution

The SQL handle_error event will also be written in Java and replaces the stored procedure that was written in SQL in the previous implementation

public String handleError( ianywhere.ml.script.InOutInteger actionCode, int errorCode, String errorMessage, String user, String table ) { if( errorCode == -196 || errorCode == -194 ) { actionCode.setValue( 1000 ); } else { actionCode.setValue( 3000 ); } return( null );}

Outline


Historical Implementation using .Net

The historical implementation using .Net is very similar to the implementation using Java

There is no access to actual data values in the upload and download scripts

Similar interfaces such as ServerContext and DBConnectionContext exist that give you access to events as they occur in the MobiLink server

Outline


Understanding Direct Row Handling in Version 10

Customers want MobiLink to synchronize directly with enterprise systems other than those currently supported consolidated databases

Some enterprise applications examples include:• ERP Systems• Data Warehouses• Application Servers• Legacy Systems• Non-Standard Relational Database• Messaging Servers


Current solution requires an ASA staging database which sends information to the enterprise application• Another database that needs to be administered• Introduces data redundancy• Can contain potentially sensitive data and therefore

introduces complexity in implementing a secure solution• Accessing various enterprise applications from stored

procedures may not be straightforward


With the introduction of Version 10, new functionality now exists which allows the actual data values in upload stream to be accessed directly in Java and .Net synchronization events

A supported RDBMS will still be needed to hold the MobiLink System tables, but the source and destination for the data in the upload and download streams can be ANYTHING you can access via Java or .Net


The data values will still not available in the upload and download scripts, but two new synchronization events called handle_uploadData and handle_downloadData were created in which an object would be passed that gave access to the data in the upload stream and the ability to add rows to the download stream using Java or .Net


The MobiLink API was also extended in Java and .Net to include the object that is available in these two events

The UploadData interface is passed into the handle_uploadData synchronization and gives you access to all the data in the uploaded tables

The DBConnectionContext interface has added the getDownloadData method which returns a result set object to which you can add rows that will be placed in the download stream

Understanding Direct Row Handling in Version 10The UploadData interface

The UploadData interface contains two methods to allow you to get the data that was uploaded in each table• The getUploadedTables method returns an array of

UploadedTableData objects representing the data in each uploaded table– It’s important to note that the order of the tables in this array is

the same as the order in which the tables were uploaded

• The getUploadedTableByName method returns a single UploadedTableData object that contains the data for that particular table

Understanding Direct Row Handling in Version 10The UploadedTableData Interface

getDeletes method• Returns a java.sql.ResultSet or DataReader object

representing delete operations uploaded by a MobiLink client• The result set contains primary key values for deleted rows

getInserts method• Returns a java.sql.ResultSet or DataReader object

representing insert operations uploaded by a MobiLink client• Each insert is represented by one row in the result set


getUpdates method• Returns a UpdateResultSet or UpdateDataReader object

representing update operations uploaded by a MobiLink client• Each update is represented by one row including all column

values• UpdateResultSet extends java.sql.ResultSet or DataReader to

include special methods for MobiLink conflict detection– The setNewRowValues method sets the mode of the ResultSet

or DataReader to access the new row values (default)– The setOldRowValues method sets the mode of the ResultSet or

DataReader to access the old row values


getName method• Returns the table name for the UploadedTableData-calling

instance

Java getMetaData method• Gets the metadata for the UploadedTableData-calling

instance. • The metadata is a standard java.sql.ResultSetMetaData

instance• If you want the ResultSetMetaData to contain column name

information, you must specify the SendColumnNames MobiLink client extended option or property


.Net GetSchemaTable method• Gets a DataTable object that describes the metadata for this

download table• If you want the DataTable object to contain column name

information, you must specify the SendColumnNames MobiLink client extended option or property

Understanding Direct Row Handling in Version 10The DownloadData Interface

Encapsulates download data operations for direct row handling

To obtain a DownloadData instance, use the DBConnectionContext getDownloadData method

The getDownloadTables method returns an array of DownloadTableData objects for the current synchronization

The getDownloadTableByName method returns the DownloadTableData object for the current synchronization for the given table name

Understanding Direct Row Handling in Version 10The DownloadTableData Interface

getName method• Returns the table name for the DownloadTableData-calling

instance

Java getMetaData method• Gets the metadata for the DownloadTableData-calling

instance• The metadata is a standard java.sql.ResultSetMetaData

object.

.Net GetSchemaTable method• Gets a DataTable that describes the column metadata


getUpsertPreparedStatement method• Returns a java.sql.PreparedStatement or DBCommand instance

which allows the user to add upsert (insert or update) operations to the download of a synchronization

• The prepared statement or command applies to the DownloadTableData-calling instance and contains a parameter for each column in the table

• To include an insert or update operation in the download, set all column values in your java.sql.PreparedStatement or DBCommand and then call the java.sql.PreparedStatement.executeUpdate or ExecuteNonQuery method

• Calling java.sql.PreparedStatement.executeUpdate or ExecuteNonQuery on the prepared statement will return 0 if the insert or update operation was filtered and will return 1 if the operation was not filtered

• You must set all column values for download insert and update operations.


getDeletePreparedStatement method• Returns a java.sql.PreparedStatement or DBCommand instance

that allows the user to add delete operations to the download• The prepared statement or command applies to the

DownloadTableData-calling instance and contains a parameter for each primary key column in the table.

• To include a delete operation in the download, set all columns in your java.sql.PreparedStatement or DBCommand and then call the java.sql.PreparedStatement.executeUpdate or ExecuteNonQuery method

• Calling java.sql.PreparedStatement.executeUpdate or ExecuteNonQuery will return 0 if the delete operation was filtered and will return 1 if the delete was not filtered

• You must set all primary key values for download delete operations

Understanding Direct Row Handling in Version 10Possible Pitfalls

When using data sources for synchronization other than the supported RDBMS, whether that be through the Direct Row API or using the traditional method and executing Java or .Net code in the end_upload event on data it holding tables, care must be taken to ensure that data is not sent to the data source multiple times or not at all by the Mobilink Server• Since these operations are taking place outside of the

commit/rollback protection provided by the database holding the MobiLink System tables, how will you guarantee that a row that was sent to the data source really arrived?

Outline


Direct Row Handling using Java

When using the Direct Row API, the only three synchronization events that need to be defined• handle_uploadData

– This event will process the upload stream• begin_download

– This event is only needed to set some global variables to capture the current MobiLink user that is synchronizing and the last download timestamp that was passed up by the remote database

• handle_downloadData– This event will create the download stream

The sample has also defined a handle_error and authenticate_user event written in Java as well

Outline


Direct Row Handling Using .Net

At the time this presentation was submitted, the implementation of the .Net Direct Row API was not yet feature complete

An updated presentation should be available from the TechWave site after the completion of the conference with information in this section

Documents

MOB342 - Writing Mobilink Synchronization Scripts Using Java and.NET Reg Domaratzki Senior Software Developer iAnywhere Solutions