MOB342 - Writing Mobilink Synchronization Scripts Using Java and.NET Reg Domaratzki Senior Software Developer iAnywhere Solutions

Outline  Introduction  Describing the Sample  Implementation using SQL  Historical Implementation using Java  Historical Implementation using.Net  Understanding Direct Row Handling in version 10  Direct Row Handling using Java  Direct Row Handing using.Net

Introduction  Since version of MobiLink, it has been possible to write your synchronization scripts in three different languages SQL Java.Net  This talk will discuss how to implement the exact same synchronization logic using all three languages  We will also discuss the new feature in version of MobiLink called Direct Row Handling and show how the same logic is implemented using this new feature as well

Introduction  If you are reading a hard copy of this presentation, it may seem a little bare  Much of this talk will be demonstrations on setting up the synchronizations, and discussions of the assocaited code  Check the notes on each slides for the actual code that performs the synchronization, as it’s often difficult to fit into a PowerPoint slide without breaking it up over multiple slides, making it difficult to understand During the actual presentation, I’ll be switching between the slides, Sybase Central, and my text editor to discuss the code

Outline  Introduction  Describing the Sample  Implementation using SQL  Historical Implementation using Java  Historical Implementation using.Net  Understanding Direct Row Handling in version 10  Direct Row Handling using Java  Direct Row Handing using.Net

Describing the Sample  The sample I’ll be using throughout this presentation will contain only three tables The entire contents of the Admin table will exist at all the remote sites. Because the data is shared by all the remotes, we need to implement conflict resolution on this table. The Parent table contains a column called “user_id”. This column will be used to partition the data in the Parent table based on the MobiLink user that is currently synchronizing. The Child table has a foreign key relationship back to the Parent table. It will also be partitioned by MobiLink user, but no “user_id” column exists on this table. A join must be made to the Parent table to determine the rows to be downloaded on the Child table

Describing the Sample  TODO : Add ERP diagram

Describing the Sample  Primary keys on all the tables are defined using the BIGINT data type with a default value of GLOBAL AUTOINCREMENT This will ensure that primary keys are unique across the entire system  The database will also contain a stored procedure calls “rep_data” that can be used to add data into each of the three tables with a single call to a stored procedure

Describing the Sample  At the remote databases, we need to define several objects to define what data from the remote will be synchronized to MobiLink server A publication is created that contains all the columns from all three tables A unique synchronization user will be defined at each remote database A synchronization subscription will be created at each remote database to identify the location of the MobiLink server, as well as the script version to use during synchronization

Outline  Introduction  Describing the Sample  Implementation using SQL  Historical Implementation using Java  Historical Implementation using.Net  Understanding Direct Row Handling in version 10  Direct Row Handling using Java  Direct Row Handing using.Net

Implementation Using SQL Schema Changes at the Consolidated  Without schema modifications to the table on the consolidated, there is no way to filter out rows that have been previously download  Each table on the consolidated will have an additional column called “last_modified” with a data type of timestamp  This column will track the last time the row was modified by using the special default “TIMESTAMP”

Implementation Using SQL Write Upload Scripts  MobiLink will need to know how to handle the inserts, updates and deletes that are being synchronized from the remote database  This is done by writing three events for each table upload_insert upload_update upload_delete

Implementation Using SQL Write the download_cursor events  The download_cursor for each table will be using timestamp synchronization, so each download_cursor will include “where last_modified >= ?” to filter out previously downloaded rows  The Parent table will reference the user_id column and the MobiLink user passed into the script to filter out previously downloaded rows and partition the data by MobiLink user  The Child table will need to execute a join back to the Parent table to partition rows by MobiLink user

Implementation Using SQL Write the download_delete_cursor  In order to download deletes to the remote database, we need create a shadow table to track the rows that are removed from the consolidated database  Since each table uses BIGINT as the primary key in the table, the sample uses a single table to track deletes on all the tables  A delete trigger is also needed on each table to capture all the deletes that occur

Implementation Using SQL Write the download_delete_cursor  Once the gear is in place to capture deletes in a shadow table, we need to write the download_delete_cursor events to send down the primary key values of rows that were deleted on each table  Note that although timestamp based synchronization is used to ensure that the same deletes are not sent multiple times, no care has been taken to partition the deletes being downloaded on the Parent and Child tables by MobiLink user

Implementation Using SQL Define a handle_error event  The sample chooses to define primary key violations and unique index violations as errors that will NOT cause the synchronization to stop, but to continue An action code of 1000 is returned in these cases  All other errors will cause the synchronization to abort by returning an action code of 3000.

Implementation Using SQL Implement conflict resolution on the Admin table  There are two ways to implement conflict resolution using the MobiLink  The first involves first writing an upload_fetch event that is used to get the current contents of the row being uploaded so that it can be compared to old row value that has been passed up in the upload stream If MobiLink determines that the old row value in the upload stream does not match the existing row on the consolidated database, then three additional events are fired –upload_old_row_insert –upload_new_row_insert –resolve_conflict

Implementation Using SQL Implement conflict resolution on the Admin table  The upload_old_row_insert and upload_new_row_insert events are used to stored the old and new row images from the remote  The resolve_conflict event will then fire to actually resolve the conflict  Note that you do not need to write a resolve_conflict event, as you can delay the actual conflict resolution for a later time, such as the end_upload table level or connection level event

Implementation Using SQL Implement conflict resolution on the Admin table  The second method of conflict resolution can be implemented using only the upload_update event  The upload_update event is actually passed both the old and new row images in the upload stream  A stored procedure can be written that handles the conflict in the upload_update event without the need for the three events mentioned in the previous slides  The sample implements the first method of conflict resolution

Implementation Using SQL Implement conflict resolution on the Admin table  The upload_fetch event takes the primary key as parameters and fetches the current value of the row from the consolidated database  If a conflict occurs, then the old and new images of the row will need to be stored somewhere so that the conflict resolution code can have access to the values  The sample uses a GLOBAL TEMPORARY TABLE for these values for several reasons Each connection to the database only sees it’s own values in the table A COMMIT will clear all the rows from the table

Implementation Using SQL Implement conflict resolution on the Admin table  The upload_old_row_insert and upload_new_row_insert events do nothing other than insert the row images into the global temporary table that we defined  The sample delays the actual conflict resolution to the end_upload event on the Admin table, so that all the conflicts can be handled at once  The code in the AdminConflict stored procedure is NOT to be used in a real production system It randomly chooses one of the three row images as the “winner” of the conflict simply to illustrate that you have a choice between the three images

Outline  Introduction  Describing the Sample  Implementation using SQL  Historical Implementation using Java  Historical Implementation using.Net  Understanding Direct Row Handling in version 10  Direct Row Handling using Java  Direct Row Handing using.Net

Historical Implementation using Java  A Java Virtual Machine can run in the MobiLink server when you add “-sl java” to the MobiLink start line  This allows you to write your synchronization logic uses Java instead of SQL  SQL Anywhere does not ship with a Java compiler, so your classes must be compiled outside of SQL Anywhere When compiling your scripts, you need to add the %SQLANY10%\java\mlscript.jar file to your CLASSPATH

Historical Implementation using Java  The important thing to note about the implementation is that the actual data values in the upload and download streams are not available to a majority of the synchronization events The upload_insert, upload_update, upload_delete, download_cursor, and download_delete_cursor events, when implemented in Java, need to return a SQL statement which is then executed against the consolidated database through the ODBC connection

Historical Implementation using Java  The benefit of the historical implementation lies in the ability to define other synchronization events, such as prepare_for_download, authenticate_user and upload_end in the Java language, which is much more flexible than SQL  You also gain access to the MobiLink Java API giving you access to events that occur in the MobiLink server

Historical Implementation using Java  A common workaround to the fact that the data values are not accessible in the upload events is to upload all your data into holding tables and then execute the Java code in your end_upload connection event and process the data that was inserted into the holding tables  A similar approach can be used for the download by placing the data you want to download using Java code in the begin_download event into holding tables, and then telling your download_cursor events to select from the holding tables

Historical Implementation using Java  To define a Java synchronization event, you use the ml_add_java_table_script and ml_add_java_connection_script stored procedures, and specify the name of the Java Class call ml_add_java_table_script( 'v1', 'Admin', 'upload_insert', 'Example.uploadInsert‘ );  The class must be found in the classpath that is specified on the MobiLink start line mlsrv10 –c “dsn=tw” –sl java(-cp c:\mlclass)

Historical Implementation using Java MobiLink Java API Overview  ServerContext Interface public java.sql.Connection makeConnection( ) –Creates and returns a new Connection to the consolidated database public void addErrorListener( LogListener ll ) –Adds a LogListener Interface allowing you to capture error messages written to the MobiLink output Log public void addWarningListener( LogListener ll ) –Adds a LogListener Interface allowing you to capture Warning messages written to the MobiLink output Log public void addShutdownListener( ShutdownListener sl ) –Add a ShutdownListener Interface allowing you to capture when the MobiLink Server is being shut down

Historical Implementation using Java MobiLink Java API Overview  DBConnectionContext interface This interface is passed to the constructor of classes containing scripts public java.sql.Connection getConnection( ) –Returns the existing connection java.sql.Connection as a JDBC connection –The connection is the same connection that MobiLink uses to execute SQL scripts –This connection must not be committed, closed or altered in any way that would affect the MobiLink server use of this connection

Historical Implementation using Java MobiLink Java API Overview  SynchronizationException class Thrown to indicate that there is an error condition that makes the completion of the current synchronization impossible Throwing this exception will force the MobiLink server to rollback

Historical Implementation using Java Creating your Java Class  The java class that you will compile must import a few classes that will need to be found at compile and run time ianywhere.ml.script.* java.sql.* java.util.* (if you want to print to the MobiLink output log)  The sample defines a few global variables for the class, as well a constructor that accepts takes a DBConnectionContext interface to initialize the global variables

Historical Implementation using Java Creating your Java Class  Once you’ve created your.java file, you compile it using javac.exe from Sun’s Java JDK javac -classpath %SQLANY10%\java\mlscript.jar Example.java  If you place the compiled Example.class file in the same directory where you start the MobiLink server, the following MobiLink start line will ensure that the class is found when needed mlsrv10 –c “dsn=tw” –sl java( -cp. )

Historical Implementation using Java Write Upload and Download Scripts  Remember that the upload_insert, upload_update, and upload_delete java synchronization events need to return a SQL statement that will be executed against the consolidated database using the existing ODBC connection  A simple method that returns a String is needed for each of the upload and download scripts that we defined when using the SQL implementation from the last section

Historical Implementation using Java Write Upload and Download Scripts  Once the Java Class contains the needed method to return the upload and download scripts, you need to define the methods that are going to be used by calling the ml_add_java_table_script stored procedure

Historical Implementation using Java Defining Conflict Resolution  The upload_fetch, upload_old_row_insert and upload_new_row_insert events are again simply returning SQL Statements that will be executed against the consolidated database  The calls to ml_add_java_table_script also need to be called to define the methods that will return the SQL

Historical Implementation using Java Defining Conflict Resolution  The end_upload event that was previously written as a stored procedure will be re-written to make use of the Connection object that we created in the constructor of the class  It will implement the same functionality as the stored procedure, but is written in Java

Historical Implementation using Java Conflict Resolution  The SQL handle_error event will also be written in Java and replaces the stored procedure that was written in SQL in the previous implementation public String handleError( ianywhere.ml.script.InOutInteger actionCode, int errorCode, String errorMessage, String user, String table ) { if( errorCode == -196 || errorCode == -194 ) { actionCode.setValue( 1000 ); } else { actionCode.setValue( 3000 ); } return( null ); }

Outline  Introduction  Describing the Sample  Implementation using SQL  Historical Implementation using Java  Historical Implementation using.Net  Understanding Direct Row Handling in version 10  Direct Row Handling using Java  Direct Row Handing using.Net

Historical Implementation using.Net  The historical implementation using.Net is very similar to the implementation using Java  There is no access to actual data values in the upload and download scripts  Similar interfaces such as ServerContext and DBConnectionContext exist that give you access to events as they occur in the MobiLink server

Outline  Introduction  Describing the Sample  Implementation using SQL  Historical Implementation using Java  Historical Implementation using.Net  Understanding Direct Row Handling in version 10  Direct Row Handling using Java  Direct Row Handing using.Net

Understanding Direct Row Handling in Version 10  Customers want MobiLink to synchronize directly with enterprise systems other than those currently supported consolidated databases  Some enterprise applications examples include: ERP Systems Data Warehouses Application Servers Legacy Systems Non-Standard Relational Database Messaging Servers

Understanding Direct Row Handling in Version 10  Current solution requires an ASA staging database which sends information to the enterprise application Another database that needs to be administered Introduces data redundancy Can contain potentially sensitive data and therefore introduces complexity in implementing a secure solution Accessing various enterprise applications from stored procedures may not be straightforward

Understanding Direct Row Handling in Version 10  With the introduction of Version 10, new functionality now exists which allows the actual data values in upload stream to be accessed directly in Java and.Net synchronization events  A supported RDBMS will still be needed to hold the MobiLink System tables, but the source and destination for the data in the upload and download streams can be ANYTHING you can access via Java or.Net

Understanding Direct Row Handling in Version 10  The data values will still not available in the upload and download scripts, but two new synchronization events called handle_uploadData and handle_downloadData were created in which an object would be passed that gave access to the data in the upload stream and the ability to add rows to the download stream using Java or.Net

Understanding Direct Row Handling in Version 10  The MobiLink API was also extended in Java and.Net to include the object that is available in these two events  The UploadData interface is passed into the handle_uploadData synchronization and gives you access to all the data in the uploaded tables  The DBConnectionContext interface has added the getDownloadData method which returns a result set object to which you can add rows that will be placed in the download stream

Understanding Direct Row Handling in Version 10 The UploadData interface  The UploadData interface contains two methods to allow you to get the data that was uploaded in each table The getUploadedTables method returns an array of UploadedTableData objects representing the data in each uploaded table –It’s important to note that the order of the tables in this array is the same as the order in which the tables were uploaded The getUploadedTableByName method returns a single UploadedTableData object that contains the data for that particular table

Understanding Direct Row Handling in Version 10 The UploadedTableData Interface  getDeletes method Returns a java.sql.ResultSet or DataReader object representing delete operations uploaded by a MobiLink client The result set contains primary key values for deleted rows  getInserts method Returns a java.sql.ResultSet or DataReader object representing insert operations uploaded by a MobiLink client Each insert is represented by one row in the result set

Understanding Direct Row Handling in Version 10 The UploadedTableData Interface  getUpdates method Returns a UpdateResultSet or UpdateDataReader object representing update operations uploaded by a MobiLink client Each update is represented by one row including all column values UpdateResultSet extends java.sql.ResultSet or DataReader to include special methods for MobiLink conflict detection –The setNewRowValues method sets the mode of the ResultSet or DataReader to access the new row values (default) –The setOldRowValues method sets the mode of the ResultSet or DataReader to access the old row values

Understanding Direct Row Handling in Version 10 The UploadedTableData Interface  getName method Returns the table name for the UploadedTableData-calling instance  Java getMetaData method Gets the metadata for the UploadedTableData-calling instance. The metadata is a standard java.sql.ResultSetMetaData instance If you want the ResultSetMetaData to contain column name information, you must specify the SendColumnNames MobiLink client extended option or property

Understanding Direct Row Handling in Version 10 The UploadedTableData Interface .Net GetSchemaTable method Gets a DataTable object that describes the metadata for this download table If you want the DataTable object to contain column name information, you must specify the SendColumnNames MobiLink client extended option or property

Understanding Direct Row Handling in Version 10 The DownloadData Interface  Encapsulates download data operations for direct row handling  To obtain a DownloadData instance, use the DBConnectionContext getDownloadData method  The getDownloadTables method returns an array of DownloadTableData objects for the current synchronization  The getDownloadTableByName method returns the DownloadTableData object for the current synchronization for the given table name

Understanding Direct Row Handling in Version 10 The DownloadTableData Interface  getName method Returns the table name for the DownloadTableData-calling instance  Java getMetaData method Gets the metadata for the DownloadTableData-calling instance The metadata is a standard java.sql.ResultSetMetaData object. .Net GetSchemaTable method Gets a DataTable that describes the column metadata

Understanding Direct Row Handling in Version 10 The DownloadTableData Interface  getUpsertPreparedStatement method Returns a java.sql.PreparedStatement or DBCommand instance which allows the user to add upsert (insert or update) operations to the download of a synchronization The prepared statement or command applies to the DownloadTableData-calling instance and contains a parameter for each column in the table To include an insert or update operation in the download, set all column values in your java.sql.PreparedStatement or DBCommand and then call the java.sql.PreparedStatement.executeUpdate or ExecuteNonQuery method Calling java.sql.PreparedStatement.executeUpdate or ExecuteNonQuery on the prepared statement will return 0 if the insert or update operation was filtered and will return 1 if the operation was not filtered You must set all column values for download insert and update operations.

Understanding Direct Row Handling in Version 10 The DownloadTableData Interface  getDeletePreparedStatement method Returns a java.sql.PreparedStatement or DBCommand instance that allows the user to add delete operations to the download The prepared statement or command applies to the DownloadTableData-calling instance and contains a parameter for each primary key column in the table. To include a delete operation in the download, set all columns in your java.sql.PreparedStatement or DBCommand and then call the java.sql.PreparedStatement.executeUpdate or ExecuteNonQuery method Calling java.sql.PreparedStatement.executeUpdate or ExecuteNonQuery will return 0 if the delete operation was filtered and will return 1 if the delete was not filtered You must set all primary key values for download delete operations

Understanding Direct Row Handling in Version 10 Possible Pitfalls  When using data sources for synchronization other than the supported RDBMS, whether that be through the Direct Row API or using the traditional method and executing Java or.Net code in the end_upload event on data it holding tables, care must be taken to ensure that data is not sent to the data source multiple times or not at all by the Mobilink Server Since these operations are taking place outside of the commit/rollback protection provided by the database holding the MobiLink System tables, how will you guarantee that a row that was sent to the data source really arrived?

Outline  Introduction  Describing the Sample  Implementation using SQL  Historical Implementation using Java  Historical Implementation using.Net  Understanding Direct Row Handling in version 10  Direct Row Handling using Java  Direct Row Handing using.Net

Direct Row Handling using Java  When using the Direct Row API, the only three synchronization events that need to be defined handle_uploadData –This event will process the upload stream begin_download –This event is only needed to set some global variables to capture the current MobiLink user that is synchronizing and the last download timestamp that was passed up by the remote database handle_downloadData –This event will create the download stream  The sample has also defined a handle_error and authenticate_user event written in Java as well

Outline  Introduction  Describing the Sample  Implementation using SQL  Historical Implementation using Java  Historical Implementation using.Net  Understanding Direct Row Handling in version 10  Direct Row Handling using Java  Direct Row Handing using.Net

Direct Row Handling Using.Net  At the time this presentation was submitted, the implementation of the.Net Direct Row API was not yet feature complete  An updated presentation should be available from the TechWave site after the completion of the conference with information in this section