Implement Domain Value Maps (DVM) with Oracle Service Bus (OSB) 10R3

December 25, 2009 gschmutz Leave a comment Go to comments

Domain Value Maps (DVM) are an interesting feature of Oracle SOA Suite for supporting Canonical

Data Models. They help to map from one vocabulary, used in a given domain, to another vocabulary

used in a different domain. For example, one domain might represent a country with a numeric code

while another domain may represent a country with the ISO-standard alphanumeric country code. In

SOA Suite 10g there were part of the “old” Oracle ESB and in SOA Suite 11g they can be used from a

Mediator component.

Unfortunately this feature is not yet available in Oracle Service Bus (OSB). It will be added in the

future, as the statement of direction states.

This might be less of a problem when using both the Oracle SOA Suite together with the Oracle Service

Bus in a larger architecture. In this case, the responsibility for mapping these values can be delegated

to the SOA Suite 11g Mediator component, where the DVM functionality is available.

But if only the OSB is used standalone, then such value mappings would be nice in the OSB as well.

With the help of XQuery the DVM functionality can be implemented in a similar way, so that if the

feature is later available in a new version of OSB, it can be replaced by that with minimal work

involved. This blog entry shows the custom DVM functionality implemented for the Oracle Service Bus.

Custom DVM functionality implemented using XQuery function in OSB

In order to show the custom DVM functionality a simple OSB service has been implemented. It only

consists of a simple Proxy Service, which accepts a request message with two static code values

(country and currency), translates the message into the canonical format and returns (echo) it to the

caller. A sample request and the corresponding response can be seen in the window below taken from

soapUI.

The message flow of the Proxy Service below uses a stage (TransformationToCanonicalState), which

is responsible to map from the source to the canonical format. First the value maps are assigned to

variables (Assign 1 for country map and Assign 2 for currency amp) and then the message is

transformed using an XQuery function (Assign 3).

For the definition of the Domain Value Maps the same format known from the SOA Suite is used. The

picture below shows the map for the currency code mapping. The <columns> element defines the

number of value domains and assigns a name to each domain, in our case Source and Canonical. The

<rows> element defines the value mappings, i.e. the value 10 in the Source domain should be mapped

to to the value CHF in the Canonical domain.

The format is based on a XML Schema which can be found in the Oracle SOA Suite documentation

here. An XSD file with the schema is also part of the example OSB project which can be downloaded

from here.

The next window shows the definition of the 3rd Assign action in the XQuery Expression Editor. The

first parameter holds the source message to be translated and the next two parameters hold the

content of the two value maps for country and currency mapping.

The XQuery script (transform.xq) implements the transformation. The next figure shows it in the

graphical (design) view. The translation of the countryCode to isoCountry and from currencyCode to

isoCountry are both using the corresponding parameters holding the content of the domain value

maps. 

The source of the transform() function is shown below

The custom DVM functionality is wrapped by the lookupValue() function, matching the DVM

implementation of the Oracle SOA Suite, where a function with the same name is available.  The only

difference is the first parameter, which in our case is an element holding the contents of the domain

value map, whereas in the SOA Suite version the first parameter only holds the name (reference) of

the DVM file.

The source of the lookupValue() function can be seen below

The function looks up the necessary source value and translates it to corresponding target value.

Unfortunately the Oracle Service Bus does not support the XQuery Modules feature, which would allow

the creation of function libraries to be reused. Therefore the lookupValue() function needs to be

copied into the transform.xq script. This copy-paste reuse is not optimal, I hope that OSB will fully

support XQuery 1.0 and especially the Module functionality in the future.

This finishes the custom implementation of the DVM functionality in Oracle Service Bus. It closely

matches the DVM functionality of the Oracle SOA Suite, which Oracle will offer in a future version of

OSB as well. The solution shown in this blog can help in the meantime and will allow for an easy

migration to the built-in function in the future. It’s only a proof-of-concept and has not yet been used in

production!

The example project can be downloaded form here.

Introduction to Domain Value Maps

Domain value maps operate on actual data values that transit through the infrastructure at runtime. They enable you to map from one vocabulary used in a given domain to another vocabulary used in a different domain. For example, one domain may represent a city with a long name (Boston), while another domain may represent a city with a short name (BO). In such cases, you can directly map the values by using domain value maps. A direct mapping of values between two or more domains is known as point-to-point mapping. Table 44-1 shows a point-to-point mapping for cities between two domains:

Table 44-1 Point-to-Point Mapping

CityCode CityName

BELG_MN_STLouis BelgradeStLouis

BELG_NC BelgradeNorthCarolina

BO Boston

NP Northport

KN_USA KensingtonUSA

CityCode CityName

KN_CAN KensingtonCanada

Each domain value map typically holds a specific category of mappings among multiple applications. For example, one domain value map may hold mappings for city codes and another may hold mappings for state codes.

Domain value map values are static. You specify the domain value map values at design time using Oracle JDeveloper, and then at runtime, the domain value map columns are looked up for values.

Implementing MDS in Oracle SOA 11G MDS is used as a repository for managing and reusing shared resources like XSD, WSDL, XSL files.MDS should be implemented from day 1 of development otherwise developers will run into XSD mismatch, also during production it will be very tough to keep the XSD same.

The concept is very simple, rather than each project picking the xsd (any other file) from there project's 'xsd' folder, pick it from the common/shared folder.

Oracle SOA suite 11G provides Central MDS and Local MDS.

Central MDS is present inside the SOA server

Local MDS is a folder inside the JDeveloper

*** Ideally all the artifacts should be inside the central MDS.

We will go through step by step creating Central MDS repository for keeping XSD, use them in project and later see how to remove XSD from Central MDS.

Creating Central MDS repository for keeping XSD:

Step1.) Making MDS connection in JDeveloper

a.) Make a Database connection to MDS schema, all the XSD will get saved inside the DEV_MDS schema, this schema gets created when we run RCU.bat file, please run rcu.bat again if you have skipped it while installation.

b.) Create a new MDS connection, this MDS connection is to the Database, set the Connection type as "DB based MDS", this is actually creating MDS in the SOA server which saves all the repository files in the database (DEV_MDS).

We can now browse to the MDS repository and see what are the files inside it, this is the place where where the XSD files will come up.

Step2.) Pick the XSD files from the local system, While doing the JDeveloper installation it create a local MDS for us which is replicated in the central MDS, we will keep the XSD files in the local directory.

Go to this location on local system "Oracle\Middleware\jdeveloper\integration\seed\" and create a new folder in it 'apps' Oracle\Middleware\jdeveloper\integration\seed\appsand paste all the XSD's you want to move to central MDS.I have pasted here 'Employee.xsd'

Step3.) Create a simple JAR deployment profile that contains the artifacts,Create a new SOA project in JDeveloper and select empty composite, double click on the project to go on the properties.

Create a new deployment profile with name "SharedSOACentralMDS" type "JAR File"

Click on contributors and add a new contributor.

Browse for the folder "Oracle\Middleware\jdeveloper\integration\seed\apps"

Now click on Filters to check which all files we want to put on Central MDS, this is the screen where we can decide which files will go inside the MDS

Next, we have to create a deployment profile of the type SOA Bundle at the level of the composite application.

Create a new deployment profile of type "SOA Bundle" and go on dependencies

Step4.) Deployment

Deployment success

Browse through the MDS connection we created in JDeveloper

Using files from MDS in project

Create a new BPEL process by dropping a BPEL process on composite.xml

For the input and output of this process we will refer to the XSD present in the central MDS.

Un-check the option for "copy to project" to use it from MDS and take a look at the

URL "oramds:/apps/Employee.xsd"

Repeat the same for output variable too.

Now we will go inside the WSDL file and check what is the difference, instead of picking it from the xsd folder it is using "oramds:/apps/Employee.xsd" and picking it from the MDS

Deleting files from MDS

We can use Weblogic scripting tool (WLST) to delete this folder from MDS,

double click "wlst.cmd" inside C:\Oracle\Middleware\Oracle_SOA1\common\bin to start

Cross Reference Tables.

Oracle Cross Reference Table In Oracle ESB the crosse reference table was introduced to create domain value maps.

Within Oracle AIA this mechnism is one of the technical foundation on which AIA runs.

The documentaion was limited on this functionality. Some core functions used in stylesheets were explained. Popular cross-reference functions are:

xref:populateXRefRow() - Insert a record in the XRef table xref:markForDelete() - Delete a record from the XRef table

Note: The xref:markForDelete() will update the delete-column in the XRef-table. In future releases, this function could change that it will physicly delete the table instead of update.

Since Oracle SOA Suite 10.1.3.4, the documentation has improved dramaticly. Detailes description and examples are now documented. It can be found here:

http://download.oracle.com/docs/cd/E12524_01/doc.1013/e12638/esb_xref.htm

From conceptualpoint of view and almost technical it can be used a reference documentation in olde versions 10.1.3.3 and AIA 2.0.x.

XREF table creation and lookups in SOA Suite 11g I was working on a requirement on XREF tables in SOA Suite 11g. I would like  to share my experience here.

Sample requirement: I created two dummy tables  SAP_EMP_TABLE

R12_EMP_TABLE

The content of SAP_EMP_TABLE was as follows:

EMP_ID EMPNAME JOBCODE

1 Sharat SAP1

2 Neeraj SAP2

3 Abhishek SAP1

And I wanted to transfer the data in R12_EMP_TABLE as so that

SAP1=ORC1

SAP2=ORC2

Approach:

So, First I created XREF Table inside a BPEL process(XrefCreationDemo), SAP_ORCL.xref.

I populated this XREF table with Reference values and their respective column names using the function:

xref:populateXRefRow("SAP_ORCL.xref","SAP","SAP2","ORCL","ORC2","ADD") in an assign activity.

At first I got confused because there was nothing inside SOA_INFRA.XREF_DATA table after this. But at the same time I was sure that it is working fine because it was not running second time. Which means that actually the values were getting stored somewhere. For this please check the XREF data source at Weblogic Admin and there you can find out where actual XREF_DATA is present. There I saw that whatever I inserted was present. Here is the content present in XREF_DATA table.

  

After this I created another process XrefDemo, in which I used the following inside transform activity between source table and destination table

xref:lookupXRef("oramds:/deployed-composites/XrefCreationDemo_rev1.0/SAP_ORCL.xref","SAP",ns0:jobcode,"ORCL",true())

Here you can see that the XREF table we inserted is present at oramds:/deployed-composites/XrefCreationDemo_rev1.0/ which is nothing but the project name of the process in which we populated the XREF Table.

After using xref:lookupXRef I was successful in inserting the data in R12_EMP_TABLE and its content was as follows:

EMP_ID

EMPNAME

JOBCODE

1 Sharat ORC1

3 Abhishek ORC1

2 Neeraj ORC2

Conclusion: The Xref tables we create is a one-time activity and we can refer to any XREF table from oramds, if it is properly populated in XREF_DATA. We will be using BPEL process for populating the data.

I was able to do this only because of Bishnu, as he was the one who came up with the correct XREF_DATA table. Thanks Bishnu.

Note that there are some challenges when you use the cross-reference table.

Check that the table has is optimzed for perfomance. That is has to correct indexes and contraints. The latest patches of the Oracle SOA Suite, applies these indexes and contraints.

One question on this cross-reference table is the date format. The table contains two columns that specifies the date since 1 january 1970. You can obtain the correct format as follows:

select xref_table_name , xref_column_name , row_number, value , is_deleted, to_char((to_date('01/01/1970','mm/dd/yyyy')+ last_modified/86400000), 'dd-mm-yyyy hh24:mi:ss'), to_char((to_date('01/01/1970','mm/dd/yyyy')+ last_accessed/86400000), 'dd-mm-yyyy hh24:mi:ss')from xref_data