Integrating SharePoint with Documentum - Dell€¦ · 2010 EMC Proven Professional Knowledge Sharing 8 Our Journey with the Integration How it started Now that we knew exactly what

EMC Proven Professional Knowledge Sharing 2010

Integrating SharePoint with Documentum

Rami A Al Ghanim and Sara A Al Mansour

Rami A Al GhanimSystems AnalystSaudi [email protected]

Sara A Al MansourSystems AnalystSaudi [email protected]

2010 EMC Proven Professional Knowledge Sharing 2

Table of Contents Introduction ................................................................................................................................... 3

Company Overview ....................................................................................................................... 4

History of Enterprise Content Management in Saudi Aramco IT, Dhahran ............................... 5

Our Journey with the Integration ................................................................................................... 8

How it started ............................................................................................................................ 8

EMC Documentum Repository Services for Microsoft SharePoint (EDRSMS) ....................... 10

Challenges .................................................................................................................................. 14

Future Goals and Integration Road Map ..................................................................................... 15

Disclaimer: The views, processes, or methodologies published in this compilation are those of the authors. They do not

necessarily reflect EMC Corporation’s views, processes, or methodologies.


Introduction When users browse any Enterprise Content Management (ECM) system, they have certain

expectations of where, what, when, and how to find the right information in the right time. Users

expect that a number of employees can work together to manage, version, edit, and store

documents. Information access is controlled, based on user roles. Users expect to be able to

easily store and retrieve their information. Users do not expect duplication or redundancy of

their documents or efforts. The feeling is, if the information exists and the users have access to

it, they, as users of the ECM system, have the right to simply find it. Users anticipate improved

communication and collaboration. And last but not least, users expect all of this to be done

through a user-friendly interface.

Saudi Aramco employees have the same expectations, and would like to use one system that

would make all their document-related issues go away. We, as service providers, could give this

to the Saudi Aramco users but we have one simple dilemma; we have two systems for us and

the user to choose from, with pros and cons to each. A way to combine the two systems would

be wonderful.

This article starts with a brief overview of Saudi Aramco and the history of ECM in Saudi

Aramco. Next, we will walk you through the journey of integration; why and how it started and

how we used EMC Documentum® Repository Services for Microsoft SharePoint (EDRSMS) to

resolve this situation. Finally, we will discuss our challenges, future goals, and the integration

road map.


Company Overview Owned by the Saudi Arabian Government, Saudi Aramco — with over 54,000 employees — is a

fully-integrated, global petroleum enterprise and a world leader in exploration and producing,

refining, distribution, shipping, and marketing. The Company manages the largest proven

reserves of conventional crude oil, 260 billion barrels, and manages the fourth-largest gas

reserves in the world, 263 trillion cubic feet. Saudi Aramco, through its affiliate, Vela Marine

International Ltd, owns and operates the world's second largest tanker fleet to transport its

crude oil production, which amounted to 1.7 million barrels per day in 2008. In addition to its

headquarters in Dhahran, Saudi Arabia, Saudi Aramco, through its affiliates, has joint ventures

and subsidiary offices in China, Japan, India, the Netherlands, the Republic of Korea, Malaysia,

Singapore, the United Arab Emirates, the United Kingdom, and the United States. Saudi

Aramco also refines and distributes oil products throughout the Kingdom of Saudi Arabia to

meet domestic daily energy demands. For more information, visit www.saudiaramco.com.


History of Enterprise Content Management in Saudi Aramco IT, Dhahran To better appreciate our current situation, and what we have been through regarding ECM, we

need to take you back to the year 2000! It was around that time when web applications were

becoming very popular in the region and Saudi Aramco was searching for a user friendly, web-

based document management system. At that time, the only one that fit Saudi Aramco’s

requirements was Livelink provided by Opentext. Livelink was acquired and rolled out

companywide as a document management and sharing application. It was used for uploading

and sharing product documentation, submitting status reports, project collaboration, simple

business process automation, and various other purposes. It was also around that time when

Documentum was acquired but used in a much smaller scale for specific applications.

As time passed, users became used to document management solutions and started to have

more challenging requirements. Documentum started to be used more widely and became more

popular within Saudi Aramco. Still, Documentum was used for specific applications that were

designed and used at a corporate level. Of course, Livelink was still there and very mature,

stable, and reliable. Livelink was very user friendly and provided most of what our users

needed. For a few years, we continued to provide Livelink as is and made a customized

application on top of Documentum.

Saudi Aramco uses SAP for business process automation. Some of these processes needed a

complex, secured document management system to store and manage the documents

associated with these business processes. So a project was started to look into the best product

in the market that can provide the needed requirements. We had two document management

solutions, Documentum and Livelink; so a technical decision had to be made. At that time,

Documentum was the only document management product certified by SAP among the two; so

the decision was to integrate SAP with Documentum. After that project was completed,

Documentum became the repository for all SAP documents. Of course, Livelink became

certified shortly after.

We also started getting requests for specific needs and requirements from the Livelink users

but, since we were only providing Livelink out of the box, we couldn’t fulfill those requests.


However, we were able to satisfy those needs with Documentum. This introduced a new

document management and sharing application along with a few questions; “Why do you have

the two applications?” “Why do they both provide almost the same features?” “Which one

should I use?” All valid questions from our customers and management. The answer was to go

with the application on which we invested the greatest amount of time and effort. So we now

had two systems; neither of them cheap in any way. We had to think right and fast. Saudi

Aramco conducted an application road map to decide on the future of many of its major

applications. Documentum was selected to be the solution for Saudi Aramco document

management since it was used in many corporate customized applications and was the

repository for Saudi Aramco SAP.

Once this decision was made, another set of questions were raised, “What to do with the

Livelink Content?” and “How do we manage the heavy users of Livelink?” We faced the

challenge of moving the Livelink Content to Documentum and converting all the Livelink users to

Documentum users. We looked at many solutions and nothing met our specific needs. We

developed our own migration tool and migrated all of our Livelink content to the customized

Documentum application, called e-Cabinet. A major awareness and change management

campaign was conducted. The result was that in 2006, Documentum became Saudi Aramco’s

document management system.

At around the same time, SharePoint 2007 Beta Release was distributed by Microsoft. We got a

request to establish a Knowledge Sharing System on top of SharePoint. Of course we all saw

the amazing features and user friendliness of SharePoint, but from a system perspective, we

didn’t want to add another application. Our users had a number of issues that a knowledge

sharing system would resolve. One of the issues that Saudi Aramco faced was a huge

retirement rush. All the extremely knowledgeable employees who have been working for the

company were scheduled to leave within the next five years. Their knowledge needed to be

captured and saved for current and future employees. Another issue was the duplication of

efforts when it came to analyzing and solving similar problems. As an example, when an

engineer working in Plant A notices a major problem with one of the machines in the plant, he or

she needs to look into the problem and analyze what could be causing it. This engineer found

the solution and proceeded to fix the problem. Solving this problem could take hours, days, or

weeks, depending on the situation. Now, if another engineer in Plant B faces the same problem.

what does he or she do? They go through the same process of analyzing and solving the issue.


This could be done more efficiently if the engineer had a place to go where he or she could

search for similar types of issues or even connect with the engineer who faced this similar

problem. We also had an issue with the huge number of information or organizational web sites

throughout the company and the need to consolidate those in one area. It was very clear that

we could resolve those problems using SharePoint collaboration and Content Management

features. So we spent some time looking into the Beta and eventually went ahead with

SharePoint 2007.

During that time, Livelink was fully migrated to e-Cabinet, the customized application on top of

Documentum. So, after all of that we are back to square one. We have two very similar

applications that provide very similar features. “Where do we put our stuff?” “Where can I find

this?” “What’s the difference between Knowledge and Content?” All were valid questions that

started to be repeated. The answers were a bit more difficult this time around. Why did we

provide the two systems? Looking into the situation and analyzing it further we discovered that

we did need the two systems. We discovered that each had its own areas that were extremely

efficient. Having the two ECM systems made it very difficult for us and our users. So we knew

what we had to do, but had no idea how to do it. The two systems–Documentum and

SharePoint–had to remain but only one would be apparent to users. Since we had invested a lot

in Documentum’s Enterprise Content Management features and modules, and after studying the

situation and looking into the market trends, we decided to keep SharePoint 2007 as the user-

friendly interface to Documentum; it would be our Knowledge and Content Management

System. That starts our story with integrating SharePoint with Documentum.


Our Journey with the Integration

How it started Now that we knew exactly what we wanted to do, we thought the rest would be easy …

WRONG!!! Due to the growing popularity of SharePoint, everyone provided some sort of

integration. We started looking at all of them!! And I do mean ALL of them. We had specific

requirements that needed to be met for us to take advantage of both of our systems; otherwise,

what was the point? We wanted to have the user continue working on SharePoint as is, with the

same look and feel of SharePoint. At the same time, we wanted all the documents to reside in

Documentum. Due to Microsoft’s recommended 100 GB limitation of the SQL database, we

knew we would face an issue there. Additionally, because we had invested in other EMC

Documentum ECM features such as Records Management, we wanted to be able to continue

using them after the integration took place.

None of the products we looked at fit our specific requirements. Most of them focused on the

idea of web-parts, which would provide a window view to Documentum from within SharePoint.

Even though that was very easy to implement and provided a very similar document view and

update interface to the one provided by SharePoint, that wasn’t acceptable to us or our users.

One of the products provided by EMC – Archive Services for SharePoint – seemed to have the

potential, so we decided to invest more time and effort on that product. We felt that Archive

Services would mature into a product that would fit our needs based on our testing and its road

map. Two of Saudi Aramco’s employees (who happen to be the two writing this article) were

sent on a year and a half business assignment in EMC. We were sent to gain knowledge about

EMC’s SharePoint integration products and actually helped build one of them. We weren’t spies

but we did learn a lot. We had the chance to work with the designers of Archive Services as it

matured and – of course – changed its name quite a few times.

Regardless of the name changes, we felt that the product we were following had reached a

point where it provided exactly what we were looking for; EMC called it EMC Documentum

Repository Services for Microsoft SharePoint (EDRSMS). To confirm our belief, a project team

was formed consisting of one project manager, two Documentum Administrators, two


SharePoint Administrators, and two Documentum Analysts. This project team was tasked with

evaluating the controlled release of the EDRSMS through the Controlled Release Program

(CRP). The CRP was very useful to Saudi Aramco, as it gave us early access to the product

and allowed us to collaborate with other companies and the product design team during the

entire process of planning, installing, and testing EDRSMS. After the completion of the CRP, it

was clear to us that it did provide exactly what we were looking for. Then the real fun started!!


EMC Documentum Repository Services for Microsoft SharePoint (EDRSMS) After evaluating the product, a quote provided by Saudi Aramco for use in the product rollout

best expresses our feeling about the product and why we decided to go with it. "EDRSMS

creates a true partnership, like Yin & Yang, SharePoint having the friendly face for everyone to

see and Documentum providing the muscle for everyone to use. EDRSMS combines best-of-

breed solutions allowing users to access their documents from the more user friendly interface

(SharePoint) while storing and managing them in a robust content management platform

(Documentum)."

EDRSMS journals SharePoint content into Documentum and provides seamless integration

between SharePoint and Documentum. Before going into how EDRSMS works, let’s go over the

points that led to choosing EDRSMS. First of all, the integration was transparent to the users. All

they had to do was log in to SharePoint to upload, update, or view their documents. Although it

appeared as if they were in SharePoint, they were all actually in Documentum. Second, it

relieved SQL Server bloat by having the content now stored in Documentum instead of the SQL

Server; this led to easier manageability of the storage. Third, it improved productivity since the

users would not need to access the two systems. Finally, it would allow us to capitalize on our

investment in the ECM features of Documentum and its add-on modules such as Records

Management (as promised in the future release of EDRSMS.)

Before installing EDRSMS, both existing SharePoint and Documentum environments needed to

be studied to properly design how the integration would fit within both environments. Taking a

closer look at our Documentum environment (Figure 1) illustrates how our web clients connect

to Documentum through the J2EE or IIS Servers which go through the connection brokers to

our Documentum Content Servers. We have five content servers hosting a number of

Repositories. The Content Server we are targeting for the integration is the one we call PDW. It

hosts the e-Cabinet Repository.


On the other side of it, we have our Microsoft SharePoint environment. Figure 2 shows how our

web clients would connect through the web interface to one of our Web Front Ends. In this case,

we have three Web Front End Servers; one of them hosts the Central Administration. All the

content and metadata is stored in the SQL Server databases.


To integrate SharePoint with Documentum, we needed to install and configure the needed

components of the product where they fit, as illustrated in Figure 3. We first needed to create a

new Repository (Rep_EDRSMS) that would host the SharePoint Content. Optionally, you could

also install an EDRSMS DAR file that would take care of Advanced Mapping. This means it

would bring a set of optional SharePoint properties back to Documentum such as SharePoint

Location, SharePoint Created By, SharePoint Modified By, SharePoint Deleted, etc. This would

be useful for reuse of the content or if you want to query Documentum for some specific

information based on this mapping. We decided to go with Advanced Mapping. We would then

need to install the Admin Component on the server that hosts Central Administration. This

would be used to configure and monitor EDRSMS through the EDRSMS Configuration page.

You would then need to install the EDRSMS Component on EACH Web Front End Server. This

Component consists of the Journaling Service (JME) and the Cleanup Service (CCS). The CCS

identifies the items that have been deleted from SharePoint and also need to be deleted from

Documentum. The Journaling Service does the actual content journaling and deletion in


Documentum. Once the Journaling Service is installed, you need to create the EDRSMS

database on the SQL Server (preferably) in its own instance. This database will host the

configuration information and contain the blob ID information for the content that is stored in

Documentum. After you have done all of this, your content will now be stored in the

Documentum Repository and only the blob information will reside in the EDRSMS database.


Challenges Our requirements didn’t end there. To be successful, we needed to consider existing content in

SharePoint and Documentum. Unfortunately, EDRSMS does not consider existing SharePoint

content, and would only integrate new content. After collaboration with the EDRSMS designers

and project team, we decided to take two approaches to this.

1- Forward Approach: We would do nothing. Once the user updated their content in

SharePoint, it would go though the journaling process. We sometimes called this the

“Lazy Approach.”

2- Backward Approach: We built a program that would force the content through the

journaling process. We would use this approach for SharePoint sites with large-sized

content that should move ASAP.

We also needed to consider the Documentum Content that already existed in e-Cabinet. If we

truly wanted to benefit from the integration and give the user one point of access, we needed to

migrate that content to SharePoint. Due to the integration, that content would go back to

Documentum. Technically, this change could be done in one step but we have to consider

change management and awareness of the e-Cabinet users. A phased approach was adopted

to minimize the impact of the change on the users.

We also had some backup/restore requirements that EDRSMS didn’t address. For example, we

needed to be able to copy or move our SharePoint environment to a new one for upgrades or in

case of disaster. Although possible, it had to be performed without the out-of-the-box solutions

provided by SharePoint. Therefore, extra consideration needed to be taken while attempting the

backup/restore operations.

Another challenge that we faced was not a technical one; it was related to segregation of the

administration function. We currently have two separate entities that administer each system

(SharePoint – Documentum). The EDRSMS configuration page is part of the SharePoint

Administration page and it is mainly used for the connection to Documentum, which requires a

Documentum administrator to set up. For the time being, we devised our own method until this

issue is addressed in future releases.


Future Goals and Integration Road Map Once we had the integration in place, our first phase was complete. The driver behind phase

one was to use the power of Documentum and enhance what we currently have in SharePoint.

Phase two was to take advantage of EMC Documentum’s features, e.g. Records Management,

through the integration. Although this is currently not achievable in the current release of the

product, it will be available in future releases of EDRSMS. With over six Terabytes of electronic

content (and growing), we cannot possibly keep this content forever and needed a solution that

would apply retention policies to these documents based on their type. We have already started

planning for this as the second phase of the integration.

Once we have completed all of this, we will have accomplished our Enterprise Content

Management goals and we can truly say that we have provided our users and our company with

what they require from content management and knowledge management.

Documents

Integrating SharePoint with Documentum - Dell€¦ · 2010 EMC Proven Professional Knowledge Sharing 8 Our Journey with the Integration How it started Now that we knew exactly what