Embed Size (px)
Citation preview
1
DOCUMENTUM CONTENT SERVER CERTIFICATE BASED SSL CONFIGURATION WITH CLIENTS
EMC WHITE PAPER
ABSTRACT
This white paper is step-by-step guide for Content Server 7.2 and above
versions installation with certificate based SSL for secure communication with
clients. It explains creation of certificates for Connection Broker and Content
Server. It also describes steps to configure with clients like standalone DFC and
Documentum Administrator.
July, 2015
2
To learn more about how EMC products, services, and solutions can help solve your business and IT challenges, contact your local
representative or authorized reseller, visit www.emc.com, or explore and compare products in the EMC Store
Copyright © 2015 EMC Corporation. All Rights Reserved.
EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without
notice.
The information in this publication is provided “as is.” EMC Corporation makes no representations or warranties of any kind with
respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a
particular purpose.
Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.
For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com.
VMware and <insert other VMware marks in alphabetical order; remove sentence if no VMware marks needed. Remove highlight and
brackets> are registered trademarks or trademarks of VMware, Inc. in the United States and/or other jurisdictions. All other
trademarks used herein are the property of their respective owners.
Part Number H14393
3
TABLE OF CONTENTS
ABSTRACT ............................................................................................................................................. 1
TABLE OF CONTENTS ............................................................................................................................. 3
INTRODUCTION ..................................................................................................................................... 4 Intended audience .............................................................................................................................. 4 Terminology ....................................................................................................................................... 4
CONFIGURATION ................................................................................................................................... 4
DOCBROKER .......................................................................................................................................... 4 Create Docbroker Keystore .................................................................................................................. 4 Create Docbroker Kestore's password file ............................................................................................... 5
SERVER .................................................................................................................................................. 5 Create Server Keystore ........................................................................................................................ 5 Create Server Kestore's password file .................................................................................................... 5 Create Server Trust Store .................................................................................................................... 6
DFC ........................................................................................................................................................ 6 Create DFC Trust-Store ....................................................................................................................... 6
CONTENT SERVER INSTALLATION ......................................................................................................... 6 Docbroker Configuration with Installer ................................................................................................... 7 Manual Steps for Docbroker Configuration .............................................................................................. 9 Server Configuration with Installer ........................................................................................................ 9 Manual Steps for Server Configuration .................................................................................................. 12
UPGRADE ............................................................................................................................................. 12
CLIENTS CONFIGURATION .................................................................................................................. 14 DFC Configuration .............................................................................................................................. 14
Steps to install DFC client with certificate based SSL by installer ........................................................ 14 Manual Steps to configure DFC client with certificate based SSL ......................................................... 15
Documentum Administrator Configuration ............................................................................................. 16
Steps to configuration of DA .......................................................................................................... 16
TROUBLESHOOTING ............................................................................................................................ 16
CONCLUSION ....................................................................................................................................... 18
REFERENCES ........................................................................................................................................ 18
4
INTRODUCTION
Documentum release 7.2 support content server and connection broker connections in both native and secure modes. For secure
connections, Anonymous SSL is used by default. Support for Non Anonymous or Certificate based SSL for communication is new
feature introduced in Content Server to further enhance communication security. For using Certificate based SSL, Content
Server, Docbroker and Clients needs to be configured either while installation or after installation. This paper explains in detail
configuration and troubleshooting steps.
All the components (Server, Docbroker and Clients) should use same mode for secure communication. Mixed environments are
not supported, i.e. either all the components have to use Anonymous SSL or Non Anonymous SSL for communication. For
Certificate based SSL deferent ciphers are supported. For all RSA libraries supported ciphers please refer 7.2 content server
installation guide.
Intended audience
The audience for this white paper comprises personnel responsible for the Installation, configuration and deployment of Content
Server in production environments. This document is intended for internal EMC teams, partners and customers.
Terminology
Connection Broker / Docbroker: A connection broker or Docbroker is name server for Content Server.
Repository / Docbase: A repository or docbase is logical entity that provides abstraction to storage that stores native content
files and RDBMS that stores content metadata.
CONFIGURATION This section describes the steps required to configure different components of the system (Docbroker, Content Server and
Clients) to use certificate based SSL for communication. Following needs to be configured:
1. Docbroker as SSL server (Content Server & DFC as SSL clients) 2. Content Server as SSL server (DFC as SSL client) and SSL Client (Docbroker as SSL Server) 3. DFC as SSL client
To enable clients to use Certificate based SSL communication for secure connections to Docbroker and Content Server, we need
to install Docbroker and Content Server in secure or 'native & secure' mode. Content Server 7.2 installer support automatic
configuration of Certificate based SSL. So this configuration can do in two ways. One method is Configure the docbroker and
docbase while installation by selecting the certificates. Second method is after installation is complete, all the services need to
be stopped (Docbroker, Content Server & Method Server) and then system needs to be configured for Certificate based SSL
manually.
DOCBROKER
Create Docbroker Keystore
For configuring Docbroker as SSL server, a private key needs to be created for Docbroker and Certificate Signing Request (CSR)
needs to be generated using private key. This CSR then needs to be presented to and signed by Certificate Authority (CA) for
generating Docbroker's public certificate. Finally, we need to create a Keystore for Docbroker to store its private key and public
certificate. In this paper we are creating self-signed certificates instead of CA signed certificates.
All the above steps are described here in detail. In this paper OpenSSL tool is used for managing Public Key Infrastructure (PKI).
Below command will generate Docbroker’s private key (brokerkey.pem) and self-signed public certificate (brokercrt.pem), both
in PEM format.
5
openssl req -newkey rsa:2048 -x509 -nodes -sha256 -keyout brokerkey.pem -out brokercrt.pem -days 365 -subj
"/C=IN/ST=Kar/L=BLR/O=emc/OU=iig/CN=broker.com/[email protected]"
Docbroker will look for private key and public certificate in Keystore which should be in PKCS #12 format. There is no restriction
on Keystore's filename or extension. It should be placed in $DOCUMENTUM/dba/secure as Docbroker will look in this directory
for locating its keystore. To generate Docbroker's keystore (broker.p12) by storing key and self-signed public certificate, use
below command:
openssl pkcs12 -export -out broker.p12 -inkey brokerkey.pem -in brokercrt.pem -name broker -descert
When executed, it will ask for key’s and Keystore's password. Option -name is specified to provide an alias for keys in keystore
Create Docbroker Keystore’s password file
Keystores are secured by making them password protected. Docbroker Keystore's password is encrypted and stored in a file
which will be used by Docbroker for accessing Keystore. Plain password also will work without any error. Docbroker will look in
directory $DOCUMENTUM/dba/secure for locating password file. There is no restriction on password file's name or extension.
Below command generates password file with keystore's password:
dm_encrypt_password –encrypt <password> -keyname <AEK Name> [-passphrase <AEK Passphrage>] -lockbox
<lockbox> -lockboxpassphrase <Lockbox passphrase> -file <file>
e.g : dm_encrypt_password -encrypt broker -file broker.pwd -keyname CSaek -passphrase Password@123 -lockbox
lockbox.lb -lockboxpassphrase Password@123
dm_encrypt_password utility is installed with Content Server and will work on machines on which Content Server is installed.
SERVER
Create Server Keystore
Similar to Docbroker, private key and public certificates needs to be generated for Server and stored in Server's Keystore from
where Server will access them.
Below command will generate Server’s private key (serverkey.pem) and self-signed public certificate (servercrt.pem), both in
PEM format.
openssl req -newkey rsa:2048 -sha256 -x509 -nodes -keyout serverkey.pem -out servercrt.pem -days 365 -subj
"/C=IN/ST=Kar/L=BLR/O=emc/OU=iig/CN=server.com/[email protected]"
Server will look for private key and public certificate in Keystore which should be in PKCS #12 formats. There is no restriction on
Keystore's filename or extension. It should be placed in $DOCUMENTUM/dba/secure where server will look in this directory for
locating its keystore. To generate Server's keystore (server.p12) by storing key and self-signed public certificate, use below
command:
openssl pkcs12 -export -out server.p12 -inkey serverkey.pem -in servercrt.pem -name server –descert
Create Server Keystore’s password file
Server Keystore's password is encrypted and stored in a file in directory $DOCUMENTUM/dba/secure. Plain password also will
work without any error. There is no restriction on password file's name or extension. Below command generates password file
with Keystore's password:
dm_encrypt_password –encrypt <password> -keyname <AEK Name> [-passphrase <AEK Passphrage>] -lockbox
<lockbox> -lockboxpassphrase <Lockbox passphrase> -file <file>
6
e.g : dm_encrypt_password -encrypt server -file server.pwd -keyname CSaek -passphrase Password@123 -lockbox
lockbox.lb -lockboxpassphrase Password@123
Create Server Trust store
When acting as SSL client to Docbroker, Server needs to verify public certificate sent by Docbroker. For verifying Docbroker's
public certificate, Docbroker’s public certificate or CA certificate which was used to sign Docbroker's public certificate needs to be
stored in a Keystore. This Keystore will act as Server's trust-store and server will verify Certificates sent by Docbroker against
certificates in this keystore.
Server expects trust-store to be in PKCS #7 binary format. There is no restriction on trust-store's name or extension. It should
be placed in location $DOCUMENTUM/dba/secure. Below command generates trust-store by importing Docbroker's certificate.
openssl crl2pkcs7 -nocrl -certfile brokercrt.pem -outform der -out server-trust.p7b
DFC
Create DFC Trust-Store
When DFC will be acting SSL client to Docbroker or Server, it will use a trust-store to store trusted certificates. Certificates sent
by Docbroker or Server will be validated against the certificates in DFC trust-store. DFC trust store should be in JKS (Java
Keystore) format. There is no restriction on store's name or extension as well as location.
We will convert the Docbroker & Server public certificates that were generated in text format to binary format using OpenSSL.
Then public certificates of Docbroker and Server in binary format will be imported to DFC trust-store. Below commands converts
certificates from PEM to DER format:
openssl x509 -outform der -in brokercrt.pem -out brokercrt.der
openssl x509 -outform der -in servercrt.pem -out servercrt.der
Following commands create DFC trust-store (dfc.keystore) by importing Docbroker's & Server's public certificates:
keytool -importcert -keystore dfc.keystore -file brokercrt.der -alias broker
keytool -importcert -keystore dfc.keystore -file servercrt.der -alias server
CONTENT SERVER INSTALLATION
First, install Content Server using serverSetup.bin. In the last step installer provides option to launch Content Server
configuration program and continue with Docbase configuration or configure this later using server configuration program.
‘Configure now’ is express mode in which configuration program does not provide options to configure Docbroker and select
connection modes. In this case docbroker is created in native mode. If we try to configure docbase in certificate based ssl then
we will get docbase connection failure error. So need to install the docbase also in native mode. After installation we need to
follow the manual steps to convert docbroker and docbase into certificate based ssl mode.
7
If we select ‘Configure later’ and exiting from the installer then installer will support automatic configuration of Docbroker and
docbase into certificate based SSL.
Docbroker Configuration with Installer
1. First create the aek key manually with following command which is mentioned in CS admin guide.
dm_crypto_create [-location <location>][-lockbox <lockbox>] [-lockboxpassphrase <lockboxpassphrase>] [-keyname
<keyname>] [-location <location>] [-passphrase <passphrase>] [-noprompt] [-move] [-check] [-algorithm] [-help]
2. Copy Docbroker Keystore file and keystore password file and placed into $DOCUMENTUM/dba/secure folder.
3. Run Server Configuration program using $DM_HOME/install/Server_Configuration_Program.exe script to install Docbroker.
4. Select “Secure” or “Native and Secure” radio button in “Select Connection Mode for Connection Broker” page
8
5. Next page will come with option Use certificates check box
6. If select Use certificates check box it will display keystore filename, keystore password file name, cipher list etc entries in
the same page. Need to change the proper keystore filename, keystore password file name and cipher list. Here installer
supported default cipher is AES128-SHA. We need to enter the dfc keystore password in the password text box. Here Use
Default Trust store check box is optional.
9
7. Proceed the further steps as normal installation.
After installation cross check the entries in Docbroker configuration file (<Docbroker>.ini) under
[DOCBROKER_CONFIGURATION] section at $DOCUMENTUM/dba directory
keystore_file=broker.p12
keystore_pwd_file=broker.pwd
cipherlist=AES128-SHA
crypto_keyname = CSaek
crypto_lockbox=lockbox.lb
And also new entries will be added in dfc.properties file at $DOCUMENTUM/config
dfc.security.ssl.truststore=C\:\\Documentum\\dba\\secure\\dfc.keystore
dfc.security.ssl.truststore_password=AAAAEEC7iXVob/bqfJE57KEqP3pQukuV0hQJrc1FRRzVTH29
Manual Steps for Docbroker Configuration
Manual steps are useful when the docbroker is already installed without enable the certificate based ssl.
1. Install docbroker and stop the docbroker.
2. Add the bellow entries in Docbroker configuration file((<Docbroker>.ini) at $DOCUMENTUM/dba directory
keystore_file=broker.p12
keystore_pwd_file=broker.pwd
cipherlist=AES128-SHA
crypto_keyname = CSaek
crypto_lockbox=lockbox.lb
3. Add the bellow entries in in dfc.properties file under [DOCBROKER_CONFIGURATION] section at $DOCUMENTUM/config
dfc.security.ssl.truststore=C\:\\Documentum\\dba\\secure\\dfc.keystore
dfc.security.ssl.truststore_password=<dfc trust store password>
4. Start the docbroker manually.
Server configuration with installer
1. Copy Server Keystore file, server trust store file, keystore password file and DFC trust store file and placed in
$DOCUMENTUM/dba/secure folder.
2. Run Server Configuration program using $DM_HOME/install/Server_Configuration_Program.exe script.
3. Enter the Connection broker details and select the check box “Use Certificates” and enter the valid DFC trust store password
password in Docbroker information page.
10
4. Select “Secure” or “Native and Secure” radio button in “Select Connection Mode” page
11
5. Select Use certificates check box it will display keystore filename, keystore password file name, Trust store file name, cipher
list etc entries in the same page. Need to change the proper keystore filename , keystore password file name, Trust store
file name and cipher list. Here installer supported default cipher is AES128-SHA. We need to enter the valid dfc keystore
password in the password text box. Here Use Default Trust store check box is optional.
6. Proceed the further steps as normal installation.
After installation cross check the entries in Server configuration file (<server>.ini) under [SERVER_STARTUP] section at
$DOCUMENTUM/dba/config/<docbasename> directory
keystore_file=server.p12
keystore_pwd_file=server.pwd
truststore_file=server-trust.p7b
cipherlist=AES128-SHA
And also new entry will be added in dfc.properties file at $DOCUMENTUM/config
dfc.session.secure_connect_default=try_secure_first
12
Manual Steps for Server configuration
Manual steps are useful when the docbase is already installed without enable the certificate based ssl
1. Install Docbase and stop the docbase service.
2. Add the bellow entries in server configuration file((<server>.ini) under [SERVER_STARTUP] section at
$DOCUMENTUM/dba/config/<docbasename> directory keystore_file=server.p12
keystore_pwd_file=server.pwd
truststore_file=server-trust.p7b
cipherlist=AES128-SHA
And also add the bellow entry in dfc.properties file at $DOCUMENTUM/config
dfc.session.secure_connect_default=try_secure_first
3. Start the docbase manually
UPGRADE:
This section explaining the content server, docbroker and docbase upgrade steps with certificate based ssl. Irrespective of
anonymous or non-anonymous ssl we need to follow the bellow steps to configure certificate based ssl in upgrade.
1. Stop docbase and docbroker.
2. Upgrade the content server.
3. Upgrade the docbroker. In upgrade docbroker, installer is not supporting the certificate based ssl settings. So we need to
stop the docbroker and do the manual steps which are mentioned above.
4. Docbase installer will support certificate based SSL settings. Enter the Docbroker port and host which is already configured
in certificate based ssl and select the “Use certificate” option in docbroker information page while upgrading the docbase.
Enter proper dfc trust store location and dfc trust store password.
5. Select “Secure” or “Native and Secure” radio button in “Select Connection Mode” page
13
6. Select Use certificates check box it will display keystore filename, keystore password file name, Trust store file name, cipher
list etc entries in the same page. Need to change the proper keystore filename, keystore password file name, Trust store file
name and cipher list. Here installer supported default cipher is AES128-SHA. We need to enter the valid dfc keystore
password in the password text box. Here Use Default Trust store check box is optional.
14
7. Proceed the further steps as normal upgrade.
After installation cross check the entries in Server configuration file (<server>.ini) at
$DOCUMENTUM/dba/config/<docbasename> directory
keystore_file=server.p12
keystore_pwd_file=server.pwd
truststore_file=server-trust.p7b
cipherlist=AES128-SHA
And also new entry will be added in dfc.properties file at $DOCUMENTUM/config
dfc.session.secure_connect_default=try_secure_first
CLIENTS CONFIGURATION:
DFC configuration
DFC will look in dfc.properties file to resolve DFC trust-store location, name and password. Property dfc.security.ssl.truststore is
used to specify trust-store's path and dfc.security.ssl.truststore_password to specify trust-store's password. Trust-store's
password can be given in plain text or encrypted format. DFC installer will support automatic selection of password and trust
store location.
Steps to install DFC client with certificate based SSL by installer:
1. Select the Use certificates check box and enter the primary connection broker details.
15
2. Browse the trust store location and enter the password. Here Use Default Java Truststore check box is optional.
3. For secure connections to Server and Docbroker, put the value of property dfc.session.secure_connect_default as secure.
Same property controls behavior of connections to both Docbroker and Server. Put below property in dfc.properties file:
dfc.session.secure_connect_default = secure
Manual steps to configure DFC client with certificate based SSL:
1. In dfc.properties, put entries for DFC trust-store and trust-store's password:
dfc.security.ssl.truststore = c\:/secure/dfc.keystore
dfc.security.ssl.truststore_password = password
DFC trust-store password can be encrypted using DFC utility as below:
java com.documentum.fc.tools.RegistryPasswordUtils <password>
E.g: java com.documentum.fc.tools.RegistryPasswordUtils password
The encrypted password generated by this command can be copied and pasted to dfc.properties file. For this command to
execute successfully, dfc.jar should be in java classpath.
16
There is an additional property dfc.security.ssl.use_existing_truststore that is mutually exclusive to above two properties. When
this property is specified, there is no need to put above two properties. In this case, Java Keystore will act as DFC trust-store or
trust-store can be specified using JVM parameter javax.net.ssl.trustStore.
2. For secure connections to Server and Docbroker, put the value of property dfc.session.secure_connect_default as secure.
Same property controls behavior of connections to both Docbroker and Server. Put below property in dfc.properties file:
dfc.session.secure_connect_default = secure
Documentum Administrator configuration
Documentum Administrator (DA) is a WDK web based client. When DA wants to connect certificate based ssl docbase DA
contained DFC will be acting SSL client to Docbroker or Server, it will use a trust-store to store trusted certificates. Certificates
sent by Docbroker or Server will be validated against the certificates in DFC trust-store. DA we will look in dfc.properties file to
resolve DFC trust-store location, name and password. Property dfc.security.ssl.truststore is used to specify trust-store's path
and dfc.security.ssl.truststore_password to specify trust-store's password. Trust-store's password can be given in plain text or
encrypted format. We need to add the bellow entries in dfc.properties at <da-home>/WEB-INF/classes folder.
dfc.session.secure_connect_default = secure
dfc.security.ssl.truststore = c\:/secure/dfc.keystore
dfc.security.ssl.truststore_password = password
Steps to configuration of DA:
1. Copy dfc.keystore file into C:\Secure folder. There is no restriction on folder name or path.
2. Stop application server.
3. Extract the the da.war file under webapps folder in application server.
4. Add the global repository details along with bellow entires in dfc.properties at <da-home>/WEB-INF/classes folder.
dfc.session.secure_connect_default = secure
dfc.security.ssl.truststore = c\:/secure/dfc.keystore
dfc.security.ssl.truststore_password = password
5. Start Application server.
TROUBLESHOOTING:
This section describes some of the common issues faced during configuration and steps to troubleshoot
Docbroker startup fails
1. Check if Docbroker Keystore and Keystore's password files are present in $DOCUMENTUM/dba/secure 2. Check if entry for above two files is there in Docbroker configuration file (<Docbroker>.ini) 3. Check if the format of Docbroker Keystore is PKCS #12
Below commands should list the keys in Keystore if it’s in PKCS #12 format, else error will be displayed
17
Using OpenSSL
openssl pkcs12 -info -in <keystore>
Using Keytool:
keytool -list –v -storetype pkcs12 -keystore <keystore>.
4. Check if password in Keystore password file is correct. For testing, password can be put in plain-text (without encryption).
Docbase startup fails
1. Check if Server Keystore, Server Keystore password and Server trust-store files are present in $DOCUMENTUM/dba/secure 2. Check if entry for above three files is there in Server configuration file (server.ini) 3. Check if the format of Server Keystore is PKCS #12. See commands in Docbroker startup fails section to verify 4. Check if server trust-store is in PKCS #7 binary (der) format
For verifying, check if below command dumps it successfully openssl pkcs7 -in <Keystore> -inform der E.g. openssl pkcs7 -in server-trust.p7b -inform der
Server not able to connect to Docbroker
1. Check whether Docbroker Keystore has proper key and public certificate
Use below command to print the keys in keystore:
Using OpenSSL
openssl pkcs12 -info -in <keystore>
Using Keytool
keytool -list -storetype pkcs12 -keystore <keystore>
2. Check whether Docbroker is sending proper certificates
Below command will start a simple client which tries to connect to SSL Server and displays certificate chain sent by Server:
openssl s_client -showcerts -debug –connect <SSL_Server_IP>:<SSL_Server_Port>
E.g. openssl s_client -showcerts -debug -connect 10.8.53.24:1490
3. Check if Server's trust-store contains Docbroker's public certificate or CA certificate chain used to sign Docbroker's public
certificate
Use below command to display all certificates in trust store:
openssl pkcs7 -in <trust-store> -inform der -print_certs –text
Clients not able to connect to Docbroker 1. Check if proper entries are present in dfc.properties and trust-store file exists. 2. Check if DFC trust-store contains Docbroker's public certificate or CA certificate used to sign Docbroker’s public certificate
To dump Trust-store's contents (no need to specify storetype as default type is JKS):
keytool -list -keystore <keystore> -storepass <storepass>.
3. Check the firewall settings are proper in client and docbroker machines.
18
Clients not able to connect to Server
1. Check if proper entries are made in dfc.properties and trust-store file exists
2. Check if DFC trust-store contains Server's public certificate or CA certificate used to sign Server’s public certificate
To dump Trust-store's contents (no need to specify storetype as default type is JKS):
keytool -list -keystore <keystore> -storepass <storepass>
3. Check if dfc.properties has right value for trust-store password. For verification, password can be given in plain-text 4. Check the firewall settings are proper in client and server machines.
CONCLUSION
This white paper provides information required for understanding and quickly getting started with the process of installing
Documentum Content Server 7.2 with certificate based SSL. For more details refer product documentation and references
provided in next section.
REFERENCES
• Documentum Content Server Installation Guide Version 7.2
• Documentum Content Server Distributed Content Configuration Guide Version 7.2
• Documentum Content Server Administration and Configuration Guide Version 7.2
www.EMC.com
