Workaround for Oracle ZFS Appliance(ZFSSA) LDAPS / TLS Reject When Using OUD

I recently patched/upgraded firmware on a Oracle ZFS Appliance. Once the upgraded was completed, the ZFS Appliance LDAPS connections stopped working, the error was due to the ZFS Appliance rejecting the Diffie-Hellman(DH) Cipher being used in the LDAPS connections. It turns out the new firmware uses a more strict security model. Below is describe the issue plus workaround options available. Note: The article below demonstrate the issue (and workaround) with the assumption you are using Oracle Unified Directory(OUD) as you LDAP source. For more information on how to install and configure OUD please check this out – OUD Directory Proxy Installation Configuration

ZFSSA LDAPS TLS (dh) key length issue

After upgrading the ZFS Appliance firmware, we realized that the LDAPs service is not working. We then tried re-initializing/re-connecting to the LDAP server. Below is the error we got on the ZFS Appliance – showing in the picture below. Basically, the error is related in using a DH key too small.

An unanticipated system error occurred:

failed to connect to ldap.domain.com:1636: dh key too small

This may be due to transient failure, or a software defect. If this problem persists, contact your service provider.

OUD and SSL/TLS – Supported Keys

There are two options to modify the OUD Ciphers

• Option 1: Using Oracle Directory Service Manager(ODSM)
• Option 2: Using dsconfig OUD commend line utility

Note: Since installing and configuring ODSM is not simple and or might not be configured/used in your environment, the option of using dsconfg is first provided as a solution. Fixing the issue by using ODSM is more simpler and is described in the end of this article. Before we jump right in to the solution, I would like to show how the initial OUD key was created. for the full detail information check this out – OUD Directory Proxy Installation Configuration

keytool -genkeypair -alias ldap -keyalg rsa -keysize 2048 -validity 3560 -dname "cn=ldap.domain.com" -keystore ldap.jks -storetype JKS

OUD encryption / cipher algorithms

First, check what encryption / cipher algorithms is enabled in your OUD configuration. To found out which ciphers are currently enabled in your OUD instillation run the below. You should see something like the below.

nmap --script ssl-enum-ciphers -p 1636 ldap.domain.com

Starting Nmap 7.11 ( https://nmap.org ) at 2017-04-24 15:37 EDT
Nmap scan report for ldap.domain.com (10.10.10.11)
Host is up (0.0018s latency).
PORT     STATE SERVICE
1636/tcp open  unknown
| ssl-enum-ciphers:
|   TLSv1.0:
|     ciphers:
|       TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA (dh 768) - E
|       TLS_DHE_RSA_WITH_AES_128_CBC_SHA (dh 768) - C
|       TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA (secp160k1) - D
|       TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (secp160k1) - A
[...] snip
|   TLSv1.1:
|     ciphers:
|       TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA (dh 768) - E
|       TLS_DHE_RSA_WITH_AES_128_CBC_SHA (dh 768) - C
[...] snip
|   TLSv1.2:
|     ciphers:
|       TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA (dh 768) - E
|       TLS_DHE_RSA_WITH_AES_128_CBC_SHA (dh 768) - C
|       TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 (dh 768) - C
|       TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA (secp160k1) - D

As you can see from the output above, a dh 768 is being presented by the server, this causes the ZFS Appliance to reject the connection, since dh keys of less then 1024 are considered being an un-secure connection. For more details on lookup up Weak Diffie-Hellman and the Logjam Attack

Setting OUD Ciphers and encryption algorithms

First lets set a password file, it will make life easier 🙂

echo 'your-password' > pwdfile.txt
export PWS=pwdfile.txt

Now lets check what Ciphers are currently set in OUD.

cd $asinst_1/OUD/bin
./dsconfig -h localhost -p 4444 -D "cn=Directory Manager" -j $PWS --trustAll --no-prompt get-connection-handler-prop --handler-name "LDAPS Connection Handler"
Property               : Value(s)
-----------------------:---------
allow-ldap-v2          : true
allow-start-tls        : false
allowed-client         : -
denied-client          : -
enabled                : true
keep-stats             : true
key-manager-provider   : JCEKS
listen-address         : 0.0.0.0
listen-port            : 1636
ssl-cert-nickname      : ldap
ssl-cipher-suite       : -
ssl-client-auth-policy : optional
ssl-protocol           : -
trust-manager-provider : JCEKS
use-ssl                : true

As you can see above ssl-cipher-suite and ssl-protocol are not set, meaning its using the default which is all available ciphers. To note, since OUD is a pure Java LDAP implementation it will support any SSL Cipher supported by your java version – described here, and here is the full java cipher supported spec.

Setting which ciphers OUD will accept

Now that we see the issue, lets try and fix it by setting which cipher’s OUD will accept. The below commend will change/set the accepted protocols and ciphers. Note: A directory restart is required for the change to take effect.

./dsconfig \
set-connection-handler-prop \
-h localhost \
-p 4444 \
-D "cn=Directory Manager" \
-j $PWS \
--trustAll \
--no-prompt \
--handler-name "LDAPS Connection Handler" \
--add ssl-protocol:SSLv3 \
--add ssl-protocol:TLSv1.1 \
--add ssl-protocol:TLSv1.2 \
--add ssl-cipher-suite:TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA \
--add ssl-cipher-suite:TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 \
--add ssl-cipher-suite:TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 \
--add ssl-cipher-suite:TLS_RSA_WITH_AES_128_CBC_SHA \
--add ssl-cipher-suite:TLS_RSA_WITH_AES_128_CBC_SHA256 \
--add ssl-cipher-suite:TLS_RSA_WITH_AES_128_GCM_SHA256

The above sets the protocols accepted to SSLv3, TLSv1.1 and TLSv1.2 but no TLSv1.0. it also sets the cipher-suite accepted. Note: None of the dh keys are being added to the accepted list. Now, a restart to OUD is required for the change to take effect, to restart run the below.

./stop-ds; sleep 1;./start-ds

Now lets check the accepted protocols and ciphers

./dsconfig -h localhost -p 4444 -D "cn=Directory Manager" -j $PWS --trustAll --no-prompt get-connection-handler-prop --handler-name "LDAPS Connection Handler"
Property               : Value(s)
-----------------------:-------------------------------------------------------
allow-ldap-v2          : true
allow-start-tls        : false
allowed-client         : -
denied-client          : -
enabled                : true
keep-stats             : true
key-manager-provider   : JCEKS
listen-address         : 0.0.0.0
listen-port            : 1636
ssl-cert-nickname      : ldap
ssl-cipher-suite       : TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,
                       : TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,
                       : TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
                       : TLS_RSA_WITH_AES_128_CBC_SHA,
                       : TLS_RSA_WITH_AES_128_CBC_SHA256,
                       : TLS_RSA_WITH_AES_128_GCM_SHA256
ssl-client-auth-policy : optional
ssl-protocol           : SSLv3, TLSv1.1, TLSv1.2
trust-manager-provider : JCEKS
use-ssl                : true

Ciphers omitted from the list

Below are a list of protocols and ciphers not being set. Note: Some of these ciphers are required not to be set if TLS 1.0 is not being used, otherwise it will end-up breaking OUD connections.

--remove ssl-protocol:TLSv1.0 \
--remove ssl-cipher-suite:TLS_DHE_RSA_WITH_AES_128_CBC_SHA \
--remove ssl-cipher-suite:TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 \

## Required to not use if TLS 1.0 is not being used
--remove ssl-cipher-suite:TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA \
--remove ssl-cipher-suite:TLS_RSA_WITH_3DES_EDE_CBC_SHA \
--remove ssl-cipher-suite:TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 \

Using Oracle Directory Service Manager(ODSM) to set the Protocols and Ciphers

You can also change the Protocols and Ciphers with Oracle Directory Service Manager(ODSM). Below is an example on how to change Protocols and Ciphers by using ODSM.

Checking SSH and TLS week DH Security issues

As a side note, below you can see how to check for the the same week DH in an SSH connection.

ssh -v -o 'KexAlgorithms diffie-hellman-group1-sha1' server
ssh -v -o 'KexAlgorithms diffie-hellman-group14-sha1' server
ssh -v -o 'KexAlgorithms diffie-hellman-group-exchange-sha1' server
ssh -v -o 'KexAlgorithms diffie-hellman-group-exchange-sha256' server

group 1 (768 bits) and group14 (1024 bits) should never be used. if you need the other two for compatibility with legacy clients, at least generate a new moduli file if you haven’t already (see the ssh-keygen documentation). To check which algorithms your system supports, just run the below.

ssh -Q kex         
diffie-hellman-group1-sha1
diffie-hellman-group14-sha1
diffie-hellman-group14-sha256
[...] snip

Understanding SSl/TLS References

Diffie-Hellman SSL/TLS SSL/TLS History OUD Ciphers and supported algorithms