This document describes some Solaris operating system tunables you may wish to apply to a production Stingray Traffic Manager instance. Note that the kernel tunables only apply to Stingray Traffic Manager software installed on a customer-provided Solaris instance; it does not apply to the Stingray Traffic Manager Virtual Appliance or Cloud instances.
Consider the tuning techniques in this document when:
For more information on performance tuning, start with the Tuning Stingray Traffic Manager article.
The ephemeral port range sets the upper limit on the the number of TCP connections a server can hold open to a particular IP address (including connections that have been closed and are in the TIME_WAIT state). Increasing it is recommended; doing so has no unwanted side effects on a typical system.
/usr/sbin/ndd -set /dev/tcp tcp_smallest_anon_port 1024
/usr/sbin/ndd -set /dev/tcp tcp_largest_anon_port 65535 # This is usually the default value
tcp_conn_req_max_q sets the maximum size of the TCP listen queue (the number of completed TCP connections waiting to be accept()ed), per port.
tcp_conn_req_max_q0 sets the maximum number of half-open TCP connections.
/usr/sbin/ndd -set /dev/tcp tcp_conn_req_max_q 1024
/usr/sbin/ndd -set /dev/tcp tcp_conn_req_max_q0 4096
If you expect a very high rate of new connections, you may find it beneficial to increase these again (by up to a factor of 10 or so).
The following two tunables force window scaling on and enable TCP timestamps (a protection against sequence number wraparound). We recommend they both be enabled.
/usr/sbin/ndd -set /dev/tcp tcp_wscale_always 1
/usr/sbin/ndd -set /dev/tcp tcp_tstamp_if_wscale 1
The following increases the size of the TCP/Send receive buffer, enabling more efficient handling of connections.
/usr/sbin/ndd -set /dev/tcp tcp_max_buf 16777216
The following three tunables set the maximum size of the TCP congestion, transmit and receive windows respectively.
/usr/sbin/ndd -set /dev/tcp tcp_cwnd_max 8388608
/usr/sbin/ndd -set /dev/tcp tcp_xmit_hiwat 4000000
/usr/sbin/ndd -set /dev/tcp tcp_recv_hiwat 4000000
Solaris' duplicate address detection can interfere with correct fail over of Traffic IP addresses. We recommend that the following tuning is applied to prevent this.
ndd -set /dev/arp arp_probe_count 0
ndd -set /dev/ip ip_dup_recovery 50
The following tunables have been found to increase performance on systems that use UltraSPARC T1 and T2 processors. We recommend that you consult Oracle for the recommended tunings for your Solaris release. The following lines should be added to /etc/system. The system will need to be rebooted before they take effect.
During our testing on OpenSolaris we encountered some issues when the tcp_fusion option was enabled. We found that the kernel could incorrectly buffer large amounts of data, which could cause the machine in question to hang. We recommend that customers using this release of OpenSolaris turn off the tcp_fusion feature to avoid this issue.
To turn off this feature you can immediately turn it off with the following command:
# echo 'do_tcp_fusion/W 0' | mdb -kw
This will not persist across system reboots. If you want 'tcp_fusion' to be permanently disabled you will need to add the following line to '/etc/system':
set ip:do_tcp_fusion = 0
The Sun UltraSPARC T1 and T2 processors contain on-board cryptographic routines that speed up SSL connections, and Stingray will use these routines if they are detected.
The T1 and T2 processors contain the ncp module which provides operations for the RSA algorithm. Additionally the T2 contains n2cp module which provides symmetric cipher support (such as AES, DES and 3DES) and digest algorithms (e.g. MD5 and SHA-1).
When Stingray starts up it will detect the presence of the ncp support and if it is present it will take full advantage of the performance improvements. In particular, don't configure PKCS#11 support on the Global Settings -> SSL Hardware support (this is used for external SSL hardware such as a PCI card or network device).
You can check Stingray is using the ncp support by running:
$ $ZEUSHOME/zxtm/bin/zeus.zxtm -vv
Version 5.0, Build date: May 20 2008 22:05:47
Compiled for platform: SunOS
Compression library : zlib v1.2.3
Regex library : PCRE v7.6
Crypto library : Built-in PKCS#11: ncp/0 Crypto Accel Asym 1.0
Compiler : CC: Sun C++ 5.8 2005/10/13
XML library : 2.6.23
XSLT library : 10115
Note the "Crypto Library" line refers to ncp. If Stingray is reporting itself as using something else you should check that the ncp support is enabled using the cryptoadm list command.