This is a spin off from ticket: https://fedorahosted.org/pki/ticket/1566

Two scenarios with same issue:
1. When CA has been installed, and a second subsystem is being installed and tries to contact the CA as security domain
2. If bypass the above issue with temporarily removed offending ciphers in CA, the 2nd subsystem (in our case, the kra) installs fine, however, doing a cert enrollment on CA, when archival is required, CA has the same issue talking to KRA.

Investigation shows that when both subsystem's SSL server and clients certs are generated on the HSM, Subsystems have issue talking to each other when TLS_ECDHE_RSA_ ciphers are present. Browsers (on nss soft token) has no problem talking the either subsystem (ssltap shows successful use of TLS_ECDHE_RSA_ ciphers)

Here is the ssltap log between CA-> KRA when doing archival:
--> [
(198 bytes of 193)
SSLRecord { [Wed Aug 19 15:16:48 2015]
0: 16 03 01 00 c1 | .....
type = 22 (handshake)
version = { 3,1 }
length = 193 (0xc1)
handshake {
0: 01 00 00 bd | ....
type = 1 (client_hello)
length = 189 (0x0000bd)
ClientHelloV3 {
client_version = {3, 3}
random = {...}
0: 7e 60 45 92 dc 6e ce 18 f0 7d ea 41 97 e0 52 aa | ~`E..n...}.A..R.
10: 10 9a 90 99 71 5f ba 48 54 4e 5c 30 91 7e ea cf | ....q_.HTN\0.~..
session ID = {
length = 0
contents = {...}
}
cipher_suites[7] = {
(0xc02f) TLS/ECDHE-RSA/AES128-GCM/SHA256
(0xc013) TLS/ECDHE-RSA/AES128-CBC/SHA
(0xc027) TLS/ECDHE-RSA/AES128-CBC/SHA256
(0xc014) TLS/ECDHE-RSA/AES256-CBC/SHA
(0xc012) TLS/ECDHE-RSA/3DES-EDE-CBC/SHA
(0x002f) TLS/RSA/AES128-CBC/SHA
(0x0035) TLS/RSA/AES256-CBC/SHA
}
compression[1] = {
(00) NULL
}
extensions[134] = {
extension type server_name, length [37] = {
0: 00 23 00 00 20 63 73 71 61 32 2e 69 64 6d 2e 6c | .#.. csqa2.idm.l
10: 61 62 2e 65 6e 67 2e 72 64 75 2e 72 65 64 68 61 | ab.eng.rdu.redha
20: 74 2e 63 6f 6d | t.com
}
extension type renegotiation_info, length [1] = {
0: 00 | .
}
extension type elliptic_curves, length [52] = {
0: 00 32 00 01 00 02 00 03 00 04 00 05 00 06 00 07 | .2..............
10: 00 08 00 09 00 0a 00 0b 00 0c 00 0d 00 0e 00 0f | ................
20: 00 10 00 11 00 12 00 13 00 14 00 15 00 16 00 17 | ................
30: 00 18 00 19 | ....
}
extension type ec_point_formats, length [2] = {
0: 01 00 | ..
}
extension type signature_algorithms, length [22] = {
0: 00 14 04 01 05 01 06 01 02 01 04 03 05 03 06 03 | ................
10: 02 03 04 02 02 02 | ......
}
}
}
}
}
]
Read EOF on Server socket. [Wed Aug 19 15:16:49 2015]

Efforts on trying to change either the subsystem server or client keys to "sign" only or "sign", "decrypt", and "unwrap" did not make any difference. All evidence seem to show that when the subsystem "client" (i.e. subsystem cert) keys are no HSM, there is an issue.