#1697 Dogtag does not accept an externally signed CA certificate, through ipa-server-install
Closed: Invalid None Opened 8 years ago by jamesmasson.

Following on from this thread - https://www.redhat.com/archives/freeipa-users/2015-October/msg00257.html

Dogtag ( as part of IPA) seems to refuse to use a signed CA cert which passes validation by IPA install (python) and openssl.

1. I use ipa-server-install in two-phase mode to create a CSR.

/sbin/ipa-server-install -U -a password -p password --hostname=foo.local -r LOCAL -n local --mkhomedir --setup-dns --forwarder=8.8.8.8 --external-ca

IPA CSR

-----BEGIN CERTIFICATE REQUEST-----
MIICpzCCAY8CAQAwMDEOMAwGA1UECgwFTE9DQUwxHjAcBgNVBAMMFUNlcnRpZmljYXRlIEF1dGhv
cml0eTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMHqggYeq+LuQGREe75VsXcu4n8b
h/BjFc0UIN+yGVvZAPh2O8a68CHCuiONxEs41CZ56MIXcVGozE0JK0DGVkI53zpsXPHiXua2GAGe
jOLKkDuci5rfNDhzNLu4rgKAVld2bFTVexHKfBGyg0qiIxWrmEpulgBzN/dMpyVL1YHfx1s1reD2
zMSUF9H0ciax7jEW0wqDyzvLTIFjka1gdyU1833DluYGhlHZKpBAltiAVsgKHplFmwB+GtAmMygO
Zb5DAw6O4eoRof80GvwvUHDdksJUh6Uz4Cz+QCP/CA28KHMZT+bB3dJJXq+7+2cSxnCJETjxXrDH
P9HD4P/MpvUCAwEAAaAyMDAGCSqGSIb3DQEJDjEjMCEwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8B
Af8EBAMCAcYwDQYJKoZIhvcNAQELBQADggEBAJgTCqog8gg47UA5JG6EsK2lWtOvJ2iYas7du4Zr
F51gjjjuiyFsO0H2jND7HurtN86h26TfE2GxTdTXZCKV24qtYp9jfNm8utuDlhivL656fVhfQ+2z
bAjCwL5n2MW8aCXma+/jP8wfz6HXwyS3s/Pej6LpcykN0IvX08/9nxm315MT7EeXPekE+Q8rkV99
DAmJzJhk5bzGzyHQFgP3oTWivmqHcrOMdtEnNskNBCpNDWjH9kdbrgl7qGuryi/DfNrowwk9AAMg
eIlwmxO+6ZHhBf41kh8cYw1YPPXYMFB3SltfanJtV/9WW3yz+LVginwi0n2kwGvOg1xbsvRdgXY=
-----END CERTIFICATE REQUEST-----

2. I sign the CSR, using a private root CA.

ROOT CA CERT

-----BEGIN CERTIFICATE-----
MIIDQzCCAiugAwIBAgIUFSRMJpupr6zOYNz1vCm2yAKS/XMwDQYJKoZIhvcNAQEL
BQAwEzERMA8GA1UEAxMIcm9vdC5jb20wHhcNMTUxMTEyMTIxOTIzWhcNMTUxMjEx
MTYxOTIzWjATMREwDwYDVQQDEwhyb290LmNvbTCCASIwDQYJKoZIhvcNAQEBBQAD
ggEPADCCAQoCggEBANY3vwn4+qiIoxfAzgzUDuuzsMZaiy/9rb4Luo+b6tXeOJYJ
UgiW4e0IAZBx1d3cxEfIsLnEznKPLAKjYfufMrvaF8k8KxpJYMVVS4eShqqRdh4e
Edc3BSltCTDRTGu1U6z9A9TJfx8UGeF1Ki5/beyJkQ6FCuXlDTcuyAWYH8/MMlpB
e8aYDxDWQeVrajPsrqRSpnObCN9i0sYVKbfW0d3fz/N0f5pD5rpwK7b200tklYSS
MERgr1BP7Up8Gcflz6SC02TE9GNDoHRJ7Z+vUk4JK9zF80dd+a9jzq/csuHO41et
rabcIcaDHV60Zm70Th3lVEnp64EYhPmV3cIFWc8CAwEAAaOBjjCBizAOBgNVHQ8B
Af8EBAMCAa4wEwYDVR0lBAwwCgYIKwYBBQUHAwkwDwYDVR0TAQH/BAUwAwEB/zAd
BgNVHQ4EFgQU3WBw8v7zv+0BZWR/NZNn9iMSsAcwHwYDVR0jBBgwFoAU3WBw8v7z
v+0BZWR/NZNn9iMSsAcwEwYDVR0RBAwwCoIIcm9vdC5jb20wDQYJKoZIhvcNAQEL
BQADggEBALArRGXjfhlz1c0G6dMQkpcC1AglTPTvUR8yG4AIETyjwg3ZLtsUS1ev
1akj2Jvq6cacYhO5o5axVZU5DQrHSVdF8VDGQsL4L5dWvsA9YBf/mVGpDzxwA6K3
giReO/nF1ziV0yqYGrjNE1PW11faJ9Vd7yfbNEHm9iZZf/s2ZGdWJVk45T+dEzEw
IPm5zhrR4dFBeYLWYXSSKNvJegdiPu7n1bDoMS5N9Z5Fhc7jpytfg7Gi/EB8/eIF
tCqbS7r0tG8IH517g9DPiqWfo6DUtP+kMudllnU5XR8f6MuDCUeiz8eUu3vQbeUj
8Jh0kOjWbZYLaNeNNE5M6UGEiMRJKR0=
-----END CERTIFICATE-----

3. I verify the issued IPA Sub-CA cert, against the ROOT ca.

IPA SUB-CA Cert

-----BEGIN CERTIFICATE-----
MIIDNDCCAhygAwIBAgIUblro3DG6S2teTzACRH19ad/QvckwDQYJKoZIhvcNAQEL
BQAwEzERMA8GA1UEAxMIcm9vdC5jb20wHhcNMTUxMTEyMTIzMzEwWhcNMTUxMTEy
MjIzMzEwWjAwMQ4wDAYDVQQKEwVMT0NBTDEeMBwGA1UEAxMVQ2VydGlmaWNhdGUg
QXV0aG9yaXR5MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAweqCBh6r
4u5AZER7vlWxdy7ifxuH8GMVzRQg37IZW9kA+HY7xrrwIcK6I43ESzjUJnnowhdx
UajMTQkrQMZWQjnfOmxc8eJe5rYYAZ6M4sqQO5yLmt80OHM0u7iuAoBWV3ZsVNV7
Ecp8EbKDSqIjFauYSm6WAHM390ynJUvVgd/HWzWt4PbMxJQX0fRyJrHuMRbTCoPL
O8tMgWORrWB3JTXzfcOW5gaGUdkqkECW2IBWyAoemUWbAH4a0CYzKA5lvkMDDo7h
6hGh/zQa/C9QcN2SwlSHpTPgLP5AI/8IDbwocxlP5sHd0kler7v7ZxLGcIkROPFe
sMc/0cPg/8ym9QIDAQABo2MwYTAdBgNVHQ4EFgQUheO7yVxMUik20Zg3MPV0wL0Z
49QwHwYDVR0jBBgwFoAU3WBw8v7zv+0BZWR/NZNn9iMSsAcwDwYDVR0TAQH/BAUw
AwEB/zAOBgNVHQ8BAf8EBAMCAcYwDQYJKoZIhvcNAQELBQADggEBABi/IsMBPVz8
D45NfJMycO9MqhGtmddlz576I+ntLRyQjUUby3CKQ2jobcHi2AHLeL24vkqt87vD
CQr7rowcVxplX9zggCK2hO2CtprT3P1iaNXnkLIJifjx8dbn57nJTHTNXueC1OBW
xSkaOWSIK7G8QSmlTLzY0ySwQ4H+ELcdQeEHg04Ur65s47V4jECWtMemlEC3/RbS
XNTa+xGVGtW8FSwaQUBPYxiM9Kqge37WoLDFkssuDuN+rvzZvLHEdfJugRoffWeS
5uPisedjeNFwJgqFwzvOpHLHrLlsCwbM2F1vsVSYE++4VbwOnlVL3J94c+dpP7FO
NlTT7fzm3jw=
-----END CERTIFICATE-----
[root@foo ~]# openssl verify -verbose -CAfile vaultca.crt  ipa.crt 
ipa.crt: OK

4. I complete the two-phase IPA install

/sbin/ipa-server-install --external-cert-file=/root/ipa.crt  --external-cert-file=/root/vaultca.crt

5. Result

Dogtag hangs on startup (as part of second-phase IPA install )

dogtag debug log

12/Nov/2015:12:41:35][localhost-startStop-1]: CertUtils: verifySystemCerts() cert tag=audit_signing
[12/Nov/2015:12:41:35][localhost-startStop-1]: CertUtils: verifySystemCertByTag(audit_signing)
[12/Nov/2015:12:41:35][localhost-startStop-1]: CertUtils: verifySystemCertByNickname(auditSigningCert cert-pki-ca,ObjectSigner)
[12/Nov/2015:12:41:35][localhost-startStop-1]: CertUtils: verifySystemCertByNickname(): calling isCertValid()
[12/Nov/2015:12:41:35][localhost-startStop-1]: CertUtils: verifySystemCertByNickname() passed: auditSigningCert cert-pki-ca
[12/Nov/2015:12:41:35][localhost-startStop-1]: SignedAuditEventFactory: create() message=[AuditEvent=CIMC_CERT_VERIFICATION][SubjectID=$System$][Outcome=Success][CertNickName=auditSigningCert cert-pki-ca] CIMC certificate verification

java.lang.Exception: SystemCertsVerification: system certs verification failure
        at com.netscape.cms.selftests.common.SystemCertsVerification.runSelfTest(SystemCertsVerification.java:198)
        at com.netscape.cmscore.selftests.SelfTestSubsystem.runSelfTestsAtStartup(SelfTestSubsystem.java:861)
        at com.netscape.cmscore.selftests.SelfTestSubsystem.startup(SelfTestSubsystem.java:1797)
        at com.netscape.cmscore.apps.CMSEngine.startupSubsystems(CMSEngine.java:1738)
        at com.netscape.cmscore.apps.CMSEngine.startup(CMSEngine.java:1185)
        at com.netscape.certsrv.apps.CMS.startup(CMS.java:200)
        at com.netscape.certsrv.apps.CMS.start(CMS.java:1602)
        at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:114)
        at javax.servlet.GenericServlet.init(GenericServlet.java:158)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:277)
        at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:274)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAsPrivileged(Subject.java:536)
        at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:309)
        at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:169)
        at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:123)
        at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1272)
        at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1197)
        at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1087)
        at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5210)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5493)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)
        at org.apache.catalina.core.ContainerBase.access$000(ContainerBase.java:133)
        at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:156)
        at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:145)
        at java.security.AccessController.doPrivileged(Native Method)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:875)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:632)
        at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:672)
        at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1862)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
[12/Nov/2015:12:41:35][localhost-startStop-1]: SignedAuditEventFactory: create() message=[AuditEvent=SELFTESTS_EXECUTION][SubjectID=$System$][Outcome=Failure] self tests execution (see selftests.log for details)

[12/Nov/2015:12:41:35][localhost-startStop-1]: CMSEngine.shutdown()

Output of certificate verification tests.

[root@foo ~]# certutil -L -d /var/lib/pki/pki-tomcat/alias

Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

caSigningCert cert-pki-ca                                    CTu,Cu,Cu
root.com                                                     CT,c,
ocspSigningCert cert-pki-ca                                  u,u,u
subsystemCert cert-pki-ca                                    u,u,u
Server-Cert cert-pki-ca                                      u,u,u
auditSigningCert cert-pki-ca                                 u,u,Pu
[root@foo ~]# certutil -V -u C -d /var/lib/pki/pki-tomcat/alias -n 'caSigningCert cert-pki-ca'
certutil: certificate is valid
[root@foo ~]# certutil -V -u C -d /var/lib/pki/pki-tomcat/alias -n 'root.com'
certutil: certificate is invalid: Certificate type not approved for application.
[root@foo ~]# certutil -V -u C -d /var/lib/pki/pki-tomcat/alias -n 'ocspSigningCert cert-pki-ca'
certutil: certificate is invalid: Certificate type not approved for application.
[root@foo ~]# certutil -V -u C -d /var/lib/pki/pki-tomcat/alias -n 'subsystemCert cert-pki-ca'
certutil: certificate is valid
[root@foo ~]# certutil -V -u C -d /var/lib/pki/pki-tomcat/alias -n 'Server-Cert cert-pki-ca'
certutil: certificate is invalid: Certificate type not approved for application.
[root@foo ~]# certutil -V -u C -d /var/lib/pki/pki-tomcat/alias -n 'auditSigningCert cert-pki-ca'
certutil: certificate is valid

Version information

[root@foo ~]# rpm -qa | grep ipa
python-iniparse-0.4-9.el7.noarch
libipa_hbac-1.12.2-58.el7_1.17.x86_64
ipa-server-4.1.0-18.el7.centos.4.x86_64
libipa_hbac-python-1.12.2-58.el7_1.17.x86_64
sssd-ipa-1.12.2-58.el7_1.17.x86_64
ipa-python-4.1.0-18.el7.centos.4.x86_64
ipa-admintools-4.1.0-18.el7.centos.4.x86_64
ipa-client-4.1.0-18.el7.centos.4.x86_64

[root@foo ~]# rpm -qa | grep pki
pki-tools-10.2.6-7.el7.centos.x86_64
dogtag-pki-server-theme-10.2.6-1.el7.centos.noarch
pki-base-10.2.6-7.el7.centos.noarch
pki-server-10.2.6-7.el7.centos.noarch
dogtag-pki-console-theme-10.2.6-1.el7.centos.noarch
pki-console-10.2.6-1.el7.centos.noarch
krb5-pkinit-1.12.2-15.el7_1.x86_64
pki-ca-10.2.6-7.el7.centos.noarch

I have this all working for IPA Sub-CA certs signed by a seperate external Dogtag instance.

There is something specific about certificates signed by this particular (non-dogtag) CA, which Dogtag (inside IPA) chokes on, but otherwise passes all validation.

The debug logs seem to indicate the 'caSigningCert cert-pki-ca'
is the certificate it has problems with.

[12/Nov/2015:12:41:35][localhost-startStop-1]: CertUtils:
verifySystemCerts() cert tag=signing
[12/Nov/2015:12:41:35][localhost-startStop-1]: CertUtils:
verifySystemCertByTag(signing)
[12/Nov/2015:12:41:35][localhost-startStop-1]: CertUtils:
verifySystemCertByNickname(caSigningCert cert-pki-ca,SSLCA)
[12/Nov/2015:12:41:35][localhost-startStop-1]: CertUtils:
verifySystemCertByNickname(): calling isCertValid()
[12/Nov/2015:12:41:35][localhost-startStop-1]: CertUtils:
verifySystemCertByNickname() failed: caSigningCert cert-pki-ca
[12/Nov/2015:12:41:35][localhost-startStop-1]: SignedAuditEventFactory:
create()
message=[AuditEvent=CIMC_CERT_VERIFICATION][SubjectID=$System$][Outcome=Failure][CertNickName=caSigningCert
cert-pki-ca] CIMC certificate verification

But further checking seems to indicate the cert passes the relevant
checks ( Y A L )

[root@foo ~]# certutil -V -u Y -d /var/lib/pki/pki-tomcat/alias -n
'caSigningCert cert-pki-ca'
certutil: certificate is valid
[root@foo ~]# certutil -V -u A -d /var/lib/pki/pki-tomcat/alias -n
'caSigningCert cert-pki-ca'
certutil: certificate is valid
[root@foo ~]# certutil -V -u L -d /var/lib/pki/pki-tomcat/alias -n
'caSigningCert cert-pki-ca'
certutil: certificate is valid

The -O option of certutil could also help see if the certs really chains up or not:

e.g.
certutil -d /var/lib/pki/pki-tomcat/alias -O -n "ocspSigningCert cert-pki-ca"

If it chains up, the output of this command would be the cert in question plus all the ca certs that chains all the way to the root cert.

Another thing worth investigation are the validities of the certs in comparison with the CA signing cert, and then in comparison with the system were you are doing installation.

All the certs chain correctly

[root@foo ~]# certutil -d /var/lib/pki/pki-tomcat/alias -L

Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

root.com                                                     CT,c,
ocspSigningCert cert-pki-ca                                  u,u,u
subsystemCert cert-pki-ca                                    u,u,u
caSigningCert cert-pki-ca                                    CTu,Cu,Cu
Server-Cert cert-pki-ca                                      u,u,u
auditSigningCert cert-pki-ca                                 u,u,Pu
[root@foo ~]# certutil -d /var/lib/pki/pki-tomcat/alias -O  -n "root.com"
"root.com" [CN=root.com]

[root@foo ~]# certutil -d /var/lib/pki/pki-tomcat/alias -O  -n "ocspSigningCert cert-pki-ca"
"root.com" [CN=root.com]

  "caSigningCert cert-pki-ca" [CN=Certificate Authority,O=LOCAL]

    "ocspSigningCert cert-pki-ca" [CN=OCSP Subsystem,O=LOCAL]

[root@foo ~]# certutil -d /var/lib/pki/pki-tomcat/alias -O  -n "caSigningCert cert-pki-ca"
"root.com" [CN=root.com]

  "caSigningCert cert-pki-ca" [CN=Certificate Authority,O=LOCAL]

[root@foo ~]# certutil -d /var/lib/pki/pki-tomcat/alias -O  -n "root.com"
"root.com" [CN=root.com]

[root@foo ~]# certutil -d /var/lib/pki/pki-tomcat/alias -O  -n "ocspSigningCert cert-pki-ca"
"root.com" [CN=root.com]

  "caSigningCert cert-pki-ca" [CN=Certificate Authority,O=LOCAL]

    "ocspSigningCert cert-pki-ca" [CN=OCSP Subsystem,O=LOCAL]

[root@foo ~]# certutil -d /var/lib/pki/pki-tomcat/alias -O  -n "subsystemCert cert-pki-ca"
"root.com" [CN=root.com]

  "caSigningCert cert-pki-ca" [CN=Certificate Authority,O=LOCAL]

    "subsystemCert cert-pki-ca" [CN=CA Subsystem,O=LOCAL]

[root@foo ~]# certutil -d /var/lib/pki/pki-tomcat/alias -O  -n "Server-Cert cert-pki-ca"
"root.com" [CN=root.com]

  "caSigningCert cert-pki-ca" [CN=Certificate Authority,O=LOCAL]

    "Server-Cert cert-pki-ca" [CN=foo.local,O=LOCAL]

[root@foo ~]# certutil -d /var/lib/pki/pki-tomcat/alias -O  -n "auditSigningCert cert-pki-ca"
"root.com" [CN=root.com]

  "caSigningCert cert-pki-ca" [CN=Certificate Authority,O=LOCAL]

    "auditSigningCert cert-pki-ca" [CN=CA Audit,O=LOCAL]

Validity is good too

[root@foo ~]# date
Tue Nov 17 16:23:09 UTC 2015
[root@foo ~]# openssl x509  -in ipa.crt -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            43:79:78:46:73:cc:25:3d:70:8d:92:1b:25:b4:6c:bb:5a:87:65:66
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN=root.com
        Validity
            Not Before: Nov 17 16:17:13 2015 GMT
            Not After : Nov 28 11:17:13 2015 GMT
        Subject: O=LOCAL, CN=Certificate Authority
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:

Would it help if I signed a CSR for you, so you can replicate this issue locally?

Root CA cert is valid too

[root@foo ~]# date
Tue Nov 17 16:48:51 UTC 2015
[root@foo ~]# openssl x509  -in vaultca.crt -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            15:24:4c:26:9b:a9:af:ac:ce:60:dc:f5:bc:29:b6:c8:02:92:fd:73
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN=root.com
        Validity
            Not Before: Nov 12 12:19:23 2015 GMT
            Not After : Dec 11 16:19:23 2015 GMT
        Subject: CN=root.com
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:

Hi, could you display cert pretty print for any of the system certs, e.g. "ocspSigningCert cert-pki-ca" so I can take a look of the validity?

Also, I noticed that you have very short validity for the CA's, is this intentional?

Replying to [comment:8 cfu]:

Hi, could you display cert pretty print for any of the system certs, e.g. "ocspSigningCert cert-pki-ca" so I can take a look of the validity?

Also, I noticed that you have very short validity for the CA's, is this intentional?

Hi, just wondering if there is an update? thanks.

Would you like a demo environment for this problem? It's 100% reproducible. Or I'm happy to sign a CSR for you.

Yes, we have deliberately short certificate lifetimes. The same length lifetimes are no problem when using certificates signed by an external Dogtag instance. Varying the lifetimes has no effect, have gone between a few days and a few months.

Here's a pretty printed OCSP cert

[root@foo ~]# certutil -d /var/lib/pki/pki-tomcat/alias -L  -n "ocspSigningCert cert-pki-ca"

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 1 (0x1)
        Signature Algorithm: PKCS #1 SHA-256 With RSA Encryption
        Issuer: "CN=Certificate Authority,O=LOCAL"
        Validity:
            Not Before: Tue Dec 01 12:48:34 2015
            Not After : Mon Nov 20 12:48:34 2017
        Subject: "CN=OCSP Subsystem,O=LOCAL"
        Subject Public Key Info:
            Public Key Algorithm: PKCS #1 RSA Encryption
            RSA Public Key:
                Modulus:
                    ef:ac:bd:25:d6:48:b3:b0:38:17:2c:d4:11:39:04:e3:
                    62:99:66:07:7c:6f:b6:84:a7:62:3b:cd:01:51:ee:e3:
                    95:37:aa:37:3f:af:dc:e7:21:ea:1b:9d:97:aa:82:6a:
                    58:fe:2e:36:a4:7a:78:e3:9f:2c:3a:1b:3f:f7:9a:d0:
                    10:e4:57:d4:dd:d6:85:30:cb:91:e6:24:32:5f:a3:b3:
                    90:64:cb:db:7d:71:b9:3b:e9:85:cd:06:39:72:04:bf:
                    bc:a0:d9:7b:88:72:b9:6b:97:74:65:fd:92:2b:23:0c:
                    8d:f4:ce:58:02:8f:46:32:b3:70:0f:e6:8c:c2:fc:6d:
                    4d:f6:cd:03:d7:e1:64:ea:a3:5e:8b:6c:86:b6:f0:73:
                    a4:37:94:71:c3:8e:a5:24:24:93:34:8b:ba:cf:6f:7e:
                    28:3c:1f:1f:9a:78:90:3e:97:cf:a5:e8:e8:c0:43:df:
                    be:eb:da:1c:0c:d0:71:01:36:de:c7:66:cc:2b:9a:00:
                    7d:52:1a:7c:65:5a:3c:a5:5f:9b:3b:de:25:dc:cf:3d:
                    0e:5e:51:c8:39:bf:06:23:e2:1d:cd:54:4c:d7:2f:89:
                    83:70:16:9a:64:49:7a:89:5a:28:61:c5:ce:f9:28:65:
                    8b:1b:89:3d:9e:5a:4c:bd:60:db:66:ec:dd:9f:73:f7
                Exponent: 65537 (0x10001)
        Signed Extensions:
            Name: Certificate Authority Key Identifier
            Key ID:
                4c:6c:57:90:2d:05:cd:57:e5:28:72:80:62:ef:56:56:
                e7:a2:18:0f

            Name: Certificate Key Usage
            Critical: True
            Usages: Digital Signature
                    Non-Repudiation
                    Certificate Signing
                    CRL Signing

            Name: Authority Information Access
            Method: PKIX Online Certificate Status Protocol
            Location: 
                URI: "http://foo.local:80/ca/ocsp"

            Name: Extended Key Usage
                OCSP Responder Certificate

    Signature Algorithm: PKCS #1 SHA-256 With RSA Encryption
    Signature:
        06:0a:f0:ab:53:42:f7:82:98:14:a5:e0:19:3e:a5:0a:
        8e:5d:b9:eb:42:e2:ad:7e:f3:e3:1a:04:f8:f4:dd:7f:
        1a:07:39:fa:a7:5b:b5:27:02:2f:ba:16:ac:71:c4:9b:
        e3:f0:af:fd:0f:ab:d2:11:42:d2:4e:ff:ea:68:9b:48:
        fd:c8:f7:d4:fd:36:0a:0e:f2:7a:3b:61:be:01:37:c0:
        69:a2:a8:c7:af:84:3d:15:50:7d:12:77:40:f1:f2:d0:
        c1:fd:5b:05:7e:4a:5b:c8:64:8d:d2:10:df:6f:46:c3:
        49:8f:ae:c7:36:42:6a:8a:49:4a:7c:0f:29:06:b2:c5:
        9f:58:bb:a6:38:97:b2:d6:95:3e:db:32:da:41:f4:b1:
        15:2a:92:0a:38:59:12:e3:b9:36:be:d2:3d:0f:08:8e:
        e6:00:14:92:44:a1:e2:9c:59:7d:b1:46:19:09:0e:67:
        fc:aa:bb:94:1e:af:47:10:fc:7d:b2:c2:68:e9:29:d9:
        15:c4:a4:e9:3e:90:c4:f2:12:a9:28:39:c9:23:9c:b0:
        fb:cd:51:56:91:d9:7c:a2:bd:9e:1f:3c:b9:89:14:e4:
        2f:48:2d:cc:24:fa:f9:c0:4a:ea:cf:c5:ae:16:d8:ba:
        71:6c:83:7a:c5:e5:e1:c7:5a:08:bb:96:1f:9b:9a:30
    Fingerprint (SHA-256):
        29:E3:70:35:87:8D:68:92:44:18:85:7A:4B:59:4A:90:1B:CE:62:CD:E9:A3:D6:BC:5F:99:37:20:8D:5C:4C:EE
    Fingerprint (SHA1):
        56:07:41:20:F0:AE:96:87:65:75:C8:75:B4:39:F4:6B:FF:08:8B:59

    Certificate Trust Flags:
        SSL Flags:
            User
        Email Flags:
            User
        Object Signing Flags:
            User

your signing CA expired before it signed the OCSP system cert? That doesn't seem right.

    Validity
        Not Before: Nov 17 16:17:13 2015 GMT
        Not After : Nov 28 11:17:13 2015 GMT
    Subject: O=LOCAL, CN=Certificate Authority

====
Validity:
Not Before: Tue Dec 01 12:48:34 2015
Not After : Mon Nov 20 12:48:34 2017
Subject: "CN=OCSP Subsystem,O=LOCAL"

I think you're confusing the previous IPA CA cert with the root cert.

See comment 7 for the root ( CA that signed the IPA cert - refered to as vaultCA ) certificate details

Replying to [comment:13 jamesmasson]:

I think you're confusing the previous IPA CA cert with the root cert.

See comment 7 for the root ( CA that signed the IPA cert - refered to as vaultCA ) certificate details

No, I was NOT talking at the root cert, as it looks fine. Please read them again again,
The OCSP signing cert was signed by "Subject: O=LOCAL, CN=Certificate Authority",
and according to comment 6, "Subject: O=LOCAL, CN=Certificate Authority" has the validity of
Not Before: Nov 17 16:17:13 2015 GMT Not After : Nov 28 11:17:13 2015 GMT ,
while in comment #10, it shows:
Issuer: "CN=Certificate Authority,O=LOCAL"
Validity:
Not Before: Tue Dec 01 12:48:34 2015
Not After : Mon Nov 20 12:48:34 2017
Subject: "CN=OCSP Subsystem,O=LOCAL"

Which tells me that the OCSP cert does not become valid until 12/1, while its issuer, "CN=Certificate Authority,O=LOCAL" already expired by Nov 28 11:17:13 2015 GMT.

The IPA CA cert is not the same one as before - as you mentioned, it expired and became invalid as expected.

Every time I update this ticket I generate a new CSR and sign it.

Here's this iteration's IPA certificate

[root@foo ~]# certutil -d /var/lib/pki/pki-tomcat/alias -L  -n "caSigningCert cert-pki-ca"
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            27:e5:19:98:a8:8f:11:b1:31:24:1f:9b:20:a9:3b:63:
            f1:31:b8:73
        Signature Algorithm: PKCS #1 SHA-256 With RSA Encryption
        Issuer: "CN=root.com"
        Validity:
            Not Before: Tue Dec 01 12:46:09 2015
            Not After : Sun Dec 06 12:46:09 2015
        Subject: "CN=Certificate Authority,O=LOCAL"
        Subject Public Key Info:
            Public Key Algorithm: PKCS #1 RSA Encryption
            RSA Public Key:
                Modulus:

ok, that explains it. thanks.
Another possibility is https://bugzilla.redhat.com/show_bug.cgi?id=1282892
In the bug, you will see that I was not sure when it became broken, so it is unclear if the issue exists in the bits used by IPA. Endi is going to add some debugging to assist with the investigation.

Replying to [comment:16 cfu]:

ok, that explains it. thanks.
Another possibility is https://bugzilla.redhat.com/show_bug.cgi?id=1282892
In the bug, you will see that I was not sure when it became broken, so it is unclear if the issue exists in the bits used by IPA. Endi is going to add some debugging to assist with the investigation.

actually, I just read the previous comments where you showed that the certs chained up. If that's case, then it's probably not 1282892. Either way, the extra debugging Endi is working on will help.

Hi James,

Could you install the following build and try again?
https://copr-be.cloud.fedoraproject.org/results/edewata/pki-epel/epel-7-x86_64/00144216-pki-core/

Please attach the debug log in /var/log/pki/pki-tomcat/ca/debug. It's supposed to show exactly where the validation fails in PKI. If this is still not sufficient, we may need to dig deeper into NSS/JSS.

It seems to be pointing the finger at the IPA CA signing cert - which is what I would expect. I've attached the full file, but here's a snippet.

[02/Dec/2015:10:21:21][localhost-startStop-1]: CertUtils: verifySystemCerts() cert tag=signing
[02/Dec/2015:10:21:21][localhost-startStop-1]: CertUtils: verifySystemCertByTag(signing)
[02/Dec/2015:10:21:21][localhost-startStop-1]: CertUtils: verifySystemCertByNickname(caSigningCert cert-pki-ca, SSLCA)
[02/Dec/2015:10:21:21][localhost-startStop-1]: CertUtils: verifySystemCertByNickname(): calling isCertValid()
[02/Dec/2015:10:21:21][localhost-startStop-1]: CertUtils: verifySystemCertByNickname() failed: caSigningCert cert-pki-ca
[02/Dec/2015:10:21:21][localhost-startStop-1]: CertUtils: verifySystemCertByNickname() failed: java.lang.Exception: Invalid certificate caSigningCert cert-pki-ca
[02/Dec/2015:10:21:21][localhost-startStop-1]: CertUtils: verifySystemCertsByTag() failed: java.lang.Exception: Invalid certificate caSigningCert cert-pki-ca
[02/Dec/2015:10:21:21][localhost-startStop-1]: SignedAuditEventFactory: create() message=[AuditEvent=CIMC_CERT_VERIFICATION][SubjectID=$System$][Outcome=Failure][CertNickName=] CIMC certificate verification

[02/Dec/2015:10:21:21][localhost-startStop-1]: SignedAuditEventFactory: create() message=[AuditEvent=CIMC_CERT_VERIFICATION][SubjectID=$System$][Outcome=Failure][CertNickName=] CIMC certificate verification

java.lang.Exception: Invalid certificate caSigningCert cert-pki-ca
        at com.netscape.cmscore.cert.CertUtils.verifySystemCertByNickname(CertUtils.java:855)
        at com.netscape.cmscore.cert.CertUtils.verifySystemCertByTag(CertUtils.java:946)
        at com.netscape.cmscore.cert.CertUtils.verifySystemCerts(CertUtils.java:1061)
        at com.netscape.cmscore.apps.CMSEngine.verifySystemCerts(CMSEngine.java:1732)
        at com.netscape.certsrv.apps.CMS.verifySystemCerts(CMS.java:1380)
        at com.netscape.cms.selftests.common.SystemCertsVerification.runSelfTest(SystemCertsVerification.java:193)
        at com.netscape.cmscore.selftests.SelfTestSubsystem.runSelfTestsAtStartup(SelfTestSubsystem.java:861)
        at com.netscape.cmscore.selftests.SelfTestSubsystem.startup(SelfTestSubsystem.java:1811)
        at com.netscape.cmscore.apps.CMSEngine.startupSubsystems(CMSEngine.java:1843)
        at com.netscape.cmscore.apps.CMSEngine.startup(CMSEngine.java:1291)
        at com.netscape.certsrv.apps.CMS.startup(CMS.java:200)
        at com.netscape.certsrv.apps.CMS.start(CMS.java:1609)
        at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:114)
        at javax.servlet.GenericServlet.init(GenericServlet.java:158)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:277)
        at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:274)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAsPrivileged(Subject.java:536)
        at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:309)
        at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:169)
        at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:123)
        at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1272)
        at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1197)
        at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1087)
        at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5210)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5493)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)
        at org.apache.catalina.core.ContainerBase.access$000(ContainerBase.java:133)
        at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:156)
        at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:145)
        at java.security.AccessController.doPrivileged(Native Method)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:875)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:632)
        at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:672)
        at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1862)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
[02/Dec/2015:10:21:21][localhost-startStop-1]: SignedAuditEventFactory: create() message=[AuditEvent=SELFTESTS_EXECUTION][SubjectID=$System$][Outcome=Failure] self tests execution (see selftests.log for details)

Here's the pretty print of the cert IPA CA cert

[root@foo ~]# certutil -d /var/lib/pki/pki-tomcat/alias -L -n 'caSigningCert cert-pki-ca'
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            51:62:8d:ea:2c:28:b6:d3:68:07:88:dc:d1:d6:18:d7:
            0a:a9:7b:6e
        Signature Algorithm: PKCS #1 SHA-256 With RSA Encryption
        Issuer: "CN=root.com"
        Validity:
            Not Before: Wed Dec 02 10:17:51 2015
            Not After : Mon Dec 07 10:17:51 2015
        Subject: "CN=Certificate Authority,O=LOCAL"
        Subject Public Key Info:
            Public Key Algorithm: PKCS #1 RSA Encryption
            RSA Public Key:
                Modulus:
                    cd:3a:7b:8a:cc:5c:8a:78:c7:e4:63:ad:e0:86:60:5e:
                    50:d5:8b:f3:15:ae:9a:5a:e9:17:14:cb:6b:b0:bc:af:
                    78:5a:4e:96:cc:a5:ef:60:e6:4a:68:18:f9:ed:98:68:
                    c8:3e:9c:8f:0e:75:3c:8c:19:f3:be:ab:9e:50:e4:c6:
                    78:13:e9:0b:2b:83:81:24:2d:ac:7d:5c:41:1d:3c:d7:
                    9b:84:82:1b:b5:6d:79:a6:f3:c9:df:88:37:3f:66:6d:
                    63:ce:3b:ed:23:76:4a:61:78:0e:a6:53:38:06:5e:63:
                    0f:fb:c4:96:96:c4:31:9f:00:c1:c9:7e:0f:d8:75:0d:
                    3f:03:62:47:49:d6:5b:f2:6e:b1:b1:60:ae:ed:10:ec:
                    13:39:5a:a7:3a:72:6a:b6:04:60:3b:2a:64:59:71:71:
                    7d:4b:a1:ff:6d:0b:50:06:e2:f3:0b:e7:ad:09:96:af:
                    a0:30:3e:de:72:ce:cf:47:4e:eb:f2:e7:97:b1:19:2a:
                    5a:18:eb:06:25:37:17:d4:73:2f:3a:53:c7:e5:6d:86:
                    0b:8a:0c:3e:1f:b8:f8:1f:f6:55:c9:b7:3e:f1:84:d7:
                    b6:37:40:31:85:5d:43:16:59:3b:9e:1b:21:3a:77:9e:
                    82:ed:e4:31:ba:94:31:cd:6f:c2:7f:cc:8c:d9:33:33
                Exponent: 65537 (0x10001)
        Signed Extensions:
            Name: Certificate Subject Key ID
            Data:
                38:af:c0:f3:7c:31:d6:79:6a:81:c0:8b:4f:17:d9:58:
                12:16:7a:ba

            Name: Certificate Authority Key Identifier
            Key ID:
                dd:60:70:f2:fe:f3:bf:ed:01:65:64:7f:35:93:67:f6:
                23:12:b0:07

            Name: Certificate Basic Constraints
            Critical: True
            Data: Is a CA with no maximum path length.

            Name: Certificate Key Usage
            Critical: True
            Usages: Digital Signature
                    Non-Repudiation
                    Certificate Signing
                    CRL Signing

    Signature Algorithm: PKCS #1 SHA-256 With RSA Encryption
    Signature:
        48:13:b3:55:91:94:db:8f:8f:63:b9:91:c9:64:5a:29:
        f4:ab:5d:61:35:47:ce:9f:40:33:6f:e0:e9:34:84:63:
        74:51:39:dc:d8:71:aa:56:23:5a:63:df:59:dc:dd:47:
        8c:f5:57:2a:16:50:c4:ed:b5:4b:ca:a3:d9:e5:f3:d4:
        af:59:a2:48:99:84:96:f1:4e:ad:a3:c6:55:b2:1c:19:
        22:4a:b5:eb:6c:13:5a:76:1a:24:82:1d:0f:e0:32:1b:
        45:25:57:6a:3a:19:06:dc:4b:33:b6:be:75:55:88:ed:
        5d:d7:99:9c:b1:6f:01:2c:29:84:7d:f2:89:2f:21:75:
        02:c8:78:68:94:cf:8c:72:b5:2c:74:24:71:33:bb:6d:
        7b:47:27:35:d1:9d:f2:52:ed:3f:f7:df:ae:4b:72:66:
        2b:3d:ba:e6:f8:ac:ce:ad:37:93:64:b2:c3:bd:52:e4:
        8c:06:e4:55:cf:76:89:42:fe:95:2f:18:60:ad:6f:ae:
        fe:69:bd:59:e1:49:94:a7:a6:f0:ce:f0:61:49:9a:e1:
        61:76:c0:4e:34:c4:be:8b:d0:6c:f0:1c:ed:95:5b:7e:
        8a:d3:08:d4:69:8e:82:8a:99:cc:05:8f:34:96:3f:a0:
        84:0f:1d:94:c9:60:18:40:f1:5c:ea:fa:e6:29:7a:ef
    Fingerprint (SHA-256):
        C7:06:6A:4B:66:1A:A9:A0:39:FD:B0:48:6D:ED:8F:19:69:FD:A8:4C:F4:ED:46:0A:60:53:30:3B:9F:4C:EA:F2
    Fingerprint (SHA1):
        50:50:47:21:CD:3E:35:DF:5E:B1:7D:D0:DD:C5:25:51:58:7E:4B:E3

    Certificate Trust Flags:
        SSL Flags:
            Valid CA
            Trusted CA
            User
            Trusted Client CA
        Email Flags:
            Valid CA
            Trusted CA
            User
        Object Signing Flags:
            Valid CA
            Trusted CA
            User

Hi James,

Thanks for the info. It looks like the certutil and JSS are calling the same validation function (i.e. CERT_VerifyCertificate()) but they are handling the result differently. I will need to do further debugging in JSS/NSS. Could you sign the following CSR so I can reproduce the problem? Thanks!

-----BEGIN NEW CERTIFICATE REQUEST-----
MIICnzCCAYcCAQAwWjE3MDUGA1UEChMuYWJjLmlkbS5sYWIuZW5nLmJycS5yZWRo
YXQuY29tIFNlY3VyaXR5IERvbWFpbjEfMB0GA1UEAxMWQ0EgU2lnbmluZyBDZXJ0
aWZpY2F0ZTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAPEzaIJd1QcL
6D+KOmU377fac31VjchHcjBunqcvGx5KS2jKWzaNg72+6FLULYczrgfJQZnPgNVE
ZxnXwbFSTzPVpXOiTlOR9FF9ohS0GR3p2kob5UK51B933FIx+rcHdqAoezJF6lFI
+LGjhEHpl7fsK/Fd5cgqaRtknJOVk0RFKrWV27FOA5xPHiltfY+qagNb4Bou2xK7
Y+cEWnlnX7z7+yMKowFCX1efl4pmYSLI9riTD9lk3KNYpvZlbf3ee1CQ5X7SfP39
zFN7jkU5W5Bk1fZIZLFGcsO1h59Pm3bSypzmhQ8kdS4v7RO++UaNo6k9BV8fV+y5
yGLwk9/S8AMCAwEAAaAAMA0GCSqGSIb3DQEBCwUAA4IBAQCXJVqH1ebNk5CXPoQE
e1CPGnmU7Y4neQn3/Cx7gy+O7M4KygfmSGybxCyIut2JXBdaapWS4K4tZcE8cxkW
a5Vit0Z1FyLqkT9Chwdz9dbTK/CO7nLhdHmJ9XLsE2hnzfxeuSk1IAKMqCGTVMkn
PzqWHXqMgzPgn6aTbhiNE/v2zvCRMLu9JmefzMyTD8ImByT1o805S3GqvkDBg6j8
3thCHk5n3VVaZM8Puldka8uYW6XyFzRD4/SitcY+CaJxYHQNMYVMkymbSJPF0ZVM
ilnA5My4RD9R72U00sId5/SYUlYq/J7OveSv4mQyG6WZCvZdBDRFLRj0gj2d6gFe
68pt
-----END NEW CERTIFICATE REQUEST-----

Here you go...

IPA cert

-----BEGIN CERTIFICATE-----
MIIDTjCCAjagAwIBAgIUZ5WLs6nR3K8vd6ycT5zV05NNYfgwDQYJKoZIhvcNAQEL
BQAwEzERMA8GA1UEAxMIcm9vdC5jb20wHhcNMTUxMjAzMjAzMzM5WhcNMTUxMjI0
MTYzMzM5WjBaMTcwNQYDVQQKEy5hYmMuaWRtLmxhYi5lbmcuYnJxLnJlZGhhdC5j
b20gU2VjdXJpdHkgRG9tYWluMR8wHQYDVQQDExZDQSBTaWduaW5nIENlcnRpZmlj
YXRlMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA8TNogl3VBwvoP4o6
ZTfvt9pzfVWNyEdyMG6epy8bHkpLaMpbNo2Dvb7oUtQthzOuB8lBmc+A1URnGdfB
sVJPM9Wlc6JOU5H0UX2iFLQZHenaShvlQrnUH3fcUjH6twd2oCh7MkXqUUj4saOE
QemXt+wr8V3lyCppG2Sck5WTREUqtZXbsU4DnE8eKW19j6pqA1vgGi7bErtj5wRa
eWdfvPv7IwqjAUJfV5+XimZhIsj2uJMP2WTco1im9mVt/d57UJDlftJ8/f3MU3uO
RTlbkGTV9khksUZyw7WHn0+bdtLKnOaFDyR1Li/tE775Ro2jqT0FXx9X7LnIYvCT
39LwAwIDAQABo1MwUTAPBgNVHRMBAf8EBTADAQH/MB0GA1UdDgQWBBSYG1d2bOXo
80R9iG0EhqvIxzEY8DAfBgNVHSMEGDAWgBQJjNngux7zFLBCiKPPvdSXGliG+DAN
BgkqhkiG9w0BAQsFAAOCAQEAfPYfp2BHDWYoU7VvFswmmHDCWZxnGIszQBq+pzD1
1wnUg6EKVVDNhsLlfIjKeDHdng/69lf7ApIbOWwYKC3sicAYpBrQVz/GuZ3CM8TI
TU6JyVJpRw9TSnzcIMEU2OQP85elkSIUg/5UrYcBKaPLZ1kvIQ3BO+Wa2+xz50K2
ApFssY+zjc6ZV0PT/8yB4VIfORiMMcEiqccOEsbPpQf+IP2HJGIaJNYAWoT7mS8Z
WwSpK5rhTTKdB5CYjmMduiibrH7b5+Jsc9EpfYEC0yLcbp/5bwLPdPcOYwBwGYMQ
tA28FDyd5IpikKNhOWQ5ZWbMpKlW9Cbj4Bn0gxtd4K6+ew==
-----END CERTIFICATE-----

New root cert - with a bit more validity time...

-----BEGIN CERTIFICATE-----
MIIDQzCCAiugAwIBAgIUQPZZ3dzbDvPykG3hqBHtmk/S3DIwDQYJKoZIhvcNAQEL
BQAwEzERMA8GA1UEAxMIcm9vdC5jb20wHhcNMTUxMjAzMjAyNzU1WhcNMTYwMTAy
MjAyNzU1WjATMREwDwYDVQQDEwhyb290LmNvbTCCASIwDQYJKoZIhvcNAQEBBQAD
ggEPADCCAQoCggEBALB2b5Z1YJ3riEUeDyiW8YPZgYbdyh0FD+AZyQr6e8lAyynE
rSOliN4k9kLTb6E42IFLxXD+OPlTy445Cd7+jUNln0WqKFH5lVi7t5h0DjmtKPQC
dy4ocxYfIUUrs7m4FL85g0m53v6gLR06fJSHLOqrpwYowio6+hV1/Fq37rp73VAF
YBK6XC3IPOdDjMuAjJFavBhtADXCJ+jJR2nulyO4DeVP03PzkTxYA9w3K88AFUL5
4Afl0VvFn+6FBJj0WCKER1SW0H9D6AWjQd7h0m3vavcURHF3vGHlTlKBoKAI4Ai1
FgdNs616ylK1jYvl+wOJOp5M7KpOzA2CzjF6MN0CAwEAAaOBjjCBizAOBgNVHQ8B
Af8EBAMCAa4wEwYDVR0lBAwwCgYIKwYBBQUHAwkwDwYDVR0TAQH/BAUwAwEB/zAd
BgNVHQ4EFgQUCYzZ4Lse8xSwQoijz73UlxpYhvgwHwYDVR0jBBgwFoAUCYzZ4Lse
8xSwQoijz73UlxpYhvgwEwYDVR0RBAwwCoIIcm9vdC5jb20wDQYJKoZIhvcNAQEL
BQADggEBAJXYS/PWVcvl2rTX0UTnmHzJuWcI7+p/QqcsPqv6y0WnaB1zQPwQ3MSj
2E5aGroPhA3pSgZV56LmVm8XOGnk0R25jWZq1JRW0unCMjLb55XVwJkRZ6yAG8P1
n3eSsf3IHxLZZLg4LOscqvT1ImZYSYWZIs2JnIOn0ndNeDN+slSydza52VGfF1Sj
vJ3cAusLD4Jsiao3pvVLY+ybtZ1q3D0JNCxIMVp4gH3V4lAb3hLiQw8XQQjmwTFh
qQUoVQc8Mq/JXR5NpIqXmXQQM4bYRnrF5xT2+AuvxnWB/23OMlWyAPuOM3dCuzNT
8B1orCn3zI/Kh2o48PD8m4Qg9Fe2L8E=
-----END CERTIFICATE-----

Thanks for the certs. Unfortunately I'm unable to reproduce the problem with these certs on Fedora 23. I may need to try this on a RHEL machine (I don't have a CentOS machine). Could you clarify which CentOS version you are using, and how did you get the PKI 10.2.6-7.el7.centos packages? To my understanding the latest RHEL only has PKI 10.2.5-x.

I've been experimenting with a couple of PKI version over the last few months I've been working on this issue.

The 10.2.6-7 version was downloaded as a SRPM from rawhide, then built locally on the Centos7 machine.

I'm happy to offer you a public VM to test with, if you wish.

I've just reverted back to a stock, fully patched Centos7 image ( which did update NSS, interestingly ), with stock IPA/PKI - and now I get a different error

[root@foo ~]# rpm -qa | sort | egrep "ipa|pki|nss"
ipa-admintools-4.1.0-18.el7.centos.4.x86_64
ipa-client-4.1.0-18.el7.centos.4.x86_64
ipa-python-4.1.0-18.el7.centos.4.x86_64
ipa-server-4.1.0-18.el7.centos.4.x86_64
jansson-2.4-6.el7.x86_64
krb5-pkinit-1.12.2-15.el7_1.x86_64
libipa_hbac-1.12.2-58.el7_1.18.x86_64
libipa_hbac-python-1.12.2-58.el7_1.18.x86_64
libsss_nss_idmap-1.12.2-58.el7_1.18.x86_64
mod_nss-1.0.8-33.el7.x86_64
nss-3.19.1-7.el7_1.2.x86_64
nss-devel-3.19.1-7.el7_1.2.x86_64
nss-softokn-3.16.2.3-13.el7_1.x86_64
nss-softokn-devel-3.16.2.3-13.el7_1.x86_64
nss-softokn-freebl-3.16.2.3-13.el7_1.x86_64
nss-softokn-freebl-devel-3.16.2.3-13.el7_1.x86_64
nss-sysinit-3.19.1-7.el7_1.2.x86_64
nss-tools-3.19.1-7.el7_1.2.x86_64
nss-util-3.19.1-4.el7_1.x86_64
nss-util-devel-3.19.1-4.el7_1.x86_64
openssh-6.6.1p1-12.el7_1.x86_64
openssh-clients-6.6.1p1-12.el7_1.x86_64
openssh-server-6.6.1p1-12.el7_1.x86_64
openssl-1.0.1e-42.el7.9.x86_64
openssl-devel-1.0.1e-42.el7.9.x86_64
openssl-libs-1.0.1e-42.el7.9.x86_64
pki-base-10.1.2-7.el7.noarch
pki-ca-10.1.2-7.el7.noarch
pki-server-10.1.2-7.el7.noarch
pki-tools-10.1.2-7.el7.x86_64
python-iniparse-0.4-9.el7.noarch
python-nss-0.16.0-2.el7.x86_64
sssd-ipa-1.12.2-58.el7_1.18.x86_64
2015-12-04T10:08:43Z DEBUG args='/usr/bin/certutil' '-d' '/etc/httpd/alias' '-f' XXXXXXXX '-R' '-k' 'rsa' '-g' '2048' '-s' 'CN=IPA RA,O=LOCAL' '-z' '/tmp/tmpQdGKqw' '-a'
2015-12-04T10:08:43Z DEBUG Process finished, return code=0
2015-12-04T10:08:43Z DEBUG stdout=
Certificate request generated by Netscape certutil
Phone: (not specified)

Common Name: IPA RA
Email: (not specified)
Organization: LOCAL
State: (not specified)
Country: (not specified)

-----BEGIN NEW CERTIFICATE REQUEST-----
MIICZjCCAU4CAQAwITEOMAwGA1UEChMFTE9DQUwxDzANBgNVBAMTBklQQSBSQTCC
ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMk4Q8QYUSDNJMctkx4OBfQT
dkOsGdd6X9aTsiQPeAfcG3gSM0uFgO/ZuvPVDvNJ9pHVzB5ZexTdkXQfbIFlG8+L
ztnFC9u+4hEEcpiOJCOvjmoxUSM4oEThMR7k1nnVERS3URsxVOIWeTsuZv7JIuq5
UU6xwuh8FQ44+dLJdW0kh0UpbwpE1vVWAYaygK6xJvsLYjN8YsVK8zxRYZ/hB8B9
o83ebjz9+79fhDDUwZ5HYHUDkitf/np53LMB6+wYxXOcIc8wKVK1IIz0sR6LjiiQ
0chC78xiTtFpdc9FAi+a92fuXinGGnC2oRjHIXZkW3OMQKMKokTm6uMShuc505sC
AwEAAaAAMA0GCSqGSIb3DQEBCwUAA4IBAQCn6KX65zJgvBf0l8NxP0jZEnFLe/mS
ijSCYukGT3oE5rKZxICV0YFSOT+hRpg1iOjK6e/6L2gTB4RJzDOv0caimJUfnFIA
geycd2Gr1M/waRZyibXG6QqvMLJaTSVVDZmEzdkEdGCSV/67r+u6M7RwVojly3aB
xpcM07Pag0/h4yrHMxUjEtokiyixZ1wdQZONo8FEuYvNGhe4uizn471jlenfYvvS
4IN83URgMeGOQ8pgXoSZVYpLCrKIYjkjPQ6OcuQmS06xAep9X/EtbDtRiKEcm4EC
gLrk1o+IpesA+WKELpPbnLpRp2onLb7vwv+HnLUJw8lViLc/Mgt81jtJ
-----END NEW CERTIFICATE REQUEST-----

2015-12-04T10:08:43Z DEBUG stderr=

Generating key.  This may take a few moments...


2015-12-04T10:08:43Z DEBUG Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 382, in start_creation
    run_step(full_msg, method)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 372, in run_step
    method()
  File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 1149, in __request_ra_certificate
    self.requestId = item_node[0].childNodes[0].data
IndexError: list index out of range

2015-12-04T10:08:43Z DEBUG   [error] IndexError: list index out of range
2015-12-04T10:08:43Z DEBUG   File "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", line 646, in run_script
    return_value = main_function()

  File "/sbin/ipa-server-install", line 1170, in main
    ca_signing_algorithm=options.ca_signing_algorithm)

  File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 520, in configure_instance
    self.start_creation(runtime=210)

  File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 382, in start_creation
    run_step(full_msg, method)

  File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 372, in run_step
    method()

  File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 1149, in __request_ra_certificate
    self.requestId = item_node[0].childNodes[0].data

2015-12-04T10:08:43Z DEBUG The ipa-server-install command failed, exception: IndexError: list index out of range

PKI does come up now - but it's the IPA install code that fails.

I've attached the IPA install and PKI debug logs.

Taking the same, stock centos7 IPA, and upgrading PKI to your debug builds, gives the original error - PKI fails to come up.

[root@foo tmp]# rpm -qa | sort | egrep "^ipa|^pki|^nss"
ipa-admintools-4.1.0-18.el7.centos.4.x86_64
ipa-client-4.1.0-18.el7.centos.4.x86_64
ipa-python-4.1.0-18.el7.centos.4.x86_64
ipa-server-4.1.0-18.el7.centos.4.x86_64
nss-3.19.1-7.el7_1.2.x86_64
nss-devel-3.19.1-7.el7_1.2.x86_64
nss-softokn-3.16.2.3-13.el7_1.x86_64
nss-softokn-devel-3.16.2.3-13.el7_1.x86_64
nss-softokn-freebl-3.16.2.3-13.el7_1.x86_64
nss-softokn-freebl-devel-3.16.2.3-13.el7_1.x86_64
nss-sysinit-3.19.1-7.el7_1.2.x86_64
nss-tools-3.19.1-7.el7_1.2.x86_64
nss-util-3.19.1-4.el7_1.x86_64
nss-util-devel-3.19.1-4.el7_1.x86_64
pki-base-10.2.6-10.el7.centos.noarch
pki-ca-10.2.6-10.el7.centos.noarch
pki-core-debuginfo-10.2.6-10.el7.centos.x86_64
pki-javadoc-10.2.6-10.el7.centos.noarch
pki-kra-10.2.6-10.el7.centos.noarch
pki-ocsp-10.2.6-10.el7.centos.noarch
pki-server-10.2.6-10.el7.centos.noarch
pki-symkey-10.2.6-10.el7.centos.x86_64
pki-tks-10.2.6-10.el7.centos.noarch
pki-tools-10.2.6-10.el7.centos.x86_64
pki-tps-10.2.6-10.el7.centos.x86_64
[04/Dec/2015:10:40:02][localhost-startStop-1]: CertUtils: verifySystemCertByNickname(): calling isCertValid()
[04/Dec/2015:10:40:02][localhost-startStop-1]: CertUtils: verifySystemCertByNickname() failed: caSigningCert cert-pki-ca
[04/Dec/2015:10:40:02][localhost-startStop-1]: CertUtils: verifySystemCertByNickname() failed: java.lang.Exception: Invalid certificate caSigningCert cert-pki-ca
[04/Dec/2015:10:40:02][localhost-startStop-1]: CertUtils: verifySystemCertsByTag() failed: java.lang.Exception: Invalid certificate caSigningCert cert-pki-ca
[04/Dec/2015:10:40:02][localhost-startStop-1]: SignedAuditEventFactory: create() message=[AuditEvent=CIMC_CERT_VERIFICATION][SubjectID=$System$][Outcome=Failure][CertNickName=] CIMC certificate verification

[04/Dec/2015:10:40:02][localhost-startStop-1]: SignedAuditEventFactory: create() message=[AuditEvent=CIMC_CERT_VERIFICATION][SubjectID=$System$][Outcome=Failure][CertNickName=] CIMC certificate verification

java.lang.Exception: Invalid certificate caSigningCert cert-pki-ca
        at com.netscape.cmscore.cert.CertUtils.verifySystemCertByNickname(CertUtils.java:855)
        at com.netscape.cmscore.cert.CertUtils.verifySystemCertByTag(CertUtils.java:946)
        at com.netscape.cmscore.cert.CertUtils.verifySystemCerts(CertUtils.java:1061)

It doesn't look like IPA 4.2 builds on EL/Centos7 yet - there was talk of a COPR repo a while back, but it's gone a bit quiet.

Again, happy to give you access to a box to demo this problem.

I've created JSS and PKI builds that would provide more details on the certificate validation error:

Would you be able to try them? Thanks.

With just the JSS RPMs, here's the debug log

[root@foo ~]# rpm -qa | sort | egrep "^ipa|^pki|^nss"
ipa-admintools-4.1.0-18.el7.centos.4.x86_64
ipa-client-4.1.0-18.el7.centos.4.x86_64
ipa-python-4.1.0-18.el7.centos.4.x86_64
ipa-server-4.1.0-18.el7.centos.4.x86_64
nss-3.19.1-7.el7_1.2.x86_64
nss-devel-3.19.1-7.el7_1.2.x86_64
nss-softokn-3.16.2.3-13.el7_1.x86_64
nss-softokn-devel-3.16.2.3-13.el7_1.x86_64
nss-softokn-freebl-3.16.2.3-13.el7_1.x86_64
nss-softokn-freebl-devel-3.16.2.3-13.el7_1.x86_64
nss-sysinit-3.19.1-7.el7_1.2.x86_64
nss-tools-3.19.1-7.el7_1.2.x86_64
nss-util-3.19.1-4.el7_1.x86_64
nss-util-devel-3.19.1-4.el7_1.x86_64
pki-base-10.2.6-10.el7.centos.noarch
pki-ca-10.2.6-10.el7.centos.noarch
pki-core-debuginfo-10.2.6-10.el7.centos.x86_64
pki-javadoc-10.2.6-10.el7.centos.noarch
pki-kra-10.2.6-10.el7.centos.noarch
pki-ocsp-10.2.6-10.el7.centos.noarch
pki-server-10.2.6-10.el7.centos.noarch
pki-symkey-10.2.6-10.el7.centos.x86_64
pki-tks-10.2.6-10.el7.centos.noarch
pki-tools-10.2.6-10.el7.centos.x86_64
pki-tps-10.2.6-10.el7.centos.x86_64

With the new PKI RPMS, the second phase of install just fails.

[root@foo ~]# rpm -qa | sort | egrep "^ipa|^pki|^nss|^jss"
ipa-admintools-4.1.0-18.el7.centos.4.x86_64
ipa-client-4.1.0-18.el7.centos.4.x86_64
ipa-python-4.1.0-18.el7.centos.4.x86_64
ipa-server-4.1.0-18.el7.centos.4.x86_64
jss-4.2.6-38.el7.centos.x86_64
jss-debuginfo-4.2.6-38.el7.centos.x86_64
jss-javadoc-4.2.6-38.el7.centos.x86_64
nss-3.19.1-7.el7_1.2.x86_64
nss-devel-3.19.1-7.el7_1.2.x86_64
nss-softokn-3.16.2.3-13.el7_1.x86_64
nss-softokn-devel-3.16.2.3-13.el7_1.x86_64
nss-softokn-freebl-3.16.2.3-13.el7_1.x86_64
nss-softokn-freebl-devel-3.16.2.3-13.el7_1.x86_64
nss-sysinit-3.19.1-7.el7_1.2.x86_64
nss-tools-3.19.1-7.el7_1.2.x86_64
nss-util-3.19.1-4.el7_1.x86_64
nss-util-devel-3.19.1-4.el7_1.x86_64
pki-base-10.2.5-6.el7.centos.noarch
pki-ca-10.2.5-6.el7.centos.noarch
pki-core-debuginfo-10.2.5-6.el7.centos.x86_64
pki-javadoc-10.2.5-6.el7.centos.noarch
pki-kra-10.2.5-6.el7.centos.noarch
pki-ocsp-10.2.5-6.el7.centos.noarch
pki-server-10.2.5-6.el7.centos.noarch
pki-symkey-10.2.5-6.el7.centos.x86_64
pki-tks-10.2.5-6.el7.centos.noarch
pki-tools-10.2.5-6.el7.centos.x86_64
pki-tps-10.2.5-6.el7.centos.x86_64
[root@foo ~]# /sbin/ipa-server-install -U -a password -p password --hostname=foo.local -r LOCAL -n local --mkhomedir --setup-dns --forwarder=8.8.8.8 --external-ca

The log file for this installation can be found in /var/log/ipaserver-install.log
==============================================================================
This program will set up the IPA Server.

This includes:
  * Configure a stand-alone CA (dogtag) for certificate management
  * Configure the Network Time Daemon (ntpd)
  * Create and configure an instance of Directory Server
  * Create and configure a Kerberos Key Distribution Center (KDC)
  * Configure Apache (httpd)
  * Configure DNS (bind)

Warning: skipping DNS resolution of host foo.local
Checking forwarders, please wait ...
Using reverse zone(s) 33.168.192.in-addr.arpa.

The IPA Master Server will be configured with:
Hostname:       foo.local
IP address(es): 192.168.33.100
Domain name:    local
Realm name:     LOCAL

BIND DNS server will be configured to serve IPA domain with:
Forwarders:    8.8.8.8
Reverse zone(s):  33.168.192.in-addr.arpa.

Configuring NTP daemon (ntpd)
  [1/4]: stopping ntpd
  [2/4]: writing configuration
  [3/4]: configuring ntpd to start on boot
  [4/4]: starting ntpd
Done configuring NTP daemon (ntpd).
Configuring directory server (dirsrv): Estimated time 1 minute
  [1/38]: creating directory server user
  [2/38]: creating directory server instance
  [3/38]: adding default schema
  [4/38]: enabling memberof plugin
  [5/38]: enabling winsync plugin
  [6/38]: configuring replication version plugin
  [7/38]: enabling IPA enrollment plugin
  [8/38]: enabling ldapi
  [9/38]: configuring uniqueness plugin
  [10/38]: configuring uuid plugin
  [11/38]: configuring modrdn plugin
  [12/38]: configuring DNS plugin
  [13/38]: enabling entryUSN plugin
  [14/38]: configuring lockout plugin
  [15/38]: creating indices
  [16/38]: enabling referential integrity plugin
  [17/38]: configuring certmap.conf
  [18/38]: configure autobind for root
  [19/38]: configure new location for managed entries
  [20/38]: configure dirsrv ccache
  [21/38]: enable SASL mapping fallback
  [22/38]: restarting directory server
  [23/38]: adding default layout
  [24/38]: adding delegation layout
  [25/38]: creating container for managed entries
  [26/38]: configuring user private groups
  [27/38]: configuring netgroups from hostgroups
  [28/38]: creating default Sudo bind user
  [29/38]: creating default Auto Member layout
  [30/38]: adding range check plugin
  [31/38]: creating default HBAC rule allow_all
  [32/38]: initializing group membership
  [33/38]: adding master entry
  [34/38]: configuring Posix uid/gid generation
  [35/38]: adding replication acis
  [36/38]: enabling compatibility plugin
  [37/38]: tuning directory server
  [38/38]: configuring directory to start on boot
Done configuring directory server (dirsrv).
Configuring certificate server (pki-tomcatd): Estimated time 3 minutes 30 seconds
  [1/8]: creating certificate server user
  [2/8]: configuring certificate server instance
The next step is to get /root/ipa.csr signed by your CA and re-run /sbin/ipa-server-install as:
/sbin/ipa-server-install --external-cert-file=/path/to/signed_certificate --external-cert-file=/path/to/external_ca_certificate
[root@foo ~]# /sbin/ipa-server-install --external-cert-file=/root/ipa.crt  --external-cert-file=/root/vaultca.crt 

The log file for this installation can be found in /var/log/ipaserver-install.log
CA is not installed yet. To install with an external CA is a two-stage process.
First run the installer with --external-ca.

I've created a VM in AWS to demo the issue. centos@54.229.193.39

Drop me your ssh public key, and I'll add you to it.

Workflow is

  1. IPA install part 1
  2. /root/sign_ipa_csr.sh
  3. Write returned cert to /root/ipa.crt
  4. IPA install part 2

I had to create new EPEL 7.1 builds:

Now I'm getting this exception during selftest:

java.security.cert.CertificateException: Invalid certificate: (-8179) Peer's Certificate issuer is not recognized.
        at org.mozilla.jss.CryptoManager.verifyCertificateNowNative(Native Method)
        at org.mozilla.jss.CryptoManager.verifyCertificate(CryptoManager.java:1548)
        at com.netscape.cmscore.cert.CertUtils.verifySystemCertByNickname(CertUtils.java:851)
        ...

Perhaps someone more familiar with NSS could explain what this error message means when validating the IPA certificate.

jmagne did some investigation and found this variable in /etc/sysconfig/pki-tomcat:

NSS_ENABLE_PKIX_VERIFY=1

The file is created after running installation step 1. I'm not sure what this variable actually does, but if you unset the variable, the selftest will no longer fail.

/etc/sysconfig/pki-tomcat seems to be updated with NSS_ENABLE_PKIX_VERIFY in step 2

  [7/27]: enable PKIX certificate path discovery and validation

However, there's no way to skip NSS_ENABLE_PKIX_VERIFY being added - using

chattr +i /etc/sysconfig/pki-tomcat

causes the installer part2 to blow up.

It's not possible to run the installer part2 a second time, or continue a previously failed part2 install.

Is there a workaround for this?

Found it... /usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py

    def enable_pkix(self):
        installutils.set_directive(self.dogtag_constants.SYSCONFIG_FILE_PATH,
                                   'DONT_NSS_ENABLE_PKIX_VERIFY', '1',
                                   quotes=False, separator='=')

.... rename the variable to something that won't be recognised :-)

Presume this is still being investigated, so eventually my workaround won't be needed?

Since the variable was added by IPA code, you might want to check with the IPA team the reason behind it, whether it's OK to disable it, and if there's a fix/workaround (e.g. by adding a CLI parameter or by moving the variable to step 1). For PKI, we will fix ticket #850 to display NSS error message during selftest, but I don't believe there is anything to fix for this particular issue.

IPA 4.2 has just been released for Centos 7 - Unfortunately, we're now back with the same problem.

[14/Dec/2015:16:47:52][localhost-startStop-1]: CertUtils: verifySystemCerts() cert tag=audit_signing
[14/Dec/2015:16:47:52][localhost-startStop-1]: CertUtils: verifySystemCertByTag(audit_signing)
[14/Dec/2015:16:47:52][localhost-startStop-1]: CertUtils: verifySystemCertByNickname(auditSigningCert cert-pki-ca,ObjectSigner)
[14/Dec/2015:16:47:52][localhost-startStop-1]: CertUtils: verifySystemCertByNickname(): calling isCertValid()
[14/Dec/2015:16:47:52][localhost-startStop-1]: CertUtils: verifySystemCertByNickname() passed: auditSigningCert cert-pki-ca
[14/Dec/2015:16:47:52][localhost-startStop-1]: SignedAuditEventFactory: create() message=[AuditEvent=CIMC_CERT_VERIFICATION][SubjectID=$System$][Outcome=Success][CertNickName=auditSigningCert cert-pki-ca] CIMC certificate verification

java.lang.Exception: SystemCertsVerification: system certs verification failure
        at com.netscape.cms.selftests.common.SystemCertsVerification.runSelfTest(SystemCertsVerification.java:198)
        at com.netscape.cmscore.selftests.SelfTestSubsystem.runSelfTestsAtStartup(SelfTestSubsystem.java:861)
        at com.netscape.cmscore.selftests.SelfTestSubsystem.startup(SelfTestSubsystem.java:1797)
        at com.netscape.cmscore.apps.CMSEngine.startupSubsystems(CMSEngine.java:1701)
        at com.netscape.cmscore.apps.CMSEngine.startup(CMSEngine.java:1148)
        at com.netscape.certsrv.apps.CMS.startup(CMS.java:200)
        at com.netscape.certsrv.apps.CMS.start(CMS.java:1602)
        at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:114)
        at javax.servlet.GenericServlet.init(GenericServlet.java:158)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:277)
        at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:274)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
        at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:309)
        at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:169)
        at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:123)
        at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1272)
        at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1197)
        at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1087)
        at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5210)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5493)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:901)
        at org.apache.catalina.core.ContainerBase.access$000(ContainerBase.java:133)
        at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:156)
        at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:145)
        at java.security.AccessController.doPrivileged(Native Method)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:875)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:632)
        at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:672)
        at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1862)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

NSS_ENABLE_PKIX_VERIFY is not enabled.

Hi:

Has the test VM mentioned above been updated to be able to show this most recent failure?
Also, if not possible to view the failure on a host, would it be possible to have the nss db in question zipped up and sent over to look at?

Apologies - this was an expired local cert that happened to occur at the same time as the IPA 4.2 upgrade.

A fresh install cycle with a new cert, and the hacked 'cainstance.py' still works.

Having read the comments on this ticket and the FreeIPA #5546 I think
we can close this with resolution "invalid" and turn our attention to
why FreeIPA ever enabled the envvar and resolving the matter there.

Metadata Update from @jamesmasson:
- Issue set to the milestone: 10.3.0

7 years ago

Dogtag PKI is moving from Pagure issues to GitHub issues. This means that existing or new
issues will be reported and tracked through Dogtag PKI's GitHub Issue tracker.

This issue has been cloned to GitHub and is available here:
https://github.com/dogtagpki/pki/issues/2256

If you want to receive further updates on the issue, please navigate to the
GitHub issue and click on Subscribe button.

Thank you for understanding, and we apologize for any inconvenience.

Login to comment on this ticket.

Metadata