Issue #5412: ipa-ca-install on promoted replica hangs on creating a temporary CA admin - freeipa

freeipa

#5412 ipa-ca-install on promoted replica hangs on creating a temporary CA admin

Closed: Fixed None Opened 8 years ago by mbabinsk.

This issue appears intermittently during testing the new replica promotion functionally.

During both stand-alone CA install and replica installation with '--setup-ca' option in domain level 1, the installation hangs on the creation of temporary CA admin:

Configuring certificate server (pki-tomcatd). Estimated time: 3 minutes 30 seconds
  [1/24]: creating certificate server user
  [2/24]: creating certificate server db
  [3/24]: setting up initial replication
Starting replication, please wait until this has completed.
Update in progress, 3 seconds elapsed
Update succeeded

  [4/24]: creating installation admin user

It seems that the replica is unable to push the created admin entry onto the master, see dirsv error logs:

master:

[27/Oct/2015:16:36:53 +0000] slapd_ldap_sasl_interactive_bind - Error: could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -1 (Can't contact LDAP server) ((null)) errno 107 (Transport endpoint is not connected)
[27/Oct/2015:16:36:53 +0000] slapi_ldap_bind - Error: could not perform interactive bind for id [] authentication mechanism [GSSAPI]: error -1 (Can't contact LDAP server)
[27/Oct/2015:16:36:59 +0000] slapd_ldap_sasl_interactive_bind - Error: could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -1 (Can't contact LDAP server) ((null)) errno 107 (Transport endpoint is not connected)
[27/Oct/2015:16:36:59 +0000] slapi_ldap_bind - Error: could not perform interactive bind for id [] authentication mechanism [GSSAPI]: error -1 (Can't contact LDAP server)
[27/Oct/2015:16:37:11 +0000] slapd_ldap_sasl_interactive_bind - Error: could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -1 (Can't contact LDAP server) ((null)) errno 107 (Transport endpoint is not connected)
[27/Oct/2015:16:37:11 +0000] slapi_ldap_bind - Error: could not perform interactive bind for id [] authentication mechanism [GSSAPI]: error -1 (Can't contact LDAP server)
[27/Oct/2015:16:37:35 +0000] slapd_ldap_sasl_interactive_bind - Error: could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -1 (Can't contact LDAP server) ((null)) errno 107 (Transport endpoint is not connected)
[27/Oct/2015:16:37:35 +0000] slapi_ldap_bind - Error: could not perform interactive bind for id [] authentication mechanism [GSSAPI]: error -1 (Can't contact LDAP server)
[27/Oct/2015:16:37:52 +0000] NSMMReplicationPlugin - conn=198 op=5 replica="o=ipaca": Unable to acquire replica: error: permission denied
[27/Oct/2015:16:37:52 +0000] NSMMReplicationPlugin - conn=199 op=5 replica="o=ipaca": Unable to acquire replica: error: permission denied
[27/Oct/2015:16:37:53 +0000] NSMMReplicationPlugin - Beginning total update of replica "agmt="cn=caToreplica1.ipa.test" (replica1:389)".
[27/Oct/2015:16:37:56 +0000] NSMMReplicationPlugin - conn=200 op=5 replica="o=ipaca": Unable to acquire replica: error: permission denied
[27/Oct/2015:16:37:56 +0000] NSMMReplicationPlugin - Finished total update of replica "agmt="cn=caToreplica1.ipa.test" (replica1:389)". Sent 179 entries.
[27/Oct/2015:16:37:57 +0000] ipa-topology-plugin - ipa_topo_util_modify: failed to modify entry (cn=caToreplica1.ipa.test,cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config): error 20
[27/Oct/2015:16:37:59 +0000] NSMMReplicationPlugin - conn=200 op=6 replica="o=ipaca": Unable to acquire replica: error: permission denied
[27/Oct/2015:16:38:05 +0000] NSMMReplicationPlugin - conn=200 op=7 replica="o=ipaca": Unable to acquire replica: error: permission denied
[27/Oct/2015:16:38:17 +0000] NSMMReplicationPlugin - conn=200 op=9 replica="o=ipaca": Unable to acquire replica: error: permission denied
[27/Oct/2015:16:38:23 +0000] NSMMReplicationPlugin - agmt="cn=meToreplica1.ipa.test" (replica1:389): Replication bind with GSSAPI auth resumed
[27/Oct/2015:16:38:41 +0000] NSMMReplicationPlugin - conn=200 op=11 replica="o=ipaca": Unable to acquire replica: error: permission denied
[27/Oct/2015:16:39:28 +0000] NSMMReplicationPlugin - conn=200 op=13 replica="o=ipaca": Unable to acquire replica: error: permission denied
[27/Oct/2015:16:41:06 +0000] NSMMReplicationPlugin - conn=202 op=5 replica="o=ipaca": Unable to acquire replica: error: permission denied
[27/Oct/2015:16:44:18 +0000] NSMMReplicationPlugin - conn=205 op=5 replica="o=ipaca": Unable to acquire replica: error: permission denied

replica:

[27/Oct/2015:16:37:56 +0000] ipa-topology-plugin - ipa_topo_be_state_change - backend ipaca is coming online; checking domain level and init shared topology
[27/Oct/2015:16:37:56 +0000] NSMMReplicationPlugin - multimaster_be_state_change: replica o=ipaca is coming online; enabling replication
[27/Oct/2015:16:37:56 +0000] - Skipping CoS Definition cn=Password Policy,cn=accounts,dc=ipa,dc=test--no CoS Templates found, which should be added before the CoS Definition.
[27/Oct/2015:16:37:56 +0000] NSMMReplicationPlugin - agmt="cn=caTomaster1.ipa.test" (master1:389): Unable to acquire replica: permission denied. The bind dn "" does not have permission to supply replication updates to the replica. Will retry later.
[27/Oct/2015:16:37:59 +0000] NSMMReplicationPlugin - agmt="cn=caTomaster1.ipa.test" (master1:389): Unable to acquire replica: permission denied. The bind dn "" does not have permission to supply replication updates to the replica. Will retry later.
[27/Oct/2015:16:38:05 +0000] NSMMReplicationPlugin - agmt="cn=caTomaster1.ipa.test" (master1:389): Unable to acquire replica: permission denied. The bind dn "" does not have permission to supply replication updates to the replica. Will retry later.
[27/Oct/2015:16:38:12 +0000] ipa-topology-plugin - ipa_topo_util_get_replica_conf: replica already exists
[27/Oct/2015:16:38:12 +0000] ipa-topology-plugin - ipa_topo_util_get_replica_conf: replica already exists
[27/Oct/2015:16:38:16 +0000] ipa-topology-plugin - ipa_topo_util_get_replica_conf: replica already exists
[27/Oct/2015:16:38:16 +0000] ipa-topology-plugin - ipa_topo_util_get_replica_conf: replica already exists
[27/Oct/2015:16:38:17 +0000] NSMMReplicationPlugin - agmt="cn=caTomaster1.ipa.test" (master1:389): Unable to acquire replica: permission denied. The bind dn "" does not have permission to supply replication updates to the replica. Will retry later.
[27/Oct/2015:16:38:42 +0000] NSMMReplicationPlugin - agmt="cn=caTomaster1.ipa.test" (master1:389): Unable to acquire replica: permission denied. The bind dn "" does not have permission to supply replication updates to the replica. Will retry later.
[27/Oct/2015:16:39:29 +0000] NSMMReplicationPlugin - agmt="cn=caTomaster1.ipa.test" (master1:389): Unable to acquire replica: permission denied. The bind dn "" does not have permission to supply replication updates to the replica. Will retry later.
[27/Oct/2015:16:41:06 +0000] NSMMReplicationPlugin - agmt="cn=caTomaster1.ipa.test" (master1:389): Unable to acquire replica: permission denied. The bind dn "" does not have permission to supply replication updates to the replica. Will retry later.
[27/Oct/2015:16:44:18 +0000] NSMMReplicationPlugin - agmt="cn=caTomaster1.ipa.test" (master1:389): Unable to acquire replica: permission denied. The bind dn "" does not have permission to supply replication updates to the replica. Will retry later.

Since the replica waits for the succesful replication of the entry on the master (as can be judged from the master's dirsrv access log below), the installation hangs indefinitely and has to be killed and attempted again:

[27/Oct/2015:16:54:58 +0000] conn=206 op=11 RESULT err=0 tag=101 nentries=0 etime=0 notes=P pr_idx=0
[27/Oct/2015:16:54:59 +0000] conn=201 op=1162 SRCH base="uid=admin-replica1.ipa.test,ou=people,o=ipaca" scope=0 filter="(objectClass=*)" attrs=ALL
[27/Oct/2015:16:54:59 +0000] conn=201 op=1162 RESULT err=32 tag=101 nentries=0 etime=0
[27/Oct/2015:16:55:00 +0000] conn=201 op=1163 SRCH base="uid=admin-replica1.ipa.test,ou=people,o=ipaca" scope=0 filter="(objectClass=*)" attrs=ALL
[27/Oct/2015:16:55:00 +0000] conn=201 op=1163 RESULT err=32 tag=101 nentries=0 etime=0
[27/Oct/2015:16:55:01 +0000] conn=201 op=1165 SRCH base="uid=admin-replica1.ipa.test,ou=people,o=ipaca" scope=0 filter="(objectClass=*)" attrs=ALL
[27/Oct/2015:16:55:01 +0000] conn=201 op=1165 RESULT err=32 tag=101 nentries=0 etime=0
[27/Oct/2015:16:55:02 +0000] conn=201 op=1166 SRCH base="uid=admin-replica1.ipa.test,ou=people,o=ipaca" scope=0 filter="(objectClass=*)" attrs=ALL
[27/Oct/2015:16:55:02 +0000] conn=201 op=1166 RESULT err=32 tag=101 nentries=0 etime=0
[27/Oct/2015:16:55:03 +0000] conn=201 op=1167 SRCH base="uid=admin-replica1.ipa.test,ou=people,o=ipaca" scope=0 filter="(objectClass=*)" attrs=ALL
[27/Oct/2015:16:55:03 +0000] conn=201 op=1167 RESULT err=32 tag=101 nentries=0 etime=0
[27/Oct/2015:16:55:04 +0000] conn=201 op=1168 SRCH base="uid=admin-replica1.ipa.test,ou=people,o=ipaca" scope=0 filter="(objectClass=*)" attrs=ALL
[27/Oct/2015:16:55:04 +0000] conn=201 op=1168 RESULT err=32 tag=101 nentries=0 etime=0
[27/Oct/2015:16:55:05 +0000] conn=201 op=1169 SRCH base="uid=admin-replica1.ipa.test,ou=people,o=ipaca" scope=0 filter="(objectClass=*)" attrs=ALL
[27/Oct/2015:16:55:05 +0000] conn=201 op=1169 RESULT err=32 tag=101 nentries=0 etime=0
[27/Oct/2015:16:55:06 +0000] conn=201 op=1170 SRCH base="uid=admin-replica1.ipa.test,ou=people,o=ipaca" scope=0 filter="(objectClass=*)" attrs=ALL
[27/Oct/2015:16:55:06 +0000] conn=201 op=1170 RESULT err=32 tag=101 nentries=0 etime=0
[27/Oct/2015:16:55:07 +0000] conn=201 op=1172 SRCH base="uid=admin-replica1.ipa.test,ou=people,o=ipaca" scope=0 filter="(objectClass=*)" attrs=ALL
[27/Oct/2015:16:55:07 +0000] conn=201 op=1172 RESULT err=32 tag=101 nentries=0 etime=0
[27/Oct/2015:16:55:08 +0000] conn=201 op=1173 SRCH base="uid=admin-replica1.ipa.test,ou=people,o=ipaca" scope=0 filter="(objectClass=*)" attrs=ALL
[27/Oct/2015:16:55:08 +0000] conn=201 op=1173 RESULT err=32 tag=101 nentries=0 etime=0

mbabinsk commented 8 years ago

This problem occurs in my test environment when raising domain level in existing topology to 1 and the adding replica w/ CA.

pvoborni commented 8 years ago

Not a blocker for 4.3, though it would be nice to fix. Reproducer is needed.

tbordaz commented 8 years ago

The installation script creates an 'admin' entry on the replica

replica

[06/Jan/2016:16:45:38 +0100] - slapd started.  Listening on All Interfaces port 389 for LDAP requests
...
[06/Jan/2016:16:45:48 +0100] conn=7 op=4 DEL dn="uid=admin-vm-replica,ou=people,o=ipaca"
[06/Jan/2016:16:45:48 +0100] conn=7 op=4 RESULT err=32 tag=107 nentries=0 etime=0
[06/Jan/2016:16:45:48 +0100] conn=7 op=5 ADD dn="uid=admin-vm-replica,ou=people,o=ipaca"
[06/Jan/2016:16:45:48 +0100] conn=7 op=5 RESULT err=0 tag=105 nentries=0 etime=0 csn=568d36ad000000060000

Then it indefinitely tested that the replica admin entry was replicated

master

[06/Jan/2016:16:45:50 +0100] conn=130 op=5 SRCH base="uid=admin-vm-replica,ou=people,o=ipaca" scope=0 filter="(objectClass=*)" attrs=ALL
[06/Jan/2016:16:45:50 +0100] conn=130 op=5 RESULT err=32 tag=101 nentries=0 etime=0
...
[06/Jan/2016:17:26:13 +0100] conn=130 op=2813 SRCH base="uid=admin-vm-replica,ou=people,o=ipaca" scope=0 filter="(objectClass=*)" attrs=ALL
[06/Jan/2016:17:26:13 +0100] conn=130 op=2813 RESULT err=32 tag=101 nentries=0 etime=0

In fact it was not replicated because the master did not grant the replication session the right to send updates

master

[06/Jan/2016:18:42:06 +0100] conn=195 op=0 BIND dn="" method=sasl version=3 mech=GSSAPI
[06/Jan/2016:18:42:06 +0100] conn=195 op=0 RESULT err=14 tag=97 nentries=0 etime=0, SASL bind in progress
[06/Jan/2016:18:42:06 +0100] conn=195 op=1 BIND dn="" method=sasl version=3 mech=GSSAPI
[06/Jan/2016:18:42:06 +0100] conn=195 op=1 RESULT err=14 tag=97 nentries=0 etime=0, SASL bind in progress
[06/Jan/2016:18:42:06 +0100] conn=195 op=2 BIND dn="" method=sasl version=3 mech=GSSAPI
[06/Jan/2016:18:42:06 +0100] conn=195 op=2 RESULT err=0 tag=97 nentries=0 etime=0 dn="krbprincipalname=ldap/<VM-replica.domain>@ipadom.org,cn=services,cn=accounts,dc=ipadom,dc=org"
[06/Jan/2016:18:42:06 +0100] conn=195 op=3 SRCH base="" scope=0 filter="(objectClass=*)" attrs="supportedControl supportedExtension"
[06/Jan/2016:18:42:06 +0100] conn=195 op=3 RESULT err=0 tag=101 nentries=1 etime=0
[06/Jan/2016:18:42:06 +0100] conn=195 op=4 SRCH base="" scope=0 filter="(objectClass=*)" attrs="supportedControl supportedExtension"
[06/Jan/2016:18:42:06 +0100] conn=195 op=4 RESULT err=0 tag=101 nentries=1 etime=0
[06/Jan/2016:18:42:06 +0100] conn=195 op=5 EXT oid="2.16.840.1.113730.3.5.12" name="replication-multimaster-extop"
[06/Jan/2016:18:42:06 +0100] conn=195 op=5 RESULT err=0 tag=120 nentries=0 etime=0
[06/Jan/2016:18:43:06 +0100] conn=195 op=6 UNBIND
[06/Jan/2016:18:43:06 +0100] conn=195 op=6 fd=128 closed - U1


[06/Jan/2016:18:42:06 +0100] NSMMReplicationPlugin - conn=195 op=5 replica="o=ipaca": Unable to acquire replica: error: permission denied

This can be fixed with:

After adding the nsDS5ReplicaBindDN in 'replica o=ipaca (like it was in dn: cn=replica,cn=dc\3Dipadom\2Cdc\3Dorg,cn=mapping tree,cn=config)

dn: cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
...
nsDS5ReplicaBindDN: krbprincipalname=ldap/<VM.domain>@REALM
 ,cn=services,cn=accounts,dc=ipadom,dc=org

Replication resumed

replica

[06/Jan/2016:19:07:06 +0100] NSMMReplicationPlugin - agmt="cn=caToVM" : Replication bind with GSSAPI auth resumed

And the replica 'admin' entry was replicated (csn=568d36ad000000060000)
master

[06/Jan/2016:19:07:06 +0100] conn=18 op=0 BIND dn="" method=sasl version=3 mech=GSSAPI
[06/Jan/2016:19:07:06 +0100] conn=18 op=0 RESULT err=14 tag=97 nentries=0 etime=0, SASL bind in progress
[06/Jan/2016:19:07:06 +0100] conn=18 op=1 BIND dn="" method=sasl version=3 mech=GSSAPI
[06/Jan/2016:19:07:06 +0100] conn=18 op=1 RESULT err=14 tag=97 nentries=0 etime=0, SASL bind in progress
[06/Jan/2016:19:07:06 +0100] conn=18 op=2 BIND dn="" method=sasl version=3 mech=GSSAPI
[06/Jan/2016:19:07:06 +0100] conn=18 op=2 RESULT err=0 tag=97 nentries=0 etime=0 dn="krbprincipalname=ldap/<VM-replica.domain>@ipadom.org,cn=services,cn=accounts,dc=ipadom,dc=org"
[06/Jan/2016:19:07:06 +0100] conn=18 op=3 SRCH base="" scope=0 filter="(objectClass=*)" attrs="supportedControl supportedExtension"
[06/Jan/2016:19:07:06 +0100] conn=18 op=3 RESULT err=0 tag=101 nentries=1 etime=0
[06/Jan/2016:19:07:06 +0100] conn=18 op=4 SRCH base="" scope=0 filter="(objectClass=*)" attrs="supportedControl supportedExtension"
[06/Jan/2016:19:07:06 +0100] conn=18 op=4 RESULT err=0 tag=101 nentries=1 etime=0
[06/Jan/2016:19:07:06 +0100] conn=18 op=5 EXT oid="2.16.840.1.113730.3.5.12" name="replication-multimaster-extop"
[06/Jan/2016:19:07:06 +0100] conn=18 op=5 RESULT err=0 tag=120 nentries=0 etime=0
[06/Jan/2016:19:07:06 +0100] conn=18 op=6 SRCH base="cn=schema" scope=0 filter="(objectClass=*)" attrs="objectClasses"
[06/Jan/2016:19:07:06 +0100] conn=18 op=6 RESULT err=0 tag=101 nentries=1 etime=0
[06/Jan/2016:19:07:06 +0100] conn=18 op=7 SRCH base="cn=schema" scope=0 filter="(objectClass=*)" attrs="attributeTypes"
[06/Jan/2016:19:07:06 +0100] conn=18 op=7 RESULT err=0 tag=101 nentries=1 etime=0
[06/Jan/2016:19:07:06 +0100] conn=18 op=8 SRCH base="cn=schema" scope=0 filter="(objectClass=*)" attrs="nsSchemaCSN"
[06/Jan/2016:19:07:06 +0100] conn=18 op=8 RESULT err=0 tag=101 nentries=1 etime=0
[06/Jan/2016:19:07:06 +0100] conn=18 op=9 SRCH base="cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config" scope=0 filter="(objectClass=*)" attrs="nsDS5ReplicaId"
[06/Jan/2016:19:07:06 +0100] conn=18 op=9 RESULT err=0 tag=101 nentries=1 etime=0
[06/Jan/2016:19:07:06 +0100] conn=18 op=10 ADD dn="uid=admin-VM-replica.domain,ou=people,o=ipaca"
[06/Jan/2016:19:07:06 +0100] conn=18 op=10 RESULT err=0 tag=105 nentries=0 etime=0 csn=568d36ad000000060000
[06/Jan/2016:19:07:06 +0100] conn=18 op=11 MOD dn="cn=Enterprise CA Administrators,ou=groups,o=ipaca"
[06/Jan/2016:19:07:06 +0100] conn=18 op=11 RESULT err=0 tag=103 nentries=0 etime=0 csn=568d36ad000100060000
[06/Jan/2016:19:07:06 +0100] conn=18 op=12 MOD dn="cn=Enterprise KRA Administrators,ou=groups,o=ipaca"
[06/Jan/2016:19:07:06 +0100] conn=18 op=12 RESULT err=0 tag=103 nentries=0 etime=0 csn=568d36ad000200060000
[06/Jan/2016:19:07:06 +0100] conn=18 op=13 MOD dn="cn=Security Domain Administrators,ou=groups,o=ipaca"
[06/Jan/2016:19:07:06 +0100] conn=18 op=13 RESULT err=0 tag=103 nentries=0 etime=0 csn=568d36ad000300060000
[06/Jan/2016:19:07:07 +0100] conn=18 op=14 EXT oid="2.16.840.1.113730.3.5.5" name="Netscape Replication End Session"
[06/Jan/2016:19:07:07 +0100] conn=18 op=14 RESULT err=0 tag=120 nentries=0 etime=0

Next steps:

- Check if  'nsDS5ReplicaBindDN' (on o=ipaca replica) was missing during the install script and if it is the expected fix.

tbordaz commented 8 years ago

The reason why replication 'replica->master' was failing is that the bindgroup was not set on 'o=ipaca' (on the master server). Setting 'nsDS5ReplicaBindDN' works but is a workaround.

On master

ldapsearch -LLL -D "cn=directory manager" -w xxx -b "cn=config" 'cn=replica' nsds5replicabinddngroup nsds5replicabinddngroupcheckinterval nsDS5ReplicaBindDN
dn: cn=replica,cn=dc\3Dipadom\2Cdc\3Dorg,cn=mapping tree,cn=config
nsds5replicabinddngroup: cn=replication managers,cn=sysaccounts,cn=etc,dc=ipad
 om,dc=org
nsds5replicabinddngroupcheckinterval: 60
nsDS5ReplicaBindDN: cn=replication manager,cn=config
nsDS5ReplicaBindDN: krbprincipalname=ldap/<VM-replica-fqdn>@IPADOM,
 cn=services,cn=accounts,dc=ipadom,dc=org

dn: cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
nsDS5ReplicaBindDN: cn=Replication Manager masterAgreement1-<vm-replica-fqdn>-pki-tomcat,ou=csusers,cn=config
nsDS5ReplicaBindDN: cn=replication manager,cn=config
nsDS5ReplicaBindDN: krbprincipalname=ldap/<VM-replica-fqdn>@IPADOM,  <--- (1) this is a value I added 
 ,cn=services,cn=accounts,dc=ipadom,dc=org

ldapsearch -LLL -D "cn=directory manager" -w xxx -b "cn=replication managers,cn=sysaccounts,cn=etc,dc=ipadom,dc=org" member
dn: cn=replication managers,cn=sysaccounts,cn=etc,dc=ipadom,dc=org
member: krbprincipalname=ldap/<VM-master-fqdn>@IPADOM
 .ORG,cn=services,cn=accounts,dc=ipadom,dc=org
member: krbprincipalname=ldap/<VM-replica-fqdn>@IPADOM
 .ORG,cn=services,cn=accounts,dc=ipadom,dc=org

On replica

ldapsearch -LLL -D "cn=directory manager" -w xxx -b "cn=config" 'cn=replica' nsds5replicabinddngroup nsds5replicabinddngroupcheckinterval nsDS5ReplicaBindDN
dn: cn=replica,cn=dc\3Dipadom\2Cdc\3Dorg,cn=mapping tree,cn=config
nsds5replicabinddngroup: cn=replication managers,cn=sysaccounts,cn=etc,dc=ipad
 om,dc=org
nsds5replicabinddngroupcheckinterval: 60
nsDS5ReplicaBindDN: cn=replication manager,cn=config
nsDS5ReplicaBindDN: krbprincipalname=ldap/<VM-replica-fqdn>@IPADOM,
 .ORG,cn=services,cn=accounts,dc=ipadom,dc=org

dn: cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
nsDS5ReplicaBindDN: cn=Replication Manager masterAgreement1-<vm-replica-fqdn>-pki-tomcat,ou=csusers,cn=config
nsDS5ReplicaBindDN: cn=replication manager,cn=config
nsDS5ReplicaBindDN: krbprincipalname=ldap/<VM-master-fqdn>@IPADOM   <-- (2) this value was already set
 ORG,cn=services,cn=accounts,dc=ipadom,dc=org

ldapsearch -LLL -D "cn=directory manager" -w dmpasswd -b "cn=replication managers,cn=sysaccounts,cn=etc,dc=ipadom,dc=org" member
dn: cn=replication managers,cn=sysaccounts,cn=etc,dc=ipadom,dc=org
member: krbprincipalname=ldap/<VM-master-fqdn>@IPADOM
 .ORG,cn=services,cn=accounts,dc=ipadom,dc=org
member: krbprincipalname=ldap/<VM-replica-fqdn>@IPADOM
 .ORG,cn=services,cn=accounts,dc=ipadom,dc=org

In conclusion:
I think the problem is that, for o=ipaca, on master AND replica the 'nds5replicabinddngroup:' was not
set. It worked master->replica because on the replica the 'nsDS5ReplicaBindDN' was set (workaround) (2).
It failed replica->master because on master the 'nsDS5ReplicaBindDN' was not set (1).

pvoborni commented 8 years ago

It should be set by:

dn: cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
onlyifexist: nsds5replicabinddngroup: cn=replication managers,cn=sysaccounts,cn=etc,$SUFFIX

in ca-topology.uldif which should be run from update_ca_topology update plugin.

Next step: investigate if there is some bug in this update process.

mbabinsk commented 8 years ago

The most straightforward way to reproduce this issue is this:

Install a domain-level 0 master using {{{--domain-level 0}}} option
prepare a replica file and install a domain level 0 replica w/ CA
raise domain level of the topology to 1: {{{ipa domainlevel-set 1}}}
install a client and promote it to replica with CA, or first make a CA-less replica and then run {{{ipa-ca-install}}}

Expected result:

We have a promoted replica with CA

Actual result:

We have a CA installation stuck at {{{ [4/24]: creating installation admin user}}}

mbasti commented 8 years ago

master:

f2b22ec correctly set LDAP bind related attributes when setting up replication

ipa-4-3:

7c8683d correctly set LDAP bind related attributes when setting up replication

jbaird commented 7 years ago

I appear to be hitting this (or at least something similar) on RHEL 7.3/IPA 4.4. My replica install is getting stuck indefinitely on 'creating installation admin user.'

pvoborni commented 7 years ago

I've seen that somebody else also encountered this/similar issue. I don't think this bug is the place to debug/discuss it. Because if there is a bug then a new one will be created.

To investigate root cause of it, it would be better to open a thread on freeipa-users, there it will reach broader audience.

Following logs from the time where the installations failed will be needed:

dirsrv access (from both master and replica)
dirsrv error (from both master and replica)
ipareplica-install.log

jbaird commented 7 years ago

I have opened a GSS case (01741580) with the data that you requested.

Metadata Update from @mbabinsk:
- Issue assigned to mbabinsk
- Issue set to the milestone: FreeIPA 4.3.1

7 years ago

Metadata

Assignee

mbabinsk

Tags

None

Blocking

None

Depending on

None

Priority

normal

Milestone

FreeIPA 4.3.1

None

affects_doc

None

source

None

knownissue

None

type

defect

blockedby

None

test_case

None

component

Replication

blocking

None

on_review

keywords

None

test_coverage

None

reviewer

mbasti

external_tracker

None

rhbz

todo

tester

None

changelog

None

design

None

freeipa

Source Code

#5412 ipa-ca-install on promoted replica hangs on creating a temporary CA admin Closed: Fixed None Opened 8 years ago by mbabinsk.

Metadata

#5412 ipa-ca-install on promoted replica hangs on creating a temporary CA admin

Closed: Fixed None Opened 8 years ago by mbabinsk.