Ticket was cloned from Red Hat Bugzilla (product Red Hat Enterprise Linux 7): Bug 1075118
Created attachment 873161 ipa replica install log file Description of problem: This i faced when i tried to create a replica from a Replica (RHEL-7.0) which was migrated from a RHEL-6.5 based Master. Version-Release number of selected component (if applicable): [root@rhel70-replica ~]# rpm -q ipa-server pki-ca 389-ds-base ipa-server-3.3.3-21.el7.x86_64 pki-ca-10.0.5-3.el7.noarch 389-ds-base-1.3.1.6-22.el7.x86_64 [root@rhel70-replica ~]# How reproducible: Always Steps to Reproduce: 1. Install Master on RHEL-6.5 2. Install Replica on RHEL-7.0 from RHEL-6.5 based Master. 3. Now again install another replica on RHEL-7.0 from replica of step (2). Actual results: Replica install fails Expected results: Replica install should be successful Additional info: Please find the attached ipareplica-install.log from replica machine where replica installation is failing
I have been investigating this issue when testing RHEL and have an idea how to fix - I can take it.
But we will definitely need to fix it in 3.3.x branch.
attachment freeipa-mkosek-461-update-dogtag-9-database-during-replica-installation.patch
Patch freeipa-mkosek-461-update-dogtag-9-database-during-replica-installation.patch sent for review
Starting review
master:
ipa-3-3:
The fix almost works perfectly but I am reopening because it misses a critical part. The script must restart the pki-tomcat instance or the changes will not have effect. I just spent hours trying to create a clone and it was always failing wth authorization errors.
As soon as I restarted pki-tomcatd on the master, I was able to successfully create a CA replica without any issue.
The patch does restart the pki-tomcat on the installed replica. Not sure what can we do about master it replicates from, we do not have control over it in ipa-replica-install/ipa-ca-install.
ipa-replica-install
ipa-ca-install
AFAIR this update is performed also at RPM upgrade time, it's at that time that pki-tomcatd need to be restarted. It is probably a good idea to always restart the CA after an rpm update of freeipa in case something changes anyway.
Ah, I think I finally understood your scenario:
Correct?
If this is correct, I am not convinced this is something we need to fix as only very limited number of deployments is affected, all new migrations do restart the PKI so we are covered there. Also, workaround is very easy.
A potential fix would not be easy as we cannot restart PKI based on *.update file contents, we would need to turn this into update plugin.
*.update
Normally, we restart PKI when any change is done, but we mostly do such changes only in ipa-upgradeconfig, you can check in:
ipa-upgradeconfig
grep ca_restart install/tools/ipa-upgradeconfig
We had another discussion with simo outside of this ticket.
Something I still wondering about is why the IPA services did not restart on the server upgrade?
We have following snippet in our spec file:
%posttrans server # This must be run in posttrans so that updates from previous # execution that may no longer be shipped are not applied. /usr/sbin/ipa-ldap-updater --upgrade --quiet >/dev/null || : /usr/sbin/ipa-upgradeconfig --quiet >/dev/null || : # Restart IPA processes. This must be also run in postrans so that plugins # and software is in consistent state python2 -c "import sys; from ipaserver.install import installutils; sys.exit(0 if installutils. is_ipa_configured() else 1);" > /dev/null 2>&1 # NOTE: systemd specific section if [ $? -eq 0 ]; then /bin/systemctl try-restart ipa.service >/dev/null 2>&1 || : fi # END
Simo, does it mean that the Python call does evaluate as "1" in your case so that IPA services are not restarted?
Replying to [comment:11 mkosek]:
We had another discussion with simo outside of this ticket. Something I still wondering about is why the IPA services did not restart on the server upgrade? We have following snippet in our spec file: {{{ %posttrans server This must be run in posttrans so that updates from previous execution that may no longer be shipped are not applied. /usr/sbin/ipa-ldap-updater --upgrade --quiet >/dev/null || : /usr/sbin/ipa-upgradeconfig --quiet >/dev/null || : Restart IPA processes. This must be also run in postrans so that plugins and software is in consistent state python2 -c "import sys; from ipaserver.install import installutils; sys.exit(0 if installutils. is_ipa_configured() else 1);" > /dev/null 2>&1 NOTE: systemd specific section if [ $? -eq 0 ]; then /bin/systemctl try-restart ipa.service >/dev/null 2>&1 || : fi END }}} Simo, does it mean that the Python call does evaluate as "1" in your case so that IPA services are not restarted?
{{{ %posttrans server
/usr/sbin/ipa-ldap-updater --upgrade --quiet >/dev/null || : /usr/sbin/ipa-upgradeconfig --quiet >/dev/null || :
python2 -c "import sys; from ipaserver.install import installutils; sys.exit(0 if installutils. is_ipa_configured() else 1);" > /dev/null 2>&1
if [ $? -eq 0 ]; then /bin/systemctl try-restart ipa.service >/dev/null 2>&1 || : fi
}}}
I've just run the command manually and it returns 0 on my system.
So it should have restarted it ... but apparently didn't
And I just ry systemctl try-restart ipa.service on my machine and it did nothing.
I would try a more robust approach, can we check if ipa.service is enabled and in that case forcibly run systemctl stop ipa.service systemctl start ipa.service
My system was saying that the ipa service is dead which caused try-restart to fail and do nothing :-(
Would you rather like to change it to "systemctl restart ipa.service" then? This would try to restart it whether it is running or not. Checking if IPA server is enabled is indeed a good idea:
# systemctl is-enabled ipa.service ; echo $? enabled 0
Speaking of the failure in your case, it seems that systemd thought that IPA service is not running. I wonder why. ipa service is a oneshot type of systemd service so I think systemd assumes whether ipa is running or not based on ipactl start result.
It makes me wonder why systemd thought the service was dead. I think that try-restart is the "correct" way to do it and doing anything else is just trying to work around systemd bugs (or bugs in the way we report status).
This was discussed on the meeting and we will workaround systemd bug for the short term in 4.0. For the long term, we would need refactoring of our systemd unit files.
This is a quick fix we should do in 4.0, thus moving to month milestone.
We should also consider backporting to ipa-3-3 as this blocks #4213.
Metadata Update from @dpal: - Issue assigned to mbasti - Issue set to the milestone: FreeIPA 4.0 - 2014/06
Login to comment on this ticket.