I'm cleaning up old testvm's/clusters where I'm checking out my bash_rc files and came across the fact that I had a cluster where it seems the CA was replicated because this was unclear in the docs back the days.
So what I did was testing this cluster out for some "recovery" as I'm quite sure lots of people might run into this issue they even don't know about they can/will have.
The situation was as follow:
ipa1: installed and created the ipa server ipa2: installed as replica prepared from ipa1
ipa-replica-install --setup-dns --no-forwarders /var/lib/ipa/replica-info-ipareplica.example.com.gpg
Now above is removed --setup-ca because this is confusing in the docs. The docs say:
Additionally, there is an option to configure a CA for the replica; while CA's are installed by default == for == servers, they are optional == for == replicas.
OK, this doesn't sound you are going to replicate the CA from the master to the replica where I guess lots of people can/will have ignored that.
As this setup is only used for ldap auth and only has 2 kinit hosts, CA is not really my issue as I only do ldap auth and use kinit to add users from my hosts.
So in my test I took down ipa1 (CA) said to myself it was dead, and removed it from ipa2. Now I wanted to prepare a new replica from ipa2 which doesn't work as there is no CA:
ipa-replica-prepare --ip-address xxx.xxx.xxx.xxx ipa3.domain.local Directory Manager (existing master) password:
Cannot issue certificates: a CA is not installed. Use the --http-cert-file, --dirsrv-cert-file options to provide custom certificates.
So now you are actually doomed, your ldap still works OK on ipa2 but you are unable to add a CA for this server.
My idea would be simple thought: Install an ipa server the normal way, so with CA, make all names the same as the old ipa1.
Now do a prepare for this ipa1 as replica, but as it's CA, ipa2 will connect to ipa1 as it's already a CA. Of course you cannot install the replica on IPA1 as it's already a server, prepare might fail too?
So the actuall problem is:
How can we create a full cluster again, with the exising LDAP data as this is most important.
Isn't backup and restore sctripts and procedures are already covering this case?
Replying to [comment:1 dpal]:
It doesn't seem so, at least not when having no CA (anymore)
Replying to [comment:2 yamakasy]:
Replying to [comment:1 dpal]: Isn't backup and restore sctripts and procedures are already covering this case? It doesn't seem so, at least not when having no CA (anymore)
Correct, but if you used backup before you would be able to recover using the restore procedures. This is why we developed them.
The procedure you are suggesting might work but it seems to be a corner case since there are already procedures to avoid this situation. We should probably improve documentation to remove ambiguity and make it clear that having replicas with CAs is important.
Replying to [comment:3 dpal]:
Replying to [comment:2 yamakasy]: Replying to [comment:1 dpal]: Isn't backup and restore sctripts and procedures are already covering this case? It doesn't seem so, at least not when having no CA (anymore) Correct, but if you used backup before you would be able to recover using the restore procedures. This is why we developed them. The procedure you are suggesting might work but it seems to be a corner case since there are already procedures to avoid this situation. We should probably improve documentation to remove ambiguity and make it clear that having replicas with CAs is important.
Hi yes this is needed and I'm willing to commit that but first I want to see what the possibilties are when people have the issue "after" it broke.
What are the options for backups ?
Let's say someone has an old backup from ipa1 (with CA) but all data is up-to-date in ipa2(ldap)
The reason why I ask is because you never know what happens, and I would like to be able as other people might want to be able to also.
As the data for CA and Ldap is seperated, this might be some workaround. Data is very important, I think the CA is something we can overcome.
Any idea about that ?
I have tried the solution from here:
http://www.freeipa.org/page/V3/Backup_and_Restore#Implementation
Before bak2db:
Directory Manager password:
ipa-02.domain.local: master
After bak2db:
ipa-02.domain.local: CA not configured
I wonder why that happens but I think it comes from the config the db2bak seems to backup too.
Ideas ?
This ticket has not been triaged. Please create new tickets in NEEDS_TRIAGE milestone to get developers attention.
The scenario was revisited during triage. It will be revisited in 4.5 planning mainly from perspective of prevention the single point of failure.
If master CA server is lost without any backup of certificate private keys then every certificate needs to be recreated.
Several related tickets were opened: #5828 #5829 #5830
WRT recovery procedure, here is an outline (possibly incomplete) of the steps that would be required. Might be a useful starting point for developing a recovery tool, if we decide to do that:
I have not performed this recovery so there the steps are only given at a high level; no actual commands, etc.
Make changes to cause FreeIPA to think it is CA-less.
Extract CA signing key from a replica info file.
Run ipa-ca-install to install the CA on one of the IPA servers, with external CA. This will generate a new private key and CSR to send to external CA.
Replace the new private key generated for the CSR, with the private key from the replica info file.
Continue the ipa-ca-install with the CA signing certificate from the replica info file.
Manually adjust serial number ranges to ensure the new CA instance does not issue certs with serial numbers that collide with certs issued by the original CA instance. (This might have to be hacked into the ipa-ca-install process).
Depending on whether your CA is self-signed, might need to tell certmonger to track the CA signing certificate.
Side-note: AFAICT, with Replica Promoting the recovery situation gets worse because there is no replica file containing the CA signing key.
Metadata Update from @yamakasy: - Issue assigned to someone - Issue set to the milestone: FreeIPA 4.5 backlog
Going to mark this as a duplicate as the work is tracked in the tickets from comment https://pagure.io/freeipa/issue/5100#comment-319958
Metadata Update from @rcritten: - Issue close_status updated to: duplicate - Issue status updated to: Closed (was: Open)
Login to comment on this ticket.