#3347 [RFE] idnssoaserial needs to be managed by a DS plugin
Closed: Fixed None Opened 11 years ago by simo.

In order to properly manage DNS SOA serials we filter replication of this attribute, however the attribute needs to be created on the target replica as it is a required attribute for a Zone.

We should manage this in a ds plugin that monitors replication of new zones.
At the same time we could manage auto-serial-increment in this plugin as well so that we do not have to rely on bind-dynd-ldap to keep the attribute updated at all times.

This is important because not all IPA servers are DNS-enabled so bind-dyndb-ldap is not always around to 'fix' it for us.
A plugin would avoid any delay in getting the attribute 'fixed' as soon as a new zone is replicated.


It will need thorough thinking...

  • BIND does direct SOA serial update (not trigerred by serial autoincrement feature) after any dynamic update. We have to catch those attempts and ignore them.

  • Any serial update have to be announced to BIND - persistent search is inevitable.

  • The timing of serial update have to be considered carefully. SOA serial have to not change during zone transfer.

  • How it would affect large searches? Would be result of subtree search for cn=dns consistent? Could serial change in half of result generation process?

  • Some way to map serial<->entryUSN should be on horizon - for incremental transfer support.

Replying to [comment:1 pspacek]:

It will need thorough thinking...

Yes

  • BIND does direct SOA serial update (not trigerred by serial autoincrement feature) after any dynamic update. We have to catch those attempts and ignore them.

A plugin can intercept any modify and manipulate it, including suppressing changes to SOA Serial.

  • Any serial update have to be announced to BIND - persistent search is inevitable.

Well isn't psearch already necessary for the serial autoincrment option ?

  • The timing of serial update have to be considered carefully. SOA serial have to not change during zone transfer.

Can't this be enforced by the bind-dyndb-plugin by searching all objects upfront and then serving only from it's cache until the AXFR is completed ?

  • How it would affect large searches? Would be result of subtree search for cn=dns consistent? Could serial change in half of result generation process?

If you do one search you'll get a snapshot of what is the DS at the time you did the search. If you do multilpe searches then yes you may have data changing between searches.

  • Some way to map serial<->entryUSN should be on horizon - for incremental transfer support.

I am not entirely sure you can use entry USN to do incremental transfers. Or at least you will still have to understand that some entries may ne 'newer' than the SOA serial you used.
Assuming old MAX_ENTRY_USN is A (point in time A) and new MAX_ENTRY_USN is B (point in time B) at the time you want to do an incremental transfer, you can't search for (&(entryUSN>A)(entryUSN<B)) because you would miss any object that has entryUSN > B, and that object my also have change between A and B and simply changed again right after point in time B.
So you need to search only for (entryUSN > A) but then you'll have entries that you'll try to transfer again in the next incremental transfer when you search for (entryUSN > B).
I am not sure how to properly ahndle this.
The only way I can see is to use the psearch to build a sort of changelog, just like DS replication does.
Or maybe we can use the DS changelog iteslf to handle incremental replication and not use the psearch.
The changelog will contain all changes even those that have been later replaced, so you would be eable to search all and only the changes between 2 points in time (from the pov of the speciifc IPA replica). However you'd have to interpret change operations not just entries, which may make it a bit more difficult to parse. Also access may be limited to very privileged users which makes it not ideal.

A third option would be to have the DS plugin read the changelog anf give you a special extended operation you can call from bind-dyndb-ldap to get all changes between time A and B for incremental replication.

Replying to [comment:2 simo]:

Replying to [comment:1 pspacek]:

  • The timing of serial update have to be considered carefully. SOA serial have to not change during zone transfer.

Can't this be enforced by the bind-dyndb-plugin by searching all objects upfront and then serving only from it's cache until the AXFR is completed ?

  • How it would affect large searches? Would be result of subtree search for cn=dns consistent? Could serial change in half of result generation process?
    If you do one search you'll get a snapshot of what is the DS at the time you did the search. If you do multilpe searches then yes you may have data changing between searches.
    In that case AXFR should not be a problem. We do LDAP search for all entries in the zone and then send complete result to the client.

  • Some way to map serial<->entryUSN should be on horizon - for incremental transfer support.

I am not entirely sure you can use entry USN to do incremental transfers. Or at least you will still have to understand that some entries may ne 'newer' than the SOA serial you used.
Assuming old MAX_ENTRY_USN is A (point in time A) and new MAX_ENTRY_USN is B (point in time B) at the time you want to do an incremental transfer, you can't search for (&(entryUSN>A)(entryUSN<B)) because you would miss any object that has entryUSN > B, and that object my also have change between A and B and simply changed again right after point in time B.
So you need to search only for (entryUSN > A) but then you'll have entries that you'll try to transfer again in the next incremental transfer when you search for (entryUSN > B).
I am not sure how to properly ahndle this.
The only way I can see is to use the psearch to build a sort of changelog, just like DS replication does.
Or maybe we can use the DS changelog iteslf to handle incremental replication and not use the psearch.
The changelog will contain all changes even those that have been later replaced, so you would be eable to search all and only the changes between 2 points in time (from the pov of the speciifc IPA replica). However you'd have to interpret change operations not just entries, which may make it a bit more difficult to parse. Also access may be limited to very privileged users which makes it not ideal.

A third option would be to have the DS plugin read the changelog anf give you a special extended operation you can call from bind-dyndb-ldap to get all changes between time A and B for incremental replication.

As you said, there are possible ways how to do it. I would defer this effort because nobody requested real IXFR support up to now (bind-dyndb-ldap ticket #64).

Moving back to Triage, this ticket is needed to fix #3458 (March milestone)

master:[[BR]]
b5b040e Configure ipa_dns DS plugin on install and upgrade[[BR]]
952a7ac Add 389 DS plugin for special idnsSOASerial attribute handling[[BR]]

ipa-3-1:[[BR]]
9cb2c75 Configure ipa_dns DS plugin on install and upgrade[[BR]]
6f395d9 Add 389 DS plugin for special idnsSOASerial attribute handling[[BR]]

This change is a bugfix - the older solution simply didn't work well, so the re-work was un-evittable. Complex bugfixes/changes also require design page.

Metadata Update from @simo:
- Issue assigned to pspacek
- Issue set to the milestone: FreeIPA 3.2 - 2013/03

7 years ago

Login to comment on this ticket.

Metadata