For example, running syncrepl with named, gives these messages in the errors log:
Internal search - Unindexed search: source (cn=server,cn=plugins,cn=config) search base="cn=changelog" filter="(&(changenumber>=42)(targetuniqueid=xxxxxxx))" etime=0 nentries=1 notes=U
Not sure which one is unindexed - could be changenumber is not indexed for range/ordering? could be targetuniqueid is not indexed for equality?
I would welcome this fix backported also to F20 Directory Server as FreeIPA DS log was swarmed with these error messages.
the problem is the missing equality index for targetuniqueid. a fix could add it to the default indexes for the changelog, but for existing deployments a reindexing would be required. Couldn't IPA if it enables the sync repl plugin also add the index and reindexthe changelog ?
attachment 0001-Ticket-47816-internal-syncrepl-searches-are-flagged-.patch
Do we also need to add the index to existing backends?
Replying to [comment:6 rmeggins]:
I see two problems with this. First: the current update scripts will just process the *ldifs in the .../updates directory, which add or modify entries in the dse.ldif whose DN is known. For adding and index to an existing backend this would require to query the dse.ldif to find the backends and then add it. I don't see how this can be done in the updateDS code. Second: Even if we succeed it would require a reindex, and if we can trigger it it would start to reindex all backends, with the result that the attribute didn't exist. If we do not trigger a reindex, I think the index will be used after a restart and newly created entries will be indexed and existing ones not.
Replying to [comment:7 lkrispen]:
Replying to [comment:6 rmeggins]: Do we also need to add the index to existing backends? I see two problems with this. First: the current update scripts will just process the *ldifs in the .../updates directory, which add or modify entries in the dse.ldif whose DN is known. For adding and index to an existing backend this would require to query the dse.ldif to find the backends and then add it. I don't see how this can be done in the updateDS code.
I see two problems with this. First: the current update scripts will just process the *ldifs in the .../updates directory, which add or modify entries in the dse.ldif whose DN is known. For adding and index to an existing backend this would require to query the dse.ldif to find the backends and then add it. I don't see how this can be done in the updateDS code.
Right. You can't do it with a .ldif file. You can do it with a .pl file. See 90subtreereindex.pl and others in ldap/admin/src/scripts
Second: Even if we succeed it would require a reindex, and if we can trigger it it would start to reindex all backends, with the result that the attribute didn't exist.
We do similar things during upgrade e.g. see 90subtreereindex.pl. I don't know if this is something that we will want to do automatically since it affects relatively few users.
If we do not trigger a reindex, I think the index will be used after a restart
A yum update/rpm update will restart the server after the updates are applied.
and newly created entries will be indexed and existing ones not.
If that is going to cause problems, then either we need to add the index and do the reindex during upgrade, or we just add the default index and tell people what they need to do to add the index to all of their existing backend instances and reindex.
Adding an index without reindexing makes searches fail. Here is an example:
Without index: [21/Aug/2014:10:34:24 +0200] conn=20 op=1 SRCH base="dc=example,dc=com" scope=2 filter="(homePhone=999-697-8966)" attrs=ALL [21/Aug/2014:10:34:24 +0200] conn=20 op=1 RESULT err=0 tag=101 '''nentries=1''' etime=0 '''notes=U'''
Adding an index for homephone, without reindexing: 21/Aug/2014:10:37:18 +0200] conn=1 op=1 SRCH base="dc=example,dc=com" scope=2 filter="(homePhone=999-697-8966)" attrs=ALL [21/Aug/2014:10:37:18 +0200] conn=1 op=1 RESULT err=0 tag=101 '''nentries=0''' etime=0
Reindexing: [21/Aug/2014:10:38:51 +0200] conn=1 op=1 SRCH base="dc=example,dc=com" scope=2 filter="(homePhone=999-697-8966)" attrs=ALL [21/Aug/2014:10:38:51 +0200] conn=1 op=1 RESULT err=0 tag=101 '''nentries=1''' etime=0
So just adding an index will result in unexpected behaviour. As this attribute is used only in the retro changelog it will not be acceptable to reindex all datbases as this can take very long.
Should we add a special script to do this only for the retro changelog backend ?
Replying to [comment:9 lkrispen]:
So just adding an index will result in unexpected behaviour. As this attribute is used only in the retro changelog it will not be acceptable to reindex all datbases as this can take very long. Should we add a special script to do this only for the retro changelog backend ?
Yes. And we should be able to do db2index -n changelogdb so that we only reindex the retro changelog. This should only affect people who are using the retrochangelog, and it should not take a long time.
ok, makes sense and avoids adding stuff to release notes, where they could be missed. It could even be db2index -n changelogdb -t targetuniqueid.
attachment 0001-Ticket-47816-v2-internal-syncrepl-searches-are-flagg.patch
]$ git merge ticket47816 Updating 50c05b4..0a69c4d Fast-forward Makefile.am | 2 ++ Makefile.in | 2 ++ ldap/admin/src/scripts/50targetuniqueid.ldif | 7 +++++++ ldap/admin/src/scripts/82targetuniqueidindex.pl | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++++ ldap/ldif/template-dse.ldif.in | 7 +++++++ 5 files changed, 70 insertions(+) create mode 100644 ldap/admin/src/scripts/50targetuniqueid.ldif create mode 100644 ldap/admin/src/scripts/82targetuniqueidindex.pl
$ git push origin master Counting objects: 21, done. Delta compression using up to 4 threads. Compressing objects: 100% (11/11), done. Writing objects: 100% (12/12), 1.97 KiB, done. Total 12 (delta 8), reused 0 (delta 0) To ssh://git.fedorahosted.org/git/389/ds.git 50c05b4..0a69c4d master -> master
committed to 1.3.2 New commits: commit abc947a
Metadata Update from @lkrispen: - Issue assigned to lkrispen - Issue set to the milestone: 1.3.3 - 8/14 (August)
389-ds-base is moving from Pagure to Github. This means that new issues and pull requests will be accepted only in 389-ds-base's github repository.
This issue has been cloned to Github and is available here: - https://github.com/389ds/389-ds-base/issues/1147
If you want to receive further updates on the issue, please navigate to the github issue and click on subscribe button.
subscribe
Thank you for understanding. We apologize for all inconvenience.
Metadata Update from @spichugi: - Issue close_status updated to: wontfix (was: Fixed)
Login to comment on this ticket.