#47358 backend performance - introduce optimization levels
Closed: wontfix None Opened 10 years ago by lkrispen.

modify performance is highly impacted by the backend lock and entry cache lock contention.
A couple of tickets are related to this and suggest changes to improve performance, but

  • it is not guranteed that each optimization id good for all deployments
  • it is a sensitive area and there is a possibility of unexpected side effects

To move forward I suggest to implement some backend related optimizations, but allow to turn them on or off, so it is possible to keep the current behaviour and deploy the optimizations after testing in a specific deployment.

For the beginning, I think the following optimizations should be considered:

  • do not update the database ruv in the modify transaction, let it be done by the calls to _replica_update_state in the event loop. This is handled specifically by#564, but it is not finally commited and some environments may want to keep the ruv update in the modify txn

  • reverse order of txn_begin/commit and backend_lock/unlock:

with the fix for #568 it is save to use batching of transaction log flush, but if the transaction is nested inside the backend lock, always only one transaction will be active, so the effect is not visible

  • move code out of the backend lock:

in a modify operation there is a lot of initial work done after taking the backend lock, but not everything requires this protection as no database access is involved, eg
-- check for illegal mods
-- check access to the entry
-- make copies of the mods to restore in a retry
-- make copies of the entry for use in plugins
some of the above depend on the enty already be found and locked in the entry cache (the entrycache itself is addressed in performance tickets #414,#614).
There is no obvious reason that the find_entry2modify cannot be done before taking the backend lock, but this would request thorough investigation and testing. For testing it could help to have a switch to turn this behaviour on


forgot the commit, thanks Thierry.

$git merge ticket47358
Updating 2b5aecb..a154ecf
Fast-forward
ldap/servers/slapd/back-ldbm/back-ldbm.h | 9 +++++++++
ldap/servers/slapd/back-ldbm/dblayer.c | 47 +++++++++++++++++++++++++++++++++++------------
ldap/servers/slapd/back-ldbm/ldbm_add.c | 2 +-
ldap/servers/slapd/back-ldbm/ldbm_config.c | 22 ++++++++++++++++++++++
ldap/servers/slapd/back-ldbm/ldbm_config.h | 1 +
ldap/servers/slapd/back-ldbm/ldbm_delete.c | 2 +-
ldap/servers/slapd/back-ldbm/ldbm_modify.c | 20 +++++++++++++++-----
ldap/servers/slapd/back-ldbm/ldbm_modrdn.c | 2 +-
8 files changed, 85 insertions(+), 20 deletions(-)
$ git push origin master
Enter passphrase for key '/home/lkrispen/.ssh/id_rsa_fedora':
Counting objects: 27, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (14/14), done.
Writing objects: 100% (14/14), 2.20 KiB, done.
Total 14 (delta 12), reused 0 (delta 0)
To ssh://git.fedorahosted.org/git/389/ds.git
2b5aecb..a154ecf master -> maste

Metadata Update from @rmeggins:
- Issue assigned to lkrispen
- Issue set to the milestone: 1.3.2 - 05/13 (May)

7 years ago

389-ds-base is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in 389-ds-base's github repository.

This issue has been cloned to Github and is available here:
- https://github.com/389ds/389-ds-base/issues/695

If you want to receive further updates on the issue, please navigate to the github issue
and click on subscribe button.

Thank you for understanding. We apologize for all inconvenience.

Metadata Update from @spichugi:
- Issue close_status updated to: wontfix (was: Fixed)

3 years ago

Login to comment on this ticket.

Metadata