Background info: The ticket #519 "Search with a complex filter including range search is slow" was a complaint against the complex filter that includes open ended range search, e.g., "(modifyTimestamp>=20120101010101Z)". To reduce the elapsed time, we introduced nsslapd-rangelookthroughlimit not to create an ID list if the range search result gets larger (5000 by default).
Regardless of the complexity of the filter, the range search is slow in general since we merge the ID list containing one ID to the primary ID list every time an entry is found to satisfy the range search filter. Rather, we should be able to implement an idl range search API that could generate an ID list without merging.
git patch file (master) 0001-Ticket-537-Improvement-of-range-search.patch
Fix description: The index range search function index_range read_ext was written to call idl_fetch_ext to get an idlist belonging to one key. Then add it to the main idlist as long as the key satisfiles the range search filter condition. This patch introduces a new range search function idl_new range_fetch to the new idl code, which generates an idlist in one idl function that eliminates the redundancy such as generating idlist and cursor per key.
This patch only implements the new idl version. If idl_new is not set, the existing code is executed.
(2013/01/10 11:22), Rich Megginson wrote:
In the new code, for fetch operations, we start a new child read transaction? Why? If we are doing this, we need to abort the transaction upon errors - then, either do a retry by beginning a new transaction, or just report the error back to the caller and let the caller decide what to do. Thanks for the review, Rich. The idl_new_range_fetch code is based upon idl_new_fetch, which also starts read transaction. It looks I introduced it when I worked on betxn enabled by default:
commit cd48bbda6b93cbe6458f6022ba6ccb15925c7ab8 Trac Ticket #351 - use betxn plugins by default
I did not leave any comments on the particular dblayer_read_txn_begin call, but I think it was needed to separate the db read access from any on-going write transaction when the search was called from plugins...
And yes, even if any error occurs in the read transaction, the code does not call abort, but just commit... I guess I should fix it.
revised git patch file (master) 0001-Ticket-537-Improvement-of-range-search.2.patch
In addition to Comment 3, idl_new_fetch did not abort the read transaction even if any error occurred in the transaction. Now, it switches between commit and abort based upon the result.
Reviewed by Rich (Thank you!!)
Pushed to master: commit 0228e57
Ticket has been cloned to Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=918710
Metadata Update from @nhosoi: - Issue assigned to nhosoi - Issue set to the milestone: 1.3.1
389-ds-base is moving from Pagure to Github. This means that new issues and pull requests will be accepted only in 389-ds-base's github repository.
This issue has been cloned to Github and is available here: - https://github.com/389ds/389-ds-base/issues/537
If you want to receive further updates on the issue, please navigate to the github issue and click on subscribe button.
subscribe
Thank you for understanding. We apologize for all inconvenience.
Metadata Update from @spichugi: - Issue close_status updated to: wontfix (was: Fixed)
Login to comment on this ticket.