A typical IPA command generally takes between 2 and 3 seconds. The breakdown in time is as follows:
- general init: .2 sec - finalize plugins: .8 sec - remote exectution: 1.2. sec - display output: .1 sec
So we can cut execution down by half if we can improve finalization. A fair bit of work was done around the edges here to skip a lot of steps if in production mode, but there is little more we can do in that regard.
What this code does is set all the parameters, create the namespaces, etc for each command. Each one takes like 3 ms to perform but with 251 commands it adds up quickly.
We should examine if there is a way to skip this altogether and just finalize the command that is to be executed.
One thing to consider is that a plugin can call other plugins. Today this is only done on the server side but there is nothing to prevent it in a forward() call, so we may need to register (or detect) dependencies.
The lazy init would be done only when context == cli.
Ondra shall look on this in a scope of his internship.
Loading of plugins is based on directory structure - it loads all plugins in specified directory.
I've measured how long does it take to load_plugins and to finalize. I was using 'ipa user-add' command and the times were measured in ipalib/cli.py in function 'run'. Current implementation took: Loading plugins: 0.19699215889 Finalization: 0.6018679142
After that I've used a tuple with names of all plugins necessary to run the 'ipa user-add' command. It consisted of 14 plugins ('user','baseldap','group','netgroup','host','hostgroup','role','sudorule','sudocmd','sudocmdgroup','hbacsvc','hbacsvcgroup','permission','xmlclient') and only those plugins were imported. Running time was reduced: Loading plugins 0.119768857956 Finalization 0.449488162994
The loading of plugins was about 40% faster, finalization only about 25% faster.
Another measurements: command 'ipa dnsrecord-find example.com'
This command needs only two plugins ('xmlclient','dns'). Here's the time reduction much bigger:
Current state:
Loading plugins 0.205113172531
Finalization 0.607607841492
Loading only necessary plugins:
Loading plugins 0.0610809326172
Finalization 0.0904400348663
Time is reduced by 70% and 85%
Reassigning to Alexander as this ticket requires much more experience with FreeIPA framework than expected and he is already involved in a solution of this ticket.
Yes, patch is sent to review, it gives 2x-3x speedup.
Mark that patch is on review.
master: e7a6d10
Metadata Update from @rcritten: - Issue assigned to abbra - Issue set to the milestone: FreeIPA 3.0 Core Effort - 2011/11
Login to comment on this ticket.