Create a test suite, similar to TET's "basic" acceptance suite, to lib389. This test suite should perform a variety of basic/common operations and tasks.
Also, start the overall framework for "suite" tests(TET porting)
Looks great. A couple of trivial comments/quiestions...
1) It'd be nice to have a clean dse.ldif in the repo: We don't need these attributes: creatorsName, modifiersName, createTimestamp, modifyTimestamp, numSubordinates. We don't need o=NetscapeRoot related acis. The value of nsslapd-pluginVersion could be none.
2) If we could also provide a standard style for checking the result might be nice? E.g., the result of each operation in test_basic_ops could be checked by search, but we could just call a standard api for verifying the expected entry's existence (or not)? Having the count of the entries in a db instance would be useful, too. Maybe, we could add a doc how to add such a shared library functions to lib389?
3) clu/db2ldif_test.py is not finished yet?
75 # test 1 function... 76 # test 2 function... 77 # ...
4) This question is more on my current task... I'm so glad you added cleanallruv testsuite. I'm now hoping to add a test case to check shutting down the servers while running the tasks (both cleanallruv and cleanruv_abort) does not crash the servers (ticket48005), do you think I could piggyback on the testsuite to check it? Or we'd better copy part of the testsuite and do the test separately?
Replying to [comment:3 nhosoi]:
Looks great. A couple of trivial comments/quiestions... 1) It'd be nice to have a clean dse.ldif in the repo: We don't need these attributes: creatorsName, modifiersName, createTimestamp, modifyTimestamp, numSubordinates. We don't need o=NetscapeRoot related acis. The value of nsslapd-pluginVersion could be none.
I can clean it up. It is a broken dse.ldif that is meant to stop the server from starting up. It probably only needs one entry :-)
2) If we could also provide a standard style for checking the result might be nice?
You mean checking for an exception?
E.g., the result of each operation in test_basic_ops could be checked by search, but we could just call a standard api for verifying the expected entry's existence (or not)? Having the count of the entries in a db instance would be useful, too.
Like a objectclass=* search on a backend?
Sorry I'm not sure I'm fully understanding your questions
Maybe, we could add a doc how to add such a shared library functions to lib389?
Like how to add a function to utils.py, or to the DirSrv object?
No, not yet, I was going to start writing skeleton scripts, but decided to wait. This patch is already quite large. I almost saw this patch as phase 1 of many phases.
75 # test 1 function... 76 # test 2 function... 77 # ... 4) This question is more on my current task... I'm so glad you added cleanallruv testsuite. I'm now hoping to add a test case to check shutting down the servers while running the tasks (both cleanallruv and cleanruv_abort) does not crash the servers (ticket48005), do you think I could piggyback on the testsuite to check it? Or we'd better copy part of the testsuite and do the test separately?
It should definitely be in the suite, because that is also a feature of cleanallruv. You can stop the server in the middle of task, and it should resume at server startup. If you don't mind, I'll write that testcase(unless you already have one).
Ah, I see. I thought it could be used as a template. If it is just a broken dse.ldif, never mind!
Well, my thought is more basic. Like an added entry is really found in the db or a moved entry is really moved to the new location, or ... that level.
Test cases are often formed like: set up a test env do some test get ther result verify the result (e.g., comparing the result with some expected result) I thought the verification part could be somehow formalized. But probably, it depends upon each test case.
Rather "Like how to add a function to utils.py, or to the DirSrv? object?" <-- this would work better -- sharing handy utilities!!
If you don't mind, I'll write that testcase(unless you already have one).
Thank you soooooo much, Mark!! I'm still working on the standalone cases. If you could cover the cleanruv tasks, I'd be much more confident on the patch being added!
$ py.test -v =============================== test session starts === platform linux2 -- Python 2.7.5 -- pytest-2.4.2 -- /usr/bin/python collected 61 items
basic/basic_test.py:72: test_basic_init PASSED basic/basic_test.py:89: test_basic_ops PASSED basic/basic_test.py:204: test_basic_import_export PASSED basic/basic_test.py:270: test_basic_backup PASSED basic/basic_test.py:306: test_basic_acl PASSED basic/basic_test.py:427: test_basic_searches PASSED basic/basic_test.py:465: test_basic_referrals PASSED basic/basic_test.py:526: test_basic_systemctl PASSED basic/basic_test.py:607: test_basic_ldapagent PASSED basic/basic_test.py:642: test_basic_dse PASSED basic/basic_test.py:662: test_basic_final PASSED betxns/betxn_test.py:51: test_betxn_init PASSED betxns/betxn_test.py:60: test_betxt_7bit PASSED betxns/betxn_test.py:115: test_betxn_attr_uniqueness PASSED betxns/betxn_test.py:169: test_betxn_final PASSED clu/clu_test.py:51: test_clu_init PASSED clu/clu_test.py:59: test_clu_pwdhash PASSED clu/clu_test.py:84: test_clu_final PASSED clu/db2ldif_test.py:51: test_db2ldif_init PASSED clu/db2ldif_test.py:59: test_db2ldif_final PASSED config/config_test.py:51: test_config_init PASSED config/config_test.py:58: test_config_listen_backport_size PASSED config/config_test.py:109: test_config_deadlock_policy PASSED config/config_test.py:164: test_config_final PASSED dynamic-plugins/test_dynamic_plugins.py:74: test_dynamic_plugins PASSED dynamic-plugins/test_dynamic_plugins.py:457: test_dynamic_plugins_final PASSED filter/filter_test.py:51: test_filter_init PASSED filter/filter_test.py:58: test_filter_escaped PASSED filter/filter_test.py:102: test_filter_search_original_attrs PASSED filter/filter_test.py:124: test_filter_final PASSED memory_leaks/range_search_test.py:52: test_range_search_init PASSED memory_leaks/range_search_test.py:76: test_range_search PASSED memory_leaks/range_search_test.py:128: test_range_search_final PASSED password/password_test.py:51: test_password_init PASSED password/password_test.py:59: test_password_delete_specific_password PASSED password/password_test.py:118: test_password_final PASSED password/pwdAdmin_test.py:62: test_pwdAdmin_init PASSED password/pwdAdmin_test.py:173: test_pwdAdmin PASSED password/pwdAdmin_test.py:388: test_pwdAdmin_final PASSED password/pwdPolicy_test.py:51: test_pwdPolicy_init PASSED password/pwdPolicy_test.py:58: test_pwdPolicy_final PASSED replication/cleanallruv_test.py:456: test_cleanallruv_init PASSED replication/cleanallruv_test.py:519: test_cleanallruv_clean PASSED replication/cleanallruv_test.py:643: test_cleanallruv_clean_restart PASSED replication/cleanallruv_test.py:791: test_cleanallruv_clean_force PASSED replication/cleanallruv_test.py:930: test_cleanallruv_abort PASSED replication/cleanallruv_test.py:1040: test_cleanallruv_abort_restart PASSED replication/cleanallruv_test.py:1157: test_cleanallruv_abort_certify PASSED replication/cleanallruv_test.py:1294: test_cleanallruv_stress_clean PASSED replication/cleanallruv_test.py:1460: test_cleanallruv_final PASSED rootdn_plugin/rootdn_plugin_test.py:54: test_rootdn_init PASSED rootdn_plugin/rootdn_plugin_test.py:113: test_rootdn_access_specific_time PASSED rootdn_plugin/rootdn_plugin_test.py:195: test_rootdn_access_day_of_week PASSED rootdn_plugin/rootdn_plugin_test.py:280: test_rootdn_access_denied_ip PASSED rootdn_plugin/rootdn_plugin_test.py:351: test_rootdn_access_denied_host PASSED rootdn_plugin/rootdn_plugin_test.py:421: test_rootdn_access_allowed_ip PASSED rootdn_plugin/rootdn_plugin_test.py:495: test_rootdn_access_allowed_host PASSED rootdn_plugin/rootdn_plugin_test.py:568: test_rootdn_config_validate PASSED rootdn_plugin/rootdn_plugin_test.py:746: test_rootdn_final PASSED schema/test_schema.py:152: test_schema_comparewithfiles PASSED schema/test_schema.py:196: test_schema_final PASSED
======================= 61 passed in 1296.17 seconds =====
revision 0001-Ticket-48003-build-suite-framework.patch
Very nice to see all the tests PASSED!
So, if we add a "new" directory in the suites and "new_test.py" under it. Is it executed automagically? dirsrvtests/suites/new/new_test.py
Replying to [comment:8 nhosoi]:
Very nice to see all the tests PASSED! So, if we add a "new" directory in the suites and "new_test.py" under it. Is it executed automagically? dirsrvtests/suites/new/new_test.py
Yes, if you use py.test. I ran "py.test -v" in /ds/dirsrvtests/suites - it recursively looks for all the files with "test" in their names, and executes them. If I would run "py.test -v" in ds/dirsrvtests, it would also execute all the ticket tests as well.
Replying to [comment:9 mreynolds]:
Replying to [comment:8 nhosoi]: Very nice to see all the tests PASSED! So, if we add a "new" directory in the suites and "new_test.py" under it. Is it executed automagically? dirsrvtests/suites/new/new_test.py Yes, if you use py.test. I ran "py.test -v" in /ds/dirsrvtests/suites - it recursively looks for all the files with "test" in their names, and executes them. If I would run "py.test -v" in ds/dirsrvtests, it would also execute all the ticket tests as well.
Beautiful! Thanks, Mark!!
1557df5..d33676f master -> master commit d33676f Author: Mark Reynolds mreynolds@redhat.com Date: Tue Feb 17 12:18:50 2015 -0500
6a315fe..8e44437 389-ds-base-1.3.3 -> 389-ds-base-1.3.3 commit 8e44437
Reopening, need to add template scripts for all the plugins, and common functional areas. That way its more obvious where community members should add tests.
add script templates 0001-Ticket-48003-add-template-scripts.patch
To ssh://git.fedorahosted.org/git/389/ds.git 430410d..6dcc12c master -> master commit 6dcc12c Author: Mark Reynolds mreynolds@redhat.com Date: Wed Feb 18 17:32:05 2015 -0500
8e44437..20888a6 389-ds-base-1.3.3 -> 389-ds-base-1.3.3 commit 20888a6
Milestone lib389 1.0 deleted
Metadata Update from @mreynolds: - Issue assigned to mreynolds
389-ds-base is moving from Pagure to Github. This means that new issues and pull requests will be accepted only in 389-ds-base's github repository.
This issue has been cloned to Github and is available here: - https://github.com/389ds/389-ds-base/issues/1334
If you want to receive further updates on the issue, please navigate to the github issue and click on subscribe button.
subscribe
Thank you for understanding. We apologize for all inconvenience.
Metadata Update from @spichugi: - Issue close_status updated to: wontfix (was: Fixed)
Login to comment on this ticket.