#47747 CI lib389: add topology fixtures
Closed: wontfix None Opened 10 years ago by tbordaz.

Test cases that use lib389, defines a topology fixture (pytest) that create/reinit the instances required by the test case.

There is an infinity kind of topology, but some of them are the most frequent:
- standalone
- single master - single consumer
- two masters
- two masters - two consumers
- masters - hub - consumer

This ticket is to enhance lib389 so that it offers standard topology deployment. In that case, most of the topology fixture in the test case will simply call the appropriate interface.


I think, it is better to have all "setUp" steps inside our test suite directory. Also, it makes sense to have it in one place.
I've create ticket and patch for this - https://fedorahosted.org/389/ticket/49055

Regarding tickets, I think we can create a few presetuped topologies (possible two for now: standalone and two masters) in lib389. It can be used by developers to improve their testing experience.

Also, it is not possible to have fixtures in lib389. For the setup and teardown purposes we can create fixture wrappers for tickets and pack our functions (create and remove the topology) in it.

After the discussion with William I've changed the test suite logic.
I've added new module to lib389/topologies.py and have put all fixtures there.

{{{
It contains:
- topology_st - with topology_st.standalone inst;
- topology_m2 and topology_m4 - with dicts
topology_mN.ms["masterX"]
topology_mN.ms["masterX_agmts"]["mX_mY"]
}}}
To use it in our test we must import the module from lib389
and use it as a usual fixture afterwards.
Also, with a new structure we can easily run through masters
and agreements in loops.

THe patch looks good but I would like to hear Mark and William if these new fixtures are fine for them.
If they are ok, we may also port create_test.py to use those fixture

This is exactly what I had in mind Simon! Ack from me, now we just need to do the import lines for the tests.

To ssh://git.fedorahosted.org/git/389/lib389.git
b9a07e2..224e200 master -> master
commit 224e2007cb58507d18dc412a45a9fb00a088adaa
Author: Simon Pichugin spichugi@redhat.com
Date: Wed Nov 30 12:34:38 2016 +0100

Again, fantastic work, this is really great for our project.

Add topology fixtures to topologies.py:
- topology_m1c1
- topology_m1h1c1
- topology_m3

To ssh://git.fedorahosted.org/git/389/lib389.git
212518a..fa2e915 master -> master
commit fa2e915ed782d62df08fe3a3e654c5dade3a9d7c
Author: Simon Pichugin spichugi@redhat.com
Date: Mon Dec 12 09:47:21 2016 +0100

is it possible to make the creation of agreements or total init optional.

we have scenarios where we need to test a specific order of creation of agreements and initialization.

Another idea is, can we make the topologies extendable. eg start with 3 masters and add 1
or have 2 masters and later add a hub and then add a set of consumers ?
This would avoid to create a specific fixture for 5,6.... masters

Replying to [comment:14 lkrispen]:

is it possible to make the creation of agreements or total init optional.

we have scenarios where we need to test a specific order of creation of agreements and initialization.

Another idea is, can we make the topologies extendable. eg start with 3 masters and add 1
or have 2 masters and later add a hub and then add a set of consumers ?
This would avoid to create a specific fixture for 5,6.... masters

I think in this case the most efficient way is to do like this:
1. I will add two more topologies to the lib389/topologies.py that contains
creation of 2 and 3 independent instances.
2. If we need additional steps for a particular test case, we add new fixture to it with the steps (enable replica, add agreements, etc.)
3. We add this fixture from step 2 to our test case.
3. If this setup is used more than once or twice along the tests, we add it to the lib389/topologies.py.
What do you think? Do you want me to add this patch?

I'm not sure if we need more fixtures, I thought of something like having a param: setup_agmt=True|False, init_replica=True|False

And to extend a topology by adding a replica do soemthing like IPA in replica-install, define a starting replica and add a new one, connecting only these two: like T.addHub("master1","hub1")

Looks like I am not looking for new fixtures but for a generic, parametrized topology creation

Replying to [comment:16 lkrispen]:

I'm not sure if we need more fixtures, I thought of something like having a param: setup_agmt=True|False, init_replica=True|False

And to extend a topology by adding a replica do soemthing like IPA in replica-install, define a starting replica and add a new one, connecting only these two: like T.addHub("master1","hub1")

Looks like I am not looking for new fixtures but for a generic, parametrized topology creation

Got it. It would be great to have such wrappers like 'T.addHub("master1","hub1")' in our lib389. I think we can create a ticket for it and I (or anyone who has time) will add it respectively.

For now, we can come up with some another solution. Unfortunately, we can't add some input parameters to the existing (imported) topology fixtures. After it is imported it can't be changed in the test case code. Even for enabling debug mode I was forced to use a system environment variable.

The solution we can use is the one I've written in the comment above. It is not a problem to add more fixtures, so after creating topology_i3 in the lib389 (with three instances), the test case will look like this:

{{{
from lib389.topologies import topology_i3

@pytest.fixture(scope="module")
def enable_replica(topology_i3):
for n in range(1, 4):
topology_i3.is["inst{}".format(n)].replica.enableReplication(suffix=SUFFIX, role=REPLICAROLE_MASTER, replicaId=SOME_ID)

You can add any fixture you like here (creating hubs, more masters etc.)

@pytest.fixture(scope="module")
def agreements(topology_i3):
for n in range(1, 4):
# Create agreements here
pass

def test_something(topology_i3, enable_replica, agreements):
# Test something with already enabled replica and create agreements
pass
}}}

And I think it is the best way for now because you can create any setup you like within your test case and lib389 wouldn't enforce you to do things you don't want.
After adding more sensable wrappers like 'T.addHub("master1","hub1")' we can do like this:
{{{
from lib389.topologies import topology_i3

def test_something(topology_i3):
# Test something
topology_i3.addHub("master1", "hub1")
# Test something else
topology_i3.addMaster("master1", "master2")
# Test something else
pass
}}}

Thanks, if you can add the _i it should do what I want.

And we can think about the more fancy addXXX later, or in another ticket

To ssh://git.fedorahosted.org/git/389/lib389.git
70e24d3..c392aca master -> master
commit c392aca364e7716b3ef4a3a4890d4fa015997e53
Author: Simon Pichugin spichugi@redhat.com
Date: Fri Jan 13 16:20:45 2017 +0100

Metadata Update from @spichugi:
- Issue assigned to spichugi
- Issue set to the milestone: lib389 1.0.3

7 years ago

Metadata Update from @mreynolds:
- Custom field reviewstatus adjusted to None
- Custom field rhbz reset (from 0)
- Issue set to the milestone: None (was: lib389 1.0.3)

4 years ago

389-ds-base is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in 389-ds-base's github repository.

This issue has been cloned to Github and is available here:
- https://github.com/389ds/389-ds-base/issues/1079

If you want to receive further updates on the issue, please navigate to the github issue
and click on subscribe button.

Thank you for understanding. We apologize for all inconvenience.

Metadata Update from @spichugi:
- Issue close_status updated to: wontfix

3 years ago

Login to comment on this ticket.

Metadata