#3160 Add more syntactical sugar around INTGCHECK_PYTEST_ARGS
Closed: Invalid None Opened 7 years ago by jhrozek.

Currently it's possible to run a single integration test with:

make intgcheck-run INTGCHECK_PYTEST_ARGS="-k test_add_empty_netgroup"

But the -k is a detail of pytest. Our Makefile targets should be independent of the underlying test framework and shouldn't expose the -k option.


Fields changed

summary: make intgcheck-run INTGCHECK_PYTEST_ARGS="-k test_add_empty_netgroup"Add more syntactical sugar around => Add more syntactical sugar around INTGCHECK_PYTEST_ARGS

Replying to [comment:1 jhrozek]:

It hasn't been done so far because Lukaš disagrees on adding a new environment variable to pass something to py.test, as we already have the INTGCHECK_PYTEST_ARGS one.

Anyways, some complications may appear in order to parser what's a file and what's a single test in order to correctly pass those arguments to pytest, as the -k argument may be used before the file or before the test itself depending on what has been passed, for instance:

  • Running a single test file

    make intgcheck-run INTGCHECK_PYTEST_ARGS="-k file.py"

  • Running a single test from a specific file

    make intgcheck-run INTGCHECK_PYTEST_ARGS="file.py -k test_foo"

But the -k is a detail of pytest. Our Makefile targets should be independent of the underlying test framework and shouldn't expose the -k option.

I have a totally different opinion. We already depend on pytest. We cannot be independent on underlying test framework. That's is a reason why we should try to avoid adding abstractions on top of pytest.

We just need to provide some examples + link to howtos for pytest.

BTW using env variables is not the best way. shell completion does not work with env variables.
Unfortunately, we currently cannot use other way.

make intgcheck-run INTGCHECK_PYTEST_ARGS="file.py -k test_foo"
Can run the test "test_foo" twice.
e.g.

 make intgcheck-run INTGCHECK_PYTEST_ARGS="test_netgroup.py -k test_add_tripled_netgroup"

It is expanded into

fakeroot /usr/bin/python2 /usr/bin/py.test -v --tb=native test_netgroup.py -k test_add_tripled_netgroup .

and the file "test_netgroup.py" is included twice by pytest. The 1st time as "test_netgroup.py" and the 2nd time as part of directory "." (at the end of line).

py.test --help
usage: py.test [options] [file_or_dir] [file_or_dir] [...]

The option "-k test_add_tripled_netgroup" just filtered
possible tests from loaded files and therefore the test was executed twice.

[user@host][.]$make intgcheck-run INTGCHECK_PYTEST_ARGS="test_netgroup.py -k test_add_tripled_netgroup"
if [ ! -d intg/pfx ]; then make intgcheck-prepare; fi; \
cd intg/bld; \
make  -C src/tests/intg intgcheck-installed; \
cd ../..
make[1]: Entering directory './intg/bld/src/tests/intg'
pipepath="/tmp/sssd-intg.rgy7ZCY9/var/lib/sss/pipes"; \
if test ${#pipepath} -gt 80; then \
    echo "error: Pipe directory path too long," \
         "D-Bus won't be able to open sockets" >&2; \
    exit 1; \
fi
set -e; \
cd "./../sssd/src/tests/intg"; \
nss_wrapper=$(pkg-config --libs nss_wrapper); \
uid_wrapper=$(pkg-config --libs uid_wrapper); \
PATH="$(dirname -- /usr/sbin/slapd):$PATH" \
PATH="/tmp/sssd-intg.rgy7ZCY9/sbin:/tmp/sssd-intg.rgy7ZCY9/bin:$PATH" \
PATH="./intg/bld/src/tests/intg:./../sssd/src/tests/intg:$PATH" \
PYTHONPATH="./intg/bld/src/tests/intg:./../sssd/src/tests/intg" \
LDB_MODULES_PATH="/tmp/sssd-intg.rgy7ZCY9/lib/ldb" \
LD_PRELOAD="$nss_wrapper $uid_wrapper" \
NSS_WRAPPER_PASSWD="./intg/bld/src/tests/intg/passwd" \
NSS_WRAPPER_GROUP="./intg/bld/src/tests/intg/group" \
NSS_WRAPPER_MODULE_SO_PATH="/tmp/sssd-intg.rgy7ZCY9/lib/libnss_sss.so.2" \
NSS_WRAPPER_MODULE_FN_PREFIX="sss" \
UID_WRAPPER=1 \
UID_WRAPPER_ROOT=1 \
    fakeroot /usr/bin/python2 /usr/bin/py.test -v --tb=native test_netgroup.py -k test_add_tripled_netgroup .
========================================== test session starts ===========================================
platform linux2 -- Python 2.7.12, pytest-2.9.2, py-1.4.31, pluggy-0.3.1 -- /usr/bin/python2
cachedir: .cache
rootdir: /dev/shm/sssd/src/tests/intg, inifile: 
collected 108 items

test_netgroup.py::test_add_tripled_netgroup PASSED
test_netgroup.py::test_add_tripled_netgroup PASSED

========================= 106 tests deselected by '-ktest_add_tripled_netgroup' ==========================
================================ 2 passed, 106 deselected in 7.71 seconds ================================
rm -f /tmp/sssd-intg.rgy7ZCY9/var/log/sssd/*
make[1]: Leaving directory './intg/bld/src/tests/intg'

Fields changed

resolution: => wontfix
status: new => closed

Metadata Update from @jhrozek:
- Issue set to the milestone: NEEDS_TRIAGE

7 years ago

SSSD is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in SSSD's github repository.

This issue has been cloned to Github and is available here:
- https://github.com/SSSD/sssd/issues/4193

If you want to receive further updates on the issue, please navigate to the github issue
and click on subscribe button.

Thank you for understanding. We apologize for all inconvenience.

Login to comment on this ticket.

Metadata