Don't commingle stderr and stdout by default, and provide a
function that provides the exit value, stderr, and stdout as a
tuple. Also, expect UTF-8 output by default from the commands (with
universal_newlines), rather than bytes.
(Nifty commands like subprocess.run() weren't added to python until
python 3.5.)
Signed-off-by: Steve Beattie <steve.beattie@canonical.com>
Acked-by: John Johansen <john.johansen@canonical.com>
MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/522
This makes several improvements to the parser caching tests to verify
that the parser is properly consuming the mtime of profiles and
abstractions when dealing with the policy cache.
It introduces a simple abstraction file and tests the mtime handling by
changing the mtime on the profile, abstraction, apparmor_parser, and
cache file in various combinations to check the parser's behavior.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: John Johansen <john.johansen@canonical.com>
This patch adds a python metaclass to wrap the test methods in the
subclasses of the template class AATestTemplate with the keep_on_fail
function, which sets the do_cleanup attribute to False when a testcase
failure occurs (i.e. an Exception is raised), and removes the manually
applied decorators to the caching tests that made use of this.
The downside to this approach is that the way metaclasses are declared
changed between python 2 and python 3 in an incompatible way. Since
python 3 is The Future™, I chose that approach and made the caching
and valgrind tests which use testlib be python3 (until this change,
they would have worked under either python 2 or python 3).
(An output message when a failure occurs is tweaked, to make the
output a little cleaner when verbose test output is requested and
failures occur.)
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: Christian Boltz <apparmor@cboltz.de>
This patch modifies testlib.write_file() to take a directory and a file
name instead of a path and return the joined result for callers to use
if necessary.
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: Christian Boltz <apparmor@cboltz.de>
This patch rewrites the caching test in python, using python's unittest
framework. It has been used with python 2.7 and python 3.3; python2.6
may have issues. It covers the tests in the existing caching.sh
test script (with the exception of the test that checks for when the
parser in $PATH is newer), as well as adding additional tests that
more extensively cover using a cache in an alternate location from
basedir. It also adds simple tests for the --create-cache-dir option
(along with that option's interaction with the alt-cache option).
(Some further work to be done is listed under TODO.)
Patch history:
v1: - initial version
v2: - create template base class
- add keep_on_fail() decorator to keep temporary test files
around after a test fails
- don't dump raw cache file to failure output in
test_cache_writing_updates_cache_file()
- push run_cmd into template class
- create run_cmd_check wrapper to run_cmd that adds an assertion
check based on whether return code matches the expected rc
(the valgrind tests only want to verify that the rc is not a
specific set of values, hence the separate wrapper function)
- similarly, add a check to run_cmd_check for verifying the output
contains a specific string, also simplifying many of the caching
tests.
- create testlib.write_file() to simplify writing file
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: Christian Boltz <apparmor@cboltz.de>
This patch adds a test wrapper that runs valgrind on the parser over the
simple_tests tree (or other directory tree if passed on the command
line). An alternate parser location can also be passed on the command
line.
Like the libapparmor python bindings test, this test uses a bit of magic
to generate tests that doesn't work with auto-detecting test utilities
like nose.
Running valgrind on the parser over all 69000+ testcases takes several
hours, so while this patch includes a make target 'make valgrind', it
does not add it to the set of tests run when 'make check' is called.
Perhaps a 'make extra-tests' target is in order.
Patch history:
v1: - initial version.
v2: - add some valgrind suppressions for overaggressive 4 byte reads
past the end of allocated storage (not completed).
v3: - add ability to dump valgrind suppressions to stdout, to use
diagnosis runs of valgrind for determining whether a given
failure is a false positive or not.
- correctly return 0 on a successful run and an error code if one
or more test cases fail.
- point LD_LIBRARY_PATH at the in-tree libapparmor build.
- split out some utility functions into testlib.py, for possible
use by other to be written test scripts
v4: - convert optparse to argparse
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: Tyler Hicks <tyhicks@canonical.com> (for v2 version)
