Merge from trunk commit revision 2907
This patch fixes the equality test script and the valgrind wrapper
script to make the parser under test use the features.all features file
from the features_files/ subdirectory. Otherwise, the equality tests
will fail on systems where the not all of the current language features
are supported. The equality fix does so in a way to make the script work
correctly regardless of the directory it is run from.
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: John Johansen <john.johansen@canonical.com>
The valgrind test script would happily chug along even if if valgrind
was not installed, not doing anything of use. This patch fixes that, and
offers up the ability to specify an alternate location for valgrind if
it does not exist in the usual /usr/bin location.
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
When passing an include directory on the command line to
apparmor_parser, valgrind emits a warning:
Invalid read of size 4
at 0x404DA6: add_search_dir(char const*) (parser_include.c:152)
by 0x40BB37: process_arg(int, char*) (parser_main.c:457)
by 0x403D43: main (parser_main.c:590)
Address 0x572207c is 28 bytes inside a block of size 29 alloc'd
at 0x4C2A420: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
by 0x53E31C9: strdup (strdup.c:42)
by 0x404D94: add_search_dir(char const*) (parser_include.c:145)
by 0x40BB37: process_arg(int, char*) (parser_main.c:457)
by 0x403D43: main (parser_main.c:590)
This patch quiets the warning by removing strlen() calls on the t char
array. Instead, it only calls strlen() on the dir char array. t is a
dupe of dir and strlen(dir) does not trigger the valgrind warning.
Additionally, this patch adds a bit of defensive programming to the
while loop to ensure that index into the t array is never negative.
Finally, the valgrind suppression is removed from valgrind_simple.py.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: Steve Beattie <steve@nxnw.org>
With commit 2364 addressing one of valgrind's false positives, we can
remove the related valgrind suppression entry from the test script.
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
Acked-by: Tyler Hicks <tyhicks@canonical.com>
With the C++-ization of the parser, some functions were renamed or
eliminated; this patch fixes the relevant valgrind false positive
suppression
pattern to match.
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
This patch adds a python metaclass to wrap the test methods in the
subclasses of the template class AATestTemplate with the keep_on_fail
function, which sets the do_cleanup attribute to False when a testcase
failure occurs (i.e. an Exception is raised), and removes the manually
applied decorators to the caching tests that made use of this.
The downside to this approach is that the way metaclasses are declared
changed between python 2 and python 3 in an incompatible way. Since
python 3 is The Future™, I chose that approach and made the caching
and valgrind tests which use testlib be python3 (until this change,
they would have worked under either python 2 or python 3).
(An output message when a failure occurs is tweaked, to make the
output a little cleaner when verbose test output is requested and
failures occur.)
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: Christian Boltz <apparmor@cboltz.de>
This patch adds the command run to the reported message when a valgrind
failure is detected. This makes reproducing the failure outside of the
test suite easier, for easier diagnosis of what problem is occurring.
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: Christian Boltz <apparmor@cboltz.de>
When converting the valgrind tests from optparse to argparse, I managed
to not verify that the resulting code actually worked :( . This patch
fixes it by adding a positional argument to handle the optional passed
directory location.
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: Christian Boltz <apparmor@cboltz.de>
This patch rewrites the caching test in python, using python's unittest
framework. It has been used with python 2.7 and python 3.3; python2.6
may have issues. It covers the tests in the existing caching.sh
test script (with the exception of the test that checks for when the
parser in $PATH is newer), as well as adding additional tests that
more extensively cover using a cache in an alternate location from
basedir. It also adds simple tests for the --create-cache-dir option
(along with that option's interaction with the alt-cache option).
(Some further work to be done is listed under TODO.)
Patch history:
v1: - initial version
v2: - create template base class
- add keep_on_fail() decorator to keep temporary test files
around after a test fails
- don't dump raw cache file to failure output in
test_cache_writing_updates_cache_file()
- push run_cmd into template class
- create run_cmd_check wrapper to run_cmd that adds an assertion
check based on whether return code matches the expected rc
(the valgrind tests only want to verify that the rc is not a
specific set of values, hence the separate wrapper function)
- similarly, add a check to run_cmd_check for verifying the output
contains a specific string, also simplifying many of the caching
tests.
- create testlib.write_file() to simplify writing file
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: Christian Boltz <apparmor@cboltz.de>
This patch adds a test wrapper that runs valgrind on the parser over the
simple_tests tree (or other directory tree if passed on the command
line). An alternate parser location can also be passed on the command
line.
Like the libapparmor python bindings test, this test uses a bit of magic
to generate tests that doesn't work with auto-detecting test utilities
like nose.
Running valgrind on the parser over all 69000+ testcases takes several
hours, so while this patch includes a make target 'make valgrind', it
does not add it to the set of tests run when 'make check' is called.
Perhaps a 'make extra-tests' target is in order.
Patch history:
v1: - initial version.
v2: - add some valgrind suppressions for overaggressive 4 byte reads
past the end of allocated storage (not completed).
v3: - add ability to dump valgrind suppressions to stdout, to use
diagnosis runs of valgrind for determining whether a given
failure is a false positive or not.
- correctly return 0 on a successful run and an error code if one
or more test cases fail.
- point LD_LIBRARY_PATH at the in-tree libapparmor build.
- split out some utility functions into testlib.py, for possible
use by other to be written test scripts
v4: - convert optparse to argparse
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: Tyler Hicks <tyhicks@canonical.com> (for v2 version)
