If a test is marked as TODO, but matches its EXRESULT, this means the
TODO is superfluous and (probably) a change fixed what the TODO was for.
Instead of more or less ignoring such superfluous TODOs, error out to
make the change visible instantly.
Without this, the system-wide parser.conf gets used, which causes test
failures if for example caching is enabled and the cache dir isn't
writeable for the user running the tests.
With the recent addition of features like ptrace and signals that
give warnings and then ignore the subset of rules when the features
directory indicates that the kernel does not support mediating such
features, at least one of the language tests fails in a chroot
environment where the apparmor securityfs tree is not mounted
inside it.
To compensate, a features file containing the current supported features
is included, and the simple.pl test driver is modified to pass it as an
argument to the parser, so that it will act as if the environment
supports all our current features.
A simple python script is included that was used to generate the
features file based on the current feature set.
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
If a parser test case causes the parser to produce a core dump, the
simple.pl test runner incorrectly treats the test as a success.
This patch treats tests that cause core dumps as failures, even when the
tests are marked as #=TODO. The only way to ignore tests that fail in
this manner is with #=DISABLED.
Note that this patch changes the meaning of the $result variable.
Previously, it held a true or false status returned when closing the
apparmor_parser pipe. Now, it holds the exit status of apparmor_parser.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: Steve Beattie <steve@nxnw.org>
This patch fixes two issue with the simple test driver. The first is
that child exec that actually ran the parser was located inside the
eval statement. This meant that if the exec failed for some reason
(like the parser didn't exist), the child wouldn't actually die,
but would pop out of the eval and continue running through the loop
of test profiles (while the parent process does the same). This meant
that if the script ran on the full testsuite with a misconfiguration,
it would explode creating O(n^2) processes, where n is the number of
testcase files -- with over 25k testcases, that's a lot. The fis is to
lift the child exec outside the eval{}, then an exec() failure causes
the child process to die correctly.
The second fix is that several of the testcases were added with the
DESCRIPTION field added in lower case (i.e. #=Description blah blah).
This fix makes the regex that pulls out the description not be
case-sensitive.
Signed-off-by: Steve Beattie <sbeattie@ubuntu.com>
Acked-By: John Johansen <john.johansen@canonical.com>
This patch replaces the hardcoded path to the in-tree apparmor parser
in several of the script based test scripts with the APPARMOR_PARSER
environment variable, keeping the hardcoded location as the default.
It also adds support for overriding the location of the parser via the
same environment variable. The make infrastructure is updated to use
this, though uses a different variable (PARSER) to drive it.
Thus 'make check PARSER=/some/path/to/an/alternate/apparmor_parser'
will run all the parser tests on that binary. This is useful for
running the testsuite in an automated post-install environment.
(It should be noted that doing so will still build and run the unit
test binaries based on the source tree.)
Signed-off-by: Steve Beattie <sbeattie@ubuntu.com>
Acked-By: John Johansen <john.johansen@canonical.com>
bashisms which were in place to get around PWD not getting set
correctly. Instead, fix the simple.pl script to call pwd directly.
Based on feedback from PLD/Arkadiusz Miskiewicz <arekm@maven.pl>.
