... without all the profiles generated by the gen-*.py scripts.
This target is meant for local manual testing, especially when working
on additional simple_tests profiles.
It makes local testing much faster (15 seconds for ~2k profiles vs.
several minutes for the additional ~70k profiles generated by gen-*.py)
Needless to say that the CI should continue to use the parser_sanity
target that includes all the generated profiles.
Ideally we would have a flag or something so the caller could choose
to handle symlinks, or traverse them. But since all callers currently
don't handle symlinks just handle them in the iterator.
Beyond fixing the early termination due to a failed symlink this also
fixes another case of failure in one job cause dir based loads to
terminate early. Which can result in partial loads.
Fixes: https://gitlab.com/apparmor/apparmor/-/issues/215
MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/850
Signed-off-by: John Johansen <john.johansen@canonical.com>
Approved-by: Georgia Garcia <georgia.garcia@canonical.com>
The generated files are exactly the same, but the code is a bit more
readable.
Additional differences:
- added test_gen_list() to verify the result of gen_list()
- null_target has a non-empty value to avoid that it gets skipped in
loops as empty value
- invert_save has an additional entry for ''
- copyright header added (based on git log of gen-xtrans.pl)
Make the error and warning message tests a first class test script; in
gitlab CI, failures would not necessarily result in output being
captured.
Signed-off-by: Steve Beattie <steve.beattie@canonical.com>
Acked-by: John Johansen <john.johansen@canonical.com>
MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/522
Instead of printing the (useless) numbers of no-longer-failing tests
marked as TODO, let `prove` print out the actual test names.
A side effect is that really unfixed TODOs and disabled tests get
printed (lists about 10 tests), but that's much better than having to
hunt down the no-longer-failing TODOs.
This change will print out lines like (the first one is still a TODO,
the second one got fixed at some time)
```
not ok 71447 - TODO: ./simple_tests//vars/vars_profile_name_13.sd: reference variables that are the profile name and attachment # TODO Unfixed testcase.
ok 71448 - TODO: ./simple_tests//vars/vars_profile_name_14.sd: reference variables in rules that also have alternations # TODO Unfixed testcase.
```
before printing the test summary.
Without this, the system-wide parser.conf gets used, which causes test
failures if for example caching is enabled and the cache dir isn't
writeable for the user running the tests.
The following patch addresses two issues on older releases:
1) In trunk commit 2911, the line 'undefine VERBOSE' was added to
parser/tst/Makefile so that the equality tests would not generate
verbose output when $VERBOSE != 1. Unfortunately, the 'undefine'
keyword was not introduced in GNU Make until version 3.82. On
distro releases like Ubuntu 12.04 LTS that include versions of Make
older than that, make check and make clean abort when VERBOSE is
not set to 1. The patch fixes that by setting VERBOSE to a zero
length string if does not already equal 1.
2) In trunk commit 2923, a workaround for systemd as init was added
to the pivot_root regression test. The workaround included a
call to ps(1) to determine if systemd is pid 1. Unfortunately,
in older versions of the procps package (such as the version in
Ubuntu 12.04 LTS), 'ps -hp1' emits the warning
Warning: bad ps syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html
The patch below converts the ps call to 'ps hp1' which does not
generate the warning.
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: John Johansen <john.johansen@canonical.com>
- make the verbose output of equality.sh honor whether or not
the environment variable VERBOSE is set
- thereby making the output verbose when 'make check V=1' or 'make
check VERBOSE=1' is given from within the parser/ directory. This
will make distribution packagers happy when diagnosing build
failures caused by test failures.
- if verbose output is not emitted and the tests were successful, emit
a newline before printing PASS.
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: John Johansen <john.johansen@canonical.com>
This patch adds a parser make variable and a make target for building
the compiler with coverage compilation flags. With this, coverage
information can be generated by running tests/test suites against the
built parser and run through tools like gcovr.
Patch History:
v1: initial version
v2: refreshed/no change
v3: address feedback from sarnold:
- mark coverage target as phony
- correct missing '.' typo in clean target
- make coverage extensions consistent in clean targets
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
This patch:
- incorporates the new python caching test into the make check/make
caching target, and removes the older shell based test script
- adjusts the python scripts to give verbose output when the VERBOSE
flag is set
- reorders the tests so that the tests that take a shorter amount of
time to run come first, leaving the language sanity test with its
69000+ testcases last
Patch history:
v1: initial revision
v2: add gen_xtrans/gen_dbus dependency to valgrind test
v3: drop gen_xtrans/gen_dbus as that was committed as a separate fix
Acked-by: Steve Beattie <steve@nxnw.org>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
The parser sanity test make target does not directly depend on the make
targets that generate the tests consumed by the sanity test, leading to
runs that did not verify all the test cases when make check is invoked
with parallelism (e.g. make check -j4). This patch against trunk fixes
the issue.
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
Acked-by: Tyler Hicks <tyhicks@canonical.com>
This patch adds a test wrapper that runs valgrind on the parser over the
simple_tests tree (or other directory tree if passed on the command
line). An alternate parser location can also be passed on the command
line.
Like the libapparmor python bindings test, this test uses a bit of magic
to generate tests that doesn't work with auto-detecting test utilities
like nose.
Running valgrind on the parser over all 69000+ testcases takes several
hours, so while this patch includes a make target 'make valgrind', it
does not add it to the set of tests run when 'make check' is called.
Perhaps a 'make extra-tests' target is in order.
Patch history:
v1: - initial version.
v2: - add some valgrind suppressions for overaggressive 4 byte reads
past the end of allocated storage (not completed).
v3: - add ability to dump valgrind suppressions to stdout, to use
diagnosis runs of valgrind for determining whether a given
failure is a false positive or not.
- correctly return 0 on a successful run and an error code if one
or more test cases fail.
- point LD_LIBRARY_PATH at the in-tree libapparmor build.
- split out some utility functions into testlib.py, for possible
use by other to be written test scripts
v4: - convert optparse to argparse
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: Tyler Hicks <tyhicks@canonical.com> (for v2 version)
With trunk commit 2205 "use libapparmor's find mountpoint fn",
the parser now builds against and uses libapparmor at runtime. However,
it currently builds against the system installed libapparmor library and
header files, which fails if either aren't installed, and is thus
painful for bootstrapping in a new environment.
Instead, the parser, like pam_apparmor and mod_apparmor, should build
against the in-tree libapparmor header and library. This patch does
that and adjusts the tests to point LD_LIBRARY_PATH at the location
of the built library as well.
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: John Johansen <john.johansen@canonical.com>
This test is to verify that a list of profiles compile down into the
same binary representation. This is useful, for example, when testing a
rule syntax that includes permission aliases, as well as implied and
explicit accesses.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
This is a test in the style of gen-xtrans.pl that attempts to run
through the most commonly constructed DBus rules. It also attempts to
run through some common mistakes to ensure that the parser fails
appropriately.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
This patch replaces the hardcoded path to the in-tree apparmor parser
in several of the script based test scripts with the APPARMOR_PARSER
environment variable, keeping the hardcoded location as the default.
It also adds support for overriding the location of the parser via the
same environment variable. The make infrastructure is updated to use
this, though uses a different variable (PARSER) to drive it.
Thus 'make check PARSER=/some/path/to/an/alternate/apparmor_parser'
will run all the parser tests on that binary. This is useful for
running the testsuite in an automated post-install environment.
(It should be noted that doing so will still build and run the unit
test binaries based on the source tree.)
Signed-off-by: Steve Beattie <sbeattie@ubuntu.com>
Acked-By: John Johansen <john.johansen@canonical.com>
Minimization was failing because it was too agressive. It was minimizing
as if there was only 1 accept condition. This allowed it to remove more
states but at the cost of loosing unique permission sets, they where
being combined into single commulative perms. This means that audit,
deny, xtrans, ... info on one path would be applied to all other paths
that it was combined with during minimization.
This means that we need to retain the unique accept states, not allowing
them to be combined into a single state. To do this we put each unique
permission set into its own partition at the start of minimization.
The states within a partition have the same permissions and can be combined
within the other states in the partition as the loss of unique path
information is will not result in a conflict.
This is similar to what perm hashing used to do but deny information is
still being correctly applied and carried.
Signed-off-by: John Johansen <john.johansen@canonical.com>
Acked-By: Steve Beattie <sbeattie@ubuntu.com>
create the generated_* directories themselves if they don't exist before
running the script to generate them.
Also modify the default invocation of prove to add -f, which reports the
details of failing test cases.
Currently apparmor provides the unsafe keyword to indicate an xtransition
is not scrubbing its environment variables. This can be used to be
explicit about which transition are unsafe instead of relying on people
remembering which of px Px is safe or unsafe.
Add the orthogonal keyword safe to allow specifying a transition is
safe.
Signed-off-by: John Johansen <john.johansen@canonical.com>
Test the leading permission form of an xrule against its trailing permission
form, to verify that they are generating the same xtransition and thus
don't conflict (assumes xtransition conflict checking is working).
eg.
px /foo,
/foo px,
should generate the same rule and thus not result in any conflicts
Signed-off-by: John Johansen <john.johansen@canonical.com>
All the combiniation of xtransition conflics where not well represented in
the regression test suite. Instead of relying on multiple static test
files, automatically generate all possible conflicts.
Signed-off-by: John Johansen <john.johansen@canonical.com>
* a non-include related syntax error (errors/modefail.sd)
* multiple successful includes followed by a failed include
(errors/multi_include.sd)
It also fixes two issues with the parser's line counting:
* the count began at 0 (demonstrated by the first testcase's error
being reporting on one line less than it should be), and
* an extra line increment when includes were detected (demonstrated
by the second testcase's error being reported at a line beyond the
correct linenumber.
The existing testcases did not catch these because they were all
based on the first include in the file failing and so the start of
the count from 0 counteracted the extra counted line.
bashisms which were in place to get around PWD not getting set
correctly. Instead, fix the simple.pl script to call pwd directly.
Based on feedback from PLD/Arkadiusz Miskiewicz <arekm@maven.pl>.
- adds a distinct includes/ subdirectory, populated with a couple of
files that test profiles can include. We purposefully choose to use
a different directory structure than the shipped profile set so that
changes to it will not break our tests.
- modifies the existing test profiles to take advantage of the
includes/ directory.
- modifies tst/Makefile with pushd/popd to get around some pwd
limitations in the parser (that need to be fixed).
