utils tests: ignore tests for 'include if exists'
See merge request apparmor/apparmor!78
Acked-by: John Johansen <john.johansen@canonical.com>
(cherry picked from commit e6ef536957)
dc7c7021 utils tests: ignore tests for 'include if exists'
The tools don't support having multiple rules in one line (they expect
\n after each rule), therefore mark some of the bare_include_tests as
known failures.
(cherry picked from commit 26af640fda)
Signed-off-by: John Johansen <john.johansen@canonical.com>
parser/tst/simple_tests/profile/profile_ns_bad8.sd was added in r3376
(trunk) / r3312 (2.10 branch) and contains the profile name ':ns/t'
which misses the terminating ':' for the namespace.
Unfortunately the tools don't understand namespaces yet and just use the
full profile name. This also means this test doesn't fail as expected
when tested against the utils code.
This patch adds profile_ns_bad8.sd to the exception list of
test-parser-simple-tests.py.
Acked-by: Steve Beattie <steve@nxnw.org> for trunk and 2.10.
Parsing variables was broken in several ways:
- empty quotes (representing an intentionally empty value) were lost,
causing parser failures
- items consisting of only one letter were lost due to a bug in RE_VARS
- RE_VARS didn't start with ^, which means leading garbage (= syntax
errors) was ignored
- trailing garbage was also ignored
This patch fixes those issues in separate_vars() and changes
var_transform() to write out empty quotes (instead of nothing) for empty
values.
Also add some tests for separate_vars() with empty quotes and adjust
several tests with invalid syntax to expect an AppArmorException.
var_transform() gets some tests added.
Finally, remove 3 testcases from the "fails to raise an exception" list
in test-parser-simple-tests.py.
Acked-by: John Johansen <john.johansen@canonical.com> for trunk and 2.9
(which also implies 2.10)
Note: 2.9 doesn't have test-parser-simple-tests.py, therefore it won't
get that part of the patch.
Add a testcase that parses all tests in the parser/tst/simple_tests/
directory with parse_profile_data() to ensure that everything with valid
syntax is accepted, and that all tests marked as FAIL raise an
exception.
This already resulted in
- several patches to fix low-hanging fruits (including some bugs in the
parser simple_tests itsself)
- a list of tests that don't behave as expected. Those files get their
expected result reverted to make sure we notice any change in the
tools behaviour, especially changing to the really expected resulted.
This method also makes sure that the testcase doesn't report any of
the known failures.
- a 5% improvement in test coverage - mostly caused by nearly completely
covering parse_profile_data.
- addition of some missing testcased (as noticed by missing coverage),
for example several "rule outside of a profile" testcases.
As indicated above, the tools don't work as expected on all test
profiles - most of the failures happen on expected-to-fail tests that
pass parse_profile_data() without raising an exception. There are also
some tests failing despite valid syntax, often with rarely used syntax
like if conditions and qualifier blocks.
Most of the failing (generated) tests are caused by features not
implemented in the tools yet:
- validating dbus rules (currently we just store them without any parsing)
- checks for conflicting x permissions
- permissions before path ("r /foo,")
- 'safe' and 'unsafe' keywords for *x rules
- 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get
rejected by the tools - ideally the generator script should create
PUx and CUx tests instead)
skip_startswith excludes several generated tests from being run. I know
that skip_startswith also excludes tests that would not fail, but the
generated filenames (especially generated_x/exact-*) don't have a
pattern that I could easily use to exclude less tests - and I'm not too
keen to add a list with 1000 single filenames ;-)
Acked-by: John Johansen <john.johansen@canonical.com>
