mirrors/apparmor
1
0
Fork 0
mirror of https://gitlab.com/apparmor/apparmor.git synced 2025-03-04 16:35:02 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions
apparmor/utils/test/test-parser-simple-tests.py

553 lines
20 KiB
Python
Raw Normal View History

Switch utils to python3 As discussed a while ago, switch the utils (including their tests) to use python3 by default. While on it, drop usage of "env" to always get the system python3 instead of a random one that happens to live somewhere in $PATH. In practise, this patch doesn't change much - AFAIK openSUSE, Debian and Ubuntu already patch aa-* to use python3. Also add a note to README to officially deprecate Python 2.x. (I won't break Python 2.x support intentionally - unless some future change gives me a very good reason to finally drop Python 2.x support.) Acked-by: Seth Arnold <seth.arnold@canonical.com> (since 2016-08-23, but the commit had to wait for the FileRule series because it touches test-file.py)
2016-10-01 20:57:09 +02:00
#! /usr/bin/python3
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
# ------------------------------------------------------------------
#
# Copyright (C) 2015 Christian Boltz <apparmor@cboltz.de>
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of version 2 of the GNU General Public
# License published by the Free Software Foundation.
#
# ------------------------------------------------------------------
Order imports and module-level dunder name assignments.
2022-08-07 20:32:07 -04:00
import os
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
import unittest
Order imports and module-level dunder name assignments.
2022-08-07 20:32:07 -04:00
import apparmor.aa as apparmor
from apparmor.common import AppArmorException, is_skippable_file, open_file_read
from common_test import AATest, setup_aa, setup_all_loops
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
# This testcase will parse all parser/tst/simple_tests with parse_profile_data(),
# except the files listed in one of the arrays below.
#
# Files listed in skip_startswith will be completely skipped.
# Files listed in the other arrays will be checked, but with the opposite of the expected result.
# XXX tests listed here will be *** SKIPPED *** XXX
skip_startswith = (
# the tools don't check for conflicting x permissions (yet?)
'generated_x/conflict-',
'generated_x/ambiguous-',
'generated_x/dominate-',
# 'safe' and 'unsafe' keywords
'generated_perms_safe/',
# Pux and Cux (which actually mean PUx and CUx) get rejected by the tools
'generated_x/exact-',
)
# testcases that should raise an exception, but don't
Speed up list creations, and change lists to tuples where appropriate..
2022-06-18 14:30:49 -04:00
exception_not_raised = (
Add most abi/bad_*.sd tests to "exception not raised" list Interestingly, abi/bad_6.sd is detected as invalid, and therefore not added to the list.
2018-10-13 20:23:57 +02:00
# most abi/bad_* aren't detected as bad by the basic implementation in the tools
'abi/bad_10.sd',
'abi/bad_11.sd',
'abi/bad_12.sd',
# invalid capabilities (like "foobar"), but syntactically correct
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
'capability/bad_1.sd',
'capability/bad_2.sd',
'capability/bad_3.sd',
'capability/bad_4.sd',
Add most abi/bad_*.sd tests to "exception not raised" list Interestingly, abi/bad_6.sd is detected as invalid, and therefore not added to the list.
2018-10-13 20:23:57 +02:00
# interesting[tm] profile name
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
'change_hat/bad_parsing.sd',
utils: Handle the safe/unsafe change_profile exec modes https://launchpad.net/bugs/1584069 This patch adds support for the safe and unsafe exec modes for change_profile rules. The logic is pretty simple at this point because the kernel's default for exec modes changed in newer versions. Therefore, this patch simply retains any specified exec mode in parsed rules. If an exec mode is not specified in a rule, there is no attempt to force the usage of "safe" because older kernels do not support it. Signed-off-by: Tyler Hicks <tyhicks@canonical.com> Acked-by: Seth Arnold <seth.arnold@canonical.com> Acked-by: Christian Boltz <apparmor@cboltz.de>
2016-07-20 17:24:11 -05:00
# The tools don't detect conflicting change_profile exec modes
'change_profile/onx_conflict_unsafe1.sd',
'change_profile/onx_conflict_unsafe2.sd',
Ignore test failures about duplicated conditionals in dbus rules Since r3634, the tools allow any order of dbus conditionals. Quoting the r3634 patch description: This patch eases the restriction on the ordering at the expense of the utils no longer being able to detect and reject a single attribute that is repeated multiple times. In that situation, only the last occurrence of the attribute will be honored by the utils. It seems nobody tested with all test profiles generated ;-) so we have to add some exceptions to the "does not raise an exception" list now. Acked-by <timeout> for trunk and 2.11
2017-04-20 13:05:53 +02:00
# duplicated conditionals aren't detected by the tools
'generated_dbus/duplicated-conditionals-45127.sd',
'generated_dbus/duplicated-conditionals-45131.sd',
'generated_dbus/duplicated-conditionals-45124.sd',
'generated_dbus/duplicated-conditionals-45130.sd',
'generated_dbus/duplicated-conditionals-45125.sd',
'generated_dbus/duplicated-conditionals-45128.sd',
'generated_dbus/duplicated-conditionals-45129.sd',
utils: Don't enforce ordering of dbus rule attributes https://launchpad.net/bugs/1628286 The utils were enforcing that the dbus rule attributes were strictly ordered in the following fashion: bus -> path -> interface -> member -> peer However, the parser has always accepted the attributes in any order. If the system contained a profile which did not use the strict ordering enforced by the utils, the utils would refuse to operate at all. This patch eases the restriction on the ordering at the expense of the utils no longer being able to detect and reject a single attribute that is repeated multiple times. In that situation, only the last occurrence of the attribute will be honored by the utils. Signed-off-by: Tyler Hicks <tyhicks@canonical.com> Acked-by: Christian Boltz <apparmor@cboltz.de>
2017-02-28 23:04:24 +00:00
'dbus/bad_modifier_2.sd',
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
'dbus/bad_regex_01.sd',
'dbus/bad_regex_02.sd',
'dbus/bad_regex_03.sd',
'dbus/bad_regex_04.sd',
'dbus/bad_regex_05.sd',
'dbus/bad_regex_06.sd',
[14/38] Use FileRule and FileRuleset Change aa.py to use FileRule and FileRuleset for parsing and saving profiles. In detail, this means: - add 'file' to the list of rule classes to enable it at various places - store file rules in aa[profile][hat]['file'] (not 'path' as before) to be consistent with the FileRule name - drop the no longer needed delete_path_duplicates() - this is now handled by FileRuleset like in all other rule classes. (same change in cleanprofile.py) - replace usage of RE_PROFILE_BARE_FILE_ENTRY and RE_PROFILE_PATH_ENTRY with FileRule.match() - drop write_path_rules() and write_paths() and replace them with the new write_file() function. - adjust several code sections to use write_file() and 'file' instead of 'path' FileRule doesn't drop optional keywords ('allow' and 'file'), therefore adjust cleanprof_test.out to the changed behaviour. (If someone insists on dropping optional keywords in aa-cleanprof, that's something for a future patch.) Also adjust the list of known failures in test-parser-simple-tests.py - switching to FileRule avoids several test failures (and introduces a few new ones ;-) IMPORTANT: This patch introduces a "brain split" which means - parsing and writing the profile and aa-cleanprof use the new location (aa[profile][hat]['file']) - aa-logprof and aa-genprof still save data to the old location (aa[profile][hat]['allow']['path']) and probably ask superfluous questions because there are no rules existing in the old location TL;DR: don't try aa-logprof or aa-genprof with only this patch applied. I know this isn't ideal, but still better than an even bigger and totally unreadable patch ;-) Acked-by: Steve Beattie <steve@nxnw.org>
2016-10-01 19:54:48 +02:00
'file/bad_re_brace_1.sd',
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
'file/bad_re_brace_2.sd',
'file/bad_re_brace_3.sd',
parser: simple_tests: mount: mark ok_[16-19] tests as bad These tests contains incompatible mount options and broken after ("parser: add conflicting flags check for options= conditionals") Signed-off-by: Alexander Mikhalitsyn <aleksandr.mikhalitsyn@canonical.com> Acked-by: John Johansen <john.johansen@canonical.com>
2023-03-23 18:56:48 +01:00
'mount/bad_1.sd',
'mount/bad_2.sd',
'mount/bad_3.sd',
'mount/bad_4.sd',
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
'mount/bad_opt_10.sd',
'mount/bad_opt_11.sd',
'mount/bad_opt_12.sd',
'mount/bad_opt_13.sd',
'mount/bad_opt_14.sd',
'mount/bad_opt_15.sd',
'mount/bad_opt_16.sd',
'mount/bad_opt_17.sd',
'mount/bad_opt_18.sd',
'mount/bad_opt_19.sd',
'mount/bad_opt_1.sd',
'mount/bad_opt_20.sd',
'mount/bad_opt_21.sd',
'mount/bad_opt_22.sd',
'mount/bad_opt_23.sd',
'mount/bad_opt_24.sd',
'mount/bad_opt_2.sd',
'mount/bad_opt_3.sd',
'mount/bad_opt_4.sd',
'mount/bad_opt_5.sd',
'mount/bad_opt_6.sd',
'mount/bad_opt_7.sd',
'mount/bad_opt_8.sd',
'mount/bad_opt_9.sd',
parser sanity tests: add more mount options tests Attempt to get clarity on what is valid syntax for mount options and fstype options. Note that simple_tests/mount/bad_opt_27.sd is marked TODO, as the parser accepts it but should not. Also mark the tests as expecting to fail to raise an exception by the python utils. Signed-off-by: Steve Beattie <steve.beattie@canonical.com> MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/607 Signed-off-by: John Johansen <john.johansen@canonical.com>
2020-09-28 19:42:06 -07:00
'mount/bad_opt_25.sd',
'mount/bad_opt_26.sd',
'mount/bad_opt_27.sd',
'mount/bad_opt_28.sd',
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
'profile/flags/flags_bad10.sd',
'profile/flags/flags_bad11.sd',
'profile/flags/flags_bad12.sd',
'profile/flags/flags_bad13.sd',
'profile/flags/flags_bad15.sd',
'profile/flags/flags_bad18.sd',
'profile/flags/flags_bad19.sd',
'profile/flags/flags_bad20.sd',
'profile/flags/flags_bad2.sd',
'profile/flags/flags_bad3.sd',
'profile/flags/flags_bad4.sd',
'profile/flags/flags_bad5.sd',
'profile/flags/flags_bad6.sd',
'profile/flags/flags_bad7.sd',
'profile/flags/flags_bad8.sd',
'profile/flags/flags_bad_debug_1.sd',
'profile/flags/flags_bad_debug_2.sd',
'profile/flags/flags_bad_debug_3.sd',
'profile/flags/flags_bad_debug_4.sd',
parser: support enforce, kill and unconfined profile modes The enforce profile mode is the default but specifying it explicitly has not been supported. Allow enforce to be specified as a mode. If no mode is specified the default is still enforce. The kernel has supported kill and unconfined profile modes for a long time now. And support to the parser so that profiles can make use of these modes. MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/440 Fixes: https://gitlab.com/apparmor/apparmor/-/issues/7 Signed-off-by: John Johansen <john.johansen@canonical.com> Acked-by: Steve Beattie <sbeattie@ubuntu.com>
2018-07-04 09:26:27 -07:00
# detection of conflicting flags not supported
'profile/flags/flags_bad30.sd',
'profile/flags/flags_bad31.sd',
'profile/flags/flags_bad32.sd',
'profile/flags/flags_bad33.sd',
'profile/flags/flags_bad34.sd',
'profile/flags/flags_bad35.sd',
'profile/flags/flags_bad36.sd',
'profile/flags/flags_bad37.sd',
'profile/flags/flags_bad38.sd',
'profile/flags/flags_bad39.sd',
'profile/flags/flags_bad40.sd',
'profile/flags/flags_bad41.sd',
'profile/flags/flags_bad42.sd',
'profile/flags/flags_bad43.sd',
'profile/flags/flags_bad44.sd',
'profile/flags/flags_bad45.sd',
'profile/flags/flags_bad46.sd',
Add simple_tests/profile/profile_ns_bad8.sd to utils test exception list parser/tst/simple_tests/profile/profile_ns_bad8.sd was added in r3376 (trunk) / r3312 (2.10 branch) and contains the profile name ':ns/t' which misses the terminating ':' for the namespace. Unfortunately the tools don't understand namespaces yet and just use the full profile name. This also means this test doesn't fail as expected when tested against the utils code. This patch adds profile_ns_bad8.sd to the exception list of test-parser-simple-tests.py. Acked-by: Steve Beattie <steve@nxnw.org> for trunk and 2.10.
2016-02-19 00:22:59 +01:00
'profile/profile_ns_bad8.sd', # 'profile :ns/t' without terminating ':'
Use PtraceRule Change aa.py to use PtraceRule and PtraceRuleset in profile_storage(), parse_profile_data() and write_ptrace(). This also means we can drop the now unused parse_ptrace_rule() and write_ptrace_rules() functions. Raw_Ptrace_Rule in rules.py is now also unused and can be dropped. Also adjust logparser.py to include the peer in the result, and shorten the list of known-failing tests in test-parser-simple-tests.py. Acked-by: John Johansen <john.johansen@canonical.com>
2015-12-27 01:20:37 +01:00
'ptrace/bad_05.sd', # actually contains a capability rule with invalid (ptrace-related) keyword
'ptrace/bad_06.sd', # actually contains a capability rule with invalid (ptrace-related) keyword
'ptrace/bad_10.sd', # peer with invalid regex
Change aa.py to use SignalRule and SignalRuleset This means: - import the classes instead of RE_PROFILE_SIGNAL - simplify signal rule parsing a lot - drop the (now unused) functions parse_signal_rule() and write_signal_rules() - change write_signal() to use the SignalRuleset class Also drop the now unused Raw_Signal_Rule from rules.py. Finally, drop most parser signal tests from the "known wrong results" blacklist in test-parser-simple-tests.py because those tests succeed with SignalRule. Acked-by: Kshitij Gupta <kgupta8592@gmail.com>
2015-11-24 00:09:37 +01:00
'signal/bad_21.sd', # invalid regex
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
'unix/bad_attr_1.sd',
'unix/bad_attr_2.sd',
'unix/bad_attr_3.sd',
'unix/bad_attr_4.sd',
'unix/bad_bind_1.sd',
'unix/bad_bind_2.sd',
'unix/bad_create_1.sd',
'unix/bad_create_2.sd',
'unix/bad_listen_1.sd',
'unix/bad_listen_2.sd',
'unix/bad_modifier_1.sd',
'unix/bad_modifier_2.sd',
'unix/bad_modifier_3.sd',
'unix/bad_modifier_4.sd',
'unix/bad_opt_1.sd',
'unix/bad_opt_2.sd',
'unix/bad_opt_3.sd',
'unix/bad_opt_4.sd',
'unix/bad_peer_1.sd',
'unix/bad_regex_01.sd',
'unix/bad_regex_02.sd',
'unix/bad_regex_03.sd',
'unix/bad_regex_04.sd',
'unix/bad_shutdown_1.sd',
'unix/bad_shutdown_2.sd',
parser: add support for autobind sockets af_unix allows for sockets to be bound to a name that is autogenerated. Currently this type of binding is only supported by a very generic rule. unix (bind) type=dgram, but this allows both sockets with specified names and anonymous sockets. Extend unix rule syntax to support specifying just an auto bind socket by specifying addr=auto eg. unix (bind) addr=auto, It is important to note that addr=auto only works for the bind permission as once the socket is bound to an autogenerated address, the addr with have a valid unique value that can be matched against with a regular addr=@name expression Fixes: https://bugs.launchpad.net/apparmor/+bug/1867216 MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/521 Signed-off-by: John Johansen <john.johansen@canonical.com>
2020-09-10 13:51:59 -07:00
'unix/bad_peer_2.sd',
'unix/bad_attr_5.sd',
'unix/bad_opt_5.sd',
'unix/bad_shutdown_3.sd',
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
'vars/vars_bad_3.sd',
'vars/vars_bad_4.sd',
'vars/vars_bad_5.sd',
'vars/vars_dbus_bad_01.sd',
'vars/vars_dbus_bad_02.sd',
'vars/vars_dbus_bad_03.sd',
utils: Don't enforce ordering of dbus rule attributes https://launchpad.net/bugs/1628286 The utils were enforcing that the dbus rule attributes were strictly ordered in the following fashion: bus -> path -> interface -> member -> peer However, the parser has always accepted the attributes in any order. If the system contained a profile which did not use the strict ordering enforced by the utils, the utils would refuse to operate at all. This patch eases the restriction on the ordering at the expense of the utils no longer being able to detect and reject a single attribute that is repeated multiple times. In that situation, only the last occurrence of the attribute will be honored by the utils. Signed-off-by: Tyler Hicks <tyhicks@canonical.com> Acked-by: Christian Boltz <apparmor@cboltz.de>
2017-02-28 23:04:24 +00:00
'vars/vars_dbus_bad_04.sd',
'vars/vars_dbus_bad_05.sd',
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
'vars/vars_dbus_bad_06.sd',
'vars/vars_dbus_bad_07.sd',
'vars/vars_file_evaluation_8.sd',
# profile name in var doesn't start with /
'vars/vars_profile_name_bad_1.sd',
# profile name is undefined variable
'vars/vars_profile_name_bad_2.sd',
'vars/vars_profile_name_23.sd',
# attachment in var doesn't start with /
'vars/vars_profile_name_07.sd',
'vars/vars_profile_name_08.sd',
'vars/vars_profile_name_12.sd',
'vars/vars_profile_name_13.sd',
'vars/vars_profile_name_15.sd',
'vars/vars_profile_name_22.sd',
'vars/vars_recursion_1.sd',
'vars/vars_recursion_2.sd',
'vars/vars_recursion_3.sd',
'vars/vars_recursion_4.sd',
'vars/vars_simple_assignment_3.sd',
'vars/vars_simple_assignment_8.sd',
'vars/vars_simple_assignment_9.sd',
'xtrans/simple_bad_conflicting_x_10.sd',
'xtrans/simple_bad_conflicting_x_11.sd',
'xtrans/simple_bad_conflicting_x_12.sd',
'xtrans/simple_bad_conflicting_x_13.sd',
'xtrans/simple_bad_conflicting_x_2.sd',
'xtrans/simple_bad_conflicting_x_4.sd',
'xtrans/simple_bad_conflicting_x_6.sd',
'xtrans/simple_bad_conflicting_x_8.sd',
'xtrans/x-conflict.sd',
Speed up list creations, and change lists to tuples where appropriate..
2022-06-18 14:30:49 -04:00
)
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
# testcases with lines that don't match any regex and end up as "unknown line"
Speed up list creations, and change lists to tuples where appropriate..
2022-06-18 14:30:49 -04:00
unknown_line = (
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
# 'other' keyword
'file/allow/ok_other_1.sd',
'file/allow/ok_other_2.sd',
'file/ok_other_1.sd',
'file/ok_other_2.sd',
'file/ok_other_3.sd',
[14/38] Use FileRule and FileRuleset Change aa.py to use FileRule and FileRuleset for parsing and saving profiles. In detail, this means: - add 'file' to the list of rule classes to enable it at various places - store file rules in aa[profile][hat]['file'] (not 'path' as before) to be consistent with the FileRule name - drop the no longer needed delete_path_duplicates() - this is now handled by FileRuleset like in all other rule classes. (same change in cleanprofile.py) - replace usage of RE_PROFILE_BARE_FILE_ENTRY and RE_PROFILE_PATH_ENTRY with FileRule.match() - drop write_path_rules() and write_paths() and replace them with the new write_file() function. - adjust several code sections to use write_file() and 'file' instead of 'path' FileRule doesn't drop optional keywords ('allow' and 'file'), therefore adjust cleanprof_test.out to the changed behaviour. (If someone insists on dropping optional keywords in aa-cleanprof, that's something for a future patch.) Also adjust the list of known failures in test-parser-simple-tests.py - switching to FileRule avoids several test failures (and introduces a few new ones ;-) IMPORTANT: This patch introduces a "brain split" which means - parsing and writing the profile and aa-cleanprof use the new location (aa[profile][hat]['file']) - aa-logprof and aa-genprof still save data to the old location (aa[profile][hat]['allow']['path']) and probably ask superfluous questions because there are no rules existing in the old location TL;DR: don't try aa-logprof or aa-genprof with only this patch applied. I know this isn't ideal, but still better than an even bigger and totally unreadable patch ;-) Acked-by: Steve Beattie <steve@nxnw.org>
2016-10-01 19:54:48 +02:00
# 'unsafe' keyword
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
'file/file/front_perms_ok_1.sd',
'file/front_perms_ok_1.sd',
'xtrans/simple_ok_cx_1.sd',
# permissions before path and owner / audit {...} blocks
'file/file/owner/ok_1.sd',
'file/owner/ok_1.sd',
'profile/entry_mods_audit_ok1.sd',
# namespace
'profile/profile_ns_named_ok1.sd', # profile keyword?
'profile/profile_ns_named_ok2.sd', # profile keyword?
'profile/profile_ns_named_ok3.sd', # profile keyword?
'profile/profile_ns_ok1.sd',
'profile/profile_ns_ok2.sd',
'profile/profile_ns_ok3.sd', # profile keyword?
'profile/re_named_ok4.sd', # profile keyword
'profile/re_ok4.sd',
tools: Mark profiles with multiple rules in one line as known-failing The tools don't support having multiple rules in one line (they expect \n after each rule), therefore mark some of the bare_include_tests as known failures.
2018-01-23 22:40:07 +01:00
# multiple rules in one line
'bare_include_tests/ok_14.sd',
'bare_include_tests/ok_19.sd',
'bare_include_tests/ok_64.sd',
'bare_include_tests/ok_69.sd',
utils tests: ignore tests for 'include if exists' ... and some exotic includes that are not supported by the tools yet
2018-03-08 20:42:57 +01:00
Better descriptions why some example profiles fail with the tools
2020-05-03 22:07:25 +02:00
# include with quoted relative path
utils tests: ignore tests for 'include if exists' ... and some exotic includes that are not supported by the tools yet
2018-03-08 20:42:57 +01:00
'bare_include_tests/ok_11.sd',
'bare_include_tests/ok_12.sd',
'bare_include_tests/ok_13.sd',
'bare_include_tests/ok_15.sd',
Better descriptions why some example profiles fail with the tools
2020-05-03 22:07:25 +02:00
# include with unquoted relative path
utils tests: ignore tests for 'include if exists' ... and some exotic includes that are not supported by the tools yet
2018-03-08 20:42:57 +01:00
'bare_include_tests/ok_16.sd',
'bare_include_tests/ok_17.sd',
'bare_include_tests/ok_18.sd',
'bare_include_tests/ok_20.sd',
Better descriptions why some example profiles fail with the tools
2020-05-03 22:07:25 +02:00
# include with quoted relative path with spaces
utils tests: ignore tests for 'include if exists' ... and some exotic includes that are not supported by the tools yet
2018-03-08 20:42:57 +01:00
'bare_include_tests/ok_26.sd',
'bare_include_tests/ok_27.sd',
Better descriptions why some example profiles fail with the tools
2020-05-03 22:07:25 +02:00
# include with quoted magic path with spaces
utils tests: ignore tests for 'include if exists' ... and some exotic includes that are not supported by the tools yet
2018-03-08 20:42:57 +01:00
'bare_include_tests/ok_28.sd',
'bare_include_tests/ok_29.sd',
Better descriptions why some example profiles fail with the tools
2020-05-03 22:07:25 +02:00
# include with magic path with spaces
utils tests: ignore tests for 'include if exists' ... and some exotic includes that are not supported by the tools yet
2018-03-08 20:42:57 +01:00
'bare_include_tests/ok_30.sd',
'bare_include_tests/ok_31.sd',
Better descriptions why some example profiles fail with the tools
2020-05-03 22:07:25 +02:00
# include if exists with quoted relative path
utils tests: ignore tests for 'include if exists' ... and some exotic includes that are not supported by the tools yet
2018-03-08 20:42:57 +01:00
'bare_include_tests/ok_61.sd',
'bare_include_tests/ok_62.sd',
'bare_include_tests/ok_63.sd',
Better descriptions why some example profiles fail with the tools
2020-05-03 22:07:25 +02:00
# include if exists with unquoted relative path
utils tests: ignore tests for 'include if exists' ... and some exotic includes that are not supported by the tools yet
2018-03-08 20:42:57 +01:00
'bare_include_tests/ok_65.sd',
'bare_include_tests/ok_66.sd',
'bare_include_tests/ok_67.sd',
'bare_include_tests/ok_68.sd',
'bare_include_tests/ok_70.sd',
Better descriptions why some example profiles fail with the tools
2020-05-03 22:07:25 +02:00
# include if exists with quoted relative path with spaces
utils tests: ignore tests for 'include if exists' ... and some exotic includes that are not supported by the tools yet
2018-03-08 20:42:57 +01:00
'bare_include_tests/ok_76.sd',
'bare_include_tests/ok_77.sd',
Better descriptions why some example profiles fail with the tools
2020-05-03 22:07:25 +02:00
# include if exists with quoted magic path with spaces
utils tests: ignore tests for 'include if exists' ... and some exotic includes that are not supported by the tools yet
2018-03-08 20:42:57 +01:00
'bare_include_tests/ok_78.sd',
'bare_include_tests/ok_79.sd',
Better descriptions why some example profiles fail with the tools
2020-05-03 22:07:25 +02:00
# include if exists with unquoted magic path with spaces
utils tests: ignore tests for 'include if exists' ... and some exotic includes that are not supported by the tools yet
2018-03-08 20:42:57 +01:00
'bare_include_tests/ok_80.sd',
'bare_include_tests/ok_81.sd',
Better descriptions why some example profiles fail with the tools
2020-05-03 22:07:25 +02:00
# include if exists with quoted relative path, non-existing include file
utils tests: ignore tests for 'include if exists' ... and some exotic includes that are not supported by the tools yet
2018-03-08 20:42:57 +01:00
'bare_include_tests/ok_82.sd',
'bare_include_tests/ok_84.sd',
'bare_include_tests/ok_85.sd',
'bare_include_tests/ok_86.sd',
Speed up list creations, and change lists to tuples where appropriate..
2022-06-18 14:30:49 -04:00
)
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
# testcases with various unexpected failures
Speed up list creations, and change lists to tuples where appropriate..
2022-06-18 14:30:49 -04:00
syntax_failure = (
Better descriptions why some example profiles fail with the tools
2020-05-03 22:07:25 +02:00
# missing profile keywords
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
'profile/re_named_ok2.sd',
# Syntax Errors caused by boolean conditions (parse_profile_data() gets confused by the closing '}')
'conditional/defined_1.sd',
'conditional/defined_2.sd',
'conditional/else_1.sd',
'conditional/else_2.sd',
'conditional/else_3.sd',
'conditional/else_if_1.sd',
'conditional/else_if_2.sd',
'conditional/else_if_3.sd',
'conditional/else_if_5.sd',
'conditional/ok_1.sd',
'conditional/ok_2.sd',
'conditional/ok_3.sd',
'conditional/ok_4.sd',
'conditional/ok_5.sd',
'conditional/ok_6.sd',
'conditional/ok_7.sd',
'conditional/ok_8.sd',
'conditional/ok_9.sd',
'conditional/stress_1.sd',
# unexpected uppercase vs. lowercase in *x rules
'file/ok_5.sd', # Invalid mode UX
'file/ok_2.sd', # Invalid mode RWM
'file/ok_4.sd', # Invalid mode iX
test-parser-simple-tests.py: No longer skip testing generated_perms_leading profiles FileRule understands leading permissions, so the reason to skip those (generated) test profiles in test-parser-simple-tests.py is gone. However, the gen-xtrans.pl script generates profiles with a not-so-valid mix of uppercase and lowercase, for example "Pux" and "Cux". The parser accepts this, but the tools complain about such rules. Therefore add the affected profiles to the exception list. In total, this means we now test 319 of the 380 generated_perms_leading test profiles. The patch also moves some lines around to get the \-escaped profiles out of the mixed uppercase/lowercase exec rule section. Acked-by: Seth Arnold <seth.arnold@canonical.com>
2017-03-03 13:14:03 +01:00
'xtrans/simple_ok_pix_1.sd', # Invalid mode pIx
'xtrans/simple_ok_pux_1.sd', # Invalid mode rPux
# unexpected uppercase vs. lowercase in *x rules - generated_perms_leading directory
'generated_perms_leading/exact-re-Puxtarget.sd',
'generated_perms_leading/dominate-ownerCuxtarget2.sd',
'generated_perms_leading/ambiguous-Cux.sd',
'generated_perms_leading/dominate-ownerPux.sd',
'generated_perms_leading/exact-re-ownerPux.sd',
'generated_perms_leading/overlap-ownerCuxtarget.sd',
'generated_perms_leading/exact-re-ownerCuxtarget.sd',
'generated_perms_leading/dominate-Puxtarget2.sd',
'generated_perms_leading/dominate-ownerCuxtarget.sd',
'generated_perms_leading/dominate-ownerPuxtarget.sd',
'generated_perms_leading/ambiguous-Pux.sd',
'generated_perms_leading/ambiguous-Cuxtarget2.sd',
'generated_perms_leading/exact-Puxtarget2.sd',
'generated_perms_leading/ambiguous-ownerCux.sd',
'generated_perms_leading/exact-ownerPux.sd',
'generated_perms_leading/ambiguous-ownerPuxtarget.sd',
'generated_perms_leading/exact-re-ownerPuxtarget.sd',
'generated_perms_leading/exact-re-Cuxtarget.sd',
'generated_perms_leading/exact-re-Puxtarget2.sd',
'generated_perms_leading/dominate-Cux.sd',
'generated_perms_leading/exact-re-ownerCuxtarget2.sd',
'generated_perms_leading/ambiguous-ownerCuxtarget.sd',
'generated_perms_leading/exact-re-Cuxtarget2.sd',
'generated_perms_leading/ambiguous-Puxtarget.sd',
'generated_perms_leading/overlap-Puxtarget.sd',
'generated_perms_leading/ambiguous-Puxtarget2.sd',
'generated_perms_leading/overlap-Puxtarget2.sd',
'generated_perms_leading/exact-Puxtarget.sd',
'generated_perms_leading/overlap-ownerPuxtarget.sd',
'generated_perms_leading/exact-ownerCuxtarget.sd',
'generated_perms_leading/exact-re-ownerCux.sd',
'generated_perms_leading/exact-ownerPuxtarget2.sd',
'generated_perms_leading/exact-ownerCux.sd',
'generated_perms_leading/overlap-Cuxtarget2.sd',
'generated_perms_leading/ambiguous-Cuxtarget.sd',
'generated_perms_leading/ambiguous-ownerPuxtarget2.sd',
'generated_perms_leading/dominate-ownerCux.sd',
'generated_perms_leading/exact-Pux.sd',
'generated_perms_leading/exact-Cuxtarget.sd',
'generated_perms_leading/overlap-ownerCuxtarget2.sd',
'generated_perms_leading/overlap-Pux.sd',
'generated_perms_leading/overlap-ownerPux.sd',
'generated_perms_leading/ambiguous-ownerCuxtarget2.sd',
'generated_perms_leading/exact-re-Cux.sd',
'generated_perms_leading/exact-re-Pux.sd',
'generated_perms_leading/overlap-Cuxtarget.sd',
'generated_perms_leading/exact-re-ownerPuxtarget2.sd',
'generated_perms_leading/exact-Cuxtarget2.sd',
'generated_perms_leading/exact-Cux.sd',
'generated_perms_leading/overlap-Cux.sd',
'generated_perms_leading/overlap-ownerCux.sd',
'generated_perms_leading/exact-ownerPuxtarget.sd',
'generated_perms_leading/dominate-Pux.sd',
'generated_perms_leading/exact-ownerCuxtarget2.sd',
'generated_perms_leading/dominate-Puxtarget.sd',
'generated_perms_leading/ambiguous-ownerPux.sd',
'generated_perms_leading/overlap-ownerPuxtarget2.sd',
'generated_perms_leading/dominate-Cuxtarget2.sd',
'generated_perms_leading/dominate-Cuxtarget.sd',
'generated_perms_leading/dominate-ownerPuxtarget2.sd',
# escaping with \
[14/38] Use FileRule and FileRuleset Change aa.py to use FileRule and FileRuleset for parsing and saving profiles. In detail, this means: - add 'file' to the list of rule classes to enable it at various places - store file rules in aa[profile][hat]['file'] (not 'path' as before) to be consistent with the FileRule name - drop the no longer needed delete_path_duplicates() - this is now handled by FileRuleset like in all other rule classes. (same change in cleanprofile.py) - replace usage of RE_PROFILE_BARE_FILE_ENTRY and RE_PROFILE_PATH_ENTRY with FileRule.match() - drop write_path_rules() and write_paths() and replace them with the new write_file() function. - adjust several code sections to use write_file() and 'file' instead of 'path' FileRule doesn't drop optional keywords ('allow' and 'file'), therefore adjust cleanprof_test.out to the changed behaviour. (If someone insists on dropping optional keywords in aa-cleanprof, that's something for a future patch.) Also adjust the list of known failures in test-parser-simple-tests.py - switching to FileRule avoids several test failures (and introduces a few new ones ;-) IMPORTANT: This patch introduces a "brain split" which means - parsing and writing the profile and aa-cleanprof use the new location (aa[profile][hat]['file']) - aa-logprof and aa-genprof still save data to the old location (aa[profile][hat]['allow']['path']) and probably ask superfluous questions because there are no rules existing in the old location TL;DR: don't try aa-logprof or aa-genprof with only this patch applied. I know this isn't ideal, but still better than an even bigger and totally unreadable patch ;-) Acked-by: Steve Beattie <steve@nxnw.org>
2016-10-01 19:54:48 +02:00
'file/ok_embedded_spaces_4.sd', # \-escaped space
'file/file/ok_embedded_spaces_4.sd', # \-escaped space
'file/ok_quoted_4.sd', # quoted string including \"
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
# misc
[6/9] Use DbusRule and DbusRuleset Change aa.py to use DbusRule and DbusRuleset in profile_storage, parse_profile_data() and write_dbus. This also means we can drop the now unused parse_dbus_rule() and write_dbus_rules() functions. Raw_DBUS_Rule in rules.py is now also unused and can be dropped. Also shorten the list of known-failing tests in test-parser-simple-tests.py. Even if the list of removals doesn't look too long, the generated_dbus/* removals mean 1989 tests now cause the expected failures. OTOH, I had to add 4 tests to the known-failing list: - 3 tests with a "wrong" order of the conditionals which the parser accepts (which is slightly surprising, because usually we enforce the order of rule parts) - one test fails because the path in the path= conditional doesn't start with / or a variable. Instead, it starts with an alternation, which wouldn't be allowed in file rules. Those 4 failures need more investigation, but shouldn't block this patchset. Finally, adjust test-regex_matches.py to import RE_PROFILE_DBUS from apparmor.regex instead of apparmor.aa. Acked-by: Seth Arnold <seth.arnold@canonical.com>
2016-05-23 23:17:37 +02:00
'vars/vars_dbus_8.sd', # Path doesn't start with / or variable: {/@{TLDS}/foo,/com/@{DOMAINS}}
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
'vars/vars_simple_assignment_12.sd', # Redefining existing variable @{BAR} ('\' not handled)
'bare_include_tests/ok_2.sd', # two #include<...> in one line
Speed up list creations, and change lists to tuples where appropriate..
2022-06-18 14:30:49 -04:00
)
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
Fix most PEP 8 whitespace, indentation, and major line length violations.
2022-08-07 12:26:24 -04:00
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
class TestParseParserTests(AATest):
tests = [] # filled by parse_test_profiles()
def _run_test(self, params, expected):
with open_file_read(params['file']) as f_in:
data = f_in.readlines()
if params['disabled']:
# skip disabled testcases
return
if params['tools_wrong']:
# if the tools are marked as being wrong about a profile, expect the opposite result
# this makes sure we notice any behaviour change, especially not being wrong anymore
expected = not expected
Check variable errors when parsing simple_tests ... by calling active_profiles.get_all_merged_variables() Also remove vars/vars_bad_add_assignment_1.sd from the exception_not_raised list again - now it raises an exception as expected.
2020-05-23 22:50:20 +02:00
# make sure the profile is known in active_profiles.files
apparmor.active_profiles.init_file(params['file'])
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
if expected:
Add in_preamble parameter to profile loading/parsing functions in_preamble keeps track of the current parsing position. It's True while parsing the preamble of a profile file, and when loading an include file that is included in the preamble (typically tunables/*). While inside a profile or parsing abstractions/*, it is False. This commit only hands the information around and keeps in_preamble updated, but it doesn't really get used yet. Also adjust the tests to hand over the additional parameter to parse_profile_data().
2021-02-22 22:22:25 +01:00
apparmor.parse_profile_data(data, params['file'], 0, True)
Drop profile_dir parameter from ProfileList get_all_merged_variables() This parameter is superfluous and unused since some commits. Also adjust all callers.
2020-06-01 23:24:22 +02:00
apparmor.active_profiles.get_all_merged_variables(params['file'], apparmor.include_list_recursive(apparmor.active_profiles.files[params['file']]))
Check variable errors when parsing simple_tests ... by calling active_profiles.get_all_merged_variables() Also remove vars/vars_bad_add_assignment_1.sd from the exception_not_raised list again - now it raises an exception as expected.
2020-05-23 22:50:20 +02:00
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
else:
with self.assertRaises(AppArmorException):
Add in_preamble parameter to profile loading/parsing functions in_preamble keeps track of the current parsing position. It's True while parsing the preamble of a profile file, and when loading an include file that is included in the preamble (typically tunables/*). While inside a profile or parsing abstractions/*, it is False. This commit only hands the information around and keeps in_preamble updated, but it doesn't really get used yet. Also adjust the tests to hand over the additional parameter to parse_profile_data().
2021-02-22 22:22:25 +01:00
apparmor.parse_profile_data(data, params['file'], 0, True)
Drop profile_dir parameter from ProfileList get_all_merged_variables() This parameter is superfluous and unused since some commits. Also adjust all callers.
2020-06-01 23:24:22 +02:00
apparmor.active_profiles.get_all_merged_variables(params['file'], apparmor.include_list_recursive(apparmor.active_profiles.files[params['file']]))
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
Fix most PEP 8 whitespace, indentation, and major line length violations.
2022-08-07 12:26:24 -04:00
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
def parse_test_profiles(file_with_path):
Use triple double-quoted strings for docstrings.
2022-08-07 14:57:30 -04:00
"""parse the test-related headers of a profile (for example EXRESULT) and add the profile to the set of tests"""
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
exresult = None
exresult_found = False
description = None
todo = False
disabled = False
with open_file_read(file_with_path) as f_in:
data = f_in.readlines()
relfile = os.path.relpath(file_with_path, apparmor.profile_dir)
for line in data:
if line.startswith('#=EXRESULT '):
exresult = line.split()[1]
if exresult == 'PASS':
Set (instead of compare) exresult Interestingly this accidentally worked because `if exresult` is true for both a non-empty string ("PASS") as well as a real `True` value. Found by Mark Grassi as part of https://gitlab.com/apparmor/apparmor/-/merge_requests/906
2022-08-14 12:33:56 +02:00
exresult = True
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
exresult_found = True
elif exresult == 'FAIL':
exresult = False
exresult_found = True
else:
Change string formatting method in Python tests
2023-02-19 16:26:14 -05:00
raise Exception('{} contains unknown EXRESULT {}'.format(file_with_path, exresult))
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
elif line.upper().startswith('#=DESCRIPTION '):
description = line.split()[1]
elif line.rstrip() == '#=TODO':
todo = True
elif line.rstrip() == '#=DISABLED':
disabled = True
if not exresult_found:
Change string formatting method in Python tests
2023-02-19 16:26:14 -05:00
raise Exception(file_with_path + ' does not contain EXRESULT')
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
if not description:
Change string formatting method in Python tests
2023-02-19 16:26:14 -05:00
raise Exception(file_with_path + ' does not contain description')
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
tools_wrong = False
if relfile in exception_not_raised:
if exresult:
Change string formatting method in Python tests
2023-02-19 16:26:14 -05:00
raise Exception(file_with_path + " listed in exception_not_raised, but has EXRESULT PASS")
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
tools_wrong = 'EXCEPTION_NOT_RAISED'
elif relfile.startswith(skip_startswith):
return 1 # XXX *** SKIP *** those tests
elif relfile in unknown_line:
if not exresult:
Change string formatting method in Python tests
2023-02-19 16:26:14 -05:00
raise Exception(file_with_path + " listed in unknown_line, but has EXRESULT FAIL")
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
tools_wrong = 'UNKNOWN_LINE'
elif relfile in syntax_failure:
if not exresult:
Change string formatting method in Python tests
2023-02-19 16:26:14 -05:00
raise Exception(file_with_path + " listed in syntax_failure, but has EXRESULT FAIL")
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
tools_wrong = 'SYNTAX_FAILURE'
params = {
'file': file_with_path,
'relfile': relfile,
'todo': todo,
'disabled': disabled,
'tools_wrong': tools_wrong,
'exresult': exresult,
}
TestParseParserTests.tests.append((params, exresult))
return 0
def find_and_setup_test_profiles(profile_dir):
Use triple double-quoted strings for docstrings.
2022-08-07 14:57:30 -04:00
"""find all profiles in the given profile_dir, excluding
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
- skippable files
- include directories
- files in the main directory (readme, todo etc.)
Use triple double-quoted strings for docstrings.
2022-08-07 14:57:30 -04:00
"""
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
skipped = 0
profile_dir = os.path.abspath(profile_dir)
apparmor.profile_dir = profile_dir
utils test: fix typo in simple_tests output Signed-off-by: Steve Beattie <steve.beattie@canonical.com>
2020-05-05 11:54:49 -07:00
print('Searching for parser simple_tests... (this will take a while)')
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
for root, dirs, files in os.walk(profile_dir):
relpath = os.path.relpath(root, profile_dir)
if relpath == '.':
# include files are checked as part of the profiles that include them (also, they don't contain EXRESULT)
dirs.remove('includes')
dirs.remove('include_tests')
dirs.remove('includes-preamble')
for file in files:
file_with_path = os.path.join(root, file)
import is_skippable_file from apparmor.common ... instead of indirectly using it via apparmor.aa
2021-08-24 22:47:39 +02:00
if not is_skippable_file(file) and relpath != '.':
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
skipped += parse_test_profiles(file_with_path)
if skipped:
Change string formatting method in Python tests
2023-02-19 16:26:14 -05:00
print('Skipping {} test profiles listed in skip_startswith.'.format(skipped))
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
Change string formatting method in Python tests
2023-02-19 16:26:14 -05:00
print('Running {} parser simple_tests...'.format(len(TestParseParserTests.tests)))
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
utils: Require apparmor.aa users to call init_aa() Introduce an apparmor.aa.init_aa() method and move the initialization code of the apparmor.aa module into it. Note that this change will break any external users of apparmor.aa because global variables that were previously initialized when importing apparmor.aa will not be initialized unless a call to the new apparmor.aa.init_aa() method is made. The main purpose of this change is to allow the utils tests to be able to set a non-default location for configuration files. Instead of hard-coding the location of logprof.conf and other utils related configuration files to /etc/apparmor/, this patch allows it to be configured by calling apparmor.aa.init_aa(confdir=PATH). This allows for the make check target to use the in-tree config file, profiles, and parser by default. A helper method, setup_aa(), is added to common_test.py that checks for an environment variable containing a non-default configuration directory path prior to calling apparmor.aa.init_aa(). All test scripts that use apparmor.aa are updated to call setup_aa(). Signed-off-by: Tyler Hicks <tyhicks@canonical.com> Suggested-by: Christian Boltz <apparmor@cboltz.de> Acked-by: Seth Arnold <seth.arnold@canonical.com> Acked-by: Christian Boltz <apparmor@cboltz.de>
2017-03-02 21:21:53 +00:00
setup_aa(apparmor)
Check variable errors when parsing simple_tests ... by calling active_profiles.get_all_merged_variables() Also remove vars/vars_bad_add_assignment_1.sd from the exception_not_raised list again - now it raises an exception as expected.
2020-05-23 22:50:20 +02:00
profile_dir = os.path.abspath('../../parser/tst/simple_tests/')
find_and_setup_test_profiles(profile_dir)
Parse all parser simple_tests with the utils code Add a testcase that parses all tests in the parser/tst/simple_tests/ directory with parse_profile_data() to ensure that everything with valid syntax is accepted, and that all tests marked as FAIL raise an exception. This already resulted in - several patches to fix low-hanging fruits (including some bugs in the parser simple_tests itsself) - a list of tests that don't behave as expected. Those files get their expected result reverted to make sure we notice any change in the tools behaviour, especially changing to the really expected resulted. This method also makes sure that the testcase doesn't report any of the known failures. - a 5% improvement in test coverage - mostly caused by nearly completely covering parse_profile_data. - addition of some missing testcased (as noticed by missing coverage), for example several "rule outside of a profile" testcases. As indicated above, the tools don't work as expected on all test profiles - most of the failures happen on expected-to-fail tests that pass parse_profile_data() without raising an exception. There are also some tests failing despite valid syntax, often with rarely used syntax like if conditions and qualifier blocks. Most of the failing (generated) tests are caused by features not implemented in the tools yet: - validating dbus rules (currently we just store them without any parsing) - checks for conflicting x permissions - permissions before path ("r /foo,") - 'safe' and 'unsafe' keywords for *x rules - 'Pux' and 'Cux' permissions (which actually mean PUx and CUx, and get rejected by the tools - ideally the generator script should create PUx and CUx tests instead) skip_startswith excludes several generated tests from being run. I know that skip_startswith also excludes tests that would not fail, but the generated filenames (especially generated_x/exact-*) don't have a pattern that I could easily use to exclude less tests - and I'm not too keen to add a list with 1000 single filenames ;-) Acked-by: John Johansen <john.johansen@canonical.com>
2015-10-20 23:00:56 +02:00
setup_all_loops(__name__)
if __name__ == '__main__':
make utils tests less verbose Given the big number of tests, printing a dot for each test (instead of multiple lines) is enough ;-)
2018-04-08 20:18:30 +02:00
unittest.main(verbosity=1)
Reference in a new issue Copy permalink