2.11: Add basic support for abi rules to the tools
Add basic "understand and keep" support for abi rules, where
"understand" means to not error out when seeing an abi rule, and "keep"
simply means to keep the original abi rule when serializing a profile.
On the long term, abi rules should be parsed (similar to include rules),
but for now, this patch is the smallest possible changeset and easy to
backport.
Note that the only added test is via cleanprof_test.* which is used by
minitools_test.py - and does not run if you do a 'make check'.
Oh, and of course the simple_tests/abi/ files also get parsed by
test-parser-simple-tests.py.
BTW: Even serialize_profile_from_old_profile() can handle abi rules ;-)
This is a backport of 072d3e04 / !202 (merged) to
2.11 (with some adjustments because that commit didn't appy cleanly)
I propose this patch for 2.10 and 2.11.
PR: https://gitlab.com/apparmor/apparmor/merge_requests/223
Acked-by: John Johansen <john.johansen@canonical.com>
2.11/2.10: is_skippable_dir(): add 'cache.d' to exclude list
This excludes the /etc/apparmor.d/cache.d/ directory from aa-logprof
parsing because parsing the binary cache, well, takes a while :-/
Reported on the opensuse-factory mailinglist by Frank Krüger and
confirmed by others.
(cherry picked from commit 5b9497a8)
While this isn't strictly needed for 2.10 or 2.11 userspace, it makes testing these branches easier ;-)
I propose this cherry-pick for 2.11 (= this merge request) and 2.10.
https://gitlab.com/apparmor/apparmor/merge_requests/222
Acked-by: John Johansen <john.johansen@canonical.com>
Add basic "understand and keep" support for abi rules, where
"understand" means to not error out when seeing an abi rule, and "keep"
simply means to keep the original abi rule when serializing a profile.
On the long term, abi rules should be parsed (similar to include rules),
but for now, this patch is the smallest possible changeset and easy to
backport.
Note that the only added test is via cleanprof_test.* which is used by
minitools_test.py - and does _not_ run if you do a 'make check'.
Oh, and of course the simple_tests/abi/ files also get parsed by
test-parser-simple-tests.py.
BTW: Even serialize_profile_from_old_profile() can handle abi rules ;-)
This is a backport of 072d3e0451 / !202 to
2.11 (with some adjustments because that commit didn't appy cleanly)
This excludes the /etc/apparmor.d/cache.d/ directory from aa-logprof
parsing because parsing the binary cache, well, takes a while :-/
Reported on the opensuse-factory mailinglist by Frank Krüger and
confirmed by others.
(cherry picked from commit 5b9497a8c6)
bare_include_tests/ok_30.sd and ok_31.sd don't fail with the 2.11 tools.
Remove them from the unknown_line exception.
(Interestingly newer branches fail on these tests, but I didn't check why.)
These tests were added with the cherry-picked commit 4184b0c363
They are expected to fail, but don't fail with the 2.11 tools because
the regex only matches #include <...> which means #include "..."
is considered to be a comment.
parse_profile_start(): Error out on nested child profiles
See merge request apparmor/apparmor!136
Acked-by: John Johansen <john.johansen@canonical.com> for 2.10..master
(cherry picked from commit b7a4f37cbb)
8462c39b parse_profile_start(): Error out on nested child profiles
write_pair() ignored the 'tail' parameter, which resulted in writing
invalid alias rules (without the trailing comma).
Also add an alias to test/cleanprof.* to ensure it doesn't break again.
(cherry picked from commit ae4ab62855)
Acked-by: John Johansen <john.johansen@canonical.com>
Acked-by: Steve Beattie <steve@nxnw.org>
PR: https://gitlab.com/apparmor/apparmor/merge_requests/119
Writing a "link subset" rule missed a space, which resulted in something
like
link subset/foo -> /bar,
Also add a test rule to tests/cleanprof.* to ensure this doesn't break
again.
(cherry picked from commit 514535608f)
Acked-by: Steve Beattie <steve@nxnw.org>
PR: https://gitlab.com/apparmor/apparmor/merge_requests/117
utils tests: ignore tests for 'include if exists'
See merge request apparmor/apparmor!78
Acked-by: John Johansen <john.johansen@canonical.com>
(cherry picked from commit e6ef536957)
dc7c7021 utils tests: ignore tests for 'include if exists'
ignore .git in is_skippable_dir()
See merge request apparmor/apparmor!77
Acked-by: John Johansen <john.johansen@canonical.com>
(cherry picked from commit 3b5683be29)
f9eb3fea ignore .git in is_skippable_dir()
The tools don't support having multiple rules in one line (they expect
\n after each rule), therefore mark some of the bare_include_tests as
known failures.
(cherry picked from commit 26af640fda)
Signed-off-by: John Johansen <john.johansen@canonical.com>
FileRule: detect that 'a' is covered by 'w'
See merge request apparmor/apparmor!23
Acked-by: John Johansen <john.johansen@canonical.com>
(cherry picked from commit 6483c627d2)
1857f07d test-file.py: Document that w doesn't cover a yet
a0d4e246 FileRule: detect that 'a' is covered by 'w'
The test-aa-easyprof.py script relies on the parser to be built so the
check target of the utils/test/Makefile should detect if the parser
exists before running any tests.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Reported-by: Christian Boltz <apparmor@cboltz.de>
get_file_perms() and propose_file_rules() happily collect all file
permissions. This could lead to proposing 'wa' permissions in
aa-logprof, which then errored out because of conflicting permissions.
This patch adds a check to both functions that removes 'a' if 'w' is
present, and extends the tests to check this.
Acked-by: Seth Arnold <seth.arnold@canonical.com> for trunk and 2.11.
Note: Both functions (including this bug) were introduced together with
FileRule, so older releases are not affected.
(garbage) ptrace events like
... apparmor="DENIED" operation="ptrace" profile="/bin/netstat" pid=1962 comm="netstat" target=""
cause an empty name2 field, which leads to a crash in the tools.
This patch lets logparser.py ignore such garbage log events, which also
avoids the crash.
As usual, add some testcases.
test-libapparmor-test_multi.py needs some special handling to ignore the
empty name2 field in one of the testcases.
References: https://bugs.launchpad.net/apparmor/+bug/1689667
Acked-by: Seth Arnold <seth.arnold@canonical.com> for trunk and 2.11.
Older releases can't handle ptrace log events and therefore can't crash ;-)
Since r3634, the tools allow any order of dbus conditionals.
Quoting the r3634 patch description:
This patch eases the restriction on the ordering at the expense of the
utils no longer being able to detect and reject a single attribute that
is repeated multiple times. In that situation, only the last occurrence
of the attribute will be honored by the utils.
It seems nobody tested with all test profiles generated ;-) so we have to
add some exceptions to the "does not raise an exception" list now.
Acked-by <timeout> for trunk and 2.11
Bug: https://launchpad.net/bugs/1658239
FileRule understands leading permissions, so the reason to skip those
(generated) test profiles in test-parser-simple-tests.py is gone.
However, the gen-xtrans.pl script generates profiles with a not-so-valid
mix of uppercase and lowercase, for example "Pux" and "Cux". The parser
accepts this, but the tools complain about such rules. Therefore add the
affected profiles to the exception list.
In total, this means we now test 319 of the 380 generated_perms_leading
test profiles.
The patch also moves some lines around to get the \-escaped profiles
out of the mixed uppercase/lowercase exec rule section.
Acked-by: Seth Arnold <seth.arnold@canonical.com>
The test-aa-easyprof.py script was attempting to do its own special
setup to import the in-tree easyprof module. However, this proved to be
very flaky and resulted in the test periodically failing due to an
AttributeError the first time easyprof.parse_args() was called.
This patch removes the flakiness by trusting that PYTHONPATH is set up
appropriately before the test script is ran. PYTHONPATH is already
initialized appropriately by utils/test/Makefile according to the
USE_SYSTEM make variable.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
Acked-by: Christian Boltz <apparmor@cboltz.de>
if USE_SYSTEM is not set, the utils make check target will instruct
test-aa-easyprof.py to provide the path of the in-tree parser executable
to aa-easyprof.
If USE_SYSTEM is set, the default parser path (/sbin/apparmor_parser or
the result of `which apparmor_parser`) is used.
The test-aa-easyprof.py script receives the parser path by checking the
__AA_PARSER environment variable. This environment variable is strictly
used by the test script and not any user-facing code so two leading
underscores were used.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: Christian Boltz <apparmor@cboltz.de>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
If USE_SYSTEM is not set, the utils make check target will instruct
test-aa-easyprof.py to provide the path of the in-tree
profiles/apparmor.d directory to aa-easyprof as the parser base
directory.
If USE_SYSTEM is set, the default base directory (/etc/apparmor.d) is
used.
The test-aa-easyprof.py script receives the base path by checking the
__AA_BASEDIR environment variable. This environment variable is strictly
used by the test script and not any user-facing code so two leading
underscores were used.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: Christian Boltz <apparmor@cboltz.de>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
Bug: https://launchpad.net/bugs/1538306
https://launchpad.net/bugs/1521031
aa-easyprof accepts a list of abstractions to include and, by default,
execs apparmor_parser to verify the generated profile including any
abstractions. However, aa-easyprof didn't provide the same flexibility
as apparmor_parser when it came to where in the filesystem the
abstraction files could exist.
The parser supports --base (defaulting to /etc/apparmor.d) and --Include
(defaulting to unset) options to specify the search paths for
abstraction files. This patch adds the same options to aa-easyprof to
aide in two different situations:
1) Some Ubuntu packages use aa-easyprof to generate AppArmor profiles
at build time. Something that has been previously needed is a way
for those packages to ship their own abstractions file(s) that are
#included in the easyprof-generated profile. That's not been
possible since the abstraction file(s) have not yet been installed
during the package build.
2) The test-aa-easyprof.py script contains some tests that specify
abstractions that should be #included. Without the ability to
specify a different --base or --Include directory, the abstractions
were required to be present in /etc/apparmor.d/abstractions/ or the
tests would fail. This prevents the Python utils from being able to
strictly test against in-tree code/profiles/etc.
I don't like the names of the command line options --base and --Include.
They're not particularly descriptive and the capital 'I' is not user
friendly. However, I decided to preserve the name of the options from
apparmor_parser.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: Christian Boltz <apparmor@cboltz.de>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
Introduce an apparmor.aa.init_aa() method and move the initialization
code of the apparmor.aa module into it. Note that this change will break
any external users of apparmor.aa because global variables that were
previously initialized when importing apparmor.aa will not be
initialized unless a call to the new apparmor.aa.init_aa() method is
made.
The main purpose of this change is to allow the utils tests to be able
to set a non-default location for configuration files. Instead of
hard-coding the location of logprof.conf and other utils related
configuration files to /etc/apparmor/, this patch allows it to be
configured by calling apparmor.aa.init_aa(confdir=PATH).
This allows for the make check target to use the in-tree config file,
profiles, and parser by default. A helper method, setup_aa(), is added
to common_test.py that checks for an environment variable containing a
non-default configuration directory path prior to calling
apparmor.aa.init_aa(). All test scripts that use apparmor.aa are updated
to call setup_aa().
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Suggested-by: Christian Boltz <apparmor@cboltz.de>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
Acked-by: Christian Boltz <apparmor@cboltz.de>
The utils tests should make use of the logprof.conf that resides in
utils/test/ when testing against the in-tree parser and profiles. When
testing against the system, it the utils tests should continue to use
the system logprof.conf.
This patch updates the parser and profiles paths to point to the in-tree
paths. Another patch is needed to get aa.py to honor a non-hardcoded
search path for logprof.conf and other configuration files.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: Christian Boltz <apparmor@cboltz.de>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
https://launchpad.net/bugs/1628286
The utils were enforcing that the dbus rule attributes were strictly
ordered in the following fashion:
bus -> path -> interface -> member -> peer
However, the parser has always accepted the attributes in any order. If
the system contained a profile which did not use the strict ordering
enforced by the utils, the utils would refuse to operate at all.
This patch eases the restriction on the ordering at the expense of the
utils no longer being able to detect and reject a single attribute that
is repeated multiple times. In that situation, only the last occurrence
of the attribute will be honored by the utils.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: Christian Boltz <apparmor@cboltz.de>
The merged /usr patches to the policy broke some utils tests due to a
change in the expected output.
Fixes: r3600 update lots of profiles for usrMerge
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: Christian Boltz <apparmor@cboltz.de>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
This allows to hand over any source instead of using the global variable.
Now that the function expects its input as parameter, get rid of the
global log_dict, which means
- change collapse_log() to initialize log_dict as local variable and
return it
- change do_logprof_pass() to catch collapse_log()'s return value and
hand it over to ask_the_questions()
- drop all references to the global log_dict variable
- update test-libapparmor-test_multi to follow the changes
Also fix an if condition that would fail if aa[profile][hat] does not
exist - get() defaults to None if the requested item doesn't exist, and
None.get('file') will raise an Exception.
Acked-by: Seth Arnold <seth.arnold@canonical.com>
ldd exits with $? == 1 if a file is 'not a dynamic executable'.
This is correct behaviour of ldd, so we should handle it instead of
raising an exception ;-)
Also extend fake_ldd and add a test to test-aa.py to cover this.
Note that 2.10 and 2.9 don't have tests for get_reqs() nor fake_ldd,
so those branches will only get the aa.py changes.
Acked-by: John Johansen <john.johansen@canonical.com> for trunk, 2.10 and 2.9.
Storing these event details depending on the operation type only makes
things more difficult because it's hard to differenciate between file
and network events.
Note that this happens at the first log parsing stage (libapparmor log
event -> temporary python array) and therefore doesn't add a serious
memory footprint. The event tree will still only contain the elements
relevant for the actual event type.
This change means that lots of testcases now get 3 more fields (all
None) when testing parse_event(), so update all affected testcases.
(test-network doesn't need a change for probably obvious reasons.)
Also rename a misnamed test in test-change_profile.
Acked-by: Seth Arnold <seth.arnold@canonical.com> for trunk and 2.10.
This patch adds profiles for all log sniplets that are expected to
result in a profile rule.
This also means some changes in test-libapparmor-test_multi.py are
needed:
- split off log_to_profile_skip from log_to_profile_known_failures to
- only skip tests in log_to_profile_skip (causing a crash or requiring
user interaction)
- run tests in log_to_profile_known_failures, but expect a non-equal
result (caused by not added rules etc.)
- add quite some tests to log_to_profile_known_failures - they were
skipped before because they didn't have a *.profile file.
- add handling for hats to shorten list of known failures
This fixes testcase24 and testcase33 (after adjusting the profiles)
and lots of the new *.profile files.
- since we now have *.profile files for all log events that should result
in a profile rule, no longer ignore FileNotFoundError
Acked-by: Seth Arnold <seth.arnold@canonical.com>
This patch adds TestLogToProfile to test-libapparmor-test_multi.py which
"translates" the test_multi log sniplets to a profile, and checks if it
matches the expected profile.
The expected profile for one log event will obviously contain only one
rule, and gets added as *.profile to the test_multi directory.
This patch includes 33 test_multi profiles - which means 83 more need to
be created. Whenever you have some time, add one or two! (Please write
those test_multi profiles manually, without using the tools.)
I know some parts of the test code looks complicated. Unfortunately this
is how things work - compare it with do_logprof_pass() in aa.py...
While on it, set tests = 'invalid' which ensures a failure in case
parse_test_profiles() doesn't set the tests array, and move printing
the test name out of parse_test_profiles() to avoid printing it twice.
A nice side effect of this patch is increased test coverage:
- 30% -> 40% in aa.py (= 250 more lines)
- 52% -> 78% in aamode.py (= 23 more lines)
- 26% -> 68% in logparser.py (= 120 more lines)
- total coverage increases from 57% to 62%
Acked-by: Seth Arnold <seth.arnold@canonical.com>
This little change means that the tests will run as part of 'make check'.
This commit is only a 'bzr mv utils/test/config_test.py utils/test/test-config.py'
without any changes in the file content.
Acked-by: Steve Beattie <steve@nxnw.org>
aa_test.py doesn't run in 'make check' because its filename doesn't
match the 'test-*.py' pattern, so this move means the tests now actually
get run.
While on it, migrate test-aamode.py to use the AATest base class, and
migrate the str_to_mode() tests to a tests[] array.
After this move, aa_test.py doesn't do anything anymore, so delete it.
Acked-by: Steve Beattie <steve@nxnw.org>.
Also add another test proposed by Steve:
(None, set()),
aa_test.py doesn't run in 'make check' because its filename doesn't
match the 'test-*.py' pattern.
mode_to_str() was dropped as part of the FileRule series, so it's
pointless to keep its tests. (The replacement is totally different and
has full test coverage already.)
loadincludes() still exists, but only testing if the function runs
without errors is not really helpful, so drop this test.
Also drop unused imports and add an explicit import for apparmor.aamode.
Acked-by: Steve Beattie <steve@nxnw.org>
aa_test.py doesn't run in 'make check' because its filename doesn't
match the 'test-*.py' pattern.
Move tests for globbing ("plain" globbing and globbing with ext) to
test-aare.py to make sure those tests actually run.
Note: This isn't an exact move - I adjusted some of the tests to make
them more useful, and added some more tests.
Also, glob_path() and glob_path_withext() no longer exist in aa.py.
They moved to the AARE class as part of the FileRule patch series.
Acked-by: Steve Beattie <steve@nxnw.org>
Add a testcase with exec-only permissions (which get ignored by
get_perms_for_path()) to increase FileRule test coverage to 100%.
Acked-by: Steve Beattie <steve@nxnw.org>
As discussed a while ago, switch the utils (including their tests) to
use python3 by default. While on it, drop usage of "env" to always get
the system python3 instead of a random one that happens to live
somewhere in $PATH.
In practise, this patch doesn't change much - AFAIK openSUSE, Debian and
Ubuntu already patch aa-* to use python3.
Also add a note to README to officially deprecate Python 2.x.
(I won't break Python 2.x support intentionally - unless some future
change gives me a very good reason to finally drop Python 2.x support.)
Acked-by: Seth Arnold <seth.arnold@canonical.com>
(since 2016-08-23, but the commit had to wait for the FileRule series
because it touches test-file.py)
If a merged profile contains additional hats or subprofiles, the "old"
aa-mergeprof silently created them as additional hasher elements (partly
buggy, because subprofiles would end up as '^/subprofile' instead of
'profile /subprofile'). After switching to FileRule, aa-mergeprof crashes
on new hats or subprofiles.
This patch adds code to ask the user if the new hat or subprofile should
be added - which means this patch replaces two bugs (crash + silently
adding subprofiles and hats) with a new feature ;-)
The new questions also add a new text CMD_ADDSUBPROFILE in ui.py.
Finally, the new "button" combinations get added to test-translations.py.
If you want to test, try to aa-mergeprof this profile (the subprofile
and hat are dummies, nothing ping would really require):
#include <tunables/global>
/{usr/,}bin/ping {
#include <abstractions/base>
#include <abstractions/consoles>
#include <abstractions/nameservice>
capability net_raw,
capability setuid,
network inet raw,
network inet6 raw,
/{,usr/}bin/ping mixr,
/etc/modules.conf r,
^hat {
/bin/hat r,
/bin/bash px,
}
profile /subprofile {
/bin/subprofile r,
/bin/bash px,
}
# Site-specific additions and overrides. See local/README for details.
#include <local/bin.ping>
}
Note that this patch is not covered by unittests, but it passed all my
manual tests.
Acked-by: Steve Beattie <steve@nxnw.org>
Bug: https://launchpad.net/bugs/1507469
FileRule uses RE_PROFILE_FILE_ENTRY, which also means
RE_PROFILE_PATH_ENTRY, RE_PROFILE_BARE_FILE_ENTRY and RE_OWNER are now
unused.
This patch drops these regexes and their tests in test-regex_matches.py.
Acked-by: Steve Beattie <steve@nxnw.org>
rank() in severity.py is a dispatcher that calls the needed function
(rank_path(), rank_capability()) based on the parameter. Since all
calling code knows what rule type it is handling, this dispatcher is
superfluous - the calling code can call rank_path() or rank_capability()
directly.
This patch drops rank() and switches the remaining users of rank() to
call the rank_*() functions directly. For the tests, this means to drop
the CAP_ prefix because rank_capability doesn't expect this prefix.
Acked-by: Steve Beattie <steve@nxnw.org>
Adding a rule to *Ruleset means it simply gets added. This also means
that then-superfluous rules will be kept.
This patch adds an optional cleanup flag to add(). If set, rules covered
by the new rule will be deleted. The difference to delete_duplicates()
is that cleanup only deletes rules that are covered by the new rule, but
keeps other, unrelated superfluous rules.
Also return the number of deleted rules to give the UI a chance to
report this number.
Finally, adjust the existing tests for FileRuleset to ensure default
mode (without cleanup) doesn't delete any rules, and add a test using
the cleanup flag.
Acked-by: Steve Beattie <steve@nxnw.org>
get_exec_rules_for_path() returns a FileRuleset with all rules matching
the given path.
get_exec_conflict_rules() returns a FileRuleset with all exec rules that
conflict with the given oldrule. This will be used by aa-mergeprof to
ask the user which rule he wants to keep.
Also add tests for both functions.
Acked-by: Steve Beattie <steve@nxnw.org>
When matching an AARE against another AARE, most AARE objects don't
contain orig_regex (only AARE instances originating from a log event
contain orig_regex).
In this case, match() will use is_equal() to error out on the safe side.
Unfortunately this also means that there are lots of false negative
cases where match() returns False errornously.
With this patch, match() checks the given AARE regex and, if it doesn't
contain any special characters (wildcards, alternations or variables),
handles it as plain path. This avoids most of the false negatives.
Also extend the AARE tests to check a bunch of plain path regexes using
AARE matching instead of only str matching.
Acked-by: Steve Beattie <steve@nxnw.org>
Merge the existing and requested permissions into a nice set of headers
that can be displayed by aa-logprof. This will look like:
Path: /foo
Old Mode: r + owner w
New Mode: rw
Also split off a _join_given_perms() function off _joint_perms() so that
we can use the permission string merging for things not stored in self.*.
Finally add some tests for logprof_header().
Acked-by: Steve Beattie <steve@nxnw.org>
Add set_options_audit_mode() to switch the audit mode in all options
offered by aa-logprof and aa-mergeprof, not only the "original" rule
(in aa-logprof, this means the non-globbed rule_obj).
As usual, add some tests to ensure the function works as expected.
Acked-by: Steve Beattie <steve@nxnw.org>
aa.py:
- add propose_file_rules() - will propose matching paths from existing
rules in the profile or one of the includes
- save user_globs if user selects '(N)ew' (will be re-used when
proposing rules)
- change user_globs to a dict so that it can carry the human-readable
path and an AARE object for it
- change order_globs() to ensure the original path (given as parameter)
is always the last item in the resulting list
- add a ruletype switch to ask_the_questions() so that it uses
propose_file_rules() for file events (I don't like this
ruletype-specific solution too much, but everything else would make
things even more complicated)
Also keep aa-mergeprof ask_the_questions() in sync with aa.py.
In FileRule, add original_perms (might be set by propose_file_rules())
Finally, add some tests to ensure propose_file_rules() does what it promises.
Acked-by: Steve Beattie <steve@nxnw.org>
get_file_perms() collects the existing permissions for a file from
various rules (exact matches, wildcards) in the main profile and the
included abstractions.
It will be used to get displaying the current permissions back, and
also to propose rules with merged permissions (next patch).
Also add some tests to make sure it does what it promises ;-)
Acked-by: Steve Beattie <steve@nxnw.org>
