I decided to use a "small" solution for now, which basically means
s/unittest.TestCase/AATest/, cleanup of some setUp() and renaming the
remaining setUp() functions to AASetup().
This doesn't mean an instant win (like in test-severity.py), but allows
to add tests with a tests[] array.
Acked-by: Steve Beattie <steve@nxnw.org>
To be able to distinguish between severity 10 and unknown severity,
change AASetup to specify 'unknown' as default rank, and change the
expected result to 'unknown' where it's expected.
Also change the "expected rank %d" to "%s" because it can be a string
now, and add a test that contains directories with different severity
in one variable.
After these changes, handle_variable_rank() errors out with
TypeError: unorderable types: str() > int()
so fix it by
- initializing rank with the default rank (instead of none)
- explicitely check that rank and rank_new are != the default rank before
doing a comparison
A side effect is another bugfix - '@{HOME}/sys/@{PROC}/overcommit_memory'
is severity 4, not 10 or unknown (confirmed by reading severity.db).
Acked-by: Steve Beattie <steve@nxnw.org>
This simplifies test-severity.py a lot:
- lots of test functions are replaced with tests[] arrays
- tempdir handling and cleanup is now done automagically
Even if test-severity.py shrunk by 65 lines, all tests are still there.
There's even an addition - SeverityTestCap now additionally verifies the
result of rank_capability().
Acked-by: Steve Beattie <steve@nxnw.org>
Change rank_capability() so that it doesn't expect the CAP_ prefix.
This makes usage easier because callers can simply hand over the
capability name.
Also change rank() to call rank_capability() without the CAP_ prefix.
Acked-by: Steve Beattie <steve@nxnw.org>
Replace rule-specific names with generic names:
- s/'capability'/ruletype/
- s/cap_obj/rule_obj/
- s/'network'/ruletype/
- s/net_obj/rule_obj/
Also set ruletype at the beginning of each block.
The long-term goal is to have
for ruletype in ['capability', 'network', ...]:
with common code to handle all rule types, and having common names makes
it easier to compare the blocks.
Acked-by: Steve Beattie <steve@nxnw.org>
aa-mergeprof has some sections where it first resets the 'deleted'
variable, and then overwrites it again a line or two later.
This patch removes the superfluous variable resets.
Acked-by: Steve Beattie <steve@nxnw.org>
Add a check to parse_profile_data() to detect if a file contains two
profiles with the same name.
Note: Two profiles with the same name, but in different files, won't be
detected by this check.
Also add basic tests to ensure that a valid profile gets parsed, and two
profiles with the same name inside the same file raise an exception.
(Sidenote: these simple tests improve aa.py coverage from 9% to 12%,
which also confirms the function is too long ;-)
Acked-by: Steve Beattie <steve@nxnw.org>
Add writeTmpfile() to AATest to write a file into the tmpdir. If no
tmpdir exists yet, automatically create one.
createTmpdir() is a separate function so that it's possible to manually
create the tmpdir (for example, if a test needs an empty tmpdir).
Also add a tearDown() function to delete the tmpdir again. This function
calls self.AATeardown() to avoid the need for super() in child classes.
Finally, simplify AaTestWithTempdir in test-aa.py to use createTmpdir()
and add an example for AATeardown() to test-example.py.
Acked-by: Steve Beattie <steve@nxnw.org>
aa-mergeprof no longer calls match_net_includes(), which means the
function can be dropped.
After that, match_net_include() is also unused, so also drop it.
Acked-by: Seth Arnold <seth.arnold@canonical.com>
Acked-by: Steve Beattie <steve@nxnw.org>
aa-mergeprof still used the old aa[profile][hat][allow]['netdomain']
which no longer gets populated. This resulted in not asking for merging
any network rules.
This patch changes ask_the_question() to the NetworkRule(set) layout.
Besides that,
- don't ask for network rules that are already covered.
Using is_known_rule() also fixes
https://bugs.launchpad.net/apparmor/+bug/1382241
- include the audit keyword in the "Network Family" headline
(I'd prefer to just use the get_clean() rule, but that's another topic)
- hide "(A)llow" when merging a deny rule
- as a side effect of using NetworkRule, fix crashes for 'network,' and
'network foo,' rules
To avoid having to repeat the list of available "buttons" and the logic
to update that list, add a available_buttons() function that returns the
list of available buttons depending on rule_obj.deny and rule_obj.audit
to aa.py, and import it into mergeprof.
I tested all changes manually.
Acked-by: Steve Beattie <steve@nxnw.org>
aa-mergeprof still used the old aa[profile][hat][allow]['capability']
which no longer gets populated - which resulted in not asking for
merging any capabilities.
Actually (and funnily),
- if other.aa[profile][hat].get(allow, False):
- continue
resulted in never merging capability rules even before the change to
CapabilityRule(set) - this was meant as optimization, but a "not" was
missing in the condition ;-) so it always skipped capability rules.
The patch changes ask_the_question to the CapabilityRule(set) layout.
Besides that,
- include the audit and deny keywords in the "Capability" headline
(I'd prefer to just use the get_clean() rule, but that's another topic)
- hide "(A)llow" when merging a deny rule
- don't ask for capabilities that are already covered
Also delete match_cap_includes() from aa.py, which is no longer used.
Acked-by: Steve Beattie <steve@nxnw.org>
Bug: https://launchpad.net/bugs/1382241
Also rename RE_PROFILE_CHANGE_PROFILE_2 to RE_PROFILE_CHANGE_PROFILE
and update apparmor/rule/change_profile.py to use the changed name.
Acked-by: Steve Beattie <steve@nxnw.org>
Change aa.py to use ChangeProfileRule and ChangeProfileRuleset instead
of a sub-hasher to store and write change_profile rules. In detail:
- drop all the change_profile rule parsing from parse_profile_data() and
serialize_profile_from_old_profile() - instead, just call
ChangeProfileRule.parse()
- change write_change_profile to use ChangeProfileRuleset
- add removal of superfluous/duplicate change_profile rules (the old
code didn't do this)
Note that this patch is much smaller than the NetworkRule and
CapabilityRule patches because aa-logprof doesn't ask for adding
change_profile rules - adding that is something for a later patch.
Acked-by: Steve Beattie <steve@nxnw.org>
Add utils/apparmor/rule/change_profile.py with the ChangeProfileRule and
ChangeProfileRuleset classes. These classes are meant to handle
change_profile rules.
In comparison to the current code in aa.py, ChangeProfileRule has some
added features:
- support for audit and allow/deny keywords (for which John promised a
parser patch really soon)
- support for change_profile rules with an exec condition
Also add the improved regex RE_PROFILE_CHANGE_PROFILE_2 to regex.py.
Acked-by: Steve Beattie <steve@nxnw.org>
It did this in the old 2.8 code, but didn't in 2.9.x (first there was a
broken hat regex, then I commented out the hat handling to avoid
breakage caused by the broken regex).
This patch makes sure the hat flags get set when setting the flags for
the main profile.
Also change RE_PROFILE_HAT_DEF to use more named matches
(leadingwhitespace and hat_keyword). Luckily all code that uses the
regex uses named matches already, which means adding another (...) pair
doesn't hurt.
Finally adjust the tests:
- change _test_set_flags to accept another optional parameter
expected_more_rules (used to specify the expected hat definition)
- add tests for hats (with '^foobar' and 'hat foobar' syntax)
- add tests for child profiles, one of them commented out (see below)
Remaining known issues (also added as TODO notes):
- The hat and child profile flags are *overwritten* with the flags used
for the main profile. (That's well-known behaviour from 2.8 :-/ but we
have more flags now, which makes this more annoying.)
The correct behaviour would be to add or remove the specified flag,
while keeping other flags unchanged.
- Child profiles are not handled/changed if you specify the 'program'
parameter. This means:
- 'aa-complain smbldap-useradd' or 'aa-complain /usr/sbin/smbldap-useradd'
_will not_ change the flags for the nscd child profile
- 'aa-complain /etc/apparmor.d/usr.sbin.smbldap-useradd' _will_ change
the flags for the nscd child profile (and any other profile and
child profile in that file)
Even with those remaining issues (which need bigger changes in
set_profile_flags() and maybe also in the whole flags handling), the
patch improves things and fixes the regression from the 2.8 code.
Acked-by: Steve Beattie <steve@nxnw.org> for trunk and 2.9
aa-mergeprof failed to fail ;-) when it should raise an AppArmorException.
Instead, it failed with
AttributeError: 'module' object has no attribute 'AppArmorException'
I confirmed this bug in trunk and 2.9.
Acked-by: Steve Beattie <steve@nxnw.org> for trunk and 2.9.
quote_if_needed() will be used by the upcoming ChangeProfileRule class,
which means it must be moved out of aa.py to avoid an import loop.
rule/__init__.py looks like a better place.
Also re-import quote_if_needed() into aa.py because it's still needed
there by various functions.
Acked-by: Seth Arnold <seth.arnold@canonical.com>
(might get re-used later ;-)
Also add two tests for profile names not starting with / - the quoted
version wasn't catched as invalid before, so this change is actually
also a bugfix.
Acked-by: Steve Beattie <steve@nxnw.org> for trunk and 2.9.
Change aa-notify parse_message() to also honor complain mode log events.
This affects both modes - desktop notifications and the summary report.
Acked-by: Steve Beattie <steve@nxnw.org>
Add setUp() to AATest that sets "self.maxDiff = None" (unlimited).
This gives us unlimited array diffs everywhere where AATest is used.
Also rename several setUp() functions in test-regex_matches.py to
AASetup() to avoid that the shiny new AATest setUp() gets overwritten.
Acked-by: Steve Beattie <steve@nxnw.org>
As requested by Steve, also add an example AASetup() to test-example.py.
Replace usage of RE_PROFILE_CAP and RE_PROFILE_NETWORK with
CapabilityRule.match() and NetworkRule.match() calls.
This also means aa.py doesn't need to import those regexes anymore.
As a side effect of this change, test-regex_matches.py needs a small
fix because it imported RE_PROFILE_CAP from apparmor.aa instead of
apparmor.regex.
Acked-by: Seth Arnold <seth.arnold@canonical.com>
Add match() and _match() class methods to rule classes:
- _match() returns a regex match object for the given raw_rule
- match() converts the _match() result to True or False
The primary usage is to get an answer to the question "is this raw_rule
your job?". (For a moment, I thought about naming the function
*Rule.myjob() instead of *Rule.match() ;-)
My next patch will change aa.py to use *Rule.match() instead of directly
using RE_*, which will make the import list much shorter and hide
another implementation detail inside the rule classes.
Also change _parse() to use _match() instead of the regex, and add some
tests for match() and _match().
Acked-by: Seth Arnold <seth.arnold@canonical.com>
Change aa.py to use NetworkRule and NetworkRuleset instead of a
sub-hasher to store, check and write network rules. In detail:
- drop profile_known_network() and use is_known_rule() instead
- replace match_net_includes() usage with match_includes() calls
- drop delete_net_duplicates(), use the code in NetworkRule and
NetworkRuleset instead
- make match_net_includes() (still used by aa-mergeprof) a wrapper for
match_includes()
- drop all the network rule parsing from parse_profile_data() and
serialize_profile_from_old_profile() - instead, just call
NetworkRule.parse()
- now that write_net_rules() got fixed, drop it ;-)
- change write_netdomain to use NetworkRuleset
- drop netrules_access_check() - that's is_covered() now
- use 'network' instead of 'netdomain' as storage keyword (log events
still use 'netdomain')
Also update cleanprofile.py to use the NetworkRuleset class.
This also means to delete the (now superfluous) delete_net_duplicates()
function.
Finally, there are some changes in regex.py:
- change RE_PROFILE_NETWORK in regex.py to named matches and to use
RE_COMMA_EOL (not only RE_EOL)
- drop the no longer needed RE_NETWORK_FAMILY and RE_NETWORK_FAMILY_TYPE
(rule/network.py has regexes that check against the list of available
keywords)
Acked-by: Kshitij Gupta <kgupta8592@gmail.com>
Add utils/test/test-network.py with tests for NetworkRule and
NetworkRuleset.
The tests are hopefully self-explaining, so let me just mention the most
important things:
- I started to play with namedtuple, which looks very useful (see "exp")
- the test loops make the tests much more readable (compare with
test-capability.py!) and make it easy to add some more tests
- 100% coverage :-)
Acked-by: Kshitij Gupta <kgupta8592@gmail.com>
Add utils/apparmor/rule/network.py with the NetworkRule and
NetworkRuleset classes. These classes are meant to handle network rules.
In comparison to the existing code in aa.py, relevant news are:
- the keywords are checked against a list of allowed domains, types and
protocols (these lists are based on what the utils/vim/Makefile
generates - on the long term an autogenerated file with the keywords
for all rule types would be nice ;-)
- there are variables for domain and type_or_protocol instead of
first_param and second_param. (If someone is bored enough to map the
protocol "shortcuts" to their expanded meaning, that shouldn't be too
hard.)
- (obviously) more readable code because we have everything at one place
now
- some bugs are fixed along the way (for example, "network foo," will now
be kept, not "network foo bar," - see my last mail about
write_net_rules() for details)
Acked-by: Kshitij Gupta <kgupta8592@gmail.com>
CleanProf.remove_duplicate_rules() didn't call
$profile['capability'].delete_duplicates()
because aa-cleanprof sets same_file=True.
Fix this by calling delete_duplicates(None) so that it
only checks the profile against itsself.
Note: this is only needed if the to-be-cleaned profile doesn't
contain any include rules - with includes present, the
"for inc in includes:" block already called delete_duplicates()
Acked-by: Kshitij Gupta <kgupta8592@gmail.com>
Implement in-profile de-duplication in BaseRuleset (currently affects
"only" CapabilityRuleset, but will also work for all future *Ruleset
classes).
Also change 'deleted' to be a simple counter and add some tests that
verify the in-profile deduplication.
Acked-by: Seth Arnold <seth.arnold@canonical.com>
test_parse_modifiers_invalid() uses a hand-broken ;-) regex to parse
only the allow/deny/audit keywords. This test applies to all rule types
and doesn't contain anything specific to capability or other rules,
therefore it should live in test-baserule.py
Moving that test also means to move the imports for parse_modifiers and
re around (nothing else in test-capability.py needs them).
Acked-by: Kshitij Gupta <kgupta8592@gmail.com>
Add some tests for the Baserule class to cover the 3 functions that must
be re-implemented in each rule class. This means we finally get 100%
test coverage for apparmor/rule/__init__.py ;-)
Acked-by: Kshitij Gupta <kgupta8592@gmail.com>
Ensure nosetests sees all tests in the tests[] tuples. This requires
some name changes because nosetests thinks all function names containing
"test" are tests. (A "not a test" docorator would be an alternative, but
that would require some try/except magic to avoid a dependency on nose.)
To avoid nosetests thinks the functions are a test,
- rename setup_all_tests() to setup_all_loops()
- rename regex_test() to _regex_test() (in test-regex_matches.py)
Also add the module_name as parameter to setup_all_loops and always run
it (not only if __name__ == '__main__').
Known issue: nosetests errors out with
ValueError: no such test method in <class ...>: stub_test
when trying to run a single test generated out of tests[].
(debugging hint: stub_test is the name used in setup_test_loop().)
But that's still an improvement over not seeing those tests at all ;-)
Acked-by: Steve Beattie <steve@nxnw.org> for trunk and 2.9.
Assume you have a profile like
/bin/foo {
/etc/ r,
network,
/usr/ r,
}
(important: there must be be a non-path rule between the two path blocks)
Then run aa-logprof and add another path event. When choosing (V)iew changes,
it will crash with a misleading
File ".../utils/apparmor/aamode.py", line 205, in split_mode
other = mode - user
TypeError: unsupported operand type(s) for -: 'collections.defaultdict' and 'set'
The reason for this is our beloved hasher, which is playing funny games
another time.
The patch wraps the hasher usage with a check for the parent element to
avoid auto-creation of empty childs, which then lead to the above crash.
BTW: This is another issue uncovered by the LibreOffice profile ;-)
Acked-by: Steve Beattie <steve@nxnw.org> for trunk and 2.9.
serialize_profile_from_old_profiles() calls store_list_var() with an
empty hasher. This fails for "+=" because in this case store_list_var()
expects a non-empty hasher with the variable already defined, and raises
an exception because of the empty hasher.
This patch sets "correct = False" if a "+=" operation appears, which
means the variable will be written in "clean" mode instead.
Adding proper support for "add to variable" needs big changes (like
storing a variable's "history" - where it was initially defined and what
got added where).
Acked-by: Steve Beattie <steve@nxnw.org> for trunk and 2.9.
the LibreOffice profile uncovered that handling of @{var} += is broken:
File ".../utils/apparmor/aa.py", line 3272, in store_list_var
var[list_var] = set(var[list_var] + vlist)
TypeError: unsupported operand type(s) for +: 'set' and 'list'
This patch fixes it:
- change separate_vars() to use and return a set instead of a list
(FYI: separate_vars() is only called by store_list_var())
- adoptstore_list_var() to expect a set
- remove some old comments in these functions
- explain the less-intuitive parameters of store_list_var()
Also add some tests for separate_vars() and store_list_var().
The tests were developed based on the old code, but not all of them
succeed with the old code.
As usual, the tests uncovered some interesting[tm] behaviour in
separate_vars() (see the XXX comments and tell me what the really
expected behaviour is ;-)
Acked-by: Steve Beattie <steve@nxnw.org> for trunk and 2.9
Move the code that does the c -> a and d -> w replacement in denied_mask
and requested_mask so that it only runs for path and exec events, but not
for other events (like dbus and ptrace). The validate_log_mode() and
log_str_to_mode() calls are also moved.
Technically, this means moving code from parse_event() to the path
and exec sections in add_event_to_tree().
This also means aa-logprof no longer crashes if it hits a ptrace or
dbus event in the log.
The "if dmask:" and "if rmask:" checks are removed - if a path event
doesn't have these two, it is totally broken and worth a aa-logprof
crash ;-)
Also adjust the parse_event() tests to expect the "raw" mask instead of
a set.
This patch fixes
https://bugs.launchpad.net/apparmor/+bug/1426651 and
https://bugs.launchpad.net/apparmor/+bug/1243932
I manually tested that
- c and d log events are still converted to a and w
- aa-logprof handles exec events correctly
- ptrace events no longer crash aa-logprof
Note: add_event_to_tree() is not covered by tests.
Acked-by: Steve Beattie <steve@nxnw.org> for trunk and 2.9
"capability foo".is_covered("deny capability foo") should return False
even if check_allow_deny is False.
Also add some tests with check_allow_deny=False.
Acked-by: Steve Beattie <steve@nxnw.org>
Thanks to the used data structure, write_net_rules() replaces bare
'network,' rules with the invalid 'network all,' when saving a profile.
This patch makes sure a correct 'network,' rule is written.
Also reset 'audit' to avoid all (remaining) rules get the audit flag
after writing an audit network rule.
Note: The first section of the function (that claims to be responsible
for bare 'network,' rules) is probably never hit - but I'm not too keen
to remove it and try it out ;-)
Acked-by: Steve Beattie <steve@nxnw.org> for trunk and 2.9.
When parsing a profile with named exec rules, the exec target included
the arrow. This resulted in two arrows when writing the profile (and one
more each time the profile was updated).
Fix this by using the match group that only contains the exec target
without the arrow in parse_profile_data() and
serialize_profile_from_old_profile().
References: https://bugs.launchpad.net/apparmor/+bug/1437901
Acked-by: Steve Beattie <steve@nxnw.org> for trunk and 2.9.
write_net_rules() doesn't add a space after 'audit' in two of three
cases, leading to invalid network rules.
This patch adds the missing spaces.
Acked-by: Seth Arnold <seth.arnold@canonical.com>
for both trunk and 2.9
write_net_rules() creates invalid rules for network rules with one
parameter (for example "network bluetooth").
Add a trailing comma to create valid rules.
Acked-by: Seth Arnold <seth.arnold@canonical.com>
for both trunk and 2.9.
Change serialize_parse_profile_start() to use parse_profile_start()
instead of using duplicated code.
The behaviour is mostly kept, with the exception that the function is
more strict now and raises exceptions instead of ignoring errors.
In practise, this won't change anything because the profiles are parsed
with parse_profile() (which calls parse_profile_start()) - and that
already errors out.
The tests are updated to match the more strict behaviour.
The next step would be to drop serialize_parse_profile_start()
completely, but this isn't urgent and can/should be done when we have
test coverage for serialize_profile_from_old_profile() one day ;-)
Acked-by: Steve Beattie <steve@nxnw.org>
Fix is_skippable_dir() - the regex also matched things like
/etc/apparmor.d/dont_disable, while it should match on the full
directory name.
Also add some tests based on a real-world aa-logprof run (with "print (path)"
in is_skippable_dir()) and some additional "funny"[tm] dirs.
Needless to say that the tests
('dont_disable', False),
('/etc/apparmor.d/cache_foo', False),
will fail with the old is_skippable_dir().
Acked-by: Steve Beattie <steve@nxnw.org>
Replace RE_PROFILE_START with RE_PROFILE_START_2 and adjust all
code sections that used RE_PROFILE_START_2.
The only real change is that test_get_flags_invalid_01 and
test_get_flags_invalid_02 now expect AppArmorException instead of
AppArmorBug.
Acked-by: Steve Beattie <steve@nxnw.org> for trunk
