Commit graph

1617 commits

Author SHA1 Message Date
John Johansen
7711baae4a Merge aa-decode: use grep -E instead of egrep
egrep and fgrep are deprecated and will print a warning in the next grep release (3.8)

https://git.savannah.gnu.org/cgit/grep.git/commit/?id=a9515624709865d480e3142fd959bccd1c9372d1

MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/792
Acked-by: John Johansen <john.johansen@canonical.com>
2021-08-24 08:29:04 +00:00
Michal Vasilek
e8f3a1b878
aa-decode: use grep -E instead of egrep
egrep and fgrep are deprecated and will print a warning in the next grep
release (3.8)

https://git.savannah.gnu.org/cgit/grep.git/commit/?id=a9515624709865d480e3142fd959bccd1c9372d1
Signed-off-by: Michal Vasilek <michal.vasilek@nic.cz>
2021-08-24 10:16:29 +02:00
Michal Vasilek
c31d49234f
utils: use internal which implementation
* removes runtime dependency on which
* fixes aa-unconfined when ss is installed outside {/usr,}/bin

Signed-off-by: Michal Vasilek <michal.vasilek@nic.cz>
2021-08-24 09:29:32 +02:00
John Johansen
1a190f2e30 Merge aa-remove-unknown: Drop superfluous $0 parameter from usage()
$0 is always the name of the script, even if using it inside a function.
Therefore use $0 directly, and no longer hand it over as a parameter.

Also `chmod +x aa-remove-unknown` to make in-tree testing easier.

MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/785
Acked-by: John Johansen <john.johansen@canonical.com>
2021-08-21 07:43:49 +00:00
John Johansen
5068f54cf1 Merge ProfileStorage: Store empty xattrs as empty string
... instead of None.

This avoids the need to allow type changes (None vs. str).

Also adjust the tests accordingly.

While on it, simplify the tests for attachment.

attachment is always a str, therefore adjust the test to expect an empty
str ('') instead of None - and later converting that None to ''.

MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/786
Acked-by: John Johansen <john.johansen@canonical.com>
2021-08-20 21:55:22 +00:00
Christian Boltz
d4b17ef904
SeverityVarsTest: Make order of variable replacements constant
... using [] instead of {}

This should keep the order of checking (and therefore code coverage)
constant, and should fix the randomly appearing partial coverage in
severity.py handle_variable_rank(). In some random cases (depending in
which order the replacements were done and checked for their severity),
the coverage report indicated that the 'elif' condition was never false.

Note: This is only "coverage cosmetics". In "real users", it doesn't
matter in which order the variable replacements are checked because the
result doesn't depend on the ordering.
2021-08-19 12:52:56 +02:00
Christian Boltz
8cecc14d2d
aa-remove-unknown: Drop superfluous $0 parameter from usage()
$0 is always the name of the script, even if using it inside a function.
Therefore use $0 directly, and no longer hand it over as a parameter.
2021-08-16 19:59:34 +02:00
Christian Boltz
169d305b2b
chmod +x aa-remove-unknown 2021-08-16 19:59:34 +02:00
Christian Boltz
e69f46ac3f
ProfileStorage: simplify tests for attachment
attachment is always a str, therefore adjust the test to expect an empty
str ('') instead of None - and later converting that None to ''.
2021-08-16 19:59:05 +02:00
Christian Boltz
337ebcd032
ProfileStorage: Store empty xattrs as empty string
... instead of None.

This avoids the need to allow type changes (None vs. str).

Also adjust the tests accordingly.
2021-08-16 19:59:02 +02:00
Christian Boltz
26d7c13f94
Add option to skip reading specific profiles
If a profile uses features not supported by the tools yet, add a
skiplist to (hopefully temporarily) exclude it from the tests.

This is meant to avoid blocking usage of new features in profiles.

When doing a release, the skip lists should be empty.
2021-07-13 21:09:36 +02:00
Christian Boltz
b02c59a730
Test if active and extra profiles can be parsed by the tools
The added test makes sure that the python code can parse all profiles
shipped with AppArmor. If this fails, read_profiles() /
read_inactive_profiles() will raise an exception.

Checking for the number of read profiles is mostly done to ensure
*something* is read (to make sure an empty or non-existing directory
won't make the test useless).
2021-07-13 21:09:36 +02:00
Christian Boltz
92883d7e6c
Fix extra profiles path in utils/test/logprof.conf 2021-07-13 21:09:35 +02:00
Christian Boltz
8d9e2489a2
Generate and keep html in utils coverage-regression
We sometimes have random coverage changes that are not reproducible and
therefore hard to debug.

Generate html coverage as part of make coverage-regression, and keep the
resulting utils/test/htmlcov/ as artifact to make debugging easier.

coverage-html needs JS files from various libjs-* packages, install them
in before_script
2021-07-13 18:40:59 +02:00
John Johansen
db94b436dd Merge Fix crash caused by ask_exec()
ask_exec still uses aa[profile][hat], therefore
- use full_profile when accessing hashlog
- correctly split the merged profile name to profile and hat
- avoid accidently initializing non-existing aa[profile][hat]

This fixes a regression from converting lots of code to use flat
profile//hat array keys.

MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/763
Acked-by: John Johansen <john@jjmx.net>
2021-06-30 06:36:55 +00:00
Christian Boltz
5aa35195f4
Make 'transitions' a dict instead of Hasher
It's used like a plain dict, therefore no further code changes are
needed.

Also simplify generating the 'context' key that is used with
'transitions'.
2021-06-27 21:37:22 +02:00
Christian Boltz
755b5d11e1
Fix crash caused by ask_exec()
ask_exec still uses aa[profile][hat], therefore
- use full_profile when accessing hashlog
- correctly split the merged profile name to profile and hat
- avoid accidently initializing non-existing aa[profile][hat]

This fixes a regression from converting lots of code to use flat
profile//hat array keys.
2021-06-27 21:21:53 +02:00
Georgia Garcia
9ff713957c Add README on tests regarding single test execution
MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/761

Signed-off-by: Georgia Garcia <georgia.garcia@canonical.com>
Acked-by: John Johansen <john.johansen@canonical.com>
2021-06-22 12:15:18 +00:00
Christian Boltz
5d8f66c9e3
get_header() tests: drop write_flags, add xattrs
Drop unused write_flags parameter from AaTest_get_header and
AaTest_get_header_01. This is a cleanup for the previous commit.

While on it, add xattrs parameter to AaTest_get_header, and add two
tests with non-empty xattrs.
2021-05-25 11:31:10 +02:00
Christian Boltz
6e6b94ab57
get_header(): remove always-True write_flags parameter
All the calling code (directly or indirectly) uses write_flags=True,
therefore drop the parameter to simplify the code.

A few tests called get_header() with write_flags=False. Adjust or drop
those tests.

Note: to keep the diff readable, the test changes are as small as
possible. The next commit will cleanup the now-superfluous write_flags
values in the tests.
2021-05-25 11:31:10 +02:00
Christian Boltz
a7b44cb1ac
write_piece(): remove always-True write_flags parameter
All the calling code (directly or indirectly) uses write_flags=True,
therefore drop the parameter to simplify the code.
2021-05-25 11:31:08 +02:00
Christian Boltz
2351bf74bf
change_profile_flags(): use ProfileStorage.parse()
... instead of "manually" creating a ProfileStorage object
2021-05-24 14:50:52 +02:00
Christian Boltz
b7e9928941
Preserve profile keyword and comment in ProfileStorage.parse() 2021-05-24 14:50:46 +02:00
John Johansen
e00e3fed85 Merge When changing flags, use correct amount of whitespace
.. instead of preserving the original leading whitespace.

This change affects the behaviour of aa-complain, aa-enforce and aa-audit.

MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/757
Acked-by: John Johansen <john@jjmx.net>
2021-05-24 10:21:07 +00:00
Christian Boltz
f0a445dd7d
When changing flags, use correct amount of whitespace
.. instead of preserving the original leading whitespace.

This change affects the behaviour of aa-complain, aa-enforce and aa-audit.
2021-05-23 19:00:06 +02:00
Christian Boltz
97aece024a
Preserve 'hat' keyword in ProfileStorage
... instead of always writing hats as '^hat'.

When writing a profile, prepending '^' or 'hat' to a hat name moves from
aa.py write_piece() to ProfileStorage.get_header().

Also extend cleanprof_test.* with 'hat bar {...}'.
2021-05-23 18:19:28 +02:00
Christian Boltz
6f15bea74c
Merge hat parsing into ProfileStorage.parse()
... and into parse_profile_start_line() (which is used by
ProfileStorage.parse()).

With this change, the section handling RE_PROFILE_HAT_DEF in
parse_profile_data() becomes superfluous.

A nice side effect is that two simple_tests parse failures get
accidently ;-) fixed.
2021-05-23 17:52:03 +02:00
Christian Boltz
055ca8f857
Add a test to ensure nested childs cause an Exception
... and add some tests for other error conditions that don't imply
nested childs, so that the intended failure gets tested.
(This is probably a leftover of the `hat == profile` -> `hat = None`
(while not in a hat/child profile) change.)
2021-05-16 18:15:03 +02:00
Christian Boltz
42fe65de71
ProfileStorage: merge parse_profile_start() into parse()
... which avoids handing several values around.

Also adjust the tests so that only the end result (of parse() gets
tested.
2021-05-16 18:15:03 +02:00
Christian Boltz
1642fea228
Move parse_profile_start{,_to_storage}() into ProfileStorage
... and make them class functions of ProfileStorage.

parse_profile_start_to_storage() gets renamed to parse().

Also move the tests for parse_profile_start() and
parse_profile_start_to_storage() to test-profile-storage.py.
2021-05-16 18:15:03 +02:00
Christian Boltz
2a97d6b6bc
ProfileStorage: change 'profile' to 'is_hat'
The 'profile' flag means "this profile is a profile or a child profile,
but not a hat". Since that's true for most cases, rename the flag to
'is_hat'.

Note that `'profile' == True` translates to `'is_hat' == False`

Also adjust all code to switch from 'profile' to 'is_hat'.
2021-05-16 18:15:03 +02:00
Christian Boltz
e1af0cdeca
parse_profile_start(): get rid of pps_set_profile
This value is True if we are in a child profile (not: hat), but that's
information we get "for free", so there's no need to hand it around.
Besides that, it was wrongly set to False for main profiles (which are
not hats).

Remove the pps_set_profile return value from parse_profile_start(), and
always assume True unless we were parsing a hat. For completeness,
explicitely set it to False when parsing a hat.

To make sure child profiles and hats don't get mixed up, add a child
profile to cleanprof_test.{in,out}.

test-libapparmor-test_multi.py always interpreted foo//bar as being
a hat, therefore explicitely mark them as such. (Technically not really
needed since this is the default, but it helps to make things clear.)
2021-04-28 21:22:08 +02:00
John Johansen
a7816e1a8f Merge Rework internal profile storage and handling in the aa-* tools to use merged profile names instead of [profile][hat]
Change the tools to use merged profile names (`var['foo//bar']`) instead of the profile/hat layout (`var[profile][hat]`) in many places. Also storage gets moved to ProfileList instead of using a hasher.

Already changed places (in this MR) are parsing profiles, writing profiles, handling and storing of extra profiles, log handling and asking the user about profile additions.

Remaining usage of the `var[profile][hat]` layout are the `aa` and `original_aa` hashers, they'll be replaced in a separate MR.

See the individual commits for details. I'd also recommend to do the review on the individual commits, because the big diff is probably unreadable ;-)

While this is a big chain of changes, each commit contains working code, converting between the two storage layouts with `split_to_merged()` and `merged_to_split()` as needed, with merged layout "bubbling up" in more and more functions.

The long-term goal of these changes is to enable support for nested child profiles in the tools, but - one step after the other ;-)

MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/736
Acked-by: John Johansen <john.johansen@canonical.com>
2021-04-27 10:16:58 +00:00
Christian Boltz
54eb2baae7
get_new_profile_filename(): use better variable name
The function decides on the filename of a profile, therefore use
'filename' as variable name instead of the somewhat confusing 'profile'
and 'full_profilename'.
2021-04-25 20:35:37 +02:00
Christian Boltz
30323a2ded
Detect endless #include loop when parsing profiles
If an include file includes itsself (for example if local/foo has
'#include <local/foo>'), print a warning instead of calling
load_include() again and again.

This fixes a crash when hitting such a case:
    RecursionError: maximum recursion depth exceeded while calling a Python object

Fixes: https://bugzilla.suse.com/show_bug.cgi?id=1184779 for the tools.
The parser will also need a fix.
2021-04-16 00:00:02 +02:00
Christian Boltz
e54b13a97a
write_piece(): use merged profile names
... and adjust the callers accordingly.

While on it, simplify the write_piece() code that prepends '^' to hats a bit.
2021-04-15 12:53:08 +02:00
Christian Boltz
0e5dca1083
serialize_profile(): use merged profile names in parameter
... and adjust all callers accordingly.
2021-04-15 12:53:08 +02:00
Christian Boltz
0802f34b35
deduplicate code in read_profile() and autodep()
if a profile doesn't have an attachment specified and the profile name
starts with '/', set the attachment to the profile name. This allows to
have one add_profile() call instead of two very similar ones.
2021-04-15 12:53:08 +02:00
Christian Boltz
8f84e125fd
simplify logic in read_profile()
exit early if profile_data is empty (which means we did read an empty
file). This allows to simplify the if conditions to "if active_profile:"
and "else:".
2021-04-15 12:53:08 +02:00
Christian Boltz
e04c8151bb
autodep(): use merged profile names
This means getting rid of several merged_to_split() and
split_to_merged() calls.
2021-04-15 12:53:08 +02:00
Christian Boltz
5cf7a28130
change get_profile() to return merged profile names
... and adjust its only caller accordingly.
2021-04-15 12:53:08 +02:00
Christian Boltz
5a68d96c11
extend create_new_profile() tests to cover required hats 2021-04-15 12:53:08 +02:00
Christian Boltz
c65206f16c
create_new_profile(): use and return merged profile names
... and adjust all callers and the tests.

For bonus points ;-) this also removes a hasher usage, and extends the
test to check that only the expected profile gets created.
2021-04-15 12:53:08 +02:00
Christian Boltz
286761271b
Store extra profiles in 'extra_profiles' ProfileList
... instead of the 'extras' hasher.

Also adjust all code that previously used 'extras' to use
'extra_profiles'. This affects get_profile() and read_profile().
2021-04-15 12:53:08 +02:00
Christian Boltz
179168f566
ProfileList: add get_profile_and_childs()
... and a test for it
2021-04-15 12:53:08 +02:00
Christian Boltz
b1a1b5dc1b
ProfileList: allow storing actual profile data
Add a prof_storage parameter to add_profile() to hand over the actual
profile data/rules as ProfileStorage.

Also adjust several tests to hand over a (dummy) ProfileStorage object.

Note: For now, the parameter is optional because it needs some more changes
in aa.py to be really useable. This will change in a later commit.
2021-04-15 12:53:08 +02:00
Christian Boltz
ba53ff3045
Change ask_the_questions() to work with merged profile names
... instead of converting log_dict to traditional [profile][hat] layout
in do_logprof_pass().

A nice side effect is that we get sorting the main profile before its
hats for free and can remove the sorting code.

Also update a comment in ask_rule_questions().

Finally, adjust aa-mergeprof so that it hands over a merged log_dict (using
split_to_merged())
2021-04-15 12:53:08 +02:00
Christian Boltz
a20865008f
collapse_log(): return merged profile names
... instead of the old [profile][hat] structure.

This needs changes in do_logprof_pass() when calling ask_the_questions()
(using merged_to_split() for now).

Also adjust test-libapparmor-test_multi.py logfile_to_profile() to
expect the merged structure.
2021-04-15 12:53:08 +02:00
Christian Boltz
bd3b62a8a1
Change collapse_log to internally use merged profile names
... and convert them back to the [profile][hat] layout at the end so
that callers still get the expected result.

As a side effect, log_dict no longer needs to be a hasher().
2021-04-15 12:53:08 +02:00
Christian Boltz
338a7774f1
parse_profile_data(): return merged profile names
... instead of the old [profile][hat] structure.

This needs changes in read_profile() (now using the merged profile name)
and attach_profile_data() (using merged_to_split() for now).

Also adjust test-aa.py to expect the merged structure.
2021-04-15 12:53:08 +02:00