The 'profile' flag means "this profile is a profile or a child profile,
but not a hat". Since that's true for most cases, rename the flag to
'is_hat'.
Note that `'profile' == True` translates to `'is_hat' == False`
Also adjust all code to switch from 'profile' to 'is_hat'.
This value is True if we are in a child profile (not: hat), but that's
information we get "for free", so there's no need to hand it around.
Besides that, it was wrongly set to False for main profiles (which are
not hats).
Remove the pps_set_profile return value from parse_profile_start(), and
always assume True unless we were parsing a hat. For completeness,
explicitely set it to False when parsing a hat.
To make sure child profiles and hats don't get mixed up, add a child
profile to cleanprof_test.{in,out}.
test-libapparmor-test_multi.py always interpreted foo//bar as being
a hat, therefore explicitely mark them as such. (Technically not really
needed since this is the default, but it helps to make things clear.)
Add a prof_storage parameter to add_profile() to hand over the actual
profile data/rules as ProfileStorage.
Also adjust several tests to hand over a (dummy) ProfileStorage object.
Note: For now, the parameter is optional because it needs some more changes
in aa.py to be really useable. This will change in a later commit.
... instead of the old [profile][hat] structure.
This needs changes in do_logprof_pass() when calling ask_the_questions()
(using merged_to_split() for now).
Also adjust test-libapparmor-test_multi.py logfile_to_profile() to
expect the merged structure.
With the exception of the documentation fixes, these should all be
invisible to users.
Signed-off-by: Steve Beattie <steve.beattie@canonical.com>
Acked-by: Christian Boltz <apparmor@cboltz.de>
MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/687
Note that the log doesn't include enough information for EXEC MODE and
EXEC COND, therefore aa-logprof will always propose ALL as EXEC COND
(comm= might give a hint about EXEC COND, but isn't good enough).
With the added support in aa-logprof, remove the changeprofile tests
from the known-failing list in test-libapparmor-test_multi.py.
Also add another test log (from darix) / expected profile to the
libapparmor testsuite.
Everything handled in 'filelist' gets handled in active_profiles now.
Note: the 'elif' branch in delete_all_duplicates() was probably never
hit because `if include.get(...)` always matched. The only possible
exception might be non-existing include files, but those cause a 'file
not found' error anyway.
... because after the previous three commits, nothing reads/needs this
anymore
Note: file_name in ask_exec() was only used in the (dropped) filelist
usage.
v2:
- parse partial log line broken at \n
- add testcase_dbus_10.* for partial log line
- remove quotes from testcasw_dbus_09.profile
The following log format has been seen in the wild, and currently results
in a RECORD_INVALID
[4835959.046111] audit: type=1107 audit(1561053426.749:186): pid=640 uid=103 auid=4294967295 ses=4294967295 msg='apparmor="ALLOWED" operation="dbus_method_call" bus="system" path="/org/freedesktop/systemd1" interface="org.freedesktop.systemd1.Manager" member="LookupDynamicUserByName" mask="send" name="org.freedesktop.systemd1" pid=20596 label="/usr/sbin/sshd" peer_pid=1 peer_label="unconfined"
exe="/usr/bin/dbus-daemon" sauid=103 hostname=? addr=? terminal=?'
Test parsing the above message with and without the \n embedded between
peer_label= and exec=
Acked-by: Seth Arnold <seth.arnold@canonical.com>
Signed-off-by: John Johansen <john.johansen@canonical.com>
If final_name still includes null-*, that's most likely caused by nested
execs which aren't supported by the tools yet. Ignoring them is better
than creating a useless null-* hat.
Note: The tools always had this restriction, so this is not a regression ;-)
Also note that test-libapparmor-test_multi expects that null-* hats get
created (which makes sense because the one-line log sniplets don't have
any exec indication), therefore add an optional parameter to keep this
behaviour for the tests.
Now that all log events arrive in hashlog, having a separate 'prelog' no
longer makes sense. Changing collapse_log() to accept 'hashlog' directly
removes that level of indirection.
It's only used by two functions:
- handle_hashlog() - writes to prelog, and now returns it
- collapse_log() - reads prelog, and now gets it as parameter
'final_name' by default is the profile name, but ask_exec() will change
it for the target profile (which is a null-* profile at this stage)
based on exec mode choice. ask_addhat() will also change it based on the
chosen hat.
Choosing "deny" or "unconfined" will result in an empty final_name and
ignoring these log events.
All other choices set final_name to the full profile name ("foo" for Px,
"foo//bar" for Cx, current profile for ix).
Also fix the order of handling log events - since ask_exec() changes the
hashlog final_name, it has to run first so that ask_addhat() (which
"only" adjusts the hat name in final_name) and handle_hashlog() can work
with the updated profile name.
Finally, update test-libapparmor-test_multi.py to ignore final_name when
checking if hashlog is empty, and fix the call order of ask_exec() etc.
... and self.pid which is also unused.
This simple change also means to adjust all the code that uses ReadLog.
We get rid of log_pid in aa.py, and have to change lots of test-*
The only remaining job of handle_children() was to handle exec events.
(And recursively calling itsself if it hits nested log events, but
logparser.py never created such a log structure.)
Therefore:
- drop the dead code handling nested log (type != str)
- rename the remaining function to ask_exec()
- drop checks for typ = 'exec' (now done as part of the for loop
- drop the "else" branch for unknown event types
- change 'return' to 'continue' because ask_exec handles all exec events
in a loop instead of being called multiple times
- oh, and of course switch over to using hashlog
Finally, change do_logprof_pass() and the tests to call ask_exec()
instead of handle_children().
While on it, update a comment in test-translations.py which held the
last reference to handle_children().
Adjust logparser.py to store change_hat events in hashlog.
In aa.py,
- split off ask_addhat() from handle_children()
- change ask_addhat() to use hashlog
- call ask_addhat() from do_logprof_pass()
Also call ask_addhat() in test-libapparmor-test_multi.py to keep it in
sync with do_logprof_pass().
In logparser.py parse_event_for_tree, convert path handling to hashlog.
While on it, include 'owner' as part of hashlog so that aa.py doesn't
need to guess.
Also switch to a simple for loop instead of using log_str_to_mode() from
aamode.py to convert denied_mask to hasher keys (which would have been
needed to allow merging of several log events for the same path anyway).
Note that the check for 'mrawlk' (intentionally without 'x') is more
strict than the validate_log_mode(), but it should still cover all file
permissions. (validate_log_mode() also allows things like 'Px', which
we'll never hit in a logfile.)
In aa.py collapse_log() update the handling of path events to match the
additional [owner] key in hashlog/prelog. This makes the owner detection
in collapse_log() superfluous.
In aa.py handle_children(), remove 'path' handling from the 'path' or
'exec' section, and add an 'if True:' to avoid lots of whitespace
changes.
In aamode.py, drop the now unused split_mode() function, and
AA_OTHER_REMOVE() that was only used by split_mode().
Finally, remove sample log events with null-* hats from the list of
known failures in test-libapparmor-test_multi.py (we no longer filter
out null-* hats), and fix whitespace in two expected profiles.
logparser.py puts each log event on a big "stack" in self.pid. Later,
handle_children() in aa.py then converts that (named 'log' in aa.py) to
the prelog hasher.
This commit changes logparser.py to create the prelog structure itsself
(named hashlog), which
- removes one level of indirection
- probably saves some memory because the hashlog automatically
de-duplicates events
This commit does this for capability, network and signal events, and
adds the infrastructure needed for all event/rule types.
In aa.py, the new function handle_hashlog() copies the hashlog content
to prelog. OTOH, the now superfluous code handling capability, network
and signal events gets removed from handle_children().
Long-term, hashlog will replace log in aa.py. When this is done,
handle_hashlog() will be replaced by a simple prelog = hashlog.
logparser.py gets a new function init_hashlog() to initialize hashlog
for each profile. It also gets changed to store capability, network and
signal events into hashlog instead of storing them in self.pid.
hashlog uses the full profile name as key, which is the first baby step
to support nested child profiles. (for now, handle_hashlog() still
splits the profile name into profile and hat.)
Known issue: The new implementation doesn't handle exec yet, which means
that events get lost at the exec boundary (= in cases aa-logprof asks
which execute mode to use). This will be fixed in a later commit.
Technical stuff first:
Replace existing_profiles (a dict with the filenames for both active and
inactive profiles) with active_profiles and extra_profiles which are
ProfileList()s and store the active profiles and those in the extra
directory separately. Thanks to ProfileList, now also the relation
between attachments and filenames is easily available.
Also replace all usage of existing_profiles with active_profiles and
extra_profiles, and adjust it to the ProfileList syntax everywhere.
With this change, several bugs in aa-complain and the other minitools
get fixed:
- aa-complain etc. never found profiles that have a profile name
(the attachment wasn't checked)
- even if the profile name was given as parameter to aa-complain, it
first did "which $parameter" so it never matched on named profiles
- profile names with alternations (without attachment specification)
also never matched because the old code didn't use AARE.
References: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=882047#92
(search for "As usual" ;-)
Just for completeness - the matching still doesn't honor/expand
variables in the profile name.
Add a check to logfile_to_profile() that checks the parsed log against a
list of input logs (log_to_profile_known_empty_log) that produce an
empty output.
If a log line contains a denial for a child profile, log_dict will
(obviously) only contain the child profile. However, serialize_profile()
expects that the parent profile is also initialized as ProfileStorage.
This patch makes sure the parent profile gets initialized.
It also removes 26 of the 37 reasons in the TODO note in aa.py :-)
test-libapparmor-test_multi.py converts the libapparmor test_multi log
examples to profiles.
This patch allows to call test-libapparmor-test-multi.py with a logfile
(containing a single log line) as parameter. It will then print the
resulting profile.
Example:
# python3 test-libapparmor-test_multi.py /path/to/libraries/libapparmor/testsuite/test_multi/testcase_dbus_01.in
/tmp/apparmor-2.8.0/tests/regression/apparmor/dbus_service {
dbus send bus=session path=/org/freedesktop/DBus interface=org.freedesktop.DBus member=Hello peer=(label=unconfined),
}
This parameter is always [], so we can simplify the ReadLog __init__()
parameters.
Note that some tests handed over '' instead of []. This was a bug, but
didn't matter because those tests only use a small portion of ReadLog.
Acked-by: Seth Arnold <seth.arnold@canonical.com>
(garbage) ptrace events like
... apparmor="DENIED" operation="ptrace" profile="/bin/netstat" pid=1962 comm="netstat" target=""
cause an empty name2 field, which leads to a crash in the tools.
This patch lets logparser.py ignore such garbage log events, which also
avoids the crash.
As usual, add some testcases.
test-libapparmor-test_multi.py needs some special handling to ignore the
empty name2 field in one of the testcases.
References: https://bugs.launchpad.net/apparmor/+bug/1689667
Acked-by: Seth Arnold <seth.arnold@canonical.com> for trunk and 2.11.
Older releases can't handle ptrace log events and therefore can't crash ;-)
Introduce an apparmor.aa.init_aa() method and move the initialization
code of the apparmor.aa module into it. Note that this change will break
any external users of apparmor.aa because global variables that were
previously initialized when importing apparmor.aa will not be
initialized unless a call to the new apparmor.aa.init_aa() method is
made.
The main purpose of this change is to allow the utils tests to be able
to set a non-default location for configuration files. Instead of
hard-coding the location of logprof.conf and other utils related
configuration files to /etc/apparmor/, this patch allows it to be
configured by calling apparmor.aa.init_aa(confdir=PATH).
This allows for the make check target to use the in-tree config file,
profiles, and parser by default. A helper method, setup_aa(), is added
to common_test.py that checks for an environment variable containing a
non-default configuration directory path prior to calling
apparmor.aa.init_aa(). All test scripts that use apparmor.aa are updated
to call setup_aa().
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Suggested-by: Christian Boltz <apparmor@cboltz.de>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
Acked-by: Christian Boltz <apparmor@cboltz.de>
This allows to hand over any source instead of using the global variable.
Now that the function expects its input as parameter, get rid of the
global log_dict, which means
- change collapse_log() to initialize log_dict as local variable and
return it
- change do_logprof_pass() to catch collapse_log()'s return value and
hand it over to ask_the_questions()
- drop all references to the global log_dict variable
- update test-libapparmor-test_multi to follow the changes
Also fix an if condition that would fail if aa[profile][hat] does not
exist - get() defaults to None if the requested item doesn't exist, and
None.get('file') will raise an Exception.
Acked-by: Seth Arnold <seth.arnold@canonical.com>
This patch adds profiles for all log sniplets that are expected to
result in a profile rule.
This also means some changes in test-libapparmor-test_multi.py are
needed:
- split off log_to_profile_skip from log_to_profile_known_failures to
- only skip tests in log_to_profile_skip (causing a crash or requiring
user interaction)
- run tests in log_to_profile_known_failures, but expect a non-equal
result (caused by not added rules etc.)
- add quite some tests to log_to_profile_known_failures - they were
skipped before because they didn't have a *.profile file.
- add handling for hats to shorten list of known failures
This fixes testcase24 and testcase33 (after adjusting the profiles)
and lots of the new *.profile files.
- since we now have *.profile files for all log events that should result
in a profile rule, no longer ignore FileNotFoundError
Acked-by: Seth Arnold <seth.arnold@canonical.com>
This patch adds TestLogToProfile to test-libapparmor-test_multi.py which
"translates" the test_multi log sniplets to a profile, and checks if it
matches the expected profile.
The expected profile for one log event will obviously contain only one
rule, and gets added as *.profile to the test_multi directory.
This patch includes 33 test_multi profiles - which means 83 more need to
be created. Whenever you have some time, add one or two! (Please write
those test_multi profiles manually, without using the tools.)
I know some parts of the test code looks complicated. Unfortunately this
is how things work - compare it with do_logprof_pass() in aa.py...
While on it, set tests = 'invalid' which ensures a failure in case
parse_test_profiles() doesn't set the tests array, and move printing
the test name out of parse_test_profiles() to avoid printing it twice.
A nice side effect of this patch is increased test coverage:
- 30% -> 40% in aa.py (= 250 more lines)
- 52% -> 78% in aamode.py (= 23 more lines)
- 26% -> 68% in logparser.py (= 120 more lines)
- total coverage increases from 57% to 62%
Acked-by: Seth Arnold <seth.arnold@canonical.com>
As discussed a while ago, switch the utils (including their tests) to
use python3 by default. While on it, drop usage of "env" to always get
the system python3 instead of a random one that happens to live
somewhere in $PATH.
In practise, this patch doesn't change much - AFAIK openSUSE, Debian and
Ubuntu already patch aa-* to use python3.
Also add a note to README to officially deprecate Python 2.x.
(I won't break Python 2.x support intentionally - unless some future
change gives me a very good reason to finally drop Python 2.x support.)
Acked-by: Seth Arnold <seth.arnold@canonical.com>
(since 2016-08-23, but the commit had to wait for the FileRule series
because it touches test-file.py)
logparser.py does a regex check on log lines as performance improvement
so that it only hands over lines that look like AppArmor events to
LibAppArmor parsing. Those regexes were incomplete and didn't cover all
log formats LibAppArmor accepts, with the end result of "overlooking"
events.
This patch splits off common parts of the regex, adds more regexes for
several log types and finally merges everything into one regex.
test-libapparmor-test_multi.py now also checks all test_multi log lines
against the regex to ensure logparser.py doesn't silently ignore events.
test-logparser.py gets adjusted to the merged RE_LOG_ALL regex.
Finally, add a new test that was posted on IRC to the test_multi set.
As already threatened nearly a month ago,
Acked by <timeout> for trunk and 2.9
This testcase will parse all libraries/libapparmor/testsuite/test_multi
tests and compare the result with the *.out files.
It also include a "ToDo list" of keywords that are not yet supported in
the python code - those are typically related to rule types not
supported in the tools yet (dbus, signal etc.).
An interesting special case are exec events with network details:
testcase01.in, testcase12.in, testcase13.in
which might be hand-made, invalid logs, but nobody remembers ;-)
Acked-by <timeout>
