the cache test is failing because it assumes that kernel features are
stored in a file instead of a directory
Signed-off-by: John Johansen <john.johansen@canonical.com>
Acked-By: Steve Beattie <sbeattie@ubuntu.com>
On newer kernels the features directory causes the creation of a
cache/.feature file that contains newline characters. This causes the
feature comparison to fail, because get_flags_string() uses fgets
which stop reading in the feature file after the first newline.
This caches the features comparision to compare a single line of the
file against the full kernel feature directory resulting in caching
failure.
Worse this also means the cache won't get updated as the parser doesn't
change what set gets caches after the .feature file gets created.
Signed-off-by: John Johansen <john.johansen@canonical.com>
Acked-By: Steve Beattie <sbeattie@ubuntu.com>
Newer versions of AppArmor use a features directory instead of a file
update the parser to use this to determine features and match string
This is just a first pass at this to get things up quickly. A much
more comprehensive rework that can parse and use the full information
set is needed.
Signed-off-by: John Johansen <john.johansen@canonical.com>
The removal of deny information is a one way operation, that can result
in a smaller dfa, but also results in a dfa that should not be used in
future operations because the deny rules from the precomputed dfa would
not get applied.
For now default filtering out of deny information to off, as it takes
extra time and seldom results in further state reduction.
Signed-off-by: John Johansen <john.johansen@canonical.com>
Acked-by: Kees Cook <kees@ubuntu.com>
Previously permission information was thrown away early and permissions
where packed to their CHFA form at the start of DFA construction. Because
of this permissions hashing to setup the initial DFA partitions was
required as x transition conflicts, etc. could not be resolved.
Move the mapping of permissions to CHFA construction, and track the full
permission set through DFA construction. This allows removal of the
perm_hashing hack, which prevented a full minimization from happening
in some DFAs. It also could result in x conflicts not being correctly
detected, and deny rules not being fully applied in some situations.
Eg.
pre full minimization
Created dfa: states 33451
Minimized dfa: final partitions 17033
with full minimization
Created dfa: states 33451
Minimized dfa: final partitions 9550
Dfa minimization no states removed: partitions 9550
The tracking of deny rules through to the completed DFA construction creates
a new class of states. That is states that are marked as being accepting
(carry permission information) but infact are non-accepting as they
only carry deny information. We add a second minimization pass where such
states have their permission information cleared and are thus moved into the
non-accepting partion.
Signed-off-by: John Johansen <john.johansen@canonical.com>
Acked-by: Kees Cook <kees@ubuntu.com>
Profile loads when specifying namespaces currently conflict with caching.
If the profile (ignoring the specified namespace) is in the cache, then
the cached profile will be loaded, replacing the profile in the current
namespace instead of loading the profile to the new namespace.
Fix this by disabling caching when a namespace is specified, forcing the
profile to be compiled.
NOTE: this will not affect profiles loaded from within a namespace using
either the same or a separate directory as the base to load a namespac
from. This only affects loading profiles directly into a child
namespace.
Signed-off-by: John Johansen <john.johansen@canonical.com>
Acked-by: Kees Cook <kees@ubuntu.com>
Currently the cache location is fixed and links are needed to move it.
Add an option that can be set in the apparmor_parser.conf file so distros
can locate the cache where ever makes sense for them.
Signed-off-by: John Johansen <john.johansen@canonical.com>
Acked-by: Kees Cook <kees@ubuntu.com>
requested and fails to be created. Also don't make the
warning output conditional on the showcache flag as we
should be showing warning/errors by default.
Signed-off-by: John Johansen <john.johansen@canonical.com>
to /etc/apparmor/parser.conf (NOTE option to allow changing this is not
provided currently).
Signed-off-by: John Johansen <john.johansen@canonical.com>
Allow dumping out which states where dropped during partition minimization
and which state became the partitions representative state.
Signed-off-by: John Johansen <john.johansen@canonical.com>
Acked-By: Steve Beattie <sbeattie@ubuntu.com>
This is a rather large rearrangement of how a subset of the parser global
variables are defined. Right now, there are unit tests built without
linking against parser_main.c. As a result, none of the globals defined in
parser_main.c could be used in the code that is built for unit tests
(misc, regex, symtab, variable). To get a clean build, either stubs needed
to be added to "#ifdef UNIT_TEST" blocks in each .c file, or we had to
depend on link-time optimizations that would throw out the unused routines.
First, this is a problem because all the compile-time warnings had to be
explicitly silenced, so reviewing the build logs becomes difficult on
failures, and we can potentially (in really unlucky situations) test
something that isn't actually part of the "real" parser.
Second, not all compilers will allow this kind of linking (e.g. mips gcc),
and the missing symbols at link time will fail the entire build even though
they're technically not needed.
To solve all of this, I've moved all of the global variables used in lex,
yacc, and main to parser_common.c, and adjusted the .h files. On top of
this, I made sure to fully link the tst builds so all symbols are resolved
(including aare lib) and removedonly tst build-log silencing (for now,
deferring to another future patchset to consolidate the build silencing).
Signed-off-by: Kees Cook <kees.cook@canonical.com>
If the apparmor_parser is updated (outside of current packaging), when
doing profile loads it will use the existing cache of compiled profiles,
instead of forcing a recompile on profiles.
This can cause apparmor to load bad policy if the parser contains a bug
fix for the previous version of the parser.
This can be worked around in packaging by invalidating the cache and
forcing a profile reload when the parser is upgraded.
Signed-off-by: John Johansen <john.johansen@canonical.com>
Older versions of the apparmor kernel patches didn't handle receiving
network tables of a larger size than expected.
Allow the parser to detect the kernel version and override the AF_MAX
value for those kernels.
This also replaces the hack using a hardcoded limit of 36 for kernels
missing the features flag.
Signed-off-by: John Johansen <john.johansen@canonical.com>
Acked-by: Kees Cook <kees.cook@canonical.com>
When doing permission merging in the dfa minimization phase the information
about whether a rule is dominant or not has been lost so the merge of
xtransitions can not be handled correctly.
When two conflicting x transitions are merged the results are unpredicitable
and not currently detected. So default dfa minimization to set up its
initial partitions with permission hashing, this ensures that dfa states
that have different xtransitions in the minimization stage will never
be merged thus will not result in a conflict.
x permission checking is still enforced at the dfa creation phase where
the originial information is available to check whether the conflicting
permissions came from exact match or re rules so that conflict resolution
can be properly applied.
The end result is that dfa minimization does not result in a truely minimal
dfa (the minimization phase is also slightly faster).
Signed-off-by: John Johansen <john.johansen@canonical.com>
add short options to turn on all stats, and all progress indicators,
also allow adding "no-" prefix to dump options to allow subtracting
individual options when short options are used.
eg.
-D stats -D no-expr-simplify
Move the -O and -D options into tables, that keep the option and its
description. This will help keep the options consistent and the description
up to date, as all information is now in one place.
Previously the options, and descriptions kept getting out of sync as all
relavent parts were spread out.
improves minimization performance, it can slow down total creation time and
result in larger compressed dfas.
This is because it results in the dfa not being completely minimized which
with the current O(n2) dfa table compression algorithm can result in slower
compressed dfa generation.
first hash does hashing on state just state transitions, which always results
in a performance improvement.
The second does hashing based off of accept permissions, which can create
more initial states but can result in not being able to achieve a true
minimum dfa. This can also lead to slowing down total dfa creation because
while minimization, compression can take longer if the dfa isn't completely
minimized.
permission hashing is currently required, as minimization does not accumulate
redundant Node permissions.
parsing, and precompilation of policy. This allows finding the most
recent text time stamp during parsing and this is then compared to
the cache file time stamp.
While this is slightly slower than the cache file check that only
validated against the profile file it fixes the bug where abstraction
updates do not cause the cache file to become invalid.
this results in
Unable to open output file - Success
to be output to standard error.
This occurs because despite specifying kernel_load = 0, the kernel load
parts are still being done, and failing.
tree. It is limited in that it doesn't currently handle the permissions of a rule.
conversion output presents an aare -> prce conversion followed by 1 or more expression
tree rules, governed by what the rule does.
eg.
aare: /** -> /[^/\x00][^\x00]*
rule: /[^/\x00][^\x00]* -> /[^\0000/]([^\0000])*
eg.
echo "/foo { /** rwlkmix, } " | ./apparmor_parser -QT -D rule-exprs -D expr-tree
aare: /foo -> /foo
aare: /** -> /[^/\x00][^\x00]*
rule: /[^/\x00][^\x00]* -> /[^\0000/]([^\0000])*
rule: /[^/\x00][^\x00]*\x00/[^/].* -> /[^\0000/]([^\0000])*\0000/[^/](.)*
DFA: Expression Tree
(/[^\0000/]([^\0000])*(((((((((((((<513>|<2>)|<4>)|<8>)|<16>)|<32>)|<64>)|<8404992>)|<32768>)|<65536>)|<131072>)|<262144>)|<524288>)|<1048576>)|/[^\0000/]([^\0000])*\0000/[^/](.)*((<16>|<32>)|<262144>))
This simple example shows many things
1. The profile name under goes pcre conversion. But since no regular expressions where found
it doesn't generate any expr rules
2. /** is converted into the pcre expression /[^\0000/]([^\0000])*
3. The pcre expression /[^\0000/]([^\0000])* is converted into two rules that are then
converted into expression trees.
The reason for this can not be seen by the output as this is actually triggered by
permissions separation for the rule. In this case the link permission is separated
into what is shown as the second rule: statement.
4. DFA: Expression Tree dump shows how these rules are combined together
You will notice that the rule conversion statement is fairly redundant currently as it just
show pcre to expression tree pcre. This will change when direct aare parsing occurs,
but currently serves to verify the pcre conversion step.
It is not the prettiest patch, as its touching some ugly code that is schedule to be cleaned
up/replaced. eg. convert_aaregex_to_pcre is going to replaced with native parse conversion
from an aare straight to the expression tree, and dfaflag passing will become part of the
rule set.
not an algorithmic improvement. It does the same basic algorithm of
test until it can insert the data, but instead of only tracking the
first free entry (and recomputing it each pass). It tracks all
free entries reducing the number of comparisons done and the table
grows in size.
This may actually result in a small loss on small tables, but is a win
for larger tables.
Add basic Hopcroft based dfa minimization. It currently does a simple
straight state comparison that can be quadratic in time to split partitions.
This is offset however by using hashing to setup the initial partitions so
that the number of states within a partition are relative few.
The hashing of states for initial partition setup is linear in time. This
means the closer the initial partition set is to the final set, the closer
the algorithm is to completing in a linear time. The hashing works as
follows: For each state we know the number of transitions that are not
the default transition. For each of of these we hash the set of letters
it can transition on using a simple djb2 hash algorithm. This creates
a unique hash based on the number of transitions and the input it can
transition on. If a state does not have the same hash we know it can not
the same as another because it either has a different number of transitions
or or transitions on a different set.
To further distiguish states, the number of transitions of each transitions
target state are added into the hash. This serves to further distiguish
states as a transition to a state with a different number of transitions
can not possibly be reduced to an equivalent state.
A further distinction of states is made for accepting states in that
we know each state with a unique set of accept permissions must be in
its own partition to ensure the unique accept permissions are in the
final dfa.
The unreachable state removal is a basic walk of the dfa from the start
state marking all states that are reached. It then sweeps any state not
reached away. This does not do dead state removal where a non accepting
state gets into a loop that will never result in an accepting state.
