apparmor/parser/libapparmor_re
John Johansen cfe20d2b63 Add support for profiles with xattrs matching
Add userland support for matching based on extended file attributes. This
leverages DFA based matching already in the kernel:

https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=8e51f908
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=73f488cd

Matching is exposed via flags on the profile:

    /usr/bin/* xattrs=(user.foo=bar user.bar=foo) {
        # ...
    }

xattr values are appended to the existing xmatch via a null transition.

    $ echo '/usr/bin/* xattrs=(user.foo=foo user.bar=bar) {}' | \
        ./parser/apparmor_parser -QT -D expr-tree
    DFA: Expression Tree
    /usr/bin/[^\0000/]([^\0000/])*(\0000bar)?(\0000foo)?< 0x1>
    DFA: Expression Tree
    (\a|(\n|(\0002|\t)))< 0x4>

Tested manually on a 4.19 kernel via QEMU+KVM.

TODO:

  * ~~Add regression tests~~ (EDIT: done)
  * ~~EDIT: add support in the tools~~ (EDIT: done)

Questions for reviewers:

  * ~~parser/libapparmor: regex construction probably needs cleaning up~~ (EDIT: done)
  * ~~parser/parser_regex.c: confused what xmatch length is for~~ (EDIT: done)

/cc @mjg59

PR: https://gitlab.com/apparmor/apparmor/merge_requests/270
Signed-off-by: John Johansen <john.johansen@canonical.com>
2019-03-21 08:12:07 +00:00
..
aare_rules.cc parser: add support for matching based on extended file attributes 2019-03-14 10:47:54 -07:00
aare_rules.h parser: add support for matching based on extended file attributes 2019-03-14 10:47:54 -07:00
apparmor_re.h Fix dfa minimization 2014-01-09 17:06:48 -08:00
chfa.cc parser: fix compilation failure on 32 bit systems 2014-01-10 11:02:59 -08:00
chfa.h Fixes to that where dropped from the diff-encode patch 2014-01-09 17:24:40 -08:00
expr-tree.cc parser: add support for matching based on extended file attributes 2019-03-14 10:47:54 -07:00
expr-tree.h parser: add support for matching based on extended file attributes 2019-03-14 10:47:54 -07:00
flex-tables.h Add Differential State Compression to the DFA 2014-01-09 16:55:55 -08:00
hfa.cc Fix compilation of audit modifiers 2015-03-18 10:05:55 -07:00
hfa.h parser/libapparmor_re: remove unnecessary throw(int) 2019-03-18 10:57:05 -07:00
Makefile parser: Honor USE_SYSTEM make variable in libapparmor_re 2015-03-25 17:09:25 -05:00
parse.h Split out parsing and expression trees from regexp.y 2011-03-13 05:46:29 -07:00
parse.y parser: revert changes from commit rev 3248 2015-10-14 13:49:26 -07:00
README Add DFA table format README. 2007-04-03 13:53:24 +00:00

Regular Expression Scanner Generator
====================================

Notes in the scanner File Format
--------------------------------

The file format used is based on the GNU flex table file format
(--tables-file option; see Table File Format in the flex info pages and
the flex sources for documentation). The magic number used in the header
is set to 0x1B5E783D insted of 0xF13C57B1 though, which is meant to
indicate that the file format logically is not the same: the YY_ID_CHK
(check) and YY_ID_DEF (default) tables are used differently.

Flex uses state compression to store only the differences between states
for states that are similar. The amount of compresion influences the parse
speed.

The following two states could be stored as in the tables outlined
below:

States and transitions on specific characters to next states
------------------------------------------------------------
 1: ('a' => 2, 'b' => 3, 'c' => 4)
 2: ('a' => 2, 'b' => 3, 'd' => 5)

Flex-like table format
----------------------
index: (default, base)
    0: (      0,    0)  <== dummy state (nonmatching)
    1: (      0,    0)
    2: (      1,  256)

  index: (next, check)
      0: (   0,     0)  <== unused entry
	 (   0,     1)  <== ord('a') identical entries
  0+'a': (   2,     1)
  0+'b': (   3,     1)
  0+'c': (   4,     1)
	 (   0,     1)  <== (255 - ord('c')) identical entries
256+'c': (   0,     2)
256+'d': (   5,     2)

Here, state 2 is described as ('c' => 0, 'd' => 5), and everything else
as in state 1. The matching algorithm is as follows.

Flex-like scanner algorithm
---------------------------
  /* current state is in <state>, input character <c> */
  while (check[base[state] + c] != state)
    state = default[state];
  state = next[state];
  /* continue with the next input character */

This state compression algorithm performs well, except when there are
many inverted or wildcard matches ("[^x]", "."). Each input character
may cause several iterations in the while loop.


We will have many inverted character classes ("[^/]") that wouldn't
compress very well. Therefore, the regexp matcher uses no state
compression, and uses the check and default tables differently. The
above states could be stored as follows:

Regexp table format
-------------------

index: (default, base)
    0: (      0,    0)  <== dummy state (nonmatching)
    1: (      0,    0)
    2: (      1,    3)

  index: (next, check)
      0: (   0,     0)  <== unused entry
	 (   0,     0)  <== ord('a') identical, unused entries
  0+'a': (   2,     1)
  0+'b': (   3,     1)
  0+'c': (   4,     1)
  3+'a': (   2,     2)
  3+'b': (   3,     2)
  3+'c': (   0,     0)  <== entry is unused
  3+'d': (   5,     2)
	 (   0,     0)  <== (255 - ord('d')) identical, unused entries

All the entries with 0 in check (except the first entry, which is
deliberately reserved) are still available for other states that
fit in there.

Regexp scanner algorithm
------------------------
  /* current state is in <state>, matching character <c> */
  if (check[base[state] + c] == state)
    state = next[state];
  else
    state = default[state];
  /* continue with the next input character */

This representation and algorithm allows states which match more
characters than they do not match to be represented as their inverse. 
For example, a third state that accepts everything other than 'a' can
be added to the tables as one entry in (default, base) and one entry in
(next, check):

State
-----
 3: ('a' => 0, everything else => 5)

Regexp tables
-------------
index: (default, base)
    0: (      0,    0)  <== dummy state (nonmatching)
    1: (      0,    0)
    2: (      1,    3)
    3: (      5,    7)

  index: (next, check)
      0: (   0,     0)  <== unused entry
	 (   0,     0)  <== ord('a') identical, unused entries
  0+'a': (   2,     1)
  0+'b': (   3,     1)
  0+'c': (   4,     1)
  3+'a': (   2,     2)
  3+'b': (   3,     2)
  3+'c': (   0,     0)  <== entry is unused
  3+'d': (   5,     2)
  7+'a': (   0,     3)
	 (   0,     0)  <== (255 - ord('a')) identical, unused entries

While the current code does not implement any form of state compression,
the flex state compression representation could be combined by
remembering (in a bit per state, for example) which default entries
refer to inverted matches, and which refer to parent states.