Split the features file into compile features and kernel features
which is needed for policy versioning and the new caching scheme.
A new flag --kernel-features was added to set the kernel features but
unfortunately -M, --features-file was setup to only specify the
compile features, when it used to effectively specify both the
compile and kernel features.
This broke existing uses of -M.
Fix this by having -M specify both the compile and kernel features,
and a new flag --compile-features that can be used to specify the
compile fature set separate from the kernel feature set.
sbeattie> fixed up error message to refer to compile features when
--compile-features argument fails.
Fixes: 9e48a5da5e ("parser: split kernel features from compile features.")
Signed-off-by: John Johansen <john.johansen@canonical.com>
Signed-off-by: Steve Beattie <steve.beattie@canonical.com>
PR: https://gitlab.com/apparmor/apparmor/merge_requests/104
The parser currently uses a fork model to do job processing. For
consistency even when the number of jobs is set to 1 a single
work process is forked. However this makes using gdb more difficult
and can be even worse for other debugging tools.
Make -j 0 disable all job spawning so all processing happens in the
main process.
PR: https://gitlab.com/apparmor/apparmor/merge_requests/105
Signed-off-by: John Johansen <john.johansen@canonical.com>
Acked-by: Steve Beattie <steve@nxnw.org>
Allow the parser to use cache overlays by extending the --cache-loc
flag to support multiple locations via a comma separated list.
eg.
--cache-loc=/var/cache/apparmor/,/etc/apparmor.d/cache.d/
The overlayed cache directories are searched in the order
specified. So in the above example /var/cache/apparmor is searched
before /etc/apparmor.d/
Time stamps are ignored in the search, the first match found wins
regardless if there exists a matching cache file with a newer timestamp
in a directory is later in the search.
Cache writes will only occur to the first dir in the list. So
/var/cache/apparmor/ in the above example.
Signed-off-by: John Johansen <john.johansen@canonical.com>
Acked-by: Christian Boltz <apparmor@cboltz.de>
The feature set needs to be split, the kernel features set determines
the cache location and controls features down grades to ensure
policy generates a policy that is usable on a given kernel.
The compile featurs set governs the feature set supported by policy
and primarily determines how policy is parsed and compiled.
Taking the intersection of the two feature sets to determine rule
downgrades for a specific kernel is left to a separate patch.
Signed-off-by: John Johansen <john.johansen@canonical.com>
Acked-by: Christian Boltz <apparmor@cboltz.de>
Add the support to have the cache be able to search multiple locations
so that the policy cache can be split into multiple locations and
that there can be a local cache that can override preshipped caches.
Signed-off-by: John Johansen <john.johansen@canonical.com>
Acked-by: Christian Boltz <apparmor@cboltz.de>
Make the internal cache dir tracking use a fixed array and update
all references to the internal dirfd to index the array.
Signed-off-by: John Johansen <john.johansen@canonical.com>
Acked-by: Christian Boltz <apparmor@cboltz.de>
Move the policy cache directory from <cacheloc>/cache/ to
<cacheloc>/cache.d/<features_id>/ where <features_id> is a unique
identifier for a set of aa_features. This allows for multiple AppArmor
policy caches exist on a system. Each policy cache will uniquely
correspond to a specific set of AppArmor kernel features. This means
that a system can reboot into a number of different kernels and the
parser will select the existing policy cache that matches each kernel's
set of AppArmor features.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: John Johansen <john.johansen@canonical.com>
The --print-cache-dir option can be used to have the parser print the
value of the cache directory that is specific to the features used (from
the current kernel, the --match-string option, or the --features-file
option). After printing the path, apparmor_parser will exit. This is
helpful because the final component in the path will become
unpredictable because it will be based on arbitrary hash function
output.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: John Johansen <john.johansen@canonical.com>
BugLink: http://bugs.launchpad.net/bugs/1566490
This patch enables to parser to scale the max jobs if new resources are
being brought online by the scheduler.
It only enables the scaling check if there is a difference between the
maximum number of cpus (CONF) and the number of online (ONLN) cpus.
Instead of checking for more resources regardless, of whether the online
cpu count is increasing it limits its checking to a maximum of
MAX CPUS + 1 - ONLN cpus times. With each check coming after fork spawns a
new work unit, giving the scheduler a chance to bring new cpus online
before the next check. The +1 ensures the checks will be done at least
once after the scheduling task sleeps waiting for its children giving
the scheduler an extra chance to bring cpus online.
Signed-off-by: John Johansen <john.johansen@canonical.com>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
Check if the current kernel supports stacking. If not, ensure that named
transitions (exec, change_profile, etc.) do not attempt to stack their
targets.
Also, set up the change_profile vector according to whether or not the
kernel supports stacking. Earlier kernels expect the policy namespace to
be in its own NUL-terminated vector element rather than passing the
entire label (namespace and profile name) as a single string to the
kernel.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
BugLink: http://bugs.launchpad.net/bugs/1534405
Patch -r 2952 switched over to using the library kernel interface, and
added a kernel_interface parameter to the dir_cb struct, that is
used to process directories.
Unfortunately kernel_interface parameter of the dir_cb struct is not being
properly initialized resulting in odd failures and sefaults when the parser
is processing directories.
Signed-off-by: John Johansen <john.johansen@canonical.com>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
Acked-by: Steve Beattie <steve@nxnw.org>
This adds a basic support for parallel compiles. It uses a fork()/wait
model due to the parsers current dependence on global variables and
structures. It has been setup in a similar manner to how cilk handles
multithreading to make it easy to port to a managed thread model once
the parser removes the dependence on global compute structures in the
backend.
This patch adds two new command line flags
-j <n> or --jobs <n>
which follows the make syntax of specifying parallel jobs currently
defaults to -jauto
-j8 or --jobs=8 allows for 8 parallel jobs
-jauto or --jobs=auto sets the jobs to the # of cpus
-jx4 or --jobs=x4 sets the jobs to # of cpus * 4
-jx1 is equivalent to -jauto
Note: unlike make -j must be accompanied by an option
--max-jobs=<n>
allows setting hard cap on the number of jobs that can be specified
by --jobs. It defaults to the number of processors in the system * 8.
It supports the "auto" and "max" keywords, and using x<n> for a
multiple of the available cpus.
additionally the -d flag has been modified to take an optional parameter
and
--debug=jobs
will output debug information for the job control logic.
In light testing on one machine the job control logic provides a nice
performance boost. On an x86 test machine with 60 profiles in the
/etc/apparmor.d/ directory, for the command
time apparmor_parser -QT /etc/apparmor.d/
old (equiv of -j1):
real 0m10.968s
user 0m10.888s
sys 0m0.088s
ubuntu parallel load using xargs:
real 0m8.003s
user 0m21.680s
sys 0m0.216s
-j:
real 0m6.547s
user 0m17.900s
sys 0m0.132s
Signed-off-by: John Johansen <john.johansen@canonical.com>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
Drop the reference to the libapparmor policy_cache pseudo object when
the parser is done.
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
The errno values libapparmor's aa_policy_cache_new() uses to indicate
when the cache directory does not exist and when an existing, invalid
cache already exists needed to be separated out. They were both ENOENT
but now the latter situation uses EEXIST.
libapparmor also needed to be updated to not print an error message to
the syslog from aa_policy_cache_new() when the max_caches parameter is
0, indicating that a new cache should not be created, and the cache
directory does not exist. This is an error situation but a debug message
is more appropriate.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: John Johansen <john.johansen@canonical.com>
The aa_policy_cache_new() and aa_policy_cache_remove() functions are
changed to accept a dirfd parameter.
The cache dirfd (by default, /etc/apparmor.d/cache) is opened earlier in
aa_policy_cache_new(). Previously, the directory wasn't accessed until
later in the following call chain:
aa_policy_cache_new() -> init_cache_features() -> create_cache()
Because of this change, the logic to create the cache dir must be moved
from create_cache() to aa_policy_cache_new().
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: John Johansen <john.johansen@canonical.com>
Instead of only accepting a path in the aa_features API, accept a
directory file descriptor and a path like then openat() family of
syscalls. This type of interface is better since it can operate exactly
like a path-only interface, by passing AT_FDCWD or -1 as the dirfd.
However, using the dirfd/path combination, it can eliminate string
allocations needed to open files in subdirectories along with the
even more important benefits mentioned in the open(2) man page.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
The _aa_dirat_for_each() function used the DIR * type for its first
parameter. It then switched back and forth between the directory file
descriptors, retrieved with dirfd(), and directory streams, retrieved
with fdopendir(), when making syscalls and calling the call back
function.
This patch greatly simplifies the function by simply using directory
file descriptors. No functionality is lost since callers can still
easily use the function after calling dirfd() to retrieve the underlying
file descriptor.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
This patch changes the aa_policy_cache_new() prototype and gets rid of
aa_policy_cache_is_valid() and aa_policy_cache_create().
The create bool of aa_policy_cache_new() is replaced with a 16 bit
unsigned int used to specify the maximum number of caches that should be
present in the specified cache directory. If the number is exceeded, the
old cache directories are reaped. The definition of "old" is private to
libapparmor and only 1 cache directory is currently supported. However,
that will change in the near future and multiple cache directories will
be supported.
If 0 is specified for the max_caches parameter, no new caches can be
created and only an existing, valid cache can be used. An error is
returned if no valid caches exist in that case.
If UINT16_MAX is specified, an unlimited amount of caches can be created
and reaping is disabled.
This means that 0 to (2^16)-2, or infinite, caches will be supported in
the future.
This change allows for the parser to continue to support the
--skip-bad-cache (by passing 0 for max_caches) and the --write-cache
option (by passing 1 or more for max_caches) without confusing
libapparmor users with the aa_policy_cache_{is_valid,create}()
functions.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: John Johansen <john.johansen@canonical.com>
Currently the cache file has its mtime set at creation time, but this
can lead to cache issues when a policy file is updated separately from
the cache. This makes it possible for an update to ship a policy file
that is newer than the what the cache file was generated from, but
result in a cache hit because the cache file was local compiled after
the policy file was package into an update (this requires the update
to set the mtime of the file when locally installed to the mtime of
the file in its update archive but this is commonly done, especially
in image based updates).
Signed-off-by: John Johansen <john.johansen@canonical.com>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
With create_cache() headed for libapparmor, we can't use the show_cache
or write_cache globals.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: John Johansen <john.johansen@canonical.com>
Create new, ref, and unref functions for aa_kernel_interface. The "new"
function allows for the caller to pass in an aa_features object that is
then used to check if the kernel supports set load operations.
Additionally, the "new" function allows for the apparmorfs path to be
discovered once instead of during every policy load.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: John Johansen <john.johansen@canonical.com>
This is the start of the kernel_interface API that allows callers to
specify a buffer, a file path, or a file descriptor that should be
copied to the proper kernel interface for loading, replacing, or
removing in-kernel policies.
Support exists for reading from a file path or file descriptor into a
buffer and then writing that buffer to the appropriate apparmorfs
interface file.
An aa_kernel_interface_write_policy() function is also provided for
callers that want to route a buffer to an arbitrary file descriptor
instead of to an apparmorfs file. This is useful when an admin instructs
apparmor_parser to write to stdout or a file.
Additionally, it removes some parser-specific globals from the
kernel_interface.c file, such as OPTION_{ADD,REPLACE,REMOVE}, in
preparation for moving the code into a library.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: John Johansen <john.johansen@canonical.com>
This function allows for a policy cache to be removed without having a
previously instatiated aa_policy_cache object. It simply works off of a
path.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: John Johansen <john.johansen@canonical.com>
This API has the same look-and-feel of the previous aa_features API.
The cache setup code was heavily dependent on globals set by CLI
options. Options such as "skip the read cache", or "skip the write
cache", or "don't clear the cache if it isn't valid", won't be useful
for all aa_policy_cache API users so some of that logic was lifted out
of the API. The constructor function still provides a bool parameter
that specifies if the cache should be created or not.
If the policy cache is invalid (currently meaning that the cache
features file doesn't match the kernel features file), then a new
aa_policy_cache object is still created but a call to
aa_policy_cache_is_valid() will return false. The caller can then decide
what to do (create a new valid cache, stop, etc.)
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: John Johansen <john.johansen@canonical.com>
This option adds unneeded complexity to the parser CLI and the upcoming
aa_policy_cache API. Get rid of it and simply create the cache dir if
--write-cache is specified.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: John Johansen <john.johansen@canonical.com>
Defines a function that can be called to test features support. It is
string based which allows the support tests to work with new kernel
features without any changes.
The use of global variables in the parser to store and check features
support is still preserved. The parser should probably move over to
passing the aa_features object around but that's left for later.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: John Johansen <john.johansen@canonical.com>
The aa_features_new_*() functions create an aa_features object. They can
be thought of as the constructor of aa_features objects. A number of
constructors are available depending on whether the features are coming
from a file in the policy cache, a string specified on the command line,
or from apparmorfs.
The aa_features_ref() and aa_features_unref() functions are used to grab
and give up references to an aa_features. When the ref count hits zero,
all allocated memory is freed. Like with free(), aa_features_unref() can
be called with a NULL pointer for convenience.
Pre-processor macros are hidden behind functions so that they don't
become part of our ABI when we move this code into libapparmor later on.
A temporary convenience function, aa_features_get_string(), is provided
while code that uses aa_features is migrated from expecting raw features
string access to something more abstract. The function will be removed
in an upcoming patch.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: John Johansen <john.johansen@canonical.com>
This keeps us from having to use the force_clear_cache global in
policy_cache.c.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: John Johansen <john.johansen@canonical.com>
Require the caller of setup_cache() to pass in a valid cache location
string. This removes the use of the basedir global from the
policy_cache.c file.
Additionally, it is no longer necessary to return the "cache dir" path
from setup_cache() since it will always be the same as the input path.
The return value is changed to an int so an error code can be returned
instead of using exit().
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: John Johansen <john.johansen@canonical.com>
Modify setup_cache() to accept the user-supplied cacheloc and return the
validated or created cache directory. The caller must then track that
variable and pass it into any parser/policy_cache.c functions that need
it.
The main reason for this change is that the cache location and the cache
directory will soon be two different paths. The cache location will
typically be the parent of the cache directory.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: John Johansen <john.johansen@canonical.com>
This patch moves the logic that sets up the policy into a new function
in policy_cache.c
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: John Johansen <john.johansen@canonical.com>
Signed-off-by: John Johansen <john.johansen@canonical.com>
[tyhicks: Fixed build failures]
[tyhicks: Fixed bug where a warning was being printed when it shouldn't]
[tyhicks: Forward ported to trunk]
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Signed-off-by: John Johansen <john.johansen@canonical.com>
[tyhicks: Handle inverted return from find_subdomainfs_mountpoint()]
[tyhicks: Link test progs to libapparmor to fix make check build fail]
[tyhicks: Migrate from opendir() to open() for opening apparmorfs]
[tyhicks: Make some of the split out functions static]
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
While some of these allocations will go away as we convert to C++,
some of these need to stay C as the are going to be moved into a
library to support loading cache from init daemons etc.
For the bits that will eventually be C++ this helps clean things up,
in the interim.
TODO: apply to libapparmor as well
Signed-off-by: John Johansen <john.johansen@canonical.com>
Acked-by: Tyler Hicks <tyhicks@canonical.com>
While some of these allocations will go away as we convert to C++,
some of these need to stay C as the are going to be moved into a
library to support loading cache from init daemons etc.
For the bits that will eventually be C++ this helps clean things up,
in the interim.
TODO: apply to libapparmor as well
Signed-off-by: John Johansen <john.johansen@canonical.com>
Signed-off-by: John Johansen <john.johansen@canonical.com>
[tyhicks: Don't move globals in favor of lifting those out later]
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
Currently the cache tracks the most recent timestamp of parsed files
and then compares that to the cache timestamp. This unfortunately
prevents the parser from being able to know which files caused the
cache check failure.
Rework the cache check so that there is a debug option, and that
the cache file timestamp is set first so that we can output
a deug message for each file that causes a cache check failure.
Signed-off-by: John Johansen <john.johansen@canonical.com>
[tyhicks: Forward ported to trunk and minor cleanups]
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: Steve Beattie <steve@nxnw.org>
The error path was being taken when openat() return 0 but openat()
returns -1 on error.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
Currently the apparmor parser warns about rules that are not enforced or
downgraded. This is a problem for distros that are not carrying the out of
tree kernel patches, as most profile loads result in warnings.
Change the behavior to not output a message unless a warn flag is passed.
This patch adds 2 different warn flags
--warn rule-downgraded # warn if a rule is downgraded
--warn rule-not-enforced # warn if a rule is not enforced at all
If the warnings are desired by default the flags can be set in the
parser.conf file.
v2 of patch
- update man page
- add --warn to usage statement
- make --quiet clear warn flags
Signed-off-by: John Johansen <john.johansen@canonical.com>
Acked-by: Steve Beattie <steve@nxnw.org>
