The new check-one-test-% pattern rule allows running individual test scripts.
This allows them to be tested in parallel across many Make worker threads or
across many distinct machines with spread.
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
... for handling pivot_root rules.
This replaces the old code that just stores the full rule as text.
We also get rid of the old ['allow'] and ['deny'] items in
ProfileStorage, the handling of old write functions, and the last usage
of _Raw_Rule (and therefore _Raw_Rule itsself).
Also delete the old test-pivot_root_parse.py which relied on the ancient
code, and even used a wrong syntax in its test rules.
Since we are using ubuntu:latest, and noble was released, some tests
are failing.
shellcheck needs python3 to run, which was possibly installed by
default in previous ubuntu images and is no longer the case.
Ignore dist-packages python files during our coverage tests.
Fixes: https://gitlab.com/apparmor/apparmor/-/issues/388
Signed-off-by: Georgia Garcia <georgia.garcia@canonical.com>
... and a simple test for a single (fake) event for ping.
Notes:
- to let aa-logprof work in the CI environment, we need to skip checking
for the AppArmor mountpoint. Introduce --no-check-mountpoint for this.
- PYTHONPATH and LD_LIBRARY_PATH need to be explicitely forwarded when
starting aa-logprof via subprocess.Popen()
- if the test runs with coverage enabled, it will also start aa-logprof
with coverage (parameters copied from Makefile).
Speaking about coverage - this test adds 4% overall coverage, and 10%
more coverage for apparmor/aa.py.
We sometimes have random coverage changes that are not reproducible and
therefore hard to debug.
Generate html coverage as part of make coverage-regression, and keep the
resulting utils/test/htmlcov/ as artifact to make debugging easier.
coverage-html needs JS files from various libjs-* packages, install them
in before_script
... by adding some new tests, and by marking two lines as "pragma: no
branch" because I didn't find a testcase that doesn't let them continue
with the next line.
Finally, remove severity.py from the "not 100% covered" list in
test/Makefile.
... to find regressions or improvements in the python code coverage.
`make coverage-regression` will error out if a file looses its 100%
coverage, or if a file improved to 100% coverage.
Other coverage changes (for example 45% -> 47%) will be ignored.
Similar to the profiles/ check using the python utilities, the
tests for the python utilities were not including the path for the
swig libapparmor library in the LD_LIBRARY_PATH variable, only in
PYTHONPATH. This commit fixes that, renaming the variable used for
the built libapparmor check.
v2:
- actually use the LIBAPPARMOR_PATH variable when defining
LD_LIBRARY_PATH
Bug: https://gitlab.com/apparmor/apparmor/-/issues/98
Signed-off-by: Steve Beattie <steve.beattie@canonical.com>
Acked-by: John Johansen <john.johansen@canonical.com>
MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/586
- Code layout based on aa-genprof example
- Extend Python dependencies to cover new need by aa-notify
- Update documentation after aa-notify is no longer in Perl
This will help ensure the future rewrite of aa-notify from Perl to Python
is less likely to introduce regressions. Tests run the command line utility
via a subprocess so it does not matter that the tests are in Python but
the aa-notify utility is in Perl (for now).
Exit rather than returning from shell snippets in Makefiles. It is
reported that returning causes the following error message with bash:
/bin/sh: line 4: return: can only `return' from a function or sourced script
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Reported-by: Christian Boltz <apparmor@cboltz.de>
The test-aa-easyprof.py script relies on the parser to be built so the
check target of the utils/test/Makefile should detect if the parser
exists before running any tests.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Reported-by: Christian Boltz <apparmor@cboltz.de>
if USE_SYSTEM is not set, the utils make check target will instruct
test-aa-easyprof.py to provide the path of the in-tree parser executable
to aa-easyprof.
If USE_SYSTEM is set, the default parser path (/sbin/apparmor_parser or
the result of `which apparmor_parser`) is used.
The test-aa-easyprof.py script receives the parser path by checking the
__AA_PARSER environment variable. This environment variable is strictly
used by the test script and not any user-facing code so two leading
underscores were used.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: Christian Boltz <apparmor@cboltz.de>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
If USE_SYSTEM is not set, the utils make check target will instruct
test-aa-easyprof.py to provide the path of the in-tree
profiles/apparmor.d directory to aa-easyprof as the parser base
directory.
If USE_SYSTEM is set, the default base directory (/etc/apparmor.d) is
used.
The test-aa-easyprof.py script receives the base path by checking the
__AA_BASEDIR environment variable. This environment variable is strictly
used by the test script and not any user-facing code so two leading
underscores were used.
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Acked-by: Christian Boltz <apparmor@cboltz.de>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
Bug: https://launchpad.net/bugs/1538306
Introduce an apparmor.aa.init_aa() method and move the initialization
code of the apparmor.aa module into it. Note that this change will break
any external users of apparmor.aa because global variables that were
previously initialized when importing apparmor.aa will not be
initialized unless a call to the new apparmor.aa.init_aa() method is
made.
The main purpose of this change is to allow the utils tests to be able
to set a non-default location for configuration files. Instead of
hard-coding the location of logprof.conf and other utils related
configuration files to /etc/apparmor/, this patch allows it to be
configured by calling apparmor.aa.init_aa(confdir=PATH).
This allows for the make check target to use the in-tree config file,
profiles, and parser by default. A helper method, setup_aa(), is added
to common_test.py that checks for an environment variable containing a
non-default configuration directory path prior to calling
apparmor.aa.init_aa(). All test scripts that use apparmor.aa are updated
to call setup_aa().
Signed-off-by: Tyler Hicks <tyhicks@canonical.com>
Suggested-by: Christian Boltz <apparmor@cboltz.de>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
Acked-by: Christian Boltz <apparmor@cboltz.de>
While running test-translations.py with the fixed german translations,
I noticed that I still get errors about hotkey conflicts
It turned out that test-translations.py reads the system-wide
apparmor-utils.mo in addition to the in-tree translations.
(I have the 2.11 beta1 translations installed, which contain hotkey
conflicts for the german translations).
This is surprising because test-translations.py explicitely sets the
locale path. Interestingly, this happens only 4 times (checked with a
temp profile with audit for those files) while test-translations.py has
9 tests).
(Any idea if this behaviour is normal or a bug?)
This patch adds LC_ALL=C to the make check and make coverage commandline
so that the system-wide translations don't get used.
I checked with a modified de.po that in-tree hotkey conflicts still get
detected.
Acked-by: Seth Arnold <seth.arnold@canonical.com>
The last utils/test/Makefile change switched to using the in-tree
libapparmor by default (unless USE_SYSTEM=1 is given). However, I missed
to add the swig/python parts of libapparmor to PYTHONPATH, so the
system-wide LibAppArmor/__init__.py was always used.
This patch adds the in-tree libapparmor python module to PYTHONPATH.
I'm sorry for the interesting[tm] way to find out that path, but
a) I don't know a better / less ugly way and
b) a similar monster already works in libapparmor/swig/python/test/ ;-)
Acked-by: John Johansen <john.johansen@canonical.com> for 2.9 and trunk
(that also implies 2.10 ;-)
Also add support for the USE_SYSTEM variable, which means:
- test against the in-tree libapparmor and python modules by default
- test against the system libapparmor and python modules if USE_SYSTEM
is set
The old behaviour was a mix of both - it always used the in-tree python
modules and the system libapparmor.
For obvious reasons, you'll need to build libapparmor before running the
tests (unless you specify USE_SYSTEM=1 as parameter to make check).
Acked-by: John Johansen <john.johansen@canonical.com> for trunk and 2.9
- drop the symlink magic of the common/ directory, and just include
files directly from there.
- update comments indicating required steps to take when including
common/Make.rules
- drop make clean steps that refer to no longer generated tarballs,
specfiles, and symlinks to the common directory/Make.rules.
- don't silence clean steps if VERBOSE is set
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: Christian "Ghostbuster" Boltz <apparmor@cboltz.de>
If one of the testcases fail, this goes unnoticed in "make coverage".
This patch changes the Makefile so that test failures let
"make coverage" fail.
You can use make COVERAGE_IGNORE_FAILURES=true coverage to build
coverage data even if some tests fail.
Signed-off-by: Steve Beattie <steve@nxnw.org>
(which was most probably meant as an Acked-by)
Also Acked-by: <timeout> ;-)
This patch adds support for generating test coverage information for the
python utils.
To view a text based report, in the test subdirectory do:
make coverage-report
To generate detailed html reports, do:
make coverage-html
And then point your web browser at
$(YOUR_CURRENT_WORKING_TREE)/utils/test/htmlcov/index.html .
An alternate output location can be specified by setting the
COVERAGE_OUT variable, e.g.
make coverage-html COVERAGE_OUT=/tmp/coverage/
(the output directory does not need to exist beforehand.)
To generate only the coverage data, do:
make coverage
or
make .coverage
(The coverage data generated by python is stored in the .coverage
file.) This essentially runs make check, using a single python
interpreter, and records which lines and branches of the python code
were exercised.
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: Christian Boltz <apparmor@cboltz.de>
Acked-by: John Johansen <john.johansen@canonical.com>
This patch moves the declaration of phony and quieted make targets
to a single section, to avoid repeated lines. It's not so useful
for just two targets, but future patches will add more targets with
similar attributes.
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: John Johansen <john.johansen@canonical.com>
This patch adds very limited support for very dumb parsing of dbus
rules. Basically, it stores dbus rules as raw strings wrapped in
a class.
There's class structure to indicate how I'd like to see fuller future
support for dbus rules to be implemented and act as a guidepost for
how to handle most rules, moving away from the giant structure of
nested dictionaries. A stub test script is included as well, with a
modification to the make check target to set the PYTHONPATH to point
in the right place.
With this patch, aa-audit, aa-autodep, aa-complain, aa-disable,
and aa-enforce all function for me. aa-logprof and aa-genprof have
functionality issues for me at the moment (one of them dumps a
backtrace even without this patch), and I'm not sure the writing out
of dbus rules is completely implemented for modified profiles.
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
packaging systems that do a make clean while trying to represent
changes.
Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: Seth Arnold <seth.arnold@canonical.com>
