We are seeing images cached and then re-constructed as if something had
changed in the meanitime. Debug image construction with make --dry-run --debug.
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
This way there's somewhat less repetition and the flow of job definitions is,
at least to me, easier to read.
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
Our cache is rather compressed already, so this should help
a little with wall-clock time.
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
A new explicit, non-parallel job is injected when the .image-garden.mk or
.spread.yaml file changes. This job warms up the cache for the subsequent
parallel testing jobs.
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
As a security measure, GitLab splits cache into two broad pools: protected and
non-protected. Any job running in a protected branch has access to the
protected cache pool. All other jobs run in the non-protected cache pool.
This effectively forces us to push to cache in non-protected branches, like all
the merge requests, in order to actually use the cache.
Ideally we'd disable this protection and only push from the default branch and
pull otherwise, as changes to dependency set is rather rare.
[1] https://docs.gitlab.com/ee/ci/caching/#use-the-same-cache-for-all-branches
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
This job is now redundant since the same operation is done when spread
builds and runs regression tests.
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
This requires a runner with the tags: linux, x86_64, kvm. One needs to
be provisioned for the AppArmor project for the pipeline to function.
It is possible to run the same tests on SAAS runners offered by GitLab
but due to issue gitlab-org/gitlab-runner#6208 there is no way to expose
/dev/kvm on the host to the guest. Without this feature emulation works
but is rather slow as to be impractical.
Note that there's some overlap between the build-all job and spread that
might be avoided in the future. At present this is made more difficult
by the fact that the path where build-all job builds libapparmor is
stored internally by autotools. This prevents us from using GitLab
artifacts from moving the built files across to the spread testing jobs
without extra work.
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
There's contention between running spread across many nodes, in chunks,
in a CI/CD pipeline, and running spread on one machine, across many
instances at the same time. The case with CI/CD needs one worker, as
parallelism is provided by GitLab. The case with local spread needs many
workers as parallelism is provided locally by spread allocating new
instances.
At present we need to focus on the CI/CD case. I have a plan on how to
avoid the problem entirely down the line, by running multiple copies of
spread locally, as if everything was done in a CI/CD pipeline.
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
A number of tests are failing and since spread does not contain a native
XFAIL facility, we have to maintain a silent-failure feature code
ourselves. A few of those have been fixed since the first iteration of
this patch. The remaining known failures are being fixed.
Later on I would like to separate XFAIL from SKIP so that if a test is
known to exercise kernel feature unavailable on the given system, the
test is just not executed.
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/1483
Approved-by: John Johansen <john@jjmx.net>
Merged-by: John Johansen <john@jjmx.net>
Python 3.13 changes the formatting of long-short option pairs that use a
meta-variable. Up until 3.13 the meta-variable was repeated. Since
Python change [1] the meta-var is only printed once.
[1] https://github.com/python/cpython/pull/103372
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/1495
Approved-by: Georgia Garcia <georgia.garcia@canonical.com>
Approved-by: Christian Boltz <apparmor@cboltz.de>
Merged-by: Zygmunt Krynicki <me@zygoon.pl>
Python 3.13 changes the formatting of long-short option pairs that use a
meta-variable. Up until 3.13 the meta-variable was repeated. Since
Python change [1] the meta-var is only printed once.
[1] https://github.com/python/cpython/pull/103372
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
This is so that we get a baseline that passes to enable testing in CI/CD
but also to spark a discussion around what to do with a profile that
indirectly relies on a kernel feature that is not available on a given
system.
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
A number of tests are failing and since spread does not contain a native
XFAIL facility, we have to maintain a silent-failure feature code
ourselves. A few of those have been fixed since the first iteration of
this patch. The remaining known failures are being fixed.
Later on I would like to separate XFAIL from SKIP so that if a test is
known to exercise kernel feature unavailable on the given system, the
test is just not executed.
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
This is a dependency of the overlayfs_fuse regression test.
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/1509
Approved-by: Georgia Garcia <georgia.garcia@canonical.com>
Merged-by: Georgia Garcia <georgia.garcia@canonical.com>
The "last" command, which was supplied by util-linux in older Ubuntu
versions, is now supplied by wtmpdb in Oracular and Plucky. Unfortunately,
this changed the output format and broke our column based parsing.
While the wtmpdb upstream has added json support at
https://github.com/thkukuk/wtmpdb/issues/20, we cannot use it because
we need to support systems that do not have this new feature added.
Signed-off-by: Ryan Lee <ryan.lee@canonical.com>
MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/1508
Approved-by: Georgia Garcia <georgia.garcia@canonical.com>
Merged-by: John Johansen <john@jjmx.net>
Since kernel commit 8c4b785a86be the class is available to check if
the log belongs to which class. This fixes cases where the logparser
is not able to distinguish between network and file operations.
This issue does not manifest previous to and including apparmor-4.0
because we did not process auditing logs then.
Fixes: https://gitlab.com/apparmor/apparmor/-/issues/478
Reported-by: vyomydv vyom.yadav@canonical.com
Signed-off-by: Georgia Garcia <georgia.garcia@canonical.com>
This patch should be cherry-picked to apparmor-4.1
Closes#478
MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/1507
Approved-by: Christian Boltz <apparmor@cboltz.de>
Merged-by: Georgia Garcia <georgia.garcia@canonical.com>
Since kernel commit 8c4b785a86be the class is available to check if
the log belongs to which class. This fixes cases where the logparser
is not able to distinguish between network and file operations.
This issue does not manifest previous to and including apparmor-4.0
because we did not process auditing logs then.
Fixes: https://gitlab.com/apparmor/apparmor/-/issues/478
Reported-by: vyomydv vyom.yadav@canonical.com
Signed-off-by: Georgia Garcia <georgia.garcia@canonical.com>
The "last" command, which was supplied by util-linux in older Ubuntu
versions, is now supplied by wtmpdb in Oracular and Plucky. Unfortunately,
this changed the output format and broke our column based parsing.
While the wtmpdb upstream has added json support at
https://github.com/thkukuk/wtmpdb/issues/20, we cannot use it because
we need to support systems that do not have this new feature added.
Signed-off-by: Ryan Lee <ryan.lee@canonical.com>
This MR includes copyediting of the `aa-load --help` text as well as a man page based on the help text.
Signed-off-by: Ryan Lee <ryan.lee@canonical.com>
MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/1505
Approved-by: John Johansen <john@jjmx.net>
Merged-by: John Johansen <john@jjmx.net>
Unfortunately we are affected by the backwards-incompatible change introduced by https://github.com/swig/swig/pull/2907
These wrappers will be needed to fix tests on systems using SWIG 4.3 or later, e.g. Ubuntu Plucky.
Signed-off-by: Ryan Lee <ryan.lee@canonical.com>
This also reorganizes the overlayfs tests slightly in order to maximize code reuse between the old test and the new one.
Signed-off-by: Ryan Lee <ryan.lee@canonical.com>
MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/1503
Approved-by: Georgia Garcia <georgia.garcia@canonical.com>
Merged-by: Georgia Garcia <georgia.garcia@canonical.com>
By making the test a file to be included as a helper, we can reuse most of the code for a fuse_overlayfs test without copy-pasting
Signed-off-by: Ryan Lee <ryan.lee@canonical.com>
We now have GitLab CI/CD pipeline co-existing with spread, coupled with
image-garden and the cloud-init profile defined for each distribution.
To avoid duplicating list of required dependencies, re-use cloud-init
profile as the reference list of dependencies (superset between build
and test) to install.
In addition to the dependency list, the build_all job now re-uses spread
prepare section in similar fashion. If it builds in spread, it should
build in CI as well.
A small quality-of-life improvement is the shape of a collapsible
section around dependency installation should make reading job logs
easier.
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/1494
Approved-by: Christian Boltz <apparmor@cboltz.de>
Merged-by: Zygmunt Krynicki <me@zygoon.pl>
Our ubuntu packaging builds Python-enabled libapparmor's in the directories `libapparmor/libapparmor.python[version_identifier]`. In order for the util's `make check` to pick up on the correct libapparmor during the Ubuntu build process, we need the ability to override its search path. This patch introduces a `LIBAPPARMOR_BASEDIR` variable to allow for that.
Signed-off-by: Ryan Lee <ryan.lee@canonical.com>
MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/1497
Approved-by: Georgia Garcia <georgia.garcia@canonical.com>
Merged-by: Georgia Garcia <georgia.garcia@canonical.com>
The first patch fixes a `test-aa-notify.py` `TypeError` when `APPARMOR_NOTIFY` and `__AA_CONFDIR` are both specified, which is something that was broken all this time.
The second patch ensures that `aa-notify` in the test suite is run using the same Python interpreter that the test suite itself is run with, which is necessary for testing the utils under different Pythons.
The third patch does analogous modifications to the minitools tests that launch `aa-audit`, `aa-complain`, etc.
Signed-off-by: Ryan Lee <ryan.lee@canonical.com>
MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/1498
Approved-by: Georgia Garcia <georgia.garcia@canonical.com>
Merged-by: Georgia Garcia <georgia.garcia@canonical.com>
