Commit graph

503 commits

Author SHA1 Message Date
John Johansen
8711c7754b Merge tests: provide better output on failures
When a test fails because of an unexpected success (XFAIL), do not display the empty error log as that may confuse the reader just as it had confused the author.

In addition, when something legitimately fails then display tail of trace log as that may show some useful information.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/1548
Approved-by: John Johansen <john@jjmx.net>
Merged-by: John Johansen <john@jjmx.net>
2025-02-18 15:06:33 +00:00
Zygmunt Krynicki
c56cbad5ea tests: remove XFAIL/mqeue, stale
There is no mqueue in Makefile TESTS anywhere. This is a red herring.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
2025-02-17 17:32:08 +01:00
Zygmunt Krynicki
5f8863c7ca tests: mark ptrace test as fixed
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
2025-02-17 16:37:16 +01:00
Zygmunt Krynicki
c268e5d11b tests: display tail of bash.trace on failure
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
2025-02-17 16:07:06 +01:00
Zygmunt Krynicki
473e791e4e tests: do not display bash.err on XFAIL passes
This makes no sense since the test has passed and there's nothing to look at in the log.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
2025-02-17 16:04:12 +01:00
Zygmunt Krynicki
083dc9652b tests: mark posix_ipc test as fixed
The test used to fail on some versions of Ubuntu but it now passes
everywhere.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
2025-02-17 16:03:46 +01:00
Zygmunt Krynicki
3987bf0f33 tests: mark attach_disconnected as fixed
The test is now passing on Ubuntu 24.04+

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
2025-02-17 16:03:32 +01:00
Alex Murray
1f2175d854
tests/profiles/tinyproxy: add test for path based attachment
The existing test checks that the tinyproxy systemd service is confined. However
it is possible that this confinement is based on systemd launching tinyproxy
with the expected profile, rather than tinyproxy running under the profile due
to path-based attachment. So add an explicit check for this as well as requested
by @zyga-aka-zygoon in
https://gitlab.com/apparmor/apparmor/-/merge_requests/1477#note_2334724042

Signed-off-by: Alex Murray <alex.murray@canonical.com>
2025-02-07 12:30:06 +10:30
John Johansen
5bc1cd763c Merge Some updates to modernize the mount regression test
Signed-off-by: Ryan Lee <ryan.lee@canonical.com>

MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/1449
Approved-by: John Johansen <john@jjmx.net>
Merged-by: John Johansen <john@jjmx.net>
2025-02-06 19:29:39 +00:00
John Johansen
5585a4fc9e Merge profiles/apparmor.d: add profile for tinyproxy
This was tested using the test-tinyproxy.py script from qa-regression-testing as
well as by running the upstream test suite with a brief hack to ensure it
invokes tinyproxy with aa-exec -p tinyproxy first.

Signed-off-by: Alex Murray <alex.murray@canonical.com>

MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/1477
Approved-by: Ryan Lee <rlee287@yahoo.com>
Approved-by: Christian Boltz <apparmor@cboltz.de>
Merged-by: John Johansen <john@jjmx.net>
2025-02-06 19:28:41 +00:00
John Johansen
e86fe8c96c Merge Add lsblk profile
AppArmor profile for the lsblk binary, developed and tested on Ubuntu 24.04.

MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/1437
Approved-by: Ryan Lee <rlee287@yahoo.com>
Merged-by: John Johansen <john@jjmx.net>
2025-02-06 19:21:14 +00:00
Alex Murray
0217af9d82
tests/profiles/tinyproxy/task.yaml: fix tinyproxy smoke test
Signed-off-by: Alex Murray <alex.murray@canonical.com>
2025-02-06 21:32:36 +10:30
Alex Murray
8729c65bfe
tests/profiles/tinyproxy: add spread smoke test for tinyproxy
Signed-off-by: Alex Murray <alex.murray@canonical.com>
2025-02-04 16:44:37 +10:30
John Johansen
4831a854fe Merge Initial profile for tar binary
Profile for `tar` package.

In order to test this, I've diffed the output of the `tar`'s testsuite with and without the profile:

```
sudo apt build-dep tar
apt source tar
cd tar-*/
./configure
cd tests/
./testsuite > without_profile.log
apparmor_parser ~/tar
./testsuite > with_profile.log
diff without_profile.log with_profile.log # should not output anything
echo $? # should be zero
```

Additionally, [the testsuite available on QRT](https://git.launchpad.net/qa-regression-testing/tree/scripts/test-tar.py) for the `tar` package should continue to pass after loading the profile.

MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/1453
Approved-by: John Johansen <john@jjmx.net>
Merged-by: John Johansen <john@jjmx.net>
2025-02-03 21:46:36 +00:00
Zygmunt Krynicki
8967dee5b9 tests/spread: fix debian system name
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
2025-01-29 19:45:45 +01:00
Zygmunt Krynicki
32bf95bb1e tests: exclude debian systems from toybox test
This is so that we get a baseline that passes to enable testing in CI/CD
but also to spark a discussion around what to do with a profile that
indirectly relies on a kernel feature that is not available on a given
system.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
2025-01-28 14:57:32 +01:00
Zygmunt Krynicki
b0422d5572 tests: mark more regression test as known-failures
A number of tests are failing and since spread does not contain a native
XFAIL facility, we have to maintain a silent-failure feature code
ourselves. A few of those have been fixed since the first iteration of
this patch. The remaining known failures are being fixed.

Later on I would like to separate XFAIL from SKIP so that if a test is
known to exercise kernel feature unavailable on the given system, the
test is just not executed.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
2025-01-28 14:56:02 +01:00
Zygmunt Krynicki
c6545b44bb tests: add smoke test for lsblk
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
Signed-off-by: Hlib Korzhynskyy <hlib.korzhynskyy@canonical.com>
2025-01-27 16:22:06 -03:30
Octavio Galland
76647b33b1 typo in tar test 2025-01-27 10:10:29 -03:00
Ryan Lee
be38da7570 Move most file setup and creation to before the overlay mount call
kernel overlayfs propagates the changes, while fuse_overlayfs doesn't

Signed-off-by: Ryan Lee <ryan.lee@canonical.com>
2025-01-24 09:00:28 -08:00
Ryan Lee
9e05668d5a Set up an overlayfs_fuse regression test by using the other path of the overlayfs_common.inc helper
Signed-off-by: Ryan Lee <ryan.lee@canonical.com>
2025-01-24 08:59:06 -08:00
Ryan Lee
a0f551d5b7 Wire up the kernel/fuse argument switch in overlayfs_common.inc regression tests
Signed-off-by: Ryan Lee <ryan.lee@canonical.com>
2025-01-24 08:59:06 -08:00
Ryan Lee
9413658277 Move overlayfs test into include helper and wrap in overlayfs_kernel
By making the test a file to be included as a helper, we can reuse most of the code for a fuse_overlayfs test without copy-pasting

Signed-off-by: Ryan Lee <ryan.lee@canonical.com>
2025-01-24 08:59:06 -08:00
Octavio Galland
8efe442717 tar test fixes 2025-01-24 09:30:49 -03:00
Octavio Galland
7a7f88ddf3 tar spread smoke-test 2025-01-23 14:03:23 -03:00
Zygmunt Krynicki
ffd38b7ac4 tests: measure toybox with actual-profile-of
This should be a more readable example to follow in other tests.  The
toybox test was special given the fact that it is a shell itself, and is
fairly programmable.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
2025-01-23 13:53:45 +01:00
Zygmunt Krynicki
23df780544 tests: add tool for observing the profile of a given command
Using gdb in batch mode, put a breakpoint on _start and spawn the
process.  Then using the built-in python interpreter print the
confinement label on the process and terminate everything.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
2025-01-23 13:53:45 +01:00
Zygmunt Krynicki
be47567d27 tests: add integration test for toybox
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
2025-01-21 12:34:42 +01:00
Ryan Lee
63c944a01a regression tests: fix the overlayfs mv test failures
The file being moved from needs rw permissions and not just w permissions

Signed-off-by: Ryan Lee <ryan.lee@canonical.com>
2025-01-16 18:10:06 -08:00
John Johansen
f171f5ebc8 Merge tests: snapd/mount-control: assorted fixes
This makes the snapd/mount-control test pass on all the currently tested systems. Note that there's a somewhat complex problem with the new mount APIs (https://lwn.net/Articles/753473/) from 2018 that are now being used on, for example, Debian 13.

I will need to make similar changes to the profiles generated by snapd, so any insight on what to do there is strongly appreciated.

MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/1479
Approved-by: John Johansen <john@jjmx.net>
Merged-by: John Johansen <john@jjmx.net>
2025-01-16 19:35:35 +00:00
Christian Boltz
b4cb33b488 Merge tests: regression: separate bash traces from errors
The BASH_XTRACEFD variable can be used to redirect "set -x" traces
to a dedicated file. We can use it to split the execution trace
(what has actually happened) from the failure messages.

On a failing test this does provide improved clarity when debugging
interactively with "spread -debug". On non-interactive runs the now
shorter error list is also implicitly printed.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/1481
Approved-by: Christian Boltz <apparmor@cboltz.de>
Approved-by: Georgia Garcia <georgia.garcia@canonical.com>
Merged-by: Christian Boltz <apparmor@cboltz.de>
2025-01-10 20:48:16 +00:00
Zygmunt Krynicki
fa33d7199b tests: run autotools test verbosely
Instead of showing just the summary, display the actual test log as well.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
2025-01-10 14:04:55 +01:00
Zygmunt Krynicki
2c2e0478f8 tests: regression: separate bash traces from errors
The BASH_XTRACEFD variable can be used to redirect "set -x" traces
to a dedicated file. We can use it to split the execution trace
(what has actually happened) from the failure messages.

On a failing test this does provide improved clarity when debugging
interactively with "spread -debug". On non-interactive runs the now
shorter error list is also implicitly printed.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
2025-01-10 12:40:17 +01:00
Zygmunt Krynicki
cff25b8d17 tests: snapd/mount-control: allow paths used on openSUSE
In addition allow linking to libeconf, generalize locale paths to cover
values other than C.UTF-8 and allow reading system-wide locale.alias and
gconv modules.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
2025-01-10 11:09:36 +01:00
Zygmunt Krynicki
8ed810756b tests: snapd/mount-control: stop/start auditd
This is needed on openSUSE Tumbleweed.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
2025-01-10 11:09:19 +01:00
Zygmunt Krynicki
5556de53c0 tests: snapd/mount-control: allow new mount APIs
This is not the best of fixes but it seems that on Debian 13, with new
libmount calling fsopen/fsconfig/move_mount, the current apparmor mount
rule is insufficient to allow the call to go through.

The key problems are:
- the fstype is not visible to LSM
- the source directory is an empty string
- the mount is moved to final position

I don't know the extent of "new" mount API coverage by LSM hooks but
I think we should either synthesize new permissions from old rules,
.e.g match each of the system calls against what the mount class
expression, or somehow allow the exceptions better.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
2025-01-10 11:08:26 +01:00
Zygmunt Krynicki
32116a50b0 tests: snapd/mount-control: fix bash syntax.
This masked failures that were already occuring.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
2025-01-10 11:08:07 +01:00
John Johansen
8e431ebcd9 Merge regression tests: make loop device size more generous
Depending on the system, copying echo to the loop device fails because the echo binary is too large.
Especially on systems that have echo be just a symlink to coreutils (e.g. busybox) (as opposed to echo being its own binary) 16k is just not enough.
2M seems fine on my system, but this might need yet a higher value depending on what coreutils other people actually run.

The crash in question:
```
cp: error writing '/tmp/sdtest.3937422-31490-Bxvi6g/mount_target/echo': No space left on device
Fatal Error (file_unbindable_mount): Unexpected shell error. Run with -x to debug
rm: cannot remove '/tmp/sdtest.3937422-31490-Bxvi6g/mount_target': Device or resource busy
```

MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/1469
Approved-by: John Johansen <john@jjmx.net>
Merged-by: John Johansen <john@jjmx.net>
2024-12-20 08:53:41 +00:00
John Johansen
cd4bb05f20 Merge Add overlayfs regression tests
These tests exercise various common file operations on files in an overlayfs.

Signed-off-by: Ryan Lee <ryan.lee@canonical.com>

MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/1461
Approved-by: Christian Boltz <apparmor@cboltz.de>
Merged-by: John Johansen <john@jjmx.net>
2024-12-20 08:25:58 +00:00
1cc2a3bd86
regression tests: make loop device size more generous
Depending on the system, copying echo to the loop device fails because the echo binary is too large.
Especially on systems that have echo be just a symlink to coreutils (e.g. busybox) 16k is just not enough.
2M seems fine on my system, but this might need yet a higher value depending on what coreutils other people actually run.
The actual loop device needs to be larger to properly fit the allocated file size. Testing shows 4M is sufficient, but this is basically arbitrary.
2024-12-19 23:48:43 +01:00
John Johansen
ba60bfff85 Merge Write a regression test for mediating file access in private mounts
This test, as is, emits an execname warning which is due to a bug in the `prologue.inc` infrastructure (see !1450 for a fix to this issue).

Signed-off-by: Ryan Lee <ryan.lee@canonical.com>

MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/1448
Approved-by: John Johansen <john@jjmx.net>
Merged-by: John Johansen <john@jjmx.net>
2024-12-19 19:44:51 +00:00
John Johansen
59957aa1d8 Merge fixes on the testing infrastructure
This MR is meant to resolve warnings such as "Warning: execname '/home/username/Documents/apparmor/tests/regression/apparmor/file_unbindable_mount': no such file or directory" when running tests like the one in the current version of !1448.

MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/1450
Approved-by: John Johansen <john@jjmx.net>
Merged-by: John Johansen <john@jjmx.net>
2024-12-19 19:38:00 +00:00
John Johansen
3ed5adb665 Merge Allow make-* flags with remount operations
While the mount syscall documentation disallows this, the kernel silently
ignores make-* flags when doing a remount, and real applications were
passing this conflicting set of flags. Because changing the kernel to
reject this combination would break userspace, we should allow them
instead.

For an example: see https://bugs.launchpad.net/apparmor/+bug/2091424.

Signed-off-by: Ryan Lee <ryan.lee@canonical.com>

MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/1466
Approved-by: Georgia Garcia <georgia.garcia@canonical.com>
Approved-by: John Johansen <john@jjmx.net>
Merged-by: John Johansen <john@jjmx.net>
2024-12-19 17:27:53 +00:00
Ryan Lee
83270fcf68 Add a regression test for allowing rprivate with conflicting options
Signed-off-by: Ryan Lee <ryan.lee@canonical.com>
2024-12-18 10:28:49 -08:00
Ryan Lee
5cd3362a81 Update fs type comment in swap regression test
Signed-off-by: Ryan Lee <ryan.lee@canonical.com>
2024-12-16 12:51:30 -08:00
Ryan Lee
1d3d48cc2a Shellcheck pass over overlayfs.sh
Signed-off-by: Ryan Lee <ryan.lee@canonical.com>
2024-12-16 09:52:38 -08:00
Ryan Lee
b24a820e7a Extend overlayfs test with more file ops
Signed-off-by: Ryan Lee <ryan.lee@canonical.com>
2024-12-16 09:52:38 -08:00
Ryan Lee
8212fa8be4 Add more operations to the regression test complain binary
This extra functionality is to be used in a different regression test that reuses the binary

Signed-off-by: Ryan Lee <ryan.lee@canonical.com>
2024-12-16 09:52:38 -08:00
Ryan Lee
e0127767fd Add the overlayfs regression test to task.yaml
Signed-off-by: Ryan Lee <ryan.lee@canonical.com>
2024-12-16 09:52:38 -08:00
Ryan Lee
1cb11f5a89 Add the overlayfs regression test to the Makefile
Signed-off-by: Ryan Lee <ryan.lee@canonical.com>
2024-12-16 09:52:38 -08:00