apparmor/utils/test
Zygmunt Krynicki 1df91e2c8c Third iteration of spread support
- Tests defined in utils/test are now described by a task.yaml in the same
  directory and can run concurrently across many machines.
- Tests for utils/ are now executed on openSUSE Tumbleweed since ttk themes is
  no longer a hard dependency in master.
- Tests no longer run on openSUSE Leap 15.6 due to the age of default
  Python (3.6) and gcc/g++. The tight integration with SWIG which does
  not seem to support other Python versions very well. Perl hard-codes
  old GCC for extension modules. The upcoming openSUSE Leap 16 should be
  a viable target. In the meantime we can still test everything through
  rolling-release Tumbleweed.
- Formatting of YAML files is now more uniform, at four spaces per tab.
- The run-spread.sh script is now in the root of the tree. The script allows
  running all spread tests sequentially on one system, while collecting logs
  and artifacts for convenient analysis after the fact.
- All systems are adjusted to run _four_ workers in parallel with _two_ virtual
  cores each and equipped with 1.5GB of virtual memory. This aims to best
  utilize the capacity of a typical CI worker with two to four cores and about
  8GB of available memory.
- Failing tests are marked as such, so that as a whole the entire spread suite
  can pass and be useful at catching regressions.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
2024-12-05 02:17:07 +01:00
..
logprof ping: allow reading /proc/sys/net/ipv6/conf/all/disable_ipv6 2024-09-27 12:05:29 +02:00
wtmp-examples Add truncated and ancient wtmp example to tests 2021-10-24 17:24:43 +02:00
cleanprof_test.complain Add aa-complain tests for profile with hats and subprofiles 2024-10-06 16:51:12 +02:00
cleanprof_test.in Adding userspace support for unix mediation 2024-03-29 13:09:06 +00:00
cleanprof_test.out Adding userspace support for unix mediation 2024-03-29 13:09:06 +00:00
common_test.py common_test setup_aa(): drop try/except 2024-10-20 20:49:03 +02:00
easyprof.conf This patchset is broken into 4 parts: 2012-05-07 22:37:48 -07:00
logprof.conf Stop calling ldd in aa-genprof and aa-autodep 2024-03-31 18:53:12 +02:00
Makefile Allow running exactly one test in utils/test 2024-12-05 02:17:07 +01:00
README.md use new build_platlib path with setuptools >= 61.2 2022-07-16 17:47:18 +02:00
runtests-py3.sh fix utils/test/runtests-py*.sh exitcode 2014-10-03 11:48:21 +02:00
severity.db Add aa-logprof test framework 2023-08-13 21:02:50 +02:00
severity_broken.db Update perl abstraction, logprof.conf, severity.db and tests for Debian/Ubuntu 2014-08-20 19:14:24 -05:00
task.yaml Third iteration of spread support 2024-12-05 02:17:07 +01:00
test-aa-cli-bootstrap.py Add option to log aa-logprof json input and output 2023-07-30 21:28:35 +02:00
test-aa-decode.py utils: fix coding style to match PEP8 2024-05-20 13:56:37 -03:00
test-aa-easyprof.py utils: fix coding style to match PEP8 2024-05-20 13:56:37 -03:00
test-aa-notify.py aa-notify: Adding support for merging notification. 2024-11-26 18:35:37 +00:00
test-aa.py utils: fix coding style to match PEP8 2024-05-20 13:56:37 -03:00
test-aare.py AARE: add __eq__() to allow checking aare1 == aare2 2023-10-11 20:38:38 +02:00
test-abi.py utils: fix coding style to match PEP8 2024-05-20 13:56:37 -03:00
test-alias.py utils: fix coding style to match PEP8 2024-05-20 13:56:37 -03:00
test-all.py utils: fix coding style to match PEP8 2024-05-20 13:56:37 -03:00
test-baserule.py utils: Simplify logparsing and rule creation from hashlog/event 2024-07-23 16:09:53 +00:00
test-boolean.py utils: fix coding style to match PEP8 2024-05-20 13:56:37 -03:00
test-capability.py logparser: adding support for comm in capability events 2024-08-12 10:23:40 +02:00
test-change_profile.py utils: fix coding style to match PEP8 2024-05-20 13:56:37 -03:00
test-common.py Change apparmor.common.combine_profname arguments from list to tuple literals. 2022-09-11 21:56:26 -04:00
test-config.py Stop calling ldd in aa-genprof and aa-autodep 2024-03-31 18:53:12 +02:00
test-dbus.py utils: Simplify logparsing and rule creation from hashlog/event 2024-07-23 16:09:53 +00:00
test-example.py Order imports and module-level dunder name assignments. 2022-08-21 11:15:07 -04:00
test-file.py test-file.py: don't expect translated file permissions 2024-09-17 14:44:20 +02:00
test-include.py utils: fix coding style to match PEP8 2024-05-20 13:56:37 -03:00
test-io_uring.py utils: fix coding style to match PEP8 2024-05-20 13:56:37 -03:00
test-libapparmor-test_multi.py test-multi: no longer expect testcase01, 12 and 13 to fail 2024-10-23 19:25:35 +02:00
test-logparser.py logparser: Add tests for create_rule_from_ev 2024-08-13 11:35:58 +02:00
test-logprof.py test-logprof: Increase timeout once more 2024-11-09 19:57:00 +01:00
test-minitools.py Merge ProfileStorage: store correct name 2024-10-29 13:48:30 +00:00
test-mount.py MountRule: Add support for empty ("") source 2024-06-09 23:09:05 +02:00
test-mqueue.py MqueueRule: allow / as mqueue name 2024-07-21 16:00:18 +02:00
test-network.py utils: fix coding style to match PEP8 2024-05-20 13:56:37 -03:00
test-notify.py Change string formatting method in Python tests 2023-02-19 16:54:38 -05:00
test-parser-simple-tests.py Quote trailing backslash in test case 2024-11-25 13:55:05 +01:00
test-pivot_root.py Add PivotRootRule class 2024-05-20 20:42:50 +02:00
test-profile-list.py utils: fix coding style to match PEP8 2024-05-20 13:56:37 -03:00
test-profile-storage.py ProfileStorage: store correct name 2024-10-06 14:44:12 +02:00
test-profiles.py Order imports and module-level dunder name assignments. 2022-08-21 11:15:07 -04:00
test-ptrace.py utils: Simplify logparsing and rule creation from hashlog/event 2024-07-23 16:09:53 +00:00
test-regex_matches.py Drop useless code in test-regex_matches.py 2024-12-02 21:33:34 +01:00
test-rlimit.py utils: fix coding style to match PEP8 2024-05-20 13:56:37 -03:00
test-severity.py Change string formatting method in Python tests 2023-02-19 16:54:38 -05:00
test-signal.py utils: Simplify logparsing and rule creation from hashlog/event 2024-07-23 16:09:53 +00:00
test-translations.py Improve utils hotkey handling 2023-01-29 13:33:44 +01:00
test-unix.py Fix handling of quoted peers in UnixRule (and others) 2024-06-19 14:01:15 +02:00
test-userns.py utils: Simplify logparsing and rule creation from hashlog/event 2024-07-23 16:09:53 +00:00
test-variable.py utils: fix coding style to match PEP8 2024-05-20 13:56:37 -03:00

Test data generated elsewhere

The tests in parser generate additional test profiles in parser/tst/simple_tests/: see gen-dbus.py and gen-xtrans.py.

utils/test/test-parser-simple-tests.py uses this test data when it is available. If this test data has not been generated, this test will not complain: it will simply exercise fewer test profiles.

Running individual tests

Python's unittest allows individual tests to be executed by specifying the class name and the test on the command line. When running tests individually, the unittest framework executes the "setUp" and "tearDown" methods automatically. For more information, refer to the unittest documentation.

Make sure to set the environment variables pointing to the in-tree apparmor modules, and the in-tree libapparmor and its python wrapper:

$ export PYTHONPATH=..:../../libraries/libapparmor/swig/python/build/$(/usr/bin/python3 ../../libraries/libapparmor/swig/python/test/buildpath.py)
$ export __AA_CONFDIR=.

To execute the test individually, run:

$ python3 ./test-tile.py ClassFoo.test_bar