mirrors/apparmor
1
0
Fork 0
mirror of https://gitlab.com/apparmor/apparmor.git synced 2025-03-04 00:14:44 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions
apparmor/.gitignore
Zygmunt Krynicki cc04181578 Allow running tests with spread 
Spread is a full-system, or integration test suite runner initially developed
to test snapd. Over time it has spread to other projects where it provides a
structured way to organize, run and debug complex full-system interactions.
Spread is documented on https://github.com/canonical/spread and is used in
production since late 2016.

Spread has a notion of backends which are responsible for allocating and
discarding test machines. For the purpose of running AppArmor regression tests,
I've combined spread with my own tool, image garden. The tool provides
off-the-shelf images, constructed on-the-fly from freely available images, and
makes them easily available to spread.

The reason for doing it this way is so that using non-free cloud systems is not
required and anyone can repeat the test process locally, on their own computer.
Vanilla spread is somewhat limited to x86-64 systems but the way I've used it
here makes it equally possible to test x86_64 *and* aarch64 systems. I've done
most of the development on an ARM single-board-computer running on my desk.

Spread requires a top-level spread.yaml file and a collection of task.yaml
files that describe individual tasks (for us, those are just tests). Tasks have
no implied dependency except that to reach a given task, spread will run all
the _prepare_ statements leading to that task, starting from the project, test
suite and then task. With proper care one can then run a specific individual
test with a one-line command, for example:

```
spread -v garden:ubuntu-cloud-24.04:tests/regression/apparmor:at_secure
```

This will prepare a fresh ubuntu-cloud-24.04 system (matching the CPU
architecture of the host), copy the project tree into the test machine, install
all the build dependencies, build all the parts of apparmor and then run one
specific variant of the regression test, namely the at_secure program.
Importantly the same test can also run on, say debian-cloud-13 (Debian Trixie),
but also, if you have a Google cloud account, on Google Compute Engine or in
one of the other backends either built into spread or available as a fork of
spread or as a helper for ad-hoc backend. Spread can also create more than one
worker per system and distribute the tests to all of the available instances.
In no way are we locking ourselves out of the ability to run our test suite on
our target of choice.

Spread has other useful switches, such as:
- `-reuse` for keeping machines around until discarded with -discard
- `-resend` for re-sending updated copy of the project (useful for -reuse)
- `-debug` for starting an interactive shell on any failure
- `-shell` for starting an interactive shell instead of the `execute` phase

This first patch contains just the spread elements, assuming that both spread
and image-garden are externally installed. A GitLab continuous integration
installing everything required and running a subset of tests will follow
shortly.

I've expanded the initial selection of systems to allow running all the tests
on several versions of Ubuntu, Debian and openSUSE, mainly as a sanity check
but also to showcase how practical spread is at covering real-world systems.

A number of systems and tests are currently failing:

- garden:debian-cloud-12:tests/regression/apparmor:attach_disconnected
- garden:debian-cloud-12:tests/regression/apparmor:deleted
- garden:debian-cloud-12:tests/regression/apparmor:unix_fd_server
- garden:debian-cloud-12:tests/regression/apparmor:unix_socket_pathname
- garden:debian-cloud-13:tests/regression/apparmor:attach_disconnected
- garden:debian-cloud-13:tests/regression/apparmor:deleted
- garden:debian-cloud-13:tests/regression/apparmor:unix_fd_server
- garden:debian-cloud-13:tests/regression/apparmor:unix_socket_pathname
- garden:opensuse-cloud-15.6:tests/regression/apparmor:attach_disconnected
- garden:opensuse-cloud-15.6:tests/regression/apparmor:deleted
- garden:opensuse-cloud-15.6:tests/regression/apparmor:e2e
- garden:opensuse-cloud-15.6:tests/regression/apparmor:unix_fd_server
- garden:opensuse-cloud-15.6:tests/regression/apparmor:unix_socket_pathname
- garden:opensuse-cloud-15.6:tests/regression/apparmor:xattrs_profile
- garden:opensuse-cloud-tumbleweed:tests/regression/apparmor:attach_disconnected
- garden:opensuse-cloud-tumbleweed:tests/regression/apparmor:deleted
- garden:opensuse-cloud-tumbleweed:tests/regression/apparmor:unix_fd_server
- garden:opensuse-cloud-tumbleweed:tests/regression/apparmor:unix_socket_pathname
- garden:ubuntu-cloud-22.04:tests/regression/apparmor:attach_disconnected

In addition, only on openSUSE, I've skipped the entire test suite of the utils
directory, as it requires python3 ttk themes, which I cannot find in packaged
form.

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
2024-12-05 02:17:07 +01:00

315 lines
11 KiB
Text
Raw Blame History

apparmor-*
cscope.*
binutils/aa-enabled
binutils/aa-enabled.1
binutils/aa-exec
binutils/aa-exec.1
binutils/aa-features-abi
binutils/aa-features-abi.1
binutils/aa-load
binutils/aa-status
binutils/aa-status.8
binutils/cJSON.o
binutils/po/*.mo
changehat/mod_apparmor/.libs
changehat/mod_apparmor/mod_apparmor.8
changehat/mod_apparmor/mod_apparmor.8.html
changehat/mod_apparmor/mod_apparmor.la
changehat/mod_apparmor/mod_apparmor.lo
changehat/mod_apparmor/mod_apparmor.slo
changehat/mod_apparmor/mod_apparmor.so
changehat/mod_apparmor/pod2htmd.tmp
changehat/pam_apparmor/get_options.o
changehat/pam_apparmor/pam_apparmor.o
changehat/pam_apparmor/pam_apparmor.so
parser/po/*.mo
parser/af_names.h
parser/cap_names.h
parser/generated_cap_names.h
parser/generated_af_names.h
parser/errnos.h
parser/tst_lib
parser/tst_misc
parser/tst_regex
parser/tst_symtab
parser/tst_variable
parser/tst/simple_tests/generated_*/*
parser/parser_lex.c
parser/parser_version.h
parser/parser_yacc.c
parser/parser_yacc.h
parser/pod2htm*.tmp
parser/libapparmor_re/*.o
parser/libapparmor_re/libapparmor_re.a
parser/*.o
parser/*.7
parser/*.5
parser/*.8
parser/*.7.html
parser/*.5.html
parser/*.8.html
parser/apparmor_parser
parser/libapparmor_re/parse.cc
parser/libapparmor_re/regexp.cc
parser/techdoc.aux
parser/techdoc.log
parser/techdoc.pdf
parser/techdoc.toc
profiles/apparmor.d/local/*
!profiles/apparmor.d/local/README
libraries/libapparmor/Makefile
libraries/libapparmor/Makefile.in
libraries/libapparmor/aclocal.m4
libraries/libapparmor/audit.log
libraries/libapparmor/autom4te.cache
libraries/libapparmor/compile
libraries/libapparmor/config.guess
libraries/libapparmor/config.log
libraries/libapparmor/config.status
libraries/libapparmor/config.sub
libraries/libapparmor/configure
libraries/libapparmor/depcomp
libraries/libapparmor/install-sh
libraries/libapparmor/libtool
libraries/libapparmor/ltmain.sh
libraries/libapparmor/missing
libraries/libapparmor/test-driver
libraries/libapparmor/ylwrap
libraries/libapparmor/doc/Makefile
libraries/libapparmor/doc/Makefile.in
libraries/libapparmor/doc/*.2
libraries/libapparmor/doc/aa_*.3
libraries/libapparmor/include/Makefile
libraries/libapparmor/include/Makefile.in
libraries/libapparmor/include/sys/Makefile
libraries/libapparmor/include/sys/Makefile.in
libraries/libapparmor/src/.deps
libraries/libapparmor/src/.libs
libraries/libapparmor/src/Makefile
libraries/libapparmor/src/Makefile.in
libraries/libapparmor/src/PMurHash.lo
libraries/libapparmor/src/PMurHash.o
libraries/libapparmor/src/af_protos.h
libraries/libapparmor/src/change_hat.lo
libraries/libapparmor/src/features.lo
libraries/libapparmor/src/features.o
libraries/libapparmor/src/grammar.lo
libraries/libapparmor/src/grammar.o
libraries/libapparmor/src/kernel.lo
libraries/libapparmor/src/kernel.o
libraries/libapparmor/src/kernel_interface.lo
libraries/libapparmor/src/kernel_interface.o
libraries/libapparmor/src/libaalogparse.lo
libraries/libapparmor/src/libaalogparse.o
libraries/libapparmor/src/libimmunix_warning.lo
libraries/libapparmor/src/policy_cache.lo
libraries/libapparmor/src/policy_cache.o
libraries/libapparmor/src/private.lo
libraries/libapparmor/src/private.o
libraries/libapparmor/src/scanner.lo
libraries/libapparmor/src/scanner.o
libraries/libapparmor/src/libapparmor.pc
libraries/libapparmor/src/libapparmor.la
libraries/libapparmor/src/libimmunix.la
libraries/libapparmor/src/grammar.c
libraries/libapparmor/src/grammar.h
libraries/libapparmor/src/scanner.c
libraries/libapparmor/src/scanner.h
libraries/libapparmor/src/test-suite.log
libraries/libapparmor/src/tst_aalogmisc
libraries/libapparmor/src/tst_aalogmisc.log
libraries/libapparmor/src/tst_aalogmisc.o
libraries/libapparmor/src/tst_aalogmisc.trs
libraries/libapparmor/src/tst_aalogparse_cpp
libraries/libapparmor/src/tst_aalogparse_cpp.log
libraries/libapparmor/src/tst_aalogparse_cpp.o
libraries/libapparmor/src/tst_aalogparse_cpp.trs
libraries/libapparmor/src/tst_aalogparse_reentrancy
libraries/libapparmor/src/tst_aalogparse_reentrancy.log
libraries/libapparmor/src/tst_aalogparse_reentrancy.o
libraries/libapparmor/src/tst_aalogparse_reentrancy.trs
libraries/libapparmor/src/tst_aalogparse_oldname
libraries/libapparmor/src/tst_aalogparse_oldname.log
libraries/libapparmor/src/tst_aalogparse_oldname.o
libraries/libapparmor/src/tst_aalogparse_oldname.trs
libraries/libapparmor/src/tst_features
libraries/libapparmor/src/tst_features.log
libraries/libapparmor/src/tst_features.o
libraries/libapparmor/src/tst_features.trs
libraries/libapparmor/src/tst_kernel
libraries/libapparmor/src/tst_kernel.log
libraries/libapparmor/src/tst_kernel.o
libraries/libapparmor/src/tst_kernel.trs
libraries/libapparmor/swig/Makefile
libraries/libapparmor/swig/Makefile.in
libraries/libapparmor/swig/perl/LibAppArmor.bs
libraries/libapparmor/swig/perl/LibAppArmor.pm
libraries/libapparmor/swig/perl/Makefile
libraries/libapparmor/swig/perl/Makefile.PL
libraries/libapparmor/swig/perl/Makefile.in
libraries/libapparmor/swig/perl/Makefile.perl
libraries/libapparmor/swig/perl/Makefile.perle
libraries/libapparmor/swig/perl/MYMETA.json
libraries/libapparmor/swig/perl/MYMETA.yml
libraries/libapparmor/swig/perl/blib
libraries/libapparmor/swig/perl/libapparmor_wrap.c
libraries/libapparmor/swig/perl/libapparmor_wrap.o
libraries/libapparmor/swig/perl/pm_to_blib
libraries/libapparmor/swig/python/LibAppArmor.py
libraries/libapparmor/swig/python/LibAppArmor.egg-info/
libraries/libapparmor/swig/python/build/
libraries/libapparmor/swig/python/libapparmor_wrap.c
libraries/libapparmor/swig/python/Makefile
libraries/libapparmor/swig/python/Makefile.in
libraries/libapparmor/swig/python/setup.py
libraries/libapparmor/swig/python/test/Makefile
libraries/libapparmor/swig/python/test/Makefile.in
libraries/libapparmor/swig/python/test/test-suite.log
libraries/libapparmor/swig/python/test/test_python.py
libraries/libapparmor/swig/python/test/test_python.py.log
libraries/libapparmor/swig/python/test/test_python.py.trs
libraries/libapparmor/swig/ruby/LibAppArmor.so
libraries/libapparmor/swig/ruby/LibAppArmor_wrap.c
libraries/libapparmor/swig/ruby/LibAppArmor_wrap.o
libraries/libapparmor/swig/ruby/Makefile
libraries/libapparmor/swig/ruby/Makefile.in
libraries/libapparmor/swig/ruby/Makefile.bak
libraries/libapparmor/swig/ruby/Makefile.ruby
libraries/libapparmor/swig/ruby/mkmf.log
libraries/libapparmor/testsuite/.deps
libraries/libapparmor/testsuite/.libs
libraries/libapparmor/testsuite/Makefile
libraries/libapparmor/testsuite/Makefile.in
libraries/libapparmor/testsuite/libaalogparse.log
libraries/libapparmor/testsuite/libaalogparse.sum
libraries/libapparmor/testsuite/site.exp
libraries/libapparmor/testsuite/test_multi.multi
libraries/libapparmor/testsuite/config/Makefile
libraries/libapparmor/testsuite/config/Makefile.in
libraries/libapparmor/testsuite/lib/Makefile
libraries/libapparmor/testsuite/lib/Makefile.in
libraries/libapparmor/testsuite/libaalogparse.test/Makefile
libraries/libapparmor/testsuite/libaalogparse.test/Makefile.in
libraries/libapparmor/testsuite/test_multi/out
libraries/libapparmor/testsuite/test_multi_multi-test_multi.o
utils/*.8
utils/*.8.html
utils/*.5
utils/*.5.html
utils/*.tmp
utils/po/*.mo
utils/apparmor/*.pyc
utils/apparmor/rule/*.pyc
utils/apparmor.egg-info/
utils/build/
utils/htmlcov/
utils/test/common_test.pyc
utils/test/.coverage
utils/test/coverage-report.txt
utils/test/htmlcov/
utils/vim/apparmor.vim
utils/vim/apparmor.vim.5
utils/vim/apparmor.vim.5.html
utils/vim/pod2htmd.tmp
tests/regression/apparmor/*.o
tests/regression/apparmor/aa_policy_cache
tests/regression/apparmor/access
tests/regression/apparmor/at_secure
tests/regression/apparmor/attach_disconnected
tests/regression/apparmor/changehat
tests/regression/apparmor/changehat_fail
tests/regression/apparmor/changehat_fork
tests/regression/apparmor/changehat_misc
tests/regression/apparmor/changehat_misc2
tests/regression/apparmor/changehat_pthread
tests/regression/apparmor/changehat_twice
tests/regression/apparmor/changehat_wrapper
tests/regression/apparmor/changeprofile
tests/regression/apparmor/chdir
tests/regression/apparmor/chgrp
tests/regression/apparmor/chmod
tests/regression/apparmor/chown
tests/regression/apparmor/clone
tests/regression/apparmor/complain
tests/regression/apparmor/dbus_eavesdrop
tests/regression/apparmor/dbus_message
tests/regression/apparmor/dbus_service
tests/regression/apparmor/dbus_unrequested_reply
tests/regression/apparmor/deleted
tests/regression/apparmor/env_check
tests/regression/apparmor/environ
tests/regression/apparmor/exec
tests/regression/apparmor/exec_qual
tests/regression/apparmor/exec_qual2
tests/regression/apparmor/fchdir
tests/regression/apparmor/fchgrp
tests/regression/apparmor/fchmod
tests/regression/apparmor/fchown
tests/regression/apparmor/fd_inheritance
tests/regression/apparmor/fd_inheritor
tests/regression/apparmor/fork
tests/regression/apparmor/introspect
tests/regression/apparmor/io_uring
tests/regression/apparmor/link
tests/regression/apparmor/link_subset
tests/regression/apparmor/mkdir
tests/regression/apparmor/mmap
tests/regression/apparmor/mount
tests/regression/apparmor/move_mount
tests/regression/apparmor/named_pipe
tests/regression/apparmor/net_inet_rcv
tests/regression/apparmor/net_inet_snd
tests/regression/apparmor/net_raw
tests/regression/apparmor/open
tests/regression/apparmor/openat
tests/regression/apparmor/pipe
tests/regression/apparmor/pivot_root
tests/regression/apparmor/posix_mq_rcv
tests/regression/apparmor/posix_mq_snd
tests/regression/apparmor/ptrace
tests/regression/apparmor/ptrace_helper
tests/regression/apparmor/pwrite
tests/regression/apparmor/query_label
tests/regression/apparmor/readdir
tests/regression/apparmor/rename
tests/regression/apparmor/rw
tests/regression/apparmor/socketpair
tests/regression/apparmor/swap
tests/regression/apparmor/symlink
tests/regression/apparmor/syscall_chroot
tests/regression/apparmor/syscall_ioperm
tests/regression/apparmor/syscall_iopl
tests/regression/apparmor/syscall_mknod
tests/regression/apparmor/syscall_mlockall
tests/regression/apparmor/syscall_ptrace
tests/regression/apparmor/syscall_reboot
tests/regression/apparmor/syscall_setdomainname
tests/regression/apparmor/syscall_sethostname
tests/regression/apparmor/syscall_setpriority
tests/regression/apparmor/syscall_setscheduler
tests/regression/apparmor/syscall_sysctl
tests/regression/apparmor/sysctl_proc
tests/regression/apparmor/sysv_mq_rcv
tests/regression/apparmor/sysv_mq_snd
tests/regression/apparmor/tcp
tests/regression/apparmor/transition
tests/regression/apparmor/unix_fd_client
tests/regression/apparmor/unix_fd_server
tests/regression/apparmor/unix_socket
tests/regression/apparmor/unix_socket_client
tests/regression/apparmor/unlink
tests/regression/apparmor/userns
tests/regression/apparmor/userns_setns
tests/regression/apparmor/uservars.inc
tests/regression/apparmor/xattrs
tests/regression/apparmor/xattrs_profile
tests/regression/apparmor/coredump
**/__pycache__/
*.orig
# Patterns related to spread integration tests
*.img
*.iso
*.log
*.qcow2
*.run
.spread-reuse.yaml
Reference in a new issue View git blame Copy permalink