mirrors/apparmor
1
0
Fork 0
mirror of https://gitlab.com/apparmor/apparmor.git synced 2025-03-04 08:24:42 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions
apparmor/utils/test/Makefile

118 lines
5.4 KiB
Makefile
Raw Permalink Normal View History

utils: fix path in test-aadecode.py and push make check into the test/ subdirectory. Signed-off-by: Steve Beattie <steve@nxnw.org> Acked-by: Seth Arnold <seth.arnold@canonical.com>
2014-02-27 16:23:35 -08:00
# ----------------------------------------------------------------------
# Copyright (c) 1999, 2004-2009 NOVELL (All rights reserved)
# Copyright (c) 2010-2014 Canonical Ltd.
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of version 2 of the GNU General Public
# License published by the Free Software Foundation.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, contact Novell, Inc.
# ----------------------------------------------------------------------
NAME = apparmor-utils
all:
COMMONDIR=../../common/
Entire tree: makefile cruft removal - drop the symlink magic of the common/ directory, and just include files directly from there. - update comments indicating required steps to take when including common/Make.rules - drop make clean steps that refer to no longer generated tarballs, specfiles, and symlinks to the common directory/Make.rules. - don't silence clean steps if VERBOSE is set Signed-off-by: Steve Beattie <steve@nxnw.org> Acked-by: Christian "Ghostbuster" Boltz <apparmor@cboltz.de>
2015-01-23 15:52:09 -08:00
include $(COMMONDIR)/Make.rules
utils: fix path in test-aadecode.py and push make check into the test/ subdirectory. Signed-off-by: Steve Beattie <steve@nxnw.org> Acked-by: Seth Arnold <seth.arnold@canonical.com>
2014-02-27 16:23:35 -08:00
Add `make coverage-regression` ... to find regressions or improvements in the python code coverage. `make coverage-regression` will error out if a file looses its 100% coverage, or if a file improved to 100% coverage. Other coverage changes (for example 45% -> 47%) will be ignored.
2021-01-10 18:14:02 +01:00
# files that don't have 100% test coverage
Use PivotRootRule and PivotRootRuleset ... for handling pivot_root rules. This replaces the old code that just stores the full rule as text. We also get rid of the old ['allow'] and ['deny'] items in ProfileStorage, the handling of old write functions, and the last usage of _Raw_Rule (and therefore _Raw_Rule itsself). Also delete the old test-pivot_root_parse.py which relied on the ancient code, and even used a wrong syntax in its test rules.
2024-05-10 22:11:28 +02:00
INCOMPLETE_COVERAGE=libraries/libapparmor/swig/python/.*/LibAppArmor/LibAppArmor.py|utils/aa-logprof|utils/apparmor/aa.py|utils/apparmor/common.py|utils/apparmor/config.py|utils/apparmor/easyprof.py|utils/apparmor/fail.py|utils/apparmor/logparser.py|utils/apparmor/ui.py|minitools_test.py
Add `make coverage-regression` ... to find regressions or improvements in the python code coverage. `make coverage-regression` will error out if a file looses its 100% coverage, or if a file improved to 100% coverage. Other coverage changes (for example 45% -> 47%) will be ignored.
2021-01-10 18:14:02 +01:00
Change utils/test/Makefile to use the in-tree libapparmor Also add support for the USE_SYSTEM variable, which means: - test against the in-tree libapparmor and python modules by default - test against the system libapparmor and python modules if USE_SYSTEM is set The old behaviour was a mix of both - it always used the in-tree python modules and the system libapparmor. For obvious reasons, you'll need to build libapparmor before running the tests (unless you specify USE_SYSTEM=1 as parameter to make check). Acked-by: John Johansen <john.johansen@canonical.com> for trunk and 2.9
2015-10-20 23:04:23 +02:00
ifdef USE_SYSTEM
Allow overrides and preservation of some environment variables in utils make check Signed-off-by: Ryan Lee <ryan.lee@canonical.com>
2025-01-17 12:55:53 -08:00
LD_LIBRARY_PATH?=
PYTHONPATH?=
utils: Require apparmor.aa users to call init_aa() Introduce an apparmor.aa.init_aa() method and move the initialization code of the apparmor.aa module into it. Note that this change will break any external users of apparmor.aa because global variables that were previously initialized when importing apparmor.aa will not be initialized unless a call to the new apparmor.aa.init_aa() method is made. The main purpose of this change is to allow the utils tests to be able to set a non-default location for configuration files. Instead of hard-coding the location of logprof.conf and other utils related configuration files to /etc/apparmor/, this patch allows it to be configured by calling apparmor.aa.init_aa(confdir=PATH). This allows for the make check target to use the in-tree config file, profiles, and parser by default. A helper method, setup_aa(), is added to common_test.py that checks for an environment variable containing a non-default configuration directory path prior to calling apparmor.aa.init_aa(). All test scripts that use apparmor.aa are updated to call setup_aa(). Signed-off-by: Tyler Hicks <tyhicks@canonical.com> Suggested-by: Christian Boltz <apparmor@cboltz.de> Acked-by: Seth Arnold <seth.arnold@canonical.com> Acked-by: Christian Boltz <apparmor@cboltz.de>
2017-03-02 21:21:53 +00:00
CONFDIR=
utils: Set parser base path according to USE_SYSTEM make variable If USE_SYSTEM is not set, the utils make check target will instruct test-aa-easyprof.py to provide the path of the in-tree profiles/apparmor.d directory to aa-easyprof as the parser base directory. If USE_SYSTEM is set, the default base directory (/etc/apparmor.d) is used. The test-aa-easyprof.py script receives the base path by checking the __AA_BASEDIR environment variable. This environment variable is strictly used by the test script and not any user-facing code so two leading underscores were used. Signed-off-by: Tyler Hicks <tyhicks@canonical.com> Acked-by: Christian Boltz <apparmor@cboltz.de> Acked-by: Seth Arnold <seth.arnold@canonical.com> Bug: https://launchpad.net/bugs/1538306
2017-03-02 21:23:32 +00:00
BASEDIR=
utils: Set parser executable path according to USE_SYSTEM make variable if USE_SYSTEM is not set, the utils make check target will instruct test-aa-easyprof.py to provide the path of the in-tree parser executable to aa-easyprof. If USE_SYSTEM is set, the default parser path (/sbin/apparmor_parser or the result of `which apparmor_parser`) is used. The test-aa-easyprof.py script receives the parser path by checking the __AA_PARSER environment variable. This environment variable is strictly used by the test script and not any user-facing code so two leading underscores were used. Signed-off-by: Tyler Hicks <tyhicks@canonical.com> Acked-by: Christian Boltz <apparmor@cboltz.de> Acked-by: Seth Arnold <seth.arnold@canonical.com>
2017-03-02 21:24:33 +00:00
PARSER=
Change utils/test/Makefile to use the in-tree libapparmor Also add support for the USE_SYSTEM variable, which means: - test against the in-tree libapparmor and python modules by default - test against the system libapparmor and python modules if USE_SYSTEM is set The old behaviour was a mix of both - it always used the in-tree python modules and the system libapparmor. For obvious reasons, you'll need to build libapparmor before running the tests (unless you specify USE_SYSTEM=1 as parameter to make check). Acked-by: John Johansen <john.johansen@canonical.com> for trunk and 2.9
2015-10-20 23:04:23 +02:00
else
Allow overrides and preservation of some environment variables in utils make check Signed-off-by: Ryan Lee <ryan.lee@canonical.com>
2025-01-17 12:55:53 -08:00
LIBAPPARMOR_BASEDIR?=../../libraries/libapparmor
PYTHON_DIST_BUILD_PATH ?= $(LIBAPPARMOR_BASEDIR)/swig/python/build/$$($(PYTHON) $(LIBAPPARMOR_BASEDIR)/swig/python/test/buildpath.py)
LIBAPPARMOR_PATH?=$(LIBAPPARMOR_BASEDIR)/src/.libs/
LD_LIBRARY_PATH:=$(LD_LIBRARY_PATH):$(LIBAPPARMOR_PATH):$(PYTHON_DIST_BUILD_PATH)
PYTHONPATH:=$(PYTHONPATH):..:$(PYTHON_DIST_BUILD_PATH)
utils: Require apparmor.aa users to call init_aa() Introduce an apparmor.aa.init_aa() method and move the initialization code of the apparmor.aa module into it. Note that this change will break any external users of apparmor.aa because global variables that were previously initialized when importing apparmor.aa will not be initialized unless a call to the new apparmor.aa.init_aa() method is made. The main purpose of this change is to allow the utils tests to be able to set a non-default location for configuration files. Instead of hard-coding the location of logprof.conf and other utils related configuration files to /etc/apparmor/, this patch allows it to be configured by calling apparmor.aa.init_aa(confdir=PATH). This allows for the make check target to use the in-tree config file, profiles, and parser by default. A helper method, setup_aa(), is added to common_test.py that checks for an environment variable containing a non-default configuration directory path prior to calling apparmor.aa.init_aa(). All test scripts that use apparmor.aa are updated to call setup_aa(). Signed-off-by: Tyler Hicks <tyhicks@canonical.com> Suggested-by: Christian Boltz <apparmor@cboltz.de> Acked-by: Seth Arnold <seth.arnold@canonical.com> Acked-by: Christian Boltz <apparmor@cboltz.de>
2017-03-02 21:21:53 +00:00
CONFDIR=$(CURDIR)
utils: Set parser base path according to USE_SYSTEM make variable If USE_SYSTEM is not set, the utils make check target will instruct test-aa-easyprof.py to provide the path of the in-tree profiles/apparmor.d directory to aa-easyprof as the parser base directory. If USE_SYSTEM is set, the default base directory (/etc/apparmor.d) is used. The test-aa-easyprof.py script receives the base path by checking the __AA_BASEDIR environment variable. This environment variable is strictly used by the test script and not any user-facing code so two leading underscores were used. Signed-off-by: Tyler Hicks <tyhicks@canonical.com> Acked-by: Christian Boltz <apparmor@cboltz.de> Acked-by: Seth Arnold <seth.arnold@canonical.com> Bug: https://launchpad.net/bugs/1538306
2017-03-02 21:23:32 +00:00
BASEDIR=../../profiles/apparmor.d
utils: Set parser executable path according to USE_SYSTEM make variable if USE_SYSTEM is not set, the utils make check target will instruct test-aa-easyprof.py to provide the path of the in-tree parser executable to aa-easyprof. If USE_SYSTEM is set, the default parser path (/sbin/apparmor_parser or the result of `which apparmor_parser`) is used. The test-aa-easyprof.py script receives the parser path by checking the __AA_PARSER environment variable. This environment variable is strictly used by the test script and not any user-facing code so two leading underscores were used. Signed-off-by: Tyler Hicks <tyhicks@canonical.com> Acked-by: Christian Boltz <apparmor@cboltz.de> Acked-by: Seth Arnold <seth.arnold@canonical.com>
2017-03-02 21:24:33 +00:00
PARSER=../../parser/apparmor_parser
Change utils/test/Makefile to use the in-tree libapparmor Also add support for the USE_SYSTEM variable, which means: - test against the in-tree libapparmor and python modules by default - test against the system libapparmor and python modules if USE_SYSTEM is set The old behaviour was a mix of both - it always used the in-tree python modules and the system libapparmor. For obvious reasons, you'll need to build libapparmor before running the tests (unless you specify USE_SYSTEM=1 as parameter to make check). Acked-by: John Johansen <john.johansen@canonical.com> for trunk and 2.9
2015-10-20 23:04:23 +02:00
endif
utils: Gracefully handle a missing parser in the check target The test-aa-easyprof.py script relies on the parser to be built so the check target of the utils/test/Makefile should detect if the parser exists before running any tests. Signed-off-by: Tyler Hicks <tyhicks@canonical.com> Reported-by: Christian Boltz <apparmor@cboltz.de>
2017-12-04 18:03:44 +00:00
.PHONY: __libapparmor __parser
Change utils/test/Makefile to use the in-tree libapparmor Also add support for the USE_SYSTEM variable, which means: - test against the in-tree libapparmor and python modules by default - test against the system libapparmor and python modules if USE_SYSTEM is set The old behaviour was a mix of both - it always used the in-tree python modules and the system libapparmor. For obvious reasons, you'll need to build libapparmor before running the tests (unless you specify USE_SYSTEM=1 as parameter to make check). Acked-by: John Johansen <john.johansen@canonical.com> for trunk and 2.9
2015-10-20 23:04:23 +02:00
__libapparmor:
ifndef USE_SYSTEM
utils/tests: add libaparmor swig library path Similar to the profiles/ check using the python utilities, the tests for the python utilities were not including the path for the swig libapparmor library in the LD_LIBRARY_PATH variable, only in PYTHONPATH. This commit fixes that, renaming the variable used for the built libapparmor check. v2: - actually use the LIBAPPARMOR_PATH variable when defining LD_LIBRARY_PATH Bug: https://gitlab.com/apparmor/apparmor/-/issues/98 Signed-off-by: Steve Beattie <steve.beattie@canonical.com> Acked-by: John Johansen <john.johansen@canonical.com> MR: https://gitlab.com/apparmor/apparmor/-/merge_requests/586
2020-07-24 00:22:25 -07:00
@if [ ! -f $(LIBAPPARMOR_PATH)libapparmor.so ]; then \
echo "error: $(LIBAPPARMOR_PATH)libapparmor.so is missing. Pick one of these possible solutions:" 1>&2; \
Change utils/test/Makefile to use the in-tree libapparmor Also add support for the USE_SYSTEM variable, which means: - test against the in-tree libapparmor and python modules by default - test against the system libapparmor and python modules if USE_SYSTEM is set The old behaviour was a mix of both - it always used the in-tree python modules and the system libapparmor. For obvious reasons, you'll need to build libapparmor before running the tests (unless you specify USE_SYSTEM=1 as parameter to make check). Acked-by: John Johansen <john.johansen@canonical.com> for trunk and 2.9
2015-10-20 23:04:23 +02:00
echo " 1) Build against the in-tree libapparmor by building it first and then trying again. See the top-level README for help." 1>&2; \
echo " 2) Build against the system libapparmor by adding USE_SYSTEM=1 to your make command." 1>&2; \
binutils, parser, utils: Exit from Makefile shell snippets Exit rather than returning from shell snippets in Makefiles. It is reported that returning causes the following error message with bash: /bin/sh: line 4: return: can only `return' from a function or sourced script Signed-off-by: Tyler Hicks <tyhicks@canonical.com> Reported-by: Christian Boltz <apparmor@cboltz.de>
2017-12-04 23:28:10 +00:00
exit 1; \
Change utils/test/Makefile to use the in-tree libapparmor Also add support for the USE_SYSTEM variable, which means: - test against the in-tree libapparmor and python modules by default - test against the system libapparmor and python modules if USE_SYSTEM is set The old behaviour was a mix of both - it always used the in-tree python modules and the system libapparmor. For obvious reasons, you'll need to build libapparmor before running the tests (unless you specify USE_SYSTEM=1 as parameter to make check). Acked-by: John Johansen <john.johansen@canonical.com> for trunk and 2.9
2015-10-20 23:04:23 +02:00
fi
endif
utils: Gracefully handle a missing parser in the check target The test-aa-easyprof.py script relies on the parser to be built so the check target of the utils/test/Makefile should detect if the parser exists before running any tests. Signed-off-by: Tyler Hicks <tyhicks@canonical.com> Reported-by: Christian Boltz <apparmor@cboltz.de>
2017-12-04 18:03:44 +00:00
__parser:
ifndef USE_SYSTEM
@if [ ! -f $(PARSER) ]; then \
echo "error: $(PARSER) is missing. Pick one of these possible solutions:" 1>&2; \
echo " 1) Test using the in-tree parser by building it first and then trying again. See the top-level README for help." 1>&2; \
echo " 2) Test using the system parser by adding USE_SYSTEM=1 to your make command." 1>&2; \
exit 1; \
fi
endif
gitlab-ci.yml: fix pipeline for ubuntu:latest (noble) Since we are using ubuntu:latest, and noble was released, some tests are failing. shellcheck needs python3 to run, which was possibly installed by default in previous ubuntu images and is no longer the case. Ignore dist-packages python files during our coverage tests. Fixes: https://gitlab.com/apparmor/apparmor/-/issues/388 Signed-off-by: Georgia Garcia <georgia.garcia@canonical.com>
2024-04-30 16:46:32 -03:00
COVERAGE_OMIT=test-*.py,common_test.py,*/dist-packages/*
utils: add python coverage generation This patch adds support for generating test coverage information for the python utils. To view a text based report, in the test subdirectory do: make coverage-report To generate detailed html reports, do: make coverage-html And then point your web browser at $(YOUR_CURRENT_WORKING_TREE)/utils/test/htmlcov/index.html . An alternate output location can be specified by setting the COVERAGE_OUT variable, e.g. make coverage-html COVERAGE_OUT=/tmp/coverage/ (the output directory does not need to exist beforehand.) To generate only the coverage data, do: make coverage or make .coverage (The coverage data generated by python is stored in the .coverage file.) This essentially runs make check, using a single python interpreter, and records which lines and branches of the python code were exercised. Signed-off-by: Steve Beattie <steve@nxnw.org> Acked-by: Christian Boltz <apparmor@cboltz.de> Acked-by: John Johansen <john.johansen@canonical.com>
2014-11-04 13:01:14 -08:00
ifneq ($(COVERAGE_OUT), )
HTML_COVR_ARGS=-d $(COVERAGE_OUT)
endif
let make coverage fail if one of the tests fail If one of the testcases fail, this goes unnoticed in "make coverage". This patch changes the Makefile so that test failures let "make coverage" fail. You can use make COVERAGE_IGNORE_FAILURES=true coverage to build coverage data even if some tests fail. Signed-off-by: Steve Beattie <steve@nxnw.org> (which was most probably meant as an Acked-by) Also Acked-by: <timeout> ;-)
2015-01-16 14:45:16 +01:00
# use make COVERAGE_IGNORE_FAILURES=true coverage to build coverage data even if some tests fail
ifeq ($(COVERAGE_IGNORE_FAILURES), true)
COVERAGE_IGNORE_FAILURES_CMD=true
else
COVERAGE_IGNORE_FAILURES_CMD=set -e
endif
utils: add python coverage generation This patch adds support for generating test coverage information for the python utils. To view a text based report, in the test subdirectory do: make coverage-report To generate detailed html reports, do: make coverage-html And then point your web browser at $(YOUR_CURRENT_WORKING_TREE)/utils/test/htmlcov/index.html . An alternate output location can be specified by setting the COVERAGE_OUT variable, e.g. make coverage-html COVERAGE_OUT=/tmp/coverage/ (the output directory does not need to exist beforehand.) To generate only the coverage data, do: make coverage or make .coverage (The coverage data generated by python is stored in the .coverage file.) This essentially runs make check, using a single python interpreter, and records which lines and branches of the python code were exercised. Signed-off-by: Steve Beattie <steve@nxnw.org> Acked-by: Christian Boltz <apparmor@cboltz.de> Acked-by: John Johansen <john.johansen@canonical.com>
2014-11-04 13:01:14 -08:00
.PHONY: clean check coverage coverage-report coverage-html
utils: fix path in test-aadecode.py and push make check into the test/ subdirectory. Signed-off-by: Steve Beattie <steve@nxnw.org> Acked-by: Seth Arnold <seth.arnold@canonical.com>
2014-02-27 16:23:35 -08:00
ifndef VERBOSE
utils: add python coverage generation This patch adds support for generating test coverage information for the python utils. To view a text based report, in the test subdirectory do: make coverage-report To generate detailed html reports, do: make coverage-html And then point your web browser at $(YOUR_CURRENT_WORKING_TREE)/utils/test/htmlcov/index.html . An alternate output location can be specified by setting the COVERAGE_OUT variable, e.g. make coverage-html COVERAGE_OUT=/tmp/coverage/ (the output directory does not need to exist beforehand.) To generate only the coverage data, do: make coverage or make .coverage (The coverage data generated by python is stored in the .coverage file.) This essentially runs make check, using a single python interpreter, and records which lines and branches of the python code were exercised. Signed-off-by: Steve Beattie <steve@nxnw.org> Acked-by: Christian Boltz <apparmor@cboltz.de> Acked-by: John Johansen <john.johansen@canonical.com>
2014-11-04 13:01:14 -08:00
.SILENT: clean check .coverage coverage coverage-report coverage-html
utils: fix path in test-aadecode.py and push make check into the test/ subdirectory. Signed-off-by: Steve Beattie <steve@nxnw.org> Acked-by: Seth Arnold <seth.arnold@canonical.com>
2014-02-27 16:23:35 -08:00
endif
utils: re-org tests Makefile a bit This patch moves the declaration of phony and quieted make targets to a single section, to avoid repeated lines. It's not so useful for just two targets, but future patches will add more targets with similar attributes. Signed-off-by: Steve Beattie <steve@nxnw.org> Acked-by: John Johansen <john.johansen@canonical.com>
2014-10-27 11:23:49 -07:00
Entire tree: makefile cruft removal - drop the symlink magic of the common/ directory, and just include files directly from there. - update comments indicating required steps to take when including common/Make.rules - drop make clean steps that refer to no longer generated tarballs, specfiles, and symlinks to the common directory/Make.rules. - don't silence clean steps if VERBOSE is set Signed-off-by: Steve Beattie <steve@nxnw.org> Acked-by: Christian "Ghostbuster" Boltz <apparmor@cboltz.de>
2015-01-23 15:52:09 -08:00
clean:
Add `make coverage-regression` ... to find regressions or improvements in the python code coverage. `make coverage-regression` will error out if a file looses its 100% coverage, or if a file improved to 100% coverage. Other coverage changes (for example 45% -> 47%) will be ignored.
2021-01-10 18:14:02 +01:00
rm -rf __pycache__/ .coverage htmlcov coverage-report.txt
utils: fix path in test-aadecode.py and push make check into the test/ subdirectory. Signed-off-by: Steve Beattie <steve@nxnw.org> Acked-by: Seth Arnold <seth.arnold@canonical.com>
2014-02-27 16:23:35 -08:00
utils: Gracefully handle a missing parser in the check target The test-aa-easyprof.py script relies on the parser to be built so the check target of the utils/test/Makefile should detect if the parser exists before running any tests. Signed-off-by: Tyler Hicks <tyhicks@canonical.com> Reported-by: Christian Boltz <apparmor@cboltz.de>
2017-12-04 18:03:44 +00:00
check: __libapparmor __parser
Re-implement aa-notify in Python (Closes: #16) - Code layout based on aa-genprof example - Extend Python dependencies to cover new need by aa-notify - Update documentation after aa-notify is no longer in Perl
2019-01-09 23:59:40 +01:00
export PYTHONPATH=$(PYTHONPATH) LD_LIBRARY_PATH=$(LD_LIBRARY_PATH) LC_ALL=C __AA_CONFDIR=$(CONFDIR) __AA_BASEDIR=$(BASEDIR) __AA_PARSER=$(PARSER) ; $(foreach test, $(wildcard test-*.py), echo ; echo === $(test) === ; $(call pyalldo, $(test)))
utils: add python coverage generation This patch adds support for generating test coverage information for the python utils. To view a text based report, in the test subdirectory do: make coverage-report To generate detailed html reports, do: make coverage-html And then point your web browser at $(YOUR_CURRENT_WORKING_TREE)/utils/test/htmlcov/index.html . An alternate output location can be specified by setting the COVERAGE_OUT variable, e.g. make coverage-html COVERAGE_OUT=/tmp/coverage/ (the output directory does not need to exist beforehand.) To generate only the coverage data, do: make coverage or make .coverage (The coverage data generated by python is stored in the .coverage file.) This essentially runs make check, using a single python interpreter, and records which lines and branches of the python code were exercised. Signed-off-by: Steve Beattie <steve@nxnw.org> Acked-by: Christian Boltz <apparmor@cboltz.de> Acked-by: John Johansen <john.johansen@canonical.com>
2014-11-04 13:01:14 -08:00
Allow running exactly one test in utils/test The new check-one-test-% pattern rule allows running individual test scripts. This allows them to be tested in parallel across many Make worker threads or across many distinct machines with spread. Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
2024-12-02 14:38:28 +01:00
# FIXME: passing -v to test-$*.py below makes some tests fail.
#
# This is a static pattern rule that allows running individual tests via the
# check-one-test-NAME target, where NAME matches an exciting file test-NAME.py.
# The way it is coded allows make <tab><tab> to complete it, making it easier
# to use interactively.
check-one-test-%: PYTHON ?= python3
$(foreach t,$(wildcard test-*.py),check-one-test-$(patsubst test-%.py,%,$t)): check-one-test-%: __libapparmor __parser
PYTHONPATH=$(PYTHONPATH) LD_LIBRARY_PATH=$(LD_LIBRARY_PATH) LC_ALL=C __AA_CONFDIR=$(CONFDIR) __AA_BASEDIR=$(BASEDIR) __AA_PARSER=$(PARSER) $(PYTHON) test-$*.py
utils: Gracefully handle a missing parser in the check target The test-aa-easyprof.py script relies on the parser to be built so the check target of the utils/test/Makefile should detect if the parser exists before running any tests. Signed-off-by: Tyler Hicks <tyhicks@canonical.com> Reported-by: Christian Boltz <apparmor@cboltz.de>
2017-12-04 18:03:44 +00:00
.coverage: $(wildcard ../aa-* ../apparmor/*.py test-*.py) __libapparmor __parser
utils: Set parser executable path according to USE_SYSTEM make variable if USE_SYSTEM is not set, the utils make check target will instruct test-aa-easyprof.py to provide the path of the in-tree parser executable to aa-easyprof. If USE_SYSTEM is set, the default parser path (/sbin/apparmor_parser or the result of `which apparmor_parser`) is used. The test-aa-easyprof.py script receives the parser path by checking the __AA_PARSER environment variable. This environment variable is strictly used by the test script and not any user-facing code so two leading underscores were used. Signed-off-by: Tyler Hicks <tyhicks@canonical.com> Acked-by: Christian Boltz <apparmor@cboltz.de> Acked-by: Seth Arnold <seth.arnold@canonical.com>
2017-03-02 21:24:33 +00:00
export PYTHONPATH=$(PYTHONPATH) LD_LIBRARY_PATH=$(LD_LIBRARY_PATH) LC_ALL=C __AA_CONFDIR=$(CONFDIR) __AA_BASEDIR=$(BASEDIR) __AA_PARSER=$(PARSER) ; $(COVERAGE_IGNORE_FAILURES_CMD) ; $(foreach test, $(wildcard test-*.py), echo ; echo === $(test) === ; $(PYTHON) -m coverage run --branch -p $(test); )
utils: add python coverage generation This patch adds support for generating test coverage information for the python utils. To view a text based report, in the test subdirectory do: make coverage-report To generate detailed html reports, do: make coverage-html And then point your web browser at $(YOUR_CURRENT_WORKING_TREE)/utils/test/htmlcov/index.html . An alternate output location can be specified by setting the COVERAGE_OUT variable, e.g. make coverage-html COVERAGE_OUT=/tmp/coverage/ (the output directory does not need to exist beforehand.) To generate only the coverage data, do: make coverage or make .coverage (The coverage data generated by python is stored in the .coverage file.) This essentially runs make check, using a single python interpreter, and records which lines and branches of the python code were exercised. Signed-off-by: Steve Beattie <steve@nxnw.org> Acked-by: Christian Boltz <apparmor@cboltz.de> Acked-by: John Johansen <john.johansen@canonical.com>
2014-11-04 13:01:14 -08:00
$(PYTHON) -m coverage combine
coverage: .coverage
coverage-report: .coverage
$(PYTHON) -m coverage report --omit="$(COVERAGE_OMIT)"
Add `make coverage-regression` ... to find regressions or improvements in the python code coverage. `make coverage-regression` will error out if a file looses its 100% coverage, or if a file improved to 100% coverage. Other coverage changes (for example 45% -> 47%) will be ignored.
2021-01-10 18:14:02 +01:00
coverage-regression: .coverage
$(PYTHON) -m coverage report --omit="$(COVERAGE_OMIT)" > coverage-report.txt
Generate and keep html in utils coverage-regression We sometimes have random coverage changes that are not reproducible and therefore hard to debug. Generate html coverage as part of make coverage-regression, and keep the resulting utils/test/htmlcov/ as artifact to make debugging easier. coverage-html needs JS files from various libjs-* packages, install them in before_script
2021-07-13 12:21:52 +02:00
$(PYTHON) -m coverage html --omit="$(COVERAGE_OMIT)" $(HTML_COVR_ARGS)
Add `make coverage-regression` ... to find regressions or improvements in the python code coverage. `make coverage-regression` will error out if a file looses its 100% coverage, or if a file improved to 100% coverage. Other coverage changes (for example 45% -> 47%) will be ignored.
2021-01-10 18:14:02 +01:00
cat coverage-report.txt
@echo 'Checking for coverage changes...'
@if grep -vE ' 100%$$|TOTAL|$(INCOMPLETE_COVERAGE)' coverage-report.txt |grep '%$$' ; then echo "ERROR: Coverage regression - the files listed above are expected to have 100% test coverage"; exit 1; fi
@if grep -E '($(INCOMPLETE_COVERAGE)).*100%$$' coverage-report.txt ; then echo 'ERROR: Coverage improvement - the files listed above now have 100% coverage. Please adjust INCOMPLETE_COVERAGE in the Makefile!'; exit 2; fi
@echo ' ... unchanged'
utils: add python coverage generation This patch adds support for generating test coverage information for the python utils. To view a text based report, in the test subdirectory do: make coverage-report To generate detailed html reports, do: make coverage-html And then point your web browser at $(YOUR_CURRENT_WORKING_TREE)/utils/test/htmlcov/index.html . An alternate output location can be specified by setting the COVERAGE_OUT variable, e.g. make coverage-html COVERAGE_OUT=/tmp/coverage/ (the output directory does not need to exist beforehand.) To generate only the coverage data, do: make coverage or make .coverage (The coverage data generated by python is stored in the .coverage file.) This essentially runs make check, using a single python interpreter, and records which lines and branches of the python code were exercised. Signed-off-by: Steve Beattie <steve@nxnw.org> Acked-by: Christian Boltz <apparmor@cboltz.de> Acked-by: John Johansen <john.johansen@canonical.com>
2014-11-04 13:01:14 -08:00
coverage-html: .coverage
$(PYTHON) -m coverage html --omit="$(COVERAGE_OMIT)" $(HTML_COVR_ARGS)
Reference in a new issue Copy permalink