apparmor/parser/tst
Steve Beattie 573cfb6b04 fix two issues for older releases
The following patch addresses two issues on older releases:

1) In trunk commit 2911, the line 'undefine VERBOSE' was added to
   parser/tst/Makefile so that the equality tests would not generate
   verbose output when $VERBOSE != 1. Unfortunately, the 'undefine'
   keyword was not introduced in GNU Make until version 3.82. On
   distro releases like Ubuntu 12.04 LTS that include versions of Make
   older than that, make check and make clean abort when VERBOSE is
   not set to 1. The patch fixes that by setting VERBOSE to a zero
   length string if does not already equal 1.

2) In trunk commit 2923, a workaround for systemd as init was added
   to the pivot_root regression test. The workaround included a
   call to ps(1) to determine if systemd is pid 1. Unfortunately,
   in older versions of the procps package (such as the version in
   Ubuntu 12.04 LTS), 'ps -hp1' emits the warning

     Warning: bad ps syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html

   The patch below converts the ps call to 'ps hp1' which does not
   generate the warning.

Signed-off-by: Steve Beattie <steve@nxnw.org>
Acked-by: John Johansen <john.johansen@canonical.com>
2015-03-31 02:46:45 -07:00
..
errors as ACKed on IRC, drop the unused $Id$ tags everywhere 2010-12-20 12:29:10 -08:00
features_files parser: update test features file for af_unix feature 2014-09-03 14:12:50 -07:00
simple_tests parser: fix compilation failure of deny link rules 2015-03-23 11:25:48 -07:00
caching.profile actually add caching tests 2009-11-11 11:07:50 -08:00
caching.py Don't use the parser time stamp to determine if policy is newer. 2014-04-23 11:01:33 -07:00
equality.sh parser: Expand Equality tests touchups 2015-03-23 11:55:48 -07:00
gen-dbus.pl parser: Test dbus eavesdrop rule parsing 2013-12-06 11:18:48 -08:00
gen-xtrans.pl Description: Improve generated test readability and build-time cleanup. 2011-05-02 13:36:55 -07:00
Makefile fix two issues for older releases 2015-03-31 02:46:45 -07:00
minimize.sh Hack rework of the feature/match file support 2014-04-23 10:53:46 -07:00
mk_features_file.py parser language tests: force using a features file 2014-04-24 12:14:54 -07:00
README parser: Detect core dumps during parser tests 2013-09-11 11:56:47 -07:00
simple.pl parser language tests: force using a features file 2014-04-24 12:14:54 -07:00
testlib.py parser testlib - use metaclass to mark all test functions keep_on_fail 2013-10-25 16:26:16 -07:00
uservars.conf as ACKed on IRC, drop the unused $Id$ tags everywhere 2010-12-20 12:29:10 -08:00
valgrind_simple.py parser: fix equality and valgrind test scripts to use features file 2015-03-19 04:35:08 -07:00

This is the README for the AppArmor parser regression testsuite.

Running the testsuite
---------------------
Running the tests is pretty easy, a simple 'make tests' should make it
go, assuming the subdomain parser and perl are installed.

There is a user configuration file 'uservars.conf'. If you wish to test
against a different parser, or use a different set of profiles for the
simple.pl test, you can change those settings in 'uservars.conf'.

You can also override which parser is used through make by specifying
the PARSER veriable. For example, to run the tests on the system parser,
run 'make PARSER=/sbin/apparmor_parser'.

Adding to the testsuite
-----------------------

The testsuite currently contains one testscript (simple.pl) and makes use
of perl's Test::Simple, Test::Harness, and prove utilities (see 'perldoc
Test::Tutorial', 'perldoc Test::Simple', 'perldoc Test::Harness', and
'man 1 prove' for more information on these).

It should be relatively easy to extend the suite with other testscripts,
as long as they're written using Test::Simple or can emulate the
Test::Harness protocol. To add a script, add it to the TESTS variable
in the Makefile, and it will included in the tests to be run.

However, in many cases, it is not necessary to add an entire new
testscript for a testcase. Instead, the simple testcase (see below)
will run all the profiles it finds on the parser, thus adding testcases
is usually as simple as writing a new profile with a couple of extra
comments.

Simple parsing tests (simple.pl)
--------------------------------
This test script tests the parser front end's ability to identify legal
profiles. It does this by running the parser against several legal and
illegal profiles (in debug mode, so as not to load them into the module
proper)

The simple script has the parser attempt to parse all of the profiles
named *.sd in the simple_tests/ subdirectory; thus, to add a new profile
to test, simply add it to the simple_tests/ directory. The simple
script also adds the testdir (simple_tests/ by default) to the parsers
include path (assuming that particular bug has been fixed :-)). There
is an includes/ subdir to place additional includes if necessary (we
purposefully choose to use different directory names versus the shipped
profiles to minimize testsuite breakage with changes in the external
policy).

The simple script looks for a few special comments in the profile,
#=DESCRIPTION, #=EXRESULT, and #=TODO:

  - #=DESCRIPTION -- all text following the keyword is considered a
    description for the test. Please try to make these meaningful.

  - #=EXRESULT -- This records the expected result of parsing this
    profile. Values can either be PASS or FAIL; if no comment is found
    that matches this pattern, then the profile is assumed to have an
    expected parse result of PASS.

  - #=TODO -- marks the test as being for a future item to implement and
    thus are expected testsuite failures and hsould be ignored.

  - #=DISABLED -- skips the test, and marks it as a failed TODO task.
    Useful if the particular testcase causes the parser to infinite
    loop or dump core.

Otherwise, the profile is passed on as-is to the subdomain parser.