test/README

Unit test

This directory contains the unit test support for the RMr
library.  The basic test is run with the follwing command:

	ksh unit_test.ksh

To run a specific test (e.g. ring_test.c) run:
	ksh unit_test.ksh ring_test.c

The script runs the unit test(s) given, and if they pass then
runs an analysis on the .gcov files generated to generate
coverage information.  By default, pass/fail of the test is
based only on the success or failure of the unit tests which
are testing functionality.  The unit test script can report
an overall failure if coverage is below the indicated threshold
when given the strict option (-s).

The analysis of .gcov files generates output shown below which
is thought to be more straight forward than the typical stuff
gcov produces:

unit_test.ksh ring_test.c
ring_test.c --------------------------------------
[OK]   100% uta_ring_insert
[OK]   100% uta_ring_extract
[OK]   100% uta_ring_free
[LOW]   76% uta_mk_ring
[PASS]  91% ../src/common/src/ring_static.c


The output shows, for each function, the coverage (column 2) and an
interpretation (ok or low) wthin an overall pass or fail.


Because of the static nature of the RMr library, tests with the
intent of providing coverage information, as opposed just to providing
functional verification, are a bit trickier.  To that end, the test
files in this directory are organised with three file name formats:

	test_*.c  tools for testing, not tests

	*_test.c  main test programmes which can be compiled in
			  a stand-alone manner (e.g. gcc foo_test.c)

	*_static_test.c  Test functions which are real tests and are
			  included by one or more stand-alone driver.

The unit_test script will search only for *_test.c and will ignore
*_static_test.c files when building it's list for testing.


Use the command 'unit_test.ksh -?' to see the usage information
and complete set of options available.


Discounting
The unit test script makes a discount pass on low coverage files in
attempt to discount the coverage rate by ignoring what are considered
to be difficult to reach blocks in the code.  Currently, these blocks
are limited to what appear to be tests for memory allocation, failure
and/or nil pointer handling.  If code blocks of this sort are found,
they are not counted against the coverage for the module.  If the -v
option is given, an augmented coverage listing is saved in .dcov which
shows the discounted lines with a string of equal signs (====) rather
than the gcov hash string (###).

The discount check is applied only if an entire module has a lower
than accepted coverage rate, and can be forced for all modules with
the -f option.

To illustrate, the following code checks the return from the system
library strdup() call which is very unlikely to fail under test without
going to extremes and substituting for the system lib.  Thus, the
block which checks for a nil pointer has been discounted:

     -:  354:
     1:  355:    dbuf = strdup( buf );
     1:  356:    if( dbuf == NULL  ) {
 =====:  357:        errno = ENOMEM;
 =====:  358:        return 0;
     -:  359:    }


Target Coverage
By default, a target coverage of 80% is used. For some modules this may
be impossible to achieve, so to prevent always failing these modules
may be listed in the .targets file with their expected minimum coverage.
Module names need to be qualified (e.g. ../src/common/src/foo.c.


-----------------------------------------------------------------------
A note about ksh (A.K.A Korn shell, or kshell)
Ksh is preferred for more complex scripts such as the unit test
script as it does not have some of the limitations that bash
(and other knock-offs) have.