QuickCheck

Files: use.m use1.m use11.m nrev.m nrev2.m
Back to main

# QuickCheck Tutorial 1

Quickcheck is a tool that aids the Mercury programmer in formulating and testing properties of programs. The programmer must supply invariant functions which test certain properties of the programs. Those invariant functions can then be automatically tested on random inputs. It is possible to alter the test data distribution or define a custom test data generator.

In order to test the validity of a predicate/function via quickcheck, you need the following components:

qcheck.m
  quickcheck module (supplied)
rnd.m
  required by quickcheck module (supplied)
XXX.m
  module containing the predicate/function the user is trying to validate
YYY.m
  main module, which calls `qcheck`/4.

## A Simple Example

Let XXX.m be "nrev.m", which contains nrev/1:

            :- func nrev(list(T)) = list(T).
            :- mode nrev(in) = out is det.
            nrev([]) = [].
            nrev([X|Xs]) = Ys :-
                    list__append(nrev(Xs), [X], Ys).

We want to test whether nrev/1 is properly implemented.

Let YYY.m be "use.m". It must contain a main/2, and an invariant function. If nrev/1 is correctly implemented then it should satisfy the following rules:

                reverse [x] = [x]
         reverse (xs ++ ys) = reverse(ys) ++ reverse(xs)
       reverse (reverse xs) = xs

To test whether 2nd rule holds for nrev/1, define the invariant function as :

            testing(Xs, Ys) =
                    nrev(Xs ++ Ys) `===` (nrev(Ys) ++ nrev(Xs)).

`===` function returns 'property:[yes]' if the left side equals right side, 'property:[no]' otherwise. In theory nrev/1 can take a list(T), however in order for quickcheck to automatically generate Xs and Ys, the programmer must specify a fixed type for which the law is to be tested, eg:

            :- func testing(list(int), list(int)) = property.
                                    Or
            :- func testing(list(char), list(char)) = property.

Now define the main/2 as

            main -->
                    qcheck(qcheck__f(testing), "sample testing").

The complete use.m looks like this:

    :- module use.

    :- interface.

    :- use_module io.

    :- pred main(io__state, io__state).
    :- mode main(di, uo) is det.

    %-------------------------------------------------------------------%

    :- implementation.

    :- import_module int, list.
    :- import_module qcheck, nrev.

    %-------------------------------------------------------------------%

    main -->
            qcheck(qcheck__f(testing), "sample testing").

    %-------------------------------------------------------------------%
    %       Invariant test functions
    %-------------------------------------------------------------------%

    :- func testing(list(int), list(int)) = property.
    testing(Xs, Ys) =
            nrev(Xs ++ Ys) `===` (nrev(Ys) ++ nrev(Xs)).

To compile the program, do "mmake use.depend", then "mmake use". After running, the statistics should be displayed like :

            Test Description : sample testing
            Number of test cases that succeeded : 100
            Number of trivial tests : 0
            Number of tests cases which failed the pre-condition : 0
            Distributions of selected argument(s) :

nrev/1 passed 100 tests and failed none (ignore the last 3 lines for now).

If the invariant function is wrongly specified, or nrev/1 is not properly implemented, then the statistics may look like :

            Test description : sample testing
            Falsifiable :
            [2]
            [-2, 1]

where running [2] as Xs and [-2, 1] as Ys failed the invariant function.

Defining an invariant function which contains only functions is more concise than one which contains predicates. Suppose that "nrev2.m" defines a predicate version of reversing list:

            :- pred nrev2(list(T), list(T)).
            :- mode nrev2(in, out) is det.
            nrev2([], []).
            nrev2([X|Xs], Ys):-
                    nrev2(Xs, Reversed),
                    list__append(Reversed, [X], Ys).

Then the invariant function looks like :

            testing2(Xs, Ys) = (Left `===` Right) :-
                    nrev2((Xs ++ Ys), Left),
                    nrev2(Ys, Part_a),
                    nrev2(Ys, Part_b),
                    Right = Part_a ++ Part_b.

Above code is effectively the same as testing/2, however it requires a few extra lines to extract the intermediate values, (use1.m):

    :- module use1.

    :- interface.

    :- use_module io.

    :- pred main(io__state, io__state).
    :- mode main(di, uo) is det.

    %-------------------------------------------------------------------%

    :- implementation.

    :- import_module int, list.
    :- import_module qcheck, nrev2.

    %-------------------------------------------------------------------%

    main -->
            qcheck(qcheck__f(testing2), "testing2").

    %-------------------------------------------------------------------%
    %       Invariant test functions
    %-------------------------------------------------------------------%

    :- func testing2(list(int), list(int)) = property.
    testing2(Xs, Ys) = (Left `===` Right) :-
            nrev2((Xs ++ Ys), Left),
            nrev2(Ys, Part_a),
            nrev2(Xs, Part_b),
            Right = Part_a ++ Part_b.

An alternative to writing a separate invariant function is to use Mercury's syntax for higher order terms. When doing this, the ``with_type`` operator can be useful to provide fixed types for the arguments of the invariant function. For example (use11.m):

    :- module use11.

    :- interface.

    :- use_module io.

    :- pred main(io__state, io__state).
    :- mode main(di, uo) is det.

    %--------------------------------------------------------------------------%

    :- implementation.

    :- import_module int, list.
    :- import_module qcheck, nrev.

    %--------------------------------------------------------------------------%

    main -->
        qcheck(qcheck__f(func(Xs `with_type` list(int), Ys `with_type` list(int))
                         = nrev(Xs ++ Ys) `===` (nrev(Ys) ++ nrev(Xs))),
               "sample testing").

In YYY.m, the main/2 should be defined as :

            main -->
                    qcheck(qcheck__f(Invariant_Function_X), Test_Desc_Y).

Where Invariant_Function_X is a higher order term (function) and Test_Desc_Y is a string which describe/name/comment Invariant_Function_X.

The invariant function is of the form

            :- mode Invariant_Function_X(in, in, in ...) = out.

The invariant function can only take 0 to 10 arguments, but the inputs can be of any type. All invariant functions must return a property, which is just a list of flags.

            :- type property == list(flag).

            :- type flag
                    --->    yes
                    ;       no
                    ;       trivial
                    ;       info(univ)
                    ;       condition.

QuickCheck does not care what happens inside Invariant_Function_X. If the output does not contain 'flag:no', the invariant function is assumed to pass the testcase satisfactory.

See part 4 of the tutorial to get the details on each individual flag.