QuickCheck

Files : use31.m use33.m nrev.m
Back to main

# QuickCheck Tutorial 3

## Monitoring Test Data: `to_trivial/3`

In Tutorial 1, the 3rd rule for reverse is that:

            reverse (reverse xs) = xs

It's not much of a test if `xs` is an empty list or a list with only 1 element.

Quickcheck can label a test being trivial via the function to_trivial/3, which does not change the meaning of a law, but it classifies some of the test cases. Without classifying, the invariant function could be:

            :- func testing1(list(float)) = property.
            testing1(Xs) =
                    nrev (nrev Xs) `===` Xs.

If the 1st argument of to_trivial/3 is equal to the 2nd argument, then that test case will be labeled trivial by pushing `flag:trivial` into the third argument (which is a list of flags). testing2/1 treats empty list as trivial test.

            :- func testing2(list(float)) = property.
            testing2(Xs) =
                    to_trivial([], Xs, nrev (nrev Xs) `===` Xs).

Use compounded to_trivial to also classify lists of 1 element as trivial

            :- func testing3(list(float)) = property.
            testing3(Xs) =
                    to_trivial(1,
                               list_length(Xs),
                               to_trivial([], Xs, nrev(nrev(Xs)) `===` Xs)
                              ).

The complete code (use31.m):

    :- module use31.

    :- interface.

    :- use_module io.

    :- pred main(io__state, io__state).
    :- mode main(di, uo) is det.

    %---------------------------------------------------------------------------%

    :- implementation.

    :- import_module int, list, bool.
    :- import_module qcheck, nrev.

    %---------------------------------------------------------------------------%

    main -->
            qcheck(qcheck__f(testing1), "testing1"),
            qcheck(qcheck__f(testing2), "testing2"),
            qcheck(qcheck__f(testing3), "testing3").

    :- func testing1(list(float)) = property.
    testing1(Xs) =
            nrev(nrev(Xs)) `===` Xs.

    :- func testing2(list(float)) = property.
    testing2(Xs) =
            to_trivial([], Xs, nrev(nrev(Xs)) `===` Xs).

    :- func testing3(list(float)) = property.
    testing3(Xs) =
            to_trivial(1,
                       list_length(Xs),
                       to_trivial([], Xs, nrev(nrev(Xs)) `===` Xs)
                      ).

A sample output :

            Test Description : testing1
            Number of test cases that succeeded : 100
            Number of trivial tests : 0
            Number of tests cases which failed the pre-condition : 0
            Distributions of selected argument(s) :

            Test Description : testing2
            Number of test cases that succeeded : 100
            Number of trivial tests : 53
            Number of tests cases which failed the pre-condition : 0
            Distributions of selected argument(s) :

            Test Description : testing3
            Number of test cases that succeeded : 100
            Number of trivial tests : 75
            Number of tests cases which failed the pre-condition : 0
            Distributions of selected argument(s) :

Note test1, the original, has no trivial cases. With test2, 53/100 tests have an empty list as its input. Test3 shows 75/100 tests have either an empty list or a list of only one element. It only tested 25/100 cases where the list is longer than 1 element.

## Monitoring Test Data: ``>>>``

The combinator ``>>>`` gathers all values that are passed to it, and prints out a histogram of these values. Let's use ``>>>`` to find out exactly what lists are generated for the previous tests:

            :- func testing4(list(float)) = property.
            testing4(Xs) =
                    list_length(Xs) `>>>` (nrev(nrev(Xs)) `===` Xs).

The combinator ``>>>`` will convert its left argument to a univ, and push info(univ) into the property list.

The complete code (use31.m):

    :- module use31.

    :- interface.

    :- use_module io.

    :- pred main(io__state, io__state).
    :- mode main(di, uo) is det.

    %---------------------------------------------------------------------------%

    :- implementation.

    :- import_module int, list, bool.
    :- import_module qcheck, nrev.

    %---------------------------------------------------------------------------%

    main -->
            qcheck(qcheck__f(testing1), "testing1"),
            qcheck(qcheck__f(testing2), "testing2"),
            qcheck(qcheck__f(testing3), "testing3"),
            qcheck(qcheck__f(testing4), "testing4").

    :- func testing1(list(float)) = property.
    testing1(Xs) =
            nrev(nrev(Xs)) `===` Xs.

    :- func testing2(list(float)) = property.
    testing2(Xs) =
            to_trivial([], Xs, nrev(nrev(Xs)) `===` Xs).

    :- func testing3(list(float)) = property.
    testing3(Xs) =
            to_trivial(1,
                       list_length(Xs),
                       to_trivial([], Xs, nrev(nrev(Xs)) `===` Xs)
                      ).

    :- func testing4(list(float)) = property.
    testing4(Xs) =
            list_length(Xs) `>>>` (nrev(nrev(Xs)) `===` Xs).

A sample output :

    Test Description : testing1
    Number of test cases that succeeded : 100
    Number of trivial tests : 0
    Number of tests cases which failed the pre-condition : 0
    Distributions of selected argument(s) :

    Test Description : testing2
    Number of test cases that succeeded : 100
    Number of trivial tests : 53
    Number of tests cases which failed the pre-condition : 0
    Distributions of selected argument(s) :

    Test Description : testing3
    Number of test cases that succeeded : 100
    Number of trivial tests : 71
    Number of tests cases which failed the pre-condition : 0
    Distributions of selected argument(s) :

    Test Description : testing4
    Number of test cases that succeeded : 100
    Number of trivial tests : 0
    Number of tests cases which failed the pre-condition : 0
    Distributions of selected argument(s) :
    1     8
    1     4
    1     6
    2     5
    8     3
    16     2
    18     1
    53     0

The display of testing4 shows that 53 cases of length == 0 18 cases of length == 1 16 cases of length == 2 ...etc... 53+18 cases = 71 cases, which were marked trivial in testing3, likewise for testing2. The numbers will add up only if all the tests were run with the same random number seed. The value passed to `>>>` does not have to be the same type, and `>>>` can be compounded like to_trivial/3, e.g. (use33.m):

    :- module use33.

    :- interface.

    :- use_module io.

    :- pred main(io__state, io__state).
    :- mode main(di, uo) is det.

    %---------------------------------------------------------------------------%

    :- implementation.

    :- import_module int, list, bool.
    :- import_module qcheck, nrev.

    %---------------------------------------------------------------------------%

    main -->
            qcheck(qcheck__f(testing5), "testing5").

    :- func testing5(list(float)) = property.
    testing5(Xs) =
            odd_even(Xs) `>>>`
                         (list_length(Xs) `>>>` (nrev(nrev(Xs)) `===` Xs)).

    :- func odd_even(list(T)) = string.
    :- mode odd_even(in) = out is det.
    odd_even(Xs) = Y :-
            (if     list_length(Xs) mod 2 = 1
             then
                    Y = "odd"
             else
                    Y = "even"
            ).

testing5 collects the list_length, and also collect "odd" or "even" A sample output :

    Test Description : testing5
    Number of test cases that succeeded : 100
    Number of trivial tests : 0
    Number of tests cases which failed the pre-condition : 0
    Distributions of selected argument(s) :
    1     7
    1     5
    2     4
    2     6
    8     3
    10     2
    29     1
    39     "odd"
    47     0
    61     "even"