Two main tasks have been carried out in this thesis as a solution to the test oracle problem. First; an independent model of a real trading system based on constraint programming; a method for solving combinatorial problems; has been created. Then; the model has been integrated as a true test oracle in automated random tests. The test oracle maintains the expected state of an order book throughout a sequence of random trade order actions; and provides the expected output of every auction triggered in the order book by generating a corresponding constraint program that is solved with the aid of a constraint programming system.
Constraint programming has allowed the development of an inexpensive; yet reliable test oracle. In 500 random test cases; the test oracle has detected two system failures. These failures correspond to defects that had been present for several years without being discovered neither by less complete oracles nor by the application of more systematic testing approaches.
The main contributions of this thesis are: (1) empirical evidence of both the suitability of applying constraint programming to solve the test oracle problem and the effectiveness of true test oracles in random testing; and (2) a first attempt; as far as the author is aware; to model a non-theoretical combinatorial double auction using constraint programming.