Date of Award


Document Type


Degree Name



Computer Science

First Advisor

Matthew J. Rutherford


design by contract, empirical study, runtime assertion checking, test oracles


Test oracles remain one of the least understood aspects of the modern

testing process. An oracle is a mechanism used by software testers and

software engineers for determining whether a test has passed or

failed. One widely-supported approach to oracles is the use of runtime

assertion checking during the testing activity. Method invariants,

pre- and postconditions help detect bugs during runtime. While

assertions are supported by virtually all programming environments,

are used widely in practice, and are often assumed to be effective as

test oracles, there are few empirical studies of their efficacy in

this role. In this thesis, we present the results of an experiment we

conducted to help understand this question. To do this, we studied

seven of the core Java classes that had been annotated by others with

assertions in the Java Modeling Language, used the muJava mutation

analysis tool to create mutant implementations of these classes,

exercised them with input-only (i.e., no oracle) test suites that

achieve branch coverage, and used a machine learning tool, Weka, to

determine which annotations were effective at ``killing'' these

mutants. We also evaluate how effective the ``null oracle'' (in our

case, the Java runtime system) is at catching these bugs. The results

of our study are interesting, and help provide software engineers with

insight into situations in which assertions can be relied upon to find

bugs, and situations in which assertions may need to be augmented with

other approaches to test oracles.


Copyright is held by the author.


Recieved from ProQuest

Rights holder

Kavir Shrestha

File size

76 p.

File format





Computer science