Automated Oracles: An Empirical Study on Cost and Effectiveness

Cu D. Nguyen, Alessandro Marchetto, and Paolo Tonella Fondazione Bruno Kessler Trento, Italy {cunduy,marchetto,tonella}

Tools (treatments):

Experiment packages:

Subject systems for false positive study:

Name Size (NCLoC) Test Size Description
commons-collections 17,590 4,601 Apache collection library for Java
commons-math3 50,924 2,392 Apache mathematic library for Java
xstream 17,202 967 Library for serializing objects to XML and back
jgap 43,501 1,415 Genetic programming library
openmrs-api 78,381 2,400 Core API of OpenMRS, a medical record system
jfreechart 94,550 2,218 Library to create charts in Java
joda-time 27,213 3,899 Date and time library

These systems are accessible as of June 2013.

Quick guide:

All these subject systems are built with Maven, compilation and test execution are easy. Moreover, Daikon online instrumentation and trace collection can be easily integrated with Maven test execution. We have all the necessary scripts for test execution, trace extraction and encoding, and experiment execution. Please contact us to get a copy if you are interested.