randoop.main

Class GenTests

java.lang.Object

randoop.main.CommandHandler

randoop.main.GenInputsAbstract

randoop.main.GenTests

public class GenTests
extends GenInputsAbstract

Test generation.

Nested Class Summary

Nested classes/interfaces inherited from class randoop.main.GenInputsAbstract

GenInputsAbstract.BehaviorType, GenInputsAbstract.BloodhoundCoverageUpdateMode, GenInputsAbstract.ClassLiteralsMode, GenInputsAbstract.FlakyTestAction, GenInputsAbstract.InputSelectionMode, GenInputsAbstract.Limits, GenInputsAbstract.MethodSelectionMode

Field Summary

Fields

Modifier and Type	Field and Description
private com.github.javaparser.ast.stmt.BlockStmt	afterAllFixtureBody
private com.github.javaparser.ast.stmt.BlockStmt	afterEachFixtureBody
private com.github.javaparser.ast.stmt.BlockStmt	beforeAllFixtureBody
private com.github.javaparser.ast.stmt.BlockStmt	beforeEachFixtureBody
private static java.lang.String	command
private static java.lang.String	commandGrammar
private static java.lang.String	example
private java.util.Map<java.net.URI,java.nio.file.FileSystem>	fileSystemCache A cache used by getResourceDirectoryPath(java.lang.String), to prevent FileSystemAlreadyExistsException.
private static java.lang.String	input
private static java.lang.String	NO_OPERATIONS_TO_TEST
private static java.util.List<RawSignature>	nonFlakyMethods Methods that are known to be non-flaky.
private static java.util.List<java.lang.String>	notes Explanations printed to the user.
private static org.plumelib.options.Options	options
private static java.lang.String	output
private static java.lang.String	pitch
static java.lang.String	POSSIBLY_FLAKY_PREFIX Is output to the user before each possibly flaky method.
private int	sequenceCompileFailureCount The count of sequences that failed to compile.
private static java.lang.String	summary
static @Regex(value=0) @Identifier java.lang.String	TEST_METHOD_NAME_PREFIX The prefix for test method names.
private static java.lang.String	where

Fields inherited from class randoop.main.GenInputsAbstract

alias_ratio, all_logs, attempted_limit, bloodhound_logging, bloodhound_update_mode, capture_output, check_compilable, checked_exception, classlist, clear, clear_memory, cm_exception, debug_checks, deterministic, dont_omit_replaced_methods, dont_output_tests, dontexecute, error_test_basename, fail_on_generation_error, flaky_test_behavior, forbid_null, generated_limit, grt_fuzzing, grt_fuzzing_stddev, ignore_condition_compilation_error, ignore_condition_exception, ignore_condition_exception_quiet, input_selection, junit_after_all, junit_after_each, junit_before_all, junit_before_each, junit_output_dir, junit_package_name, junit_reflection_allowed, jvm_max_memory, literals_file, literals_level, log, maxsize, method_selection, methodlist, minimize_error_test, ncdf_exception, no_error_revealing_tests, no_regression_assertions, no_regression_tests, nondeterministic_methods_to_output, npe_on_non_null_input, npe_on_null_input, null_ratio, omit_classes, omit_classes_file, omit_classes_no_defaults, omit_field, omit_field_file, omit_field_list, omit_methods, omit_methods_file, omit_methods_no_defaults, only_test_public_members, oom_exception, operation_history_log, output_limit, print_non_compiling_file, progressdisplay, progressintervalmillis, PROGRESSINTERVALMILLIS_DEFAULT, progressintervalsteps, randomseed, regression_test_basename, repeat_heuristic, require_classname_in_test, require_covered_classes, selection_log, side_effect_free_methods, silently_ignore_bad_class_names, sof_exception, specifications, stop_on_error_test, string_maxlen, system_props, test_package, testclass, testjar, testsperfile, time_limit, unchecked_exception, use_jdk_specifications, visitor

Fields inherited from class randoop.main.CommandHandler

fcommand, fcommandGrammar, fexample, finput, fnotes, foptions, foutput, fpitch, fsummary, fwhere

Constructor Summary

Constructors

Constructor and Description
GenTests() GenTests constructor that uses default messages.

Method Summary

Modifier and Type	Method and Description
private java.lang.String	convertClasspathToAbsolute(java.lang.String classpath) Convert each element of the given classpath from a relative to an absolute path.
private java.util.Map<TypedClassOperation,java.lang.Integer>	countSequencesPerOperation(java.util.List<ExecutableSequence> sequences, MultiMap<Type,TypedClassOperation> assertableSideEffectFreeMethods) Counts the number of sequences each operation occurs in.
private java.util.List<java.util.regex.Pattern>	createPatternsFromSignatures(java.util.List<java.lang.String> signatures) Creates a list of signature strings (see RawSignature.toString() to a list of Pattern.
static TestCheckGenerator	createTestCheckGenerator(AccessibilityPredicate accessibility, ContractSet contracts, MultiMap<Type,TypedClassOperation> sideEffectFreeMethodsByType, OmitMethodsPredicate omitMethodsPredicate) Creates the test check generator for this run based on the command-line arguments.
java.util.function.Predicate<ExecutableSequence>	createTestOutputPredicate(java.util.Set<Sequence> excludeSet, java.util.Set<java.lang.Class<?>> coveredClasses, java.util.regex.Pattern includePattern) Builds the test predicate that determines whether a particular sequence will be included in the output based on command-line arguments.
private static @PolyNull java.util.List<java.lang.String>	getFileText(@PolyNull java.lang.String filename) Return the text of the given file, as a list of lines.
private boolean	getFixtureCode() Create fixture code from GenInputsAbstract.junit_after_all, GenInputsAbstract.junit_after_each, GenInputsAbstract.junit_before_all, and GenInputsAbstract.junit_before_each and set fixture body variables.
private java.util.Collection<? extends java.nio.file.Path>	getJDKSpecificationFiles() Returns the list of JDK specification files from the specifications/jdk resources directory in the Randoop jar file.
private java.util.Set<TypedClassOperation>	getOperationsInSequence(ExecutableSequence es) Constructs a set of method-call operations appearing in an Executable Sequence.
private java.nio.file.Path	getResourceDirectoryPath(java.lang.String resourceDirectory) Returns the path for the resource directory in the jar file.
boolean	handle(java.lang.String[] args)
void	incrementSequenceCompileFailureCount() Increments the count of sequence compilation failures.
private static boolean	isNonFlaky(TypedClassOperation op) Returns true if the operation is definitely non-flaky.
private void	printSequenceExceptionError(AbstractGenerator explorer, SequenceExceptionError e) Prints information about a SequenceExceptionError that indicates a flaky test has been found.
private void	processAndOutputFlakyMethods(java.util.List<ExecutableSequence> flakySequences, java.util.List<ExecutableSequence> sequences, MultiMap<Type,TypedClassOperation> sideEffectFreeMethodsByType, OmitMethodsPredicate omitMethodsPredicate, AccessibilityPredicate accessibilityPredicate) Outputs names of suspected flaky methods by using the tf-idf metric (Term Frequency - Inverse Document Frequency), which is:
private java.util.List<java.util.regex.Pattern>	readPatterns(org.plumelib.util.EntryReader er) Returns patterns read from the given EntryReader.
private java.util.List<java.util.regex.Pattern>	readPatterns(java.io.InputStream is, java.lang.String filename) Returns patterns read from the given stream.
private java.util.List<java.util.regex.Pattern>	readPatterns(java.nio.file.Path path) Returns patterns read from the given user-provided file.
private java.util.List<java.util.regex.Pattern>	readPatternsFromResource(java.lang.String filename) Returns patterns read from the given resource.
static MultiMap<Type,TypedClassOperation>	readSideEffectFreeMethods() Read side-effect-free methods from the default JDK side-effect-free method list, and from a user-provided method list if provided.
private static java.util.regex.Pattern	signatureToPattern(java.lang.String signatureString) Converts a signature string (see RawSignature.toString() to a Pattern that matches that string (with spaces removed).
private java.util.List<ExecutableSequence>	testNamesToSequences(java.lang.Iterable<java.lang.String> testNames, java.util.List<ExecutableSequence> sequences) Given a collection of test names of the form "test005", returns the corresponding elements from the given list.
private static void	usage(java.lang.String format, java.lang.Object... args) Print message, then print usage information, then exit.
private void	writeTestFiles(JUnitCreator junitCreator, java.util.List<ExecutableSequence> testSequences, CodeWriter codeWriter, java.lang.String classNamePrefix, java.lang.String testKind) Creates the test classes for the test sequences using the JUnitCreator and then writes the files using the CodeWriter.

Methods inherited from class randoop.main.GenInputsAbstract

checkOptionsValid, getClassnamesFromArgs, getClassNamesFromFile, getClassnamesFromJarFile, getStringSetFromFile, getStringSetFromFile, validateClassName

Methods inherited from class randoop.main.CommandHandler

handles, printHTML, usageMessage

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

NO_OPERATIONS_TO_TEST

private static final java.lang.String NO_OPERATIONS_TO_TEST

See Also:

Constant Field Values

command

private static final java.lang.String command

See Also:

Constant Field Values

pitch

private static final java.lang.String pitch

See Also:

Constant Field Values

commandGrammar

private static final java.lang.String commandGrammar

See Also:

Constant Field Values

where

private static final java.lang.String where

See Also:

Constant Field Values

summary

private static final java.lang.String summary

See Also:

Constant Field Values

input

private static final java.lang.String input

See Also:

Constant Field Values

output

private static final java.lang.String output

See Also:

Constant Field Values

example

private static final java.lang.String example

See Also:

Constant Field Values

notes

private static final java.util.List<java.lang.String> notes

Explanations printed to the user.

TEST_METHOD_NAME_PREFIX

public static final @Regex(value=0) @Identifier java.lang.String TEST_METHOD_NAME_PREFIX

The prefix for test method names.

See Also:

Constant Field Values

afterAllFixtureBody

private com.github.javaparser.ast.stmt.BlockStmt afterAllFixtureBody

afterEachFixtureBody

private com.github.javaparser.ast.stmt.BlockStmt afterEachFixtureBody

beforeAllFixtureBody

private com.github.javaparser.ast.stmt.BlockStmt beforeAllFixtureBody

beforeEachFixtureBody

private com.github.javaparser.ast.stmt.BlockStmt beforeEachFixtureBody

options

private static org.plumelib.options.Options options

sequenceCompileFailureCount

private int sequenceCompileFailureCount

The count of sequences that failed to compile.

POSSIBLY_FLAKY_PREFIX

public static final java.lang.String POSSIBLY_FLAKY_PREFIX

Is output to the user before each possibly flaky method.

See Also:

Constant Field Values

nonFlakyMethods

private static final java.util.List<RawSignature> nonFlakyMethods

Methods that are known to be non-flaky.

fileSystemCache

private java.util.Map<java.net.URI,java.nio.file.FileSystem> fileSystemCache

A cache used by getResourceDirectoryPath(java.lang.String), to prevent FileSystemAlreadyExistsException.

Constructor Detail

GenTests

public GenTests()

GenTests constructor that uses default messages.

Method Detail

handle

public boolean handle(java.lang.String[] args)

Specified by:

handle in class CommandHandler

readSideEffectFreeMethods

public static MultiMap<Type,TypedClassOperation> readSideEffectFreeMethods()

Read side-effect-free methods from the default JDK side-effect-free method list, and from a user-provided method list if provided.

Returns:

a map from a Type to a set of side-effect-free methods that take that type as their only argument

isNonFlaky

private static boolean isNonFlaky(TypedClassOperation op)

Returns true if the operation is definitely non-flaky.

Parameters:

op - an operation

Returns:

true if the operation is definitely non-flaky

processAndOutputFlakyMethods

private void processAndOutputFlakyMethods(java.util.List<ExecutableSequence> flakySequences,
                                          java.util.List<ExecutableSequence> sequences,
                                          MultiMap<Type,TypedClassOperation> sideEffectFreeMethodsByType,
                                          OmitMethodsPredicate omitMethodsPredicate,
                                          AccessibilityPredicate accessibilityPredicate)

Outputs names of suspected flaky methods by using the tf-idf metric (Term Frequency - Inverse Document Frequency), which is:

(number of flaky tests M occurs in) / (number of total tests M occurs in)

Parameters:

flakySequences - the flaky test sequences

sequences - all the sequences (flaky and non-flaky)

sideEffectFreeMethodsByType - side-effect-free methods to use in assertions

omitMethodsPredicate - the user-supplied predicate for which methods should not be used during test generation

accessibilityPredicate - accessibility predicate for side-effect-free methods

testNamesToSequences

private java.util.List<ExecutableSequence> testNamesToSequences(java.lang.Iterable<java.lang.String> testNames,
                                                                java.util.List<ExecutableSequence> sequences)

Given a collection of test names of the form "test005", returns the corresponding elements from the given list.

Parameters:

testNames - names of the form "test005"

sequences - test sequences (error or regression), numbered sequentially

Returns:

the sequences corresponding to the test names

countSequencesPerOperation

private java.util.Map<TypedClassOperation,java.lang.Integer> countSequencesPerOperation(java.util.List<ExecutableSequence> sequences,
                                                                                        MultiMap<Type,TypedClassOperation> assertableSideEffectFreeMethods)

Counts the number of sequences each operation occurs in.

Parameters:

sequences - a list of sequences

assertableSideEffectFreeMethods - a map from a type to all its side-effect-free methods that can be used in assertions

Returns:

a map from operation to the number of sequences in which the operation occurs at least once

getOperationsInSequence

private java.util.Set<TypedClassOperation> getOperationsInSequence(ExecutableSequence es)

Constructs a set of method-call operations appearing in an Executable Sequence. Non-method-call operations are excluded.

Parameters:

es - an ExecutableSequence

Returns:

the set of method call operations in es

convertClasspathToAbsolute

private java.lang.String convertClasspathToAbsolute(java.lang.String classpath)

Convert each element of the given classpath from a relative to an absolute path.

Parameters:

classpath - the classpath to replace

Returns:

a version of classpath with relative paths replaced by absolute paths

writeTestFiles

private void writeTestFiles(JUnitCreator junitCreator,
                            java.util.List<ExecutableSequence> testSequences,
                            CodeWriter codeWriter,
                            java.lang.String classNamePrefix,
                            java.lang.String testKind)

Creates the test classes for the test sequences using the JUnitCreator and then writes the files using the CodeWriter. Writes the test suite if GenInputsAbstract.junit_reflection_allowed is true, or the test driver, otherwise.

Class names are numbered with classNamePrefix as the prefix. The package for tests is GenInputsAbstract.junit_package_name.

Parameters:

junitCreator - the JUnitCreator to create the test class source

testSequences - a list of ExecutableSequence objects for test methods

codeWriter - the CodeWriter to output the test classes

classNamePrefix - the prefix for the class name

testKind - a String indicating the kind of tests for logging and error messages

getFixtureCode

private boolean getFixtureCode()

Create fixture code from GenInputsAbstract.junit_after_all, GenInputsAbstract.junit_after_each, GenInputsAbstract.junit_before_all, and GenInputsAbstract.junit_before_each and set fixture body variables.

Returns:

true if all fixtures were read without error, false, otherwise

readPatterns

private java.util.List<java.util.regex.Pattern> readPatterns(java.nio.file.Path path)

Returns patterns read from the given user-provided file.

Parameters:

path - the file to read from, may be null (in which case this returns an empty list)

Returns:

contents of the file, as a list of Patterns

readPatternsFromResource

private java.util.List<java.util.regex.Pattern> readPatternsFromResource(java.lang.String filename)

Returns patterns read from the given resource.

Parameters:

filename - the resource from which to read

Returns:

contents of the resource, as a list of Patterns

readPatterns

private java.util.List<java.util.regex.Pattern> readPatterns(java.io.InputStream is,
                                                             java.lang.String filename)

Returns patterns read from the given stream.

Parameters:

is - the stream from which to read

filename - the file name to use in diagnostic messages

Returns:

contents of the file, as a list of Patterns

readPatterns

private java.util.List<java.util.regex.Pattern> readPatterns(org.plumelib.util.EntryReader er)

Returns patterns read from the given EntryReader.

Parameters:

er - the EntryReader to read from

Returns:

contents of the file, as a list of Patterns

createPatternsFromSignatures

private java.util.List<java.util.regex.Pattern> createPatternsFromSignatures(java.util.List<java.lang.String> signatures)

Creates a list of signature strings (see RawSignature.toString() to a list of Pattern.

Parameters:

signatures - the list of signature strings

Returns:

the list of patterns for the signature strings

signatureToPattern

private static java.util.regex.Pattern signatureToPattern(java.lang.String signatureString)

Converts a signature string (see RawSignature.toString() to a Pattern that matches that string (with spaces removed).

Parameters:

signatureString - the string representation of a signature

Returns:

the pattern to match signatureString

printSequenceExceptionError

private void printSequenceExceptionError(AbstractGenerator explorer,
                                         SequenceExceptionError e)

Prints information about a SequenceExceptionError that indicates a flaky test has been found. Prints information to help user identify source of flakiness, including exception, statement that threw the exception, the full sequence where exception was thrown, and the input subsequence.

Parameters:

explorer - the test generator

e - the sequence exception

createTestOutputPredicate

public java.util.function.Predicate<ExecutableSequence> createTestOutputPredicate(java.util.Set<Sequence> excludeSet,
                                                                                  java.util.Set<java.lang.Class<?>> coveredClasses,
                                                                                  java.util.regex.Pattern includePattern)

Builds the test predicate that determines whether a particular sequence will be included in the output based on command-line arguments. A true result means the test is a candidate for output.

Parameters:

excludeSet - the set of sequences to exclude

coveredClasses - the list of classes to test for coverage

includePattern - the pattern for method name inclusion

Returns:

the predicate

createTestCheckGenerator

public static TestCheckGenerator createTestCheckGenerator(AccessibilityPredicate accessibility,
                                                          ContractSet contracts,
                                                          MultiMap<Type,TypedClassOperation> sideEffectFreeMethodsByType,
                                                          OmitMethodsPredicate omitMethodsPredicate)

Creates the test check generator for this run based on the command-line arguments. The goal of the generator is to produce all appropriate checks for each sequence it is applied to.

The generator always contains validity and contract checks. If regression tests are to be generated, it also contains the regression checks generator.

Parameters:

accessibility - the accessibility predicate

contracts - the contract checks

sideEffectFreeMethodsByType - the map from types to side-effect-free methods

omitMethodsPredicate - the user-supplied predicate for which methods should not be used during test generation

Returns:

the TestCheckGenerator that reflects command line arguments

usage

private static void usage(java.lang.String format,
                          java.lang.Object... args)

Print message, then print usage information, then exit.

Parameters:

format - the string format

args - the arguments

getFileText

private static @PolyNull java.util.List<java.lang.String> getFileText(@PolyNull java.lang.String filename)

Return the text of the given file, as a list of lines. Returns null if the filename argument is null. Terminates execution if the filename file cannot be read.

Parameters:

filename - the file to read

Returns:

the contents of filename, as a list of strings

getJDKSpecificationFiles

private java.util.Collection<? extends java.nio.file.Path> getJDKSpecificationFiles()

Returns the list of JDK specification files from the specifications/jdk resources directory in the Randoop jar file.

Returns:

the list of JDK specification files

Throws:

RandoopBug - if there is an error locating the specification files

getResourceDirectoryPath

private java.nio.file.Path getResourceDirectoryPath(java.lang.String resourceDirectory)

Returns the path for the resource directory in the jar file.

Parameters:

resourceDirectory - the resource directory relative to the root of the jar file, should start with "/"

Returns:

the Path for the resource directory

Throws:

RandoopBug - if an error occurs when locating the directory

incrementSequenceCompileFailureCount

public void incrementSequenceCompileFailureCount()

Increments the count of sequence compilation failures.