Package randoop.main
Class GenTests
- java.lang.Object
-
- randoop.main.CommandHandler
-
- randoop.main.GenInputsAbstract
-
- randoop.main.GenTests
-
public class GenTests extends GenInputsAbstract
Test generation.
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class randoop.main.GenInputsAbstract
GenInputsAbstract.BehaviorType, GenInputsAbstract.BloodhoundCoverageUpdateMode, GenInputsAbstract.ClassLiteralsMode, GenInputsAbstract.FlakyTestAction, GenInputsAbstract.InputSelectionMode, GenInputsAbstract.Limits, GenInputsAbstract.MethodSelectionMode
-
-
Field Summary
Fields Modifier and Type Field Description private com.github.javaparser.ast.stmt.BlockStmt
afterAllFixtureBody
private com.github.javaparser.ast.stmt.BlockStmt
afterEachFixtureBody
private com.github.javaparser.ast.stmt.BlockStmt
beforeAllFixtureBody
private com.github.javaparser.ast.stmt.BlockStmt
beforeEachFixtureBody
private static String
command
private static String
commandGrammar
private static String
example
private Map<URI,FileSystem>
fileSystemCache
A cache used bygetResourceDirectoryPath(java.lang.String)
, to preventFileSystemAlreadyExistsException
.private static String
input
private static String
NO_OPERATIONS_TO_TEST
private static List<RawSignature>
nonFlakyMethods
Methods that are known to be non-flaky.private static List<String>
notes
Explanations printed to the user.private static org.plumelib.options.Options
options
private static String
output
private static String
pitch
static String
POSSIBLY_FLAKY_PREFIX
Is output to the user before each possibly flaky method.private int
sequenceCompileFailureCount
The count of sequences that failed to compile.private static String
summary
static @Identifier String
TEST_METHOD_NAME_PREFIX
The prefix for test method names.private static String
where
-
Fields inherited from class randoop.main.GenInputsAbstract
alias_ratio, attempted_limit, bloodhound_logging, bloodhound_update_mode, capture_output, check_compilable, checked_exception, classlist, clear, clear_memory, cm_exception, debug_checks, deterministic, dont_omit_replaced_methods, dont_output_tests, dontexecute, error_test_basename, fail_on_generation_error, flaky_test_behavior, forbid_null, generated_limit, ignore_condition_compilation_error, ignore_condition_exception, ignore_condition_exception_quiet, input_selection, junit_after_all, junit_after_each, junit_before_all, junit_before_each, junit_output_dir, junit_package_name, junit_reflection_allowed, jvm_max_memory, literals_file, literals_level, log, maxsize, method_selection, methodlist, minimize_error_test, ncdf_exception, no_error_revealing_tests, no_regression_assertions, no_regression_tests, nondeterministic_methods_to_output, npe_on_non_null_input, npe_on_null_input, null_ratio, omit_classes, omit_classes_file, omit_classes_no_defaults, omit_field, omit_field_file, omit_field_list, omit_methods, omit_methods_file, omit_methods_no_defaults, only_test_public_members, oom_exception, operation_history_log, output_limit, print_non_compiling_file, progressdisplay, progressintervalmillis, PROGRESSINTERVALMILLIS_DEFAULT, progressintervalsteps, randomseed, regression_test_basename, repeat_heuristic, require_classname_in_test, require_covered_classes, selection_log, side_effect_free_methods, silently_ignore_bad_class_names, sof_exception, specifications, stop_on_error_test, string_maxlen, system_props, test_package, testclass, testjar, testsperfile, time_limit, unchecked_exception, use_jdk_specifications, visitor
-
-
Constructor Summary
Constructors Constructor Description GenTests()
GenTests constructor that uses default messages.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description private String
convertClasspathToAbsolute(String classpath)
Convert each element of the given classpath from a relative to an absolute path.private Map<TypedClassOperation,Integer>
countSequencesPerOperation(List<ExecutableSequence> sequences, MultiMap<Type,TypedClassOperation> assertableSideEffectFreeMethods)
Counts the number of sequences each operation occurs in.private List<Pattern>
createPatternsFromSignatures(List<String> signatures)
Creates a list of signature strings (seeRawSignature.toString()
to a list ofPattern
.static TestCheckGenerator
createTestCheckGenerator(AccessibilityPredicate accessibility, ContractSet contracts, MultiMap<Type,TypedClassOperation> sideEffectFreeMethodsByType, OmitMethodsPredicate omitMethodsPredicate)
Creates the test check generator for this run based on the command-line arguments.Predicate<ExecutableSequence>
createTestOutputPredicate(Set<Sequence> excludeSet, Set<Class<?>> coveredClasses, Pattern includePattern)
Builds the test predicate that determines whether a particular sequence will be included in the output based on command-line arguments.private static @PolyNull List<String>
getFileText(@PolyNull String filename)
Return the text of the given file, as a list of lines.private boolean
getFixtureCode()
Create fixture code fromGenInputsAbstract.junit_after_all
,GenInputsAbstract.junit_after_each
,GenInputsAbstract.junit_before_all
, andGenInputsAbstract.junit_before_each
and set fixture body variables.private Collection<? extends Path>
getJDKSpecificationFiles()
Returns the list of JDK specification files from thespecifications/jdk
resources directory in the Randoop jar file.private Set<TypedClassOperation>
getOperationsInSequence(ExecutableSequence es)
Constructs a set of method-call operations appearing in an Executable Sequence.private Path
getResourceDirectoryPath(String resourceDirectory)
Returns the path for the resource directory in the jar file.boolean
handle(String[] args)
void
incrementSequenceCompileFailureCount()
Increments the count of sequence compilation failures.private static boolean
isNonFlaky(TypedClassOperation op)
Returns true if the operation is definitely non-flaky.private void
printSequenceExceptionError(AbstractGenerator explorer, SequenceExceptionError e)
Prints information about aSequenceExceptionError
that indicates a flaky test has been found.private void
processAndOutputFlakyMethods(List<ExecutableSequence> flakySequences, List<ExecutableSequence> sequences, MultiMap<Type,TypedClassOperation> sideEffectFreeMethodsByType, OmitMethodsPredicate omitMethodsPredicate, AccessibilityPredicate accessibilityPredicate)
Outputs names of suspected flaky methods by using the tf-idf metric (Term Frequency - Inverse Document Frequency), which is:private List<Pattern>
readPatterns(InputStream is, String filename)
Returns patterns read from the given stream.private List<Pattern>
readPatterns(Path path)
Returns patterns read from the given user-provided file.private List<Pattern>
readPatterns(org.plumelib.util.EntryReader er)
Returns patterns read from the given EntryReader.private List<Pattern>
readPatternsFromResource(String filename)
Returns patterns read from the given resource.static MultiMap<Type,TypedClassOperation>
readSideEffectFreeMethods()
Read side-effect-free methods from the default JDK side-effect-free method list, and from a user-provided method list if provided.private static Pattern
signatureToPattern(String signatureString)
Converts a signature string (seeRawSignature.toString()
to aPattern
that matches that string.private List<ExecutableSequence>
testNamesToSequences(Iterable<String> testNames, List<ExecutableSequence> sequences)
Given a collection of test names of the form "test005", returns the corresponding elements from the given list.private static void
usage(String format, Object... args)
Print message, then print usage information, then exit.private void
writeTestFiles(JUnitCreator junitCreator, List<ExecutableSequence> testSequences, CodeWriter codeWriter, String classNamePrefix, String testKind)
Creates the test classes for the test sequences using theJUnitCreator
and then writes the files using theCodeWriter
.-
Methods inherited from class randoop.main.GenInputsAbstract
checkOptionsValid, getClassnamesFromArgs, getClassNamesFromFile, getClassnamesFromJarFile, getStringSetFromFile, getStringSetFromFile, validateClassName
-
Methods inherited from class randoop.main.CommandHandler
handles, printHTML, usageMessage
-
-
-
-
Field Detail
-
NO_OPERATIONS_TO_TEST
private static final String NO_OPERATIONS_TO_TEST
- See Also:
- Constant Field Values
-
command
private static final String command
- See Also:
- Constant Field Values
-
pitch
private static final String pitch
- See Also:
- Constant Field Values
-
commandGrammar
private static final String commandGrammar
- See Also:
- Constant Field Values
-
where
private static final String where
- See Also:
- Constant Field Values
-
summary
private static final String summary
- See Also:
- Constant Field Values
-
input
private static final String input
- See Also:
- Constant Field Values
-
output
private static final String output
- See Also:
- Constant Field Values
-
example
private static final String example
- See Also:
- Constant Field Values
-
TEST_METHOD_NAME_PREFIX
public static final @Identifier String TEST_METHOD_NAME_PREFIX
The prefix for test method names.- See Also:
- Constant Field Values
-
afterAllFixtureBody
private com.github.javaparser.ast.stmt.BlockStmt afterAllFixtureBody
-
afterEachFixtureBody
private com.github.javaparser.ast.stmt.BlockStmt afterEachFixtureBody
-
beforeAllFixtureBody
private com.github.javaparser.ast.stmt.BlockStmt beforeAllFixtureBody
-
beforeEachFixtureBody
private com.github.javaparser.ast.stmt.BlockStmt beforeEachFixtureBody
-
options
private static org.plumelib.options.Options options
-
sequenceCompileFailureCount
private int sequenceCompileFailureCount
The count of sequences that failed to compile.
-
POSSIBLY_FLAKY_PREFIX
public static final String POSSIBLY_FLAKY_PREFIX
Is output to the user before each possibly flaky method.- See Also:
- Constant Field Values
-
nonFlakyMethods
private static final List<RawSignature> nonFlakyMethods
Methods that are known to be non-flaky.
-
fileSystemCache
private Map<URI,FileSystem> fileSystemCache
A cache used bygetResourceDirectoryPath(java.lang.String)
, to preventFileSystemAlreadyExistsException
.
-
-
Method Detail
-
handle
public boolean handle(String[] args)
- Specified by:
handle
in classCommandHandler
-
readSideEffectFreeMethods
public static MultiMap<Type,TypedClassOperation> readSideEffectFreeMethods()
Read side-effect-free methods from the default JDK side-effect-free method list, and from a user-provided method list if provided.- Returns:
- a map from a Type to a set of side-effect-free methods that take that type as their only argument
-
isNonFlaky
private static boolean isNonFlaky(TypedClassOperation op)
Returns true if the operation is definitely non-flaky.- Parameters:
op
- an operation- Returns:
- true if the operation is definitely non-flaky
-
processAndOutputFlakyMethods
private void processAndOutputFlakyMethods(List<ExecutableSequence> flakySequences, List<ExecutableSequence> sequences, MultiMap<Type,TypedClassOperation> sideEffectFreeMethodsByType, OmitMethodsPredicate omitMethodsPredicate, AccessibilityPredicate accessibilityPredicate)
Outputs names of suspected flaky methods by using the tf-idf metric (Term Frequency - Inverse Document Frequency), which is:(number of flaky tests M occurs in) / (number of total tests M occurs in)
- Parameters:
flakySequences
- the flaky test sequencessequences
- all the sequences (flaky and non-flaky)sideEffectFreeMethodsByType
- side-effect-free methods to use in assertionsomitMethodsPredicate
- the user-supplied predicate for which methods should not be used during test generationaccessibilityPredicate
- accessibility predicate for side-effect-free methods
-
testNamesToSequences
private List<ExecutableSequence> testNamesToSequences(Iterable<String> testNames, List<ExecutableSequence> sequences)
Given a collection of test names of the form "test005", returns the corresponding elements from the given list.- Parameters:
testNames
- names of the form "test005"sequences
- test sequences (error or regression), numbered sequentially- Returns:
- the sequences corresponding to the test names
-
countSequencesPerOperation
private Map<TypedClassOperation,Integer> countSequencesPerOperation(List<ExecutableSequence> sequences, MultiMap<Type,TypedClassOperation> assertableSideEffectFreeMethods)
Counts the number of sequences each operation occurs in.- Parameters:
sequences
- a list of sequencesassertableSideEffectFreeMethods
- a map from a type to all its side-effect-free methods that can be used in assertions- Returns:
- a map from operation to the number of sequences in which the operation occurs at least once
-
getOperationsInSequence
private Set<TypedClassOperation> getOperationsInSequence(ExecutableSequence es)
Constructs a set of method-call operations appearing in an Executable Sequence. Non-method-call operations are excluded.- Parameters:
es
- an ExecutableSequence- Returns:
- the set of method call operations in
es
-
convertClasspathToAbsolute
private String convertClasspathToAbsolute(String classpath)
Convert each element of the given classpath from a relative to an absolute path.- Parameters:
classpath
- the classpath to replace- Returns:
- a version of classpath with relative paths replaced by absolute paths
-
writeTestFiles
private void writeTestFiles(JUnitCreator junitCreator, List<ExecutableSequence> testSequences, CodeWriter codeWriter, String classNamePrefix, String testKind)
Creates the test classes for the test sequences using theJUnitCreator
and then writes the files using theCodeWriter
. Writes the test suite ifGenInputsAbstract.junit_reflection_allowed
is true, or the test driver, otherwise.Class names are numbered with
classNamePrefix
as the prefix. The package for tests isGenInputsAbstract.junit_package_name
.- Parameters:
junitCreator
- theJUnitCreator
to create the test class sourcetestSequences
- a list ofExecutableSequence
objects for test methodscodeWriter
- theCodeWriter
to output the test classesclassNamePrefix
- the prefix for the class nametestKind
- aString
indicating the kind of tests for logging and error messages
-
getFixtureCode
private boolean getFixtureCode()
Create fixture code fromGenInputsAbstract.junit_after_all
,GenInputsAbstract.junit_after_each
,GenInputsAbstract.junit_before_all
, andGenInputsAbstract.junit_before_each
and set fixture body variables.- Returns:
- true if all fixtures were read without error, false, otherwise
-
readPatterns
private List<Pattern> readPatterns(Path path)
Returns patterns read from the given user-provided file.- Parameters:
path
- the file to read from, may be null (in which case this returns an empty list)- Returns:
- contents of the file, as a list of Patterns
-
readPatternsFromResource
private List<Pattern> readPatternsFromResource(String filename)
Returns patterns read from the given resource.- Parameters:
filename
- the resource from which to read- Returns:
- contents of the resource, as a list of Patterns
-
readPatterns
private List<Pattern> readPatterns(InputStream is, String filename)
Returns patterns read from the given stream.- Parameters:
is
- the stream from which to readfilename
- the file name to use in diagnostic messages- Returns:
- contents of the file, as a list of Patterns
-
readPatterns
private List<Pattern> readPatterns(org.plumelib.util.EntryReader er)
Returns patterns read from the given EntryReader.- Parameters:
er
- the EntryReader to read from- Returns:
- contents of the file, as a list of Patterns
-
createPatternsFromSignatures
private List<Pattern> createPatternsFromSignatures(List<String> signatures)
Creates a list of signature strings (seeRawSignature.toString()
to a list ofPattern
.- Parameters:
signatures
- the list of signature strings- Returns:
- the list of patterns for the signature strings
-
signatureToPattern
private static Pattern signatureToPattern(String signatureString)
Converts a signature string (seeRawSignature.toString()
to aPattern
that matches that string.- Parameters:
signatureString
- the string representation of a signature- Returns:
- the pattern to match
signatureString
-
printSequenceExceptionError
private void printSequenceExceptionError(AbstractGenerator explorer, SequenceExceptionError e)
Prints information about aSequenceExceptionError
that indicates a flaky test has been found. Prints information to help user identify source of flakiness, including exception, statement that threw the exception, the full sequence where exception was thrown, and the input subsequence.- Parameters:
explorer
- the test generatore
- the sequence exception
-
createTestOutputPredicate
public Predicate<ExecutableSequence> createTestOutputPredicate(Set<Sequence> excludeSet, Set<Class<?>> coveredClasses, Pattern includePattern)
Builds the test predicate that determines whether a particular sequence will be included in the output based on command-line arguments. A true result means the test is a candidate for output.- Parameters:
excludeSet
- the set of sequences to excludecoveredClasses
- the list of classes to test for coverageincludePattern
- the pattern for method name inclusion- Returns:
- the predicate
-
createTestCheckGenerator
public static TestCheckGenerator createTestCheckGenerator(AccessibilityPredicate accessibility, ContractSet contracts, MultiMap<Type,TypedClassOperation> sideEffectFreeMethodsByType, OmitMethodsPredicate omitMethodsPredicate)
Creates the test check generator for this run based on the command-line arguments. The goal of the generator is to produce all appropriate checks for each sequence it is applied to.The generator always contains validity and contract checks. If regression tests are to be generated, it also contains the regression checks generator.
- Parameters:
accessibility
- the accessibility predicatecontracts
- the contract checkssideEffectFreeMethodsByType
- the map from types to side-effect-free methodsomitMethodsPredicate
- the user-supplied predicate for which methods should not be used during test generation- Returns:
- the
TestCheckGenerator
that reflects command line arguments
-
usage
private static void usage(String format, Object... args)
Print message, then print usage information, then exit.- Parameters:
format
- the string formatargs
- the arguments
-
getFileText
private static @PolyNull List<String> getFileText(@PolyNull String filename)
Return the text of the given file, as a list of lines. Returns null if thefilename
argument is null. Terminates execution if thefilename
file cannot be read.- Parameters:
filename
- the file to read- Returns:
- the contents of
filename
, as a list of strings
-
getJDKSpecificationFiles
private Collection<? extends Path> getJDKSpecificationFiles()
Returns the list of JDK specification files from thespecifications/jdk
resources directory in the Randoop jar file.- Returns:
- the list of JDK specification files
- Throws:
RandoopBug
- if there is an error locating the specification files
-
getResourceDirectoryPath
private Path getResourceDirectoryPath(String resourceDirectory)
Returns the path for the resource directory in the jar file.- Parameters:
resourceDirectory
- the resource directory relative to the root of the jar file, should start with "/"- Returns:
- the
Path
for the resource directory - Throws:
RandoopBug
- if an error occurs when locating the directory
-
incrementSequenceCompileFailureCount
public void incrementSequenceCompileFailureCount()
Increments the count of sequence compilation failures.
-
-