Package randoop.main

Class GenInputsAbstract

java.lang.Object
randoop.main.CommandHandler
randoop.main.GenInputsAbstract
Direct Known Subclasses:
GenTests
public abstract class GenInputsAbstract extends CommandHandler
Container for Randoop options. They are stored as static variables, not instance variables.

  • Field Details

    • testjar

      public static List<Path> testjar
      Treat every class in the given jar file as a class to test. The jarfile must be on the classpath.

      See the notes about specifying methods that may appear in a test.

    • test_package

      public static List<String> test_package
      Package to test. All classes on the classpath within the package are classes to test. Does not include classes in sub-packages.

      The classes in the package are tested in addition to any specified using --testjar, --testclass, and --classlist

    • classlist

      public static Path classlist
      File that lists classes to test. All of their methods are methods under test.

      In the file, each class under test is specified by its binary name on a separate line. See an example. These classes are tested in addition to any specified using --testjar, --test-package, and --testclass.

      Using --classlist is less common than --testjar. See the notes about specifying methods that may appear in a test.

    • testclass

      public static List<@BinaryName String> testclass
      The fully-qualified raw name of a class to test; for example, --testclass=java.util.TreeSet. All of its methods are methods under test. This class is tested in addition to any specified using --testjar, --test-package, and --classlist.

      It is unusual to specify just one or a few classes to test. See the notes about specifying methods that may appear in a test.

    • methodlist

      public static Path methodlist
      A file containing a list of methods and constructors to call in tests, each given as a fully-qualified signature on a separate line.

      These methods augment any methods from classes given by the --testjar, --classlist, and --testclass options.

      See an example file.

      Using --methodlist is less common, and more error-prone, than --testjar, --test-package, --classlist, or --testclass. See the notes about specifying methods that may appear in a test.

    • omit_classes

      public static List<Pattern> omit_classes
      A regex that indicates classes that should not be used in tests, even if included by some other command-line option. The regex is matched against fully-qualified class names. If the regular expression contains anchors "^" or "$", they refer to the beginning and the end of the class name.

    • omit_classes_file

      public static List<Path> omit_classes_file
      A file containing a list of regular expressions that indicate classes not to call in a test. These patterns are used along with those provided with --omit-classes.

    • omit_methods

      public static List<Pattern> omit_methods
      A regex that indicates methods that should not be called directly in generated tests. This does not prevent indirect calls to such methods from other, allowed methods; to prevent them, see the replacecall agent.

      Randoop will not directly call a method whose fully-qualified signature matches the regular expression, or a method inherited from a superclass or interface whose signature matches the regular expression.

      If the regular expression contains anchors "^" or "$", they refer to the beginning and the end of the signature string.

      Methods replaced by the replacecall agent are also automatically omitted.

    • omit_methods_file

      public static List<Path> omit_methods_file
      A file containing a list of regular expressions that indicate methods that should not be included in generated tests. These patterns are used along with those provided with --omit-methods, and the default omissions.

    • omit_methods_no_defaults

      public static boolean omit_methods_no_defaults
      Include methods that are otherwise omitted by default. Unless you set this to true, every method replaced by the replacecall agent is treated as if it had been supplied as an argument to --omit-methods.

    • omit_classes_no_defaults

      public static boolean omit_classes_no_defaults
      Include classes that are otherwise omitted by default.

    • dont_omit_replaced_methods

      public static boolean dont_omit_replaced_methods
      Include methods that are otherwise omitted by default. Unless you set this to true, every method replaced by the replacecall agent is treated as if it had been supplied as an argument to --omit-methods.

    • omit_field

      public static List<String> omit_field
      A fully-qualified field name of a field to be excluded from test generation. An accessible field is used unless it is omitted by this or the --omit-field-file option.

    • omit_field_file

      public static Path omit_field_file
      File that contains fully-qualified field names to be excluded from test generation. An accessible field is used unless it is omitted by this or the --omit-field option.

    • omit_field_list

      public static Path omit_field_list
      File that contains fully-qualified field names to be excluded from test generation. An accessible field is used unless it is omitted by this or the --omit-field option.

    • only_test_public_members

      public static boolean only_test_public_members
      Restrict tests to only include public members of classes.

      When this is false, the setting of --junit-package-name and package accessibility is used to determine which members will be used in tests.

    • silently_ignore_bad_class_names

      public static boolean silently_ignore_bad_class_names

    • fail_on_generation_error

      public static boolean fail_on_generation_error
      (For debugging.) If an error or exception is thrown during type instantiation or input selection, this option allows the error to be passed through to GenTests.handle(String[]) where a comprehensive error message is printed.

    • flaky_test_behavior

      public static GenInputsAbstract.FlakyTestAction flaky_test_behavior
      What to do if Randoop generates a flaky test. A flaky test is one that behaves differently on different executions.

      Flaky tests are usually due to calling Randoop on side-effecting or nondeterministic methods, and ultimately, the solution is not to call Randoop on such methods; see section Nondeterminism in the Randoop manual.

    • nondeterministic_methods_to_output

      public static int nondeterministic_methods_to_output
      How many suspected side-effecting or nondeterministic methods (from the program under test) to print.

    • no_error_revealing_tests

      public static boolean no_error_revealing_tests
      Whether to output error-revealing tests. Disables all output when used with --no-regression-tests. Restricting output can result in long runs if the default values of --generated-limit and --time-limit are used.

    • no_regression_tests

      public static boolean no_regression_tests
      Whether to output regression tests. Disables all output when used with --no-error-revealing-tests. Restricting output can result in long runs if the default values of --generated-limit and --time-limit are used.

    • no_regression_assertions

      public static boolean no_regression_assertions
      Whether to include assertions in regression tests. If false, then the regression tests contain no assertions (except that if the test throws an exception, it should continue to throw an exception of the same type). Tests without assertions can be used to exercise the code, but they do not enforce any particular behavior, such as values returned.

    • check_compilable

      public static boolean check_compilable
      Whether to check that generated sequences can be compiled. If true, the code for each generated sequence is compiled, and the sequence is only kept if the compilation succeeds without error. This check is useful because the assumptions in Randoop generation heuristics are sometimes violated by input methods, and, as a result, a generated test may not compile. This check does increases the runtime by approximately 50%.

    • require_classname_in_test

      public static Pattern require_classname_in_test
      Classes that must occur in a test. Randoop will only output tests whose source code has at least one use of a member of a class whose name matches the regular expression.

    • require_covered_classes

      public static Path require_covered_classes
      File containing binary names of classes that the tests must use, directly or indirectly. This option only works if Randoop is run using the covered-class javaagent to instrument the classes. A test is output only if it uses at least one of the class names in the file. A test uses a class if it invokes any constructor or method of the class, directly or indirectly (the constructor or method might not appear in the source code of the test). Included classes may be abstract.

    • minimize_error_test

      public static boolean minimize_error_test
      If true, Randoop outputs both original error-revealing tests and a minimized version. Setting this option may cause long Randoop run times if Randoop outputs and minimizes more than about 100 error-revealing tests; consider using --stop-on-error-test=true. Also see the test case minimization options.

    • checked_exception

      public static GenInputsAbstract.BehaviorType checked_exception
      If a test throws a checked exception, should it be included in the error-revealing test suite (value: ERROR), regression test suite (value: EXPECTED), or should it be discarded (value: INVALID)?

    • unchecked_exception

      public static GenInputsAbstract.BehaviorType unchecked_exception
      If a test throws an unchecked exception other than ConcurrentModificationException, NoClassDefFoundError, NullPointerException, OutOfMemoryError, and StackOverflowError, should the test be included in the error-revealing test suite (value: ERROR), regression test suite (value: EXPECTED), or should it be discarded (value: INVALID)?

      The arguments --cm-exception, --ncdf-exception, --npe-on-null-input, --npe-on-non-null-input, --oom-exception, and --sof-exception handle special cases of unchecked exceptions.

    • cm_exception

      public static GenInputsAbstract.BehaviorType cm_exception
      If a test throws a ConcurrentModificationException exception, should it be included in the error-revealing test suite (value: ERROR), regression test suite (value: EXPECTED), or should it be discarded (value: INVALID)?

    • ncdf_exception

      public static GenInputsAbstract.BehaviorType ncdf_exception
      If a test throws a NoClassDefFoundError exception, should it be included in the error-revealing test suite (value: ERROR), regression test suite (value: EXPECTED), or should it be discarded (value: INVALID)?

    • npe_on_null_input

      public static GenInputsAbstract.BehaviorType npe_on_null_input
      If a test that passes null as an argument throws a NullPointerException, should the test be be included in the error-revealing test suite (value: ERROR), regression test suite (value: EXPECTED), or should it be discarded (value: INVALID)?

    • npe_on_non_null_input

      public static GenInputsAbstract.BehaviorType npe_on_non_null_input
      If a test that never passes null as an argument throws a NullPointerException, should the test be included in the error-revealing test suite (value: ERROR), regression test suite (value: EXPECTED), or should it be discarded (value: INVALID)?

    • oom_exception

      public static GenInputsAbstract.BehaviorType oom_exception
      If a test throws an OutOfMemoryError exception, should it be included in the error-revealing test suite (value: ERROR), regression test suite (value: EXPECTED), or should it be discarded (value: INVALID)?

    • sof_exception

      public static GenInputsAbstract.BehaviorType sof_exception
      If a test throws a StackOverflowError exception, should it be included in the error-revealing test suite (value: ERROR), regression test suite (value: EXPECTED), or should it be discarded (value: INVALID)?

    • specifications

      public static List<Path> specifications
      Read file of specifications; see manual section "Specifying expected code behavior".

    • use_jdk_specifications

      public static boolean use_jdk_specifications
      Use built-in specifications for JDK classes and for classes that inherit from them, as if they had been supplied using the --specifications command-line argument.

    • ignore_condition_compilation_error

      public static boolean ignore_condition_compilation_error
      Make Randoop proceed, instead of failing, if the Java condition text of a specification cannot be compiled.

    • ignore_condition_exception

      public static boolean ignore_condition_exception
      Make Randoop treat a specification whose execution throws an exception as returning false. If true, Randoop treats x.f == 22 equivalently to the wordier x != null && x.f == 22. If false, Randoop halts when a specification throws an exception.

    • ignore_condition_exception_quiet

      public static boolean ignore_condition_exception_quiet
      If true, don't print diagnostics about specification that throw an exception. Has no effect unless --ignore-condition-exception is set.

    • side_effect_free_methods

      public static Path side_effect_free_methods
      File containing side-effect-free methods, each given as a fully-qualified signature on a separate line. Specifying side-effect-free methods has two benefits: it makes regression tests stronger, and it helps Randoop create smaller tests.

    • time_limit

      public static int time_limit
      Maximum number of seconds to spend generating tests. Zero means no limit. If nonzero, Randoop is nondeterministic: it may generate different test suites on different runs.

      This is the overall limit, not the limit per class under test. The default value is too small to be effective for generating tests for an entire project. If you are testing multiple classes, you may wish to multiply the default value by the number of classes under test.

      Randoop may run for longer than this because of a long-running test. The elapsed time is checked after each test, not during a test's execution.

    • LIMIT_DEFAULT

      private static int LIMIT_DEFAULT

    • attempted_limit

      public static int attempted_limit
      Maximum number of attempts to generate a test method candidate.

    • generated_limit

      public static int generated_limit
      Maximum number of test method candidates generated internally.

    • output_limit

      public static int output_limit
      The maximum number of regression and error-revealing tests to output. If there is no output, this limit has no effect. There is no output when using either --dont-output-tests or --no-error-revealing-tests together with --no-regression-tests.

      In the current implementation, the number of tests in the output can be substantially smaller than this limit. One reason is that Randoop does not output subsumed tests, which appear as a subsequence of some longer test.

    • maxsize

      public static int maxsize
      Do not generate tests with more than this many statements.

    • stop_on_error_test

      public static boolean stop_on_error_test
      Stop generation as soon as one error-revealing test has been generated. Implies --minimize-error-test.

    • null_ratio

      public static double null_ratio
      Use null with the given frequency as an argument to method calls.

      For example, a null ratio of 0.05 directs Randoop to use null as an input 5 percent of the time when a non-null value of the appropriate type is available.

      Unless --forbid_null is true, a null value will still be used if no other value can be passed as an argument even if --null-ratio=0.

      Randoop never uses null for receiver values.

    • forbid_null

      public static boolean forbid_null
      Do not use null as input to methods or constructors, even when no other argument value can be generated.

      If true, Randoop will not generate a test when unable to find a non-null value of appropriate type as an input. This could result in certain class members being untested.

    • literals_file

      public static List<String> literals_file
      A file containing literal values to be used as inputs to methods under test, or "CLASSES".

      Literals in these files are used in addition to all other constants in the pool. For the format of this file, see documentation in class LiteralFileReader. The special value "CLASSES" (with no quotes) means to read literals from all classes under test.

    • literals_level

      public static GenInputsAbstract.ClassLiteralsMode literals_level
      How to use literal values that are specified via the --literals-file command-line option.
      See Also:

    • method_selection

      public static GenInputsAbstract.MethodSelectionMode method_selection
      Randoop generates new tests by choosing from a set of methods under test. This controls how the next method is chosen, from among all methods under test.

    • bloodhound_logging

      public static boolean bloodhound_logging
      Print to standard out, method weights and method uncovered ratios.

    • bloodhound_update_mode

      public static GenInputsAbstract.BloodhoundCoverageUpdateMode bloodhound_update_mode
      Bloodhound can update coverage information at a regular interval that is either based on time or on the number of successful invocations.

    • string_maxlen

      public static int string_maxlen
      Maximum length of strings in generated tests, including in assertions. Strings longer than 65KB (or about 10,000 characters) may be rejected by the Java compiler, according to the Java Virtual Machine specification.

    • alias_ratio

      public static double alias_ratio
      Try to reuse values from a sequence with the given frequency. If an alias ratio is given, it should be between 0 and 1.

      A ratio of 0 results in tests where each value created within a test input is typically used at most once as an argument in a method call. A ratio of 1 tries to maximize the number of times values are used as inputs to parameters within a test.

    • input_selection

      public static GenInputsAbstract.InputSelectionMode input_selection
      Randoop generates new tests by combining old previously-generated tests. This controls how the old tests are chosen, from among all existing tests.

    • clear

      public static int clear
      Clear the component set each time it contains the given number of inputs.

      Randoop stores previously-generated tests in a "component" set, and uses them to generate new tests. Setting this variable to a small number can sometimes result in a greater variety of tests generated during a single run.

    • clear_memory

      public static long clear_memory
      Clear the component set each time Randoop uses this much memory.

      Setting this variable to a smaller number may prevent an out-of-memory exception or a run that is slow due to thrashing and garbage collection.

    • testsperfile

      public static int testsperfile
      Maximum number of tests to write to each JUnit file.

    • error_test_basename

      public static String error_test_basename
      Base name (no ".java" suffix) of the JUnit file containing error-revealing tests

    • regression_test_basename

      public static String regression_test_basename
      Base name (no ".java" suffix) of the JUnit file containing regression tests

    • junit_package_name

      public static String junit_package_name
      Name of the package for the generated JUnit files. Enables testing non-public members. Tests can be restricted to public members only by also using the option --only-test-public-members.

    • junit_before_each

      public static String junit_before_each
      Name of file containing code text to be added to the @Before-annotated method of each generated test class. Code is uninterpreted, and, so, is not run during generation. Intended for use when run-time behavior of classes under test requires setup behavior that is not needed for execution by reflection. (The annotation @Before is JUnit 4, and @BeforeEach is JUnit 5.)

    • junit_after_each

      public static String junit_after_each
      Name of file containing code text to be added to the @After -annotated method of each generated test class. Intended for use when run-time behavior of classes under test requires tear-down behavior that is not needed for execution by reflection. Code is uninterpreted, and, so, is not run during generation. (The annotation @After is JUnit 4, and @AfterEach is JUnit 5.)

    • junit_before_all

      public static String junit_before_all
      Name of file containing code text to be added to the @BeforeClass-annotated method of each generated test class. Intended for use when run-time behavior of classes under test requires setup behavior that is not needed for execution by reflection. Code is uninterpreted, and, so, is not run during generation. (The annotation @BeforeClass is JUnit 4, and @BeforeAll is JUnit 5.)

    • junit_after_all

      public static String junit_after_all
      Name of file containing code text to be added to the @AfterClass-annotated method of each generated test class. Intended for use when run-time behavior of classes under test requires tear-down behavior that is not needed for execution by reflection. Code is uninterpreted, and, so, is not run during generation. (The annotation @AfterClass is JUnit 4, and @AfterAll is JUnit 5.)

    • junit_output_dir

      public static String junit_output_dir
      Name of the directory in which JUnit files should be written.

    • dont_output_tests

      public static boolean dont_output_tests
      Run test generation without output. May be desirable when running with a visitor.

      NOTE: Because there is no output, the value of --output-limit will never be met, so be sure to set --generated-limit or --time-limit to a reasonable value when using this option.

    • junit_reflection_allowed

      public static boolean junit_reflection_allowed
      Whether to use JUnit's standard reflective mechanisms for invoking tests. JUnit's reflective invocations can interfere with code instrumentation, such as by the DynComp tool. If that is a problem, then set this to false and Randoop will output tests that use direct method calls instead of reflection. The tests will include a main method and will execute methods and assertions, but won't be JUnit suites.

    • system_props

      public static List<String> system_props
      System properties that Randoop will set similarly to java -D, of the form "x=y".

    • jvm_max_memory

      public static String jvm_max_memory
      How much memory Randoop should use when starting new JVMs. This only affects new JVMs; you still need to supply -Xmx... when starting Randoop itself.

    • capture_output

      public static boolean capture_output

    • randomseed

      public static int randomseed
      The random seed to use in the generation process. You do not need to provide this option to make Randoop deterministic, because Randoop is deterministic by default. It is recommended to run Randoop multiple times with a different random seed, in order to produce multiple different test suites.

    • deterministic

      public static boolean deterministic
      If true, Randoop is deterministic: running Randoop twice with the same arguments (including --randomseed) will produce the same test suite, so long as the program under test is deterministic. If false, Randoop may or may not produce the same test suite. To produce multiple different test suites, use the --randomseed command-line option.

    • progressdisplay

      public static boolean progressdisplay

    • PROGRESSINTERVALMILLIS_DEFAULT

      public static long PROGRESSINTERVALMILLIS_DEFAULT

    • progressintervalmillis

      public static long progressintervalmillis

    • progressintervalsteps

      public static long progressintervalsteps

    • debug_checks

      public static boolean debug_checks

    • log

      public static org.plumelib.util.FileWriterWithName log
      A file to which to log lots of information. If not specified, no logging is done. Enabling the logs slows down Randoop.

    • selection_log

      public static org.plumelib.util.FileWriterWithName selection_log
      A file to which to log selections; helps find sources of non-determinism. If not specified, no logging is done.

    • operation_history_log

      public static org.plumelib.util.FileWriterWithName operation_history_log
      A file to which to write operation usage, when Randoop exits.

    • dontexecute

      public static boolean dontexecute
      Create sequences but never execute them. Used to test performance of Randoop's sequence generation code.

    • visitor

      public static List<@ClassGetName String> visitor
      Install the given runtime visitor. See class randoop.ExecutionVisitor.

    • repeat_heuristic

      public static boolean repeat_heuristic

  • Constructor Details

  • Method Details

    • checkOptionsValid

      public void checkOptionsValid()
      Check that the options given satisfy any specified constraints, and fail if they do not.

    • shouldOmitClass

      private static boolean shouldOmitClass(String classname)
      Returns true if the class should be omitted, according to the omit_classes field.
      Parameters:
      classname - a class name
      Returns:
      true if the class should be omitted

    • getClassnamesFromArgs

      public static Set<@ClassGetName String> getClassnamesFromArgs(AccessibilityPredicate accessibility)
      Read names of classes under test, as provided with the --classlist or --testjar command-line argument.
      Parameters:
      accessibility - the accessibility predicate
      Returns:
      the classes provided via the --classlist or --testjar command-line argument

    • getClassnamesFromJarFile

      public static Set<@ClassGetName String> getClassnamesFromJarFile(Path jarFile, AccessibilityPredicate accessibility)
      Read names of classes from a jar file. Ignores interfaces, abstract classes, non-accessible classes, and those that cannot be loaded.
      Parameters:
      jarFile - the jar file from which to read classes
      accessibility - the accessibility predicate
      Returns:
      the names of classes in the jar file

    • getClassnamesFromPackage

      private static List<@ClassGetName String> getClassnamesFromPackage(String packageName, AccessibilityPredicate accessibility)
      Returns classes from the classpath that are in the given package. Does not include classes in sub-packages. Ignores non-accessible classes.
      Parameters:
      packageName - a package name; may be the empty string
      accessibility - the accessibility predicate
      Returns:
      classes in package packageName

    • getClassesWithPackageFromDirectory

      private static List<@ClassGetName String> getClassesWithPackageFromDirectory(File directory, String packageName, AccessibilityPredicate accessibility)
      Given a directory on the CLASSPATH, returns classes in the given package. These classes are found in a subdirectory of the given directory, whose name depends on the given package.
      Parameters:
      directory - a directory on the CLASSPATH
      packageName - a package name
      accessibility - the accessibility predicate
      Returns:
      classes with the given package

    • getClassesWithPackageFromJar

      private static List<@ClassGetName String> getClassesWithPackageFromJar(File jarFile, String packageName, AccessibilityPredicate accessibility)
      Returns classes with the given package in the given jar file.
      Parameters:
      jarFile - a jar file containing classes
      packageName - a package name
      accessibility - the accessibility predicate
      Returns:
      classes in package packageName in the given jar file

    • getClassNamesFromFile

      public static Set<@ClassGetName String> getClassNamesFromFile(Path file)
      Returns the class names listed in the file.
      Parameters:
      file - the file containing the strings
      Returns:
      the lines in the file, or null if listFile is null

    • getStringSetFromFile

      public static Set<String> getStringSetFromFile(Path listFile, String fileDescription)
      Returns a set consisting of the lines of the file, except those starting with "#". Returns empty set if listFile is null.
      Parameters:
      listFile - the file containing the strings
      fileDescription - string used in error messages
      Returns:
      the lines in the file, or null if listFile is null

    • getStringSetFromFile

      public static Set<String> getStringSetFromFile(@Nullable Path listFile, String fileDescription, String commentRegex, String includeRegex)
      Returns a set consisting of the lines of the file. Returns empty set if listFile is null.
      Parameters:
      listFile - the file containing the strings
      fileDescription - string used in error messages
      commentRegex - indicates which lines are comments that should be ignored
      includeRegex - if this string appears in the file, then another file is recursively read
      Returns:
      the strings in the file, or null if listFile is null