Package randoop.main

Class GenInputsAbstract

  • Direct Known Subclasses:
    GenTests

    public abstract class GenInputsAbstract
    extends CommandHandler
    Container for Randoop options. They are stored as static variables, not instance variables.
    • Field Detail

      • test_package

        public static List<String> test_package
        Package to test. All classes on the classpath within the package are classes to test. Does not include classes in sub-packages.

        The classes in the package are tested in addition to any specified using --testjar, --testclass, and --classlist

      • classlist

        public static Path classlist
        File that lists classes to test. All of their methods are methods under test.

        In the file, each class under test is specified by its binary name on a separate line. See an example. These classes are tested in addition to any specified using --testjar, --test-package, and --testclass.

        Using --classlist is less common than --testjar. See the notes about specifying methods that may appear in a test.

      • testclass

        public static List<@BinaryName String> testclass
        The fully-qualified raw name of a class to test; for example, --testclass=java.util.TreeSet. All of its methods are methods under test. This class is tested in addition to any specified using --testjar, --test-package, and --classlist.

        It is unusual to specify just one or a few classes to test. See the notes about specifying methods that may appear in a test.

      • methodlist

        public static Path methodlist
        A file containing a list of methods and constructors to call in tests, each given as a fully-qualified signature on a separate line.

        These methods augment any methods from classes given by the --testjar, --classlist, and --testclass options.

        See an example file.

        Using --methodlist is less common, and more error-prone, than --testjar, --test-package, --classlist, or --testclass. See the notes about specifying methods that may appear in a test.

      • omit_classes

        public static List<Pattern> omit_classes
        A regex that indicates classes that should not be used in tests, even if included by some other command-line option. The regex is matched against fully-qualified class names. If the regular expression contains anchors "^" or "$", they refer to the beginning and the end of the class name.
      • omit_classes_file

        public static List<Path> omit_classes_file
        A file containing a list of regular expressions that indicate classes not to call in a test. These patterns are used along with those provided with --omit-classes.
      • omit_methods

        public static List<Pattern> omit_methods
        A regex that indicates methods that should not be called directly in generated tests. This does not prevent indirect calls to such methods from other, allowed methods; to prevent them, see the replacecall agent.

        Randoop will not directly call a method whose fully-qualified signature matches the regular expression, or a method inherited from a superclass or interface whose signature matches the regular expression.

        If the regular expression contains anchors "^" or "$", they refer to the beginning and the end of the signature string.

        Methods replaced by the replacecall agent are also automatically omitted.

      • omit_methods_file

        public static List<Path> omit_methods_file
        A file containing a list of regular expressions that indicate methods that should not be included in generated tests. These patterns are used along with those provided with --omit-methods, and the default omissions.
      • omit_methods_no_defaults

        public static boolean omit_methods_no_defaults
        Include methods that are otherwise omitted by default. Unless you set this to true, every method replaced by the replacecall agent is treated as if it had been supplied as an argument to --omit-methods.
      • omit_classes_no_defaults

        public static boolean omit_classes_no_defaults
        Include classes that are otherwise omitted by default.
      • dont_omit_replaced_methods

        public static boolean dont_omit_replaced_methods
        Include methods that are otherwise omitted by default. Unless you set this to true, every method replaced by the replacecall agent is treated as if it had been supplied as an argument to --omit-methods.
      • omit_field

        public static List<String> omit_field
        A fully-qualified field name of a field to be excluded from test generation. An accessible field is used unless it is omitted by this or the --omit-field-file option.
      • omit_field_file

        public static Path omit_field_file
        File that contains fully-qualified field names to be excluded from test generation. An accessible field is used unless it is omitted by this or the --omit-field option.
      • omit_field_list

        public static Path omit_field_list
        File that contains fully-qualified field names to be excluded from test generation. An accessible field is used unless it is omitted by this or the --omit-field option.
      • only_test_public_members

        public static boolean only_test_public_members
        Restrict tests to only call public members of classes.

        When this is false, the setting of --junit-package-name and package accessibility is used to determine which members will be used in tests.

      • silently_ignore_bad_class_names

        public static boolean silently_ignore_bad_class_names
      • fail_on_generation_error

        public static boolean fail_on_generation_error
        (For debugging.) If an error or exception is thrown during type instantiation or input selection, this option allows the error to be passed through to GenTests.handle(String[]) where a comprehensive error message is printed.
      • flaky_test_behavior

        public static GenInputsAbstract.FlakyTestAction flaky_test_behavior
        What to do if Randoop generates a flaky test. A flaky test is one that behaves differently on different executions.

        Flaky tests are usually due to calling Randoop on side-effecting or nondeterministic methods, and ultimately, the solution is not to call Randoop on such methods; see section Nondeterminism in the Randoop manual.

      • nondeterministic_methods_to_output

        public static int nondeterministic_methods_to_output
        How many suspected side-effecting or nondeterministic methods (from the program under test) to print.
      • no_error_revealing_tests

        public static boolean no_error_revealing_tests
        Whether to output error-revealing tests. Disables all output when used with --no-regression-tests. Restricting output can result in long runs if the default values of --generated-limit and --time-limit are used.
      • no_regression_tests

        public static boolean no_regression_tests
        Whether to output regression tests. Disables all output when used with --no-error-revealing-tests. Restricting output can result in long runs if the default values of --generated-limit and --time-limit are used.
      • no_regression_assertions

        public static boolean no_regression_assertions
        Whether to include assertions in regression tests. If false, then the regression tests contain no assertions (except that if the test throws an exception, it should continue to throw an exception of the same type). Tests without assertions can be used to exercise the code, but they do not enforce any particular behavior, such as values returned.
      • check_compilable

        public static boolean check_compilable
        Whether to check that generated sequences can be compiled. If true, the code for each generated sequence is compiled, and the sequence is only kept if the compilation succeeds without error. This check is useful because the assumptions in Randoop generation heuristics are sometimes violated by input methods, and, as a result, a generated test may not compile. This check does increases the runtime by approximately 50%.
      • require_classname_in_test

        public static Pattern require_classname_in_test
        Classes that must occur in a test. Randoop will only output tests whose source code has at least one use of a member of a class whose name matches the regular expression.
      • require_covered_classes

        public static Path require_covered_classes
        File containing binary names of classes that the tests must use, directly or indirectly. This option only works if Randoop is run using the covered-class javaagent to instrument the classes. A test is output only if it uses at least one of the class names in the file. A test uses a class if it invokes any constructor or method of the class, directly or indirectly (the constructor or method might not appear in the source code of the test). Included classes may be abstract.
      • minimize_error_test

        public static boolean minimize_error_test
        If true, Randoop outputs both original error-revealing tests and a minimized version. Setting this option may cause long Randoop run times if Randoop outputs and minimizes more than about 100 error-revealing tests; consider using --stop-on-error-test=true. Also see the test case minimization options.
      • checked_exception

        public static GenInputsAbstract.BehaviorType checked_exception
        If a test throws a checked exception, should it be included in the error-revealing test suite (value: ERROR), regression test suite (value: EXPECTED), or should it be discarded (value: INVALID)?
      • unchecked_exception

        public static GenInputsAbstract.BehaviorType unchecked_exception
        If a test throws an unchecked exception other than ConcurrentModificationException, NoClassDefFoundError, NullPointerException, OutOfMemoryError, and StackOverflowError, should the test be included in the error-revealing test suite (value: ERROR), regression test suite (value: EXPECTED), or should it be discarded (value: INVALID)?

        The arguments --cm-exception, --ncdf-exception, --npe-on-null-input, --npe-on-non-null-input, --oom-exception, and --sof-exception handle special cases of unchecked exceptions.

      • cm_exception

        public static GenInputsAbstract.BehaviorType cm_exception
        If a test throws a ConcurrentModificationException exception, should it be included in the error-revealing test suite (value: ERROR), regression test suite (value: EXPECTED), or should it be discarded (value: INVALID)?
      • ncdf_exception

        public static GenInputsAbstract.BehaviorType ncdf_exception
        If a test throws a NoClassDefFoundError exception, should it be included in the error-revealing test suite (value: ERROR), regression test suite (value: EXPECTED), or should it be discarded (value: INVALID)?
      • npe_on_null_input

        public static GenInputsAbstract.BehaviorType npe_on_null_input
        If a test that passes null as an argument throws a NullPointerException, should the test be be included in the error-revealing test suite (value: ERROR), regression test suite (value: EXPECTED), or should it be discarded (value: INVALID)?
      • npe_on_non_null_input

        public static GenInputsAbstract.BehaviorType npe_on_non_null_input
        If a test that never passes null as an argument throws a NullPointerException, should the test be included in the error-revealing test suite (value: ERROR), regression test suite (value: EXPECTED), or should it be discarded (value: INVALID)?
      • oom_exception

        public static GenInputsAbstract.BehaviorType oom_exception
        If a test throws an OutOfMemoryError exception, should it be included in the error-revealing test suite (value: ERROR), regression test suite (value: EXPECTED), or should it be discarded (value: INVALID)?
      • sof_exception

        public static GenInputsAbstract.BehaviorType sof_exception
        If a test throws a StackOverflowError exception, should it be included in the error-revealing test suite (value: ERROR), regression test suite (value: EXPECTED), or should it be discarded (value: INVALID)?
      • use_jdk_specifications

        public static boolean use_jdk_specifications
        Use built-in specifications for JDK classes and for classes that inherit from them, as if they had been supplied using the --specifications command-line argument.
      • ignore_condition_compilation_error

        public static boolean ignore_condition_compilation_error
        Make Randoop proceed, instead of failing, if the Java condition text of a specification cannot be compiled.
      • ignore_condition_exception

        public static boolean ignore_condition_exception
        Make Randoop treat a specification whose execution throws an exception as returning false. If true, Randoop treats x.f == 22 equivalently to the wordier x != null && x.f == 22. If false, Randoop halts when a specification throws an exception.
      • ignore_condition_exception_quiet

        public static boolean ignore_condition_exception_quiet
        If true, don't print diagnostics about specification that throw an exception. Has no effect unless --ignore-condition-exception is set.
      • side_effect_free_methods

        public static Path side_effect_free_methods
        File containing side-effect-free methods, each given as a fully-qualified signature on a separate line. Specifying side-effect-free methods has two benefits: it makes regression tests stronger, and it helps Randoop create smaller tests.
      • time_limit

        public static int time_limit
        Maximum number of seconds to spend generating tests. Zero means no limit. If nonzero, Randoop is nondeterministic: it may generate different test suites on different runs.

        This is the overall limit, not the limit per class under test. The default value is too small to be effective for generating tests for an entire project. If you are testing multiple classes, you may wish to multiply the default value by the number of classes under test.

        Randoop may run for longer than this because of a long-running test. The elapsed time is checked after each test, not during a test's execution.

      • LIMIT_DEFAULT

        private static int LIMIT_DEFAULT
      • attempted_limit

        public static int attempted_limit
        Maximum number of attempts to generate a test method candidate.
      • generated_limit

        public static int generated_limit
        Maximum number of test method candidates generated internally.
      • output_limit

        public static int output_limit
        The maximum number of regression and error-revealing tests to output. If there is no output, this limit has no effect. There is no output when using either --dont-output-tests or --no-error-revealing-tests together with --no-regression-tests.

        In the current implementation, the number of tests in the output can be substantially smaller than this limit. One reason is that Randoop does not output subsumed tests, which appear as a subsequence of some longer test.

      • maxsize

        public static int maxsize
        Do not generate tests with more than this many statements.
      • stop_on_error_test

        public static boolean stop_on_error_test
        Stop generation as soon as one error-revealing test has been generated. Implies --minimize-error-test.
      • null_ratio

        public static double null_ratio
        Use null with the given frequency as an argument to method calls.

        For example, a null ratio of 0.05 directs Randoop to use null as an input 5 percent of the time when a non-null value of the appropriate type is available.

        Unless --forbid_null is true, a null value will still be used if no other value can be passed as an argument even if --null-ratio=0.

        Randoop never uses null for receiver values.

      • forbid_null

        public static boolean forbid_null
        Do not use null as input to methods or constructors, even when no other argument value can be generated.

        If true, Randoop will not generate a test when unable to find a non-null value of appropriate type as an input. This could result in certain class members being untested.

      • literals_file

        public static List<String> literals_file
        A file containing literal values to be used as inputs to methods under test, or "CLASSES".

        Literals in these files are used in addition to all other constants in the pool. For the format of this file, see documentation in class LiteralFileReader. The special value "CLASSES" (with no quotes) means to read literals from all classes under test.

      • method_selection

        public static GenInputsAbstract.MethodSelectionMode method_selection
        Randoop generates new tests by choosing from a set of methods under test. This controls how the next method is chosen, from among all methods under test.
      • bloodhound_logging

        public static boolean bloodhound_logging
        Print to standard out, method weights and method uncovered ratios.
      • bloodhound_update_mode

        public static GenInputsAbstract.BloodhoundCoverageUpdateMode bloodhound_update_mode
        Bloodhound can update coverage information at a regular interval that is either based on time or on the number of successful invocations.
      • string_maxlen

        public static int string_maxlen
        Maximum length of strings in generated tests, including in assertions. Strings longer than 65KB (or about 10,000 characters) may be rejected by the Java compiler, according to the Java Virtual Machine specification.
      • alias_ratio

        public static double alias_ratio
        Try to reuse values from a sequence with the given frequency. If an alias ratio is given, it should be between 0 and 1.

        A ratio of 0 results in tests where each value created within a test input is typically used at most once as an argument in a method call. A ratio of 1 tries to maximize the number of times values are used as inputs to parameters within a test.

      • input_selection

        public static GenInputsAbstract.InputSelectionMode input_selection
        Randoop generates new tests by combining old previously-generated tests. This controls how the old tests are chosen, from among all existing tests.
      • clear

        public static int clear
        Clear the component set each time it contains the given number of inputs.

        Randoop stores previously-generated tests in a "component" set, and uses them to generate new tests. Setting this variable to a small number can sometimes result in a greater variety of tests generated during a single run.

      • clear_memory

        public static long clear_memory
        Clear the component set each time Randoop uses this much memory.

        Setting this variable to a smaller number may prevent an out-of-memory exception or a run that is slow due to thrashing and garbage collection.

      • testsperfile

        public static int testsperfile
        Maximum number of tests to write to each JUnit file.
      • error_test_basename

        public static String error_test_basename
        Base name (no ".java" suffix) of the JUnit file containing error-revealing tests
      • regression_test_basename

        public static String regression_test_basename
        Class name for the JUnit regression tests. Equivalently, the base name (no ".java" suffix) of the JUnit file containing regression tests
      • junit_package_name

        public static String junit_package_name
        Name of the package for the generated JUnit files. Enables testing non-public members. Tests can be restricted to public members only by also using the option --only-test-public-members.
      • junit_before_each

        public static String junit_before_each
        Name of file containing code text to be added to the @Before-annotated method of each generated test class. Code is uninterpreted, and, so, is not run during generation. Intended for use when run-time behavior of classes under test requires setup behavior that is not needed for execution by reflection. (The annotation @Before is JUnit 4, and @BeforeEach is JUnit 5.)
      • junit_after_each

        public static String junit_after_each
        Name of file containing code text to be added to the @After -annotated method of each generated test class. Intended for use when run-time behavior of classes under test requires tear-down behavior that is not needed for execution by reflection. Code is uninterpreted, and, so, is not run during generation. (The annotation @After is JUnit 4, and @AfterEach is JUnit 5.)
      • junit_before_all

        public static String junit_before_all
        Name of file containing code text to be added to the @BeforeClass-annotated method of each generated test class. Intended for use when run-time behavior of classes under test requires setup behavior that is not needed for execution by reflection. Code is uninterpreted, and, so, is not run during generation. (The annotation @BeforeClass is JUnit 4, and @BeforeAll is JUnit 5.)
      • junit_after_all

        public static String junit_after_all
        Name of file containing code text to be added to the @AfterClass-annotated method of each generated test class. Intended for use when run-time behavior of classes under test requires tear-down behavior that is not needed for execution by reflection. Code is uninterpreted, and, so, is not run during generation. (The annotation @AfterClass is JUnit 4, and @AfterAll is JUnit 5.)
      • junit_output_dir

        public static String junit_output_dir
        Name of the directory in which JUnit files should be written.
      • dont_output_tests

        public static boolean dont_output_tests
        Run test generation without output. May be desirable when running with a visitor.

        NOTE: Because there is no output, the value of --output-limit will never be met, so be sure to set --generated-limit or --time-limit to a reasonable value when using this option.

      • junit_reflection_allowed

        public static boolean junit_reflection_allowed
        Whether to use JUnit's standard reflective mechanisms for invoking tests. JUnit's reflective invocations can interfere with code instrumentation, such as by the DynComp tool. If that is a problem, then set this to false and Randoop will output tests that use direct method calls instead of reflection. The tests will include a main method and will execute methods and assertions, but won't be JUnit suites.
      • system_props

        public static List<String> system_props
        System properties that Randoop will set similarly to java -D, of the form "x=y".
      • jvm_max_memory

        public static String jvm_max_memory
        How much memory Randoop should use when starting new JVMs. This only affects new JVMs; you still need to supply -Xmx... when starting Randoop itself.
      • capture_output

        public static boolean capture_output
      • randomseed

        public static int randomseed
        The random seed to use in the generation process. You do not need to provide this option to make Randoop deterministic, because Randoop is deterministic by default. It is recommended to run Randoop multiple times with a different random seed, in order to produce multiple different test suites.
      • deterministic

        public static boolean deterministic
        If true, Randoop is deterministic: running Randoop twice with the same arguments (including --randomseed) will produce the same test suite, so long as the program under test is deterministic. If false, Randoop may or may not produce the same test suite. To produce multiple different test suites, use the --randomseed command-line option.
      • progressdisplay

        public static boolean progressdisplay
      • PROGRESSINTERVALMILLIS_DEFAULT

        public static long PROGRESSINTERVALMILLIS_DEFAULT
      • progressintervalmillis

        public static long progressintervalmillis
      • progressintervalsteps

        public static long progressintervalsteps
      • debug_checks

        public static boolean debug_checks
      • log

        public static org.plumelib.util.FileWriterWithName log
        A file to which to log lots of information. If not specified, no logging is done. Enabling the logs slows down Randoop.
      • selection_log

        public static org.plumelib.util.FileWriterWithName selection_log
        A file to which to log selections; helps find sources of non-determinism. If not specified, no logging is done.
      • operation_history_log

        public static org.plumelib.util.FileWriterWithName operation_history_log
        A file to which to write operation usage, when Randoop exits.
      • print_non_compiling_file

        public static boolean print_non_compiling_file
        True if Randoop should print generated tests that do not compile, which indicate Randoop bugs.
      • dontexecute

        public static boolean dontexecute
        Create sequences but never execute them. Used to test performance of Randoop's sequence generation code.
      • visitor

        public static List<@ClassGetName String> visitor
        Install the given runtime visitor. See class randoop.ExecutionVisitor.
      • repeat_heuristic

        public static boolean repeat_heuristic
    • Method Detail

      • checkOptionsValid

        public void checkOptionsValid()
        Check that the options given satisfy any specified constraints, and fail if they do not.
      • validateClassName

        static void validateClassName​(String className,
                                      String commandLineOption)
        Validates an argument that should be a class name. Throws RandoopUsageError if it is not valid.
        Parameters:
        className - an argument that should be a class name
        commandLineOption - the command line option name, without leading "--"
      • shouldOmitClass

        private static boolean shouldOmitClass​(String classname)
        Returns true if the class should be omitted, according to the omit_classes field.
        Parameters:
        classname - a class name
        Returns:
        true if the class should be omitted
      • getClassnamesFromArgs

        public static Set<@ClassGetName String> getClassnamesFromArgs​(AccessibilityPredicate accessibility)
        Read names of classes under test, as provided with the --classlist or --testjar command-line argument.
        Parameters:
        accessibility - the accessibility predicate
        Returns:
        the classes provided via the --classlist or --testjar command-line argument
      • getClassnamesFromJarFile

        public static Set<@ClassGetName String> getClassnamesFromJarFile​(Path jarFile,
                                                                         AccessibilityPredicate accessibility)
        Read names of classes from a jar file. Ignores interfaces, abstract classes, non-accessible classes, and those that cannot be loaded.
        Parameters:
        jarFile - the jar file from which to read classes
        accessibility - the accessibility predicate
        Returns:
        the names of classes in the jar file
      • getClassnamesFromPackage

        private static List<@ClassGetName String> getClassnamesFromPackage​(String packageName,
                                                                           AccessibilityPredicate accessibility)
        Returns classes from the classpath that are in the given package. Does not include classes in sub-packages. Ignores non-accessible classes.
        Parameters:
        packageName - a package name; may be the empty string
        accessibility - the accessibility predicate
        Returns:
        classes in package packageName
      • getClassesWithPackageFromDirectory

        private static List<@ClassGetName String> getClassesWithPackageFromDirectory​(File directory,
                                                                                     String packageName,
                                                                                     AccessibilityPredicate accessibility)
        Given a directory on the CLASSPATH, returns classes in the given package. These classes are found in a subdirectory of the given directory, whose name depends on the given package.
        Parameters:
        directory - a directory on the CLASSPATH
        packageName - a package name
        accessibility - the accessibility predicate
        Returns:
        classes with the given package
      • getClassesWithPackageFromJar

        private static List<@ClassGetName String> getClassesWithPackageFromJar​(File jarFile,
                                                                               String packageName,
                                                                               AccessibilityPredicate accessibility)
        Returns classes with the given package in the given jar file.
        Parameters:
        jarFile - a jar file containing classes
        packageName - a package name
        accessibility - the accessibility predicate
        Returns:
        classes in package packageName in the given jar file
      • getClassNamesFromFile

        public static Set<@ClassGetName String> getClassNamesFromFile​(Path file)
        Returns the class names listed in the file.
        Parameters:
        file - the file containing the strings
        Returns:
        the lines in the file, or null if listFile is null
      • getStringSetFromFile

        public static Set<String> getStringSetFromFile​(Path listFile,
                                                       String fileDescription)
        Returns a set consisting of the lines of the file, except those starting with "#". Returns empty set if listFile is null.
        Parameters:
        listFile - the file containing the strings
        fileDescription - string used in error messages
        Returns:
        the lines in the file, or null if listFile is null
      • getStringSetFromFile

        public static Set<String> getStringSetFromFile​(@Nullable Path listFile,
                                                       String fileDescription,
                                                       String commentRegex,
                                                       String includeRegex)
        Returns a set consisting of the lines of the file. Returns empty set if listFile is null.
        Parameters:
        listFile - the file containing the strings
        fileDescription - string used in error messages
        commentRegex - indicates which lines are comments that should be ignored
        includeRegex - if this string appears in the file, then another file is recursively read
        Returns:
        the strings in the file, or null if listFile is null